Text matching method, device and system and storage medium

Abstract

The invention provides a text matching method, device and system and a storage medium, and belongs to the technical field of text matching, the method comprises the following steps: importing original texts, and taking every two original texts as one group to obtain an original statement group; constructing an original language model, and performing model analysis on the original language model according to the original statement group to obtain a target language model; and importing a to-be-processed text, and matching the to-be-processed text according to the target language model to obtain a text matching result. According to the method, the diversity and richness of the text are enhanced, the over-smoothing problem is effectively relieved, the more accurate semantic matching capability is provided, a breakthrough is also realized in the semantic search field, the model can quickly and accurately obtain a result with the closest semantics from a large amount of text data, and the text matching accuracy is improved.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of text matching, and in particular to a text matching method, device, system and storage medium. Background Art

[0002] With the development of the field of natural language processing (NLP), effective text representation learning has become a key research direction. Text representation not only needs to capture the rich semantics of the language, but also needs to accurately reflect the subtle differences between sentences. With the development of deep learning technology, language models (PLMs) such as BERT and RoBERTa have been able to provide powerful text encoding capabilities through pre-training on massive text data. However, as the depth of the model increases, these models face the problem of over-smoothing in text representation, that is, the representations of different texts tend to be similar, resulting in information loss and reduced differentiation, which also reduces the accuracy of text matching.

[0003] Traditional language models usually adopt simple methods, such as using only the embedding of the last layer or the average embedding of all layers to generate text representations. However, this method may ignore the important information contained in other layers, and the fixed pooling strategy cannot adapt to all situations because different fields or tasks may require information from different layers, which limits the performance improvement and also causes text mismatch. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a text matching method, device, system and storage medium in view of the deficiencies in the prior art.

[0005] The technical solution of the present invention to solve the above technical problem is as follows: a text matching method, comprising the following steps:

[0006] Importing multiple original texts, and grouping all the original texts in pairs, thereby obtaining multiple original sentence groups;

[0007] Constructing an original language model, and performing model analysis on the original language model according to all the original sentence groups to obtain a target language model;

[0008] Import multiple texts to be processed, match all the texts to be processed according to the target language model, and obtain text matching results.

[0009] Another technical solution of the present invention to solve the above technical problem is as follows: a text matching device, comprising:

[0010] Import module, used to import multiple raw texts;

[0011] A grouping module, used for grouping all the original texts into groups of two each, thereby obtaining a plurality of original sentence groups;

[0012] A model analysis module, used for constructing an original language model, performing model analysis on the original language model according to all the original sentence groups, and obtaining a target language model;

[0013] The import module is also used to import multiple texts to be processed;

[0014] The text matching result obtaining module is used to match all the to-be-processed texts according to the target language model to obtain text matching results.

[0015] Based on the above text matching method, the present invention also provides a text matching system.

[0016] Another technical solution of the present invention to solve the above technical problem is as follows: a text matching system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the text matching method described above is implemented.

[0017] Based on the above text matching method, the present invention also provides a computer-readable storage medium.

[0018] Another technical solution of the present invention to solve the above technical problem is as follows: a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the text matching method as described above is implemented.

[0019] The beneficial effects of the present invention are as follows: by grouping all original texts in pairs, original sentence groups are obtained, a target language model is obtained by analyzing the original language model according to the original sentence groups, and a text matching result is obtained by matching the text to be processed according to the target language model, thereby enhancing the diversity and richness of the text, effectively alleviating the over-smoothing problem, providing more accurate semantic matching capabilities, and achieving a breakthrough in the field of semantic search, so that the model can quickly and accurately obtain the most semantically similar results from a large amount of text data, thereby improving the accuracy of text matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of a text matching method provided by an embodiment of the present invention;

[0021] Figure 2 A module block diagram of a text matching device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0023] Figure 1 A flowchart of a text matching method provided by an embodiment of the present invention.

[0024] like Figure 1 As shown, a text matching method comprises the following steps:

[0025] Importing multiple original texts, and grouping all the original texts in pairs, thereby obtaining multiple original sentence groups;

[0026] Constructing an original language model, and performing model analysis on the original language model according to all the original sentence groups to obtain a target language model;

[0027] Import multiple texts to be processed, match all the texts to be processed according to the target language model, and obtain text matching results.

[0028] It should be understood that the acquired corpus (ie, original text) is used to generate sentence pairs (ie, original sentence groups).

[0029] In the above embodiment, all original texts are grouped together in pairs to obtain original sentence groups, and a target language model is obtained by analyzing the original language model based on the original sentence groups. The text to be processed is matched with the target language model to obtain a text matching result, thereby enhancing the diversity and richness of the text, effectively alleviating the over-smoothing problem, providing more accurate semantic matching capabilities, and achieving a breakthrough in the field of semantic search, so that the model can quickly and accurately obtain the most semantically similar results from a large amount of text data, thereby improving the accuracy of text matching.

[0030] Optionally, as an embodiment of the present invention, the process of performing model analysis on the original language model according to all the original sentence groups to obtain the target language model includes:

[0031] Inputting each of the original sentence groups into the original language model respectively, obtaining a sentence feature vector group corresponding to each of the original sentence groups, wherein the sentence feature vector group includes a plurality of first sentence feature vectors arranged in an output order and a plurality of second sentence feature vectors arranged in an output order, wherein the plurality of first sentence feature vectors are feature vectors after multiple CLS pooling in the original language model, and the plurality of second sentence feature vectors are feature vectors after multiple average pooling in the original language model;

[0032] Calculating target importance scores for the multiple first sentence feature vectors corresponding to each of the original sentence groups and the multiple second sentence feature vectors corresponding to each of the original sentence groups, respectively, to obtain target importance scores corresponding to each of the original sentence groups;

[0033] The third sentence feature vector is calculated for the last first sentence feature vector corresponding to each of the original sentence groups and the target importance score corresponding to each of the original sentence groups through the first formula, and the third sentence feature vector corresponding to each of the original sentence groups is obtained, and the third sentence feature vector is used as a model positive sample. The first formula is:

[0034]

[0035] in, is the third sentence feature vector corresponding to the e-th original sentence group, is the last first sentence feature vector corresponding to the e-th original sentence group, is the target importance score corresponding to the e-th original sentence group;

[0036] The fourth sentence feature vector is calculated for each of the third sentence feature vectors by the second formula to obtain the fourth sentence feature vector corresponding to each of the original sentence groups, and the fourth sentence feature vector is used as a model negative sample. The second formula is:

[0037]

[0038] Among them, y e is the fourth sentence feature vector corresponding to the e-th original sentence group, MLP() is the multi-layer perceptron function, is the third sentence feature vector corresponding to the e-th original sentence group;

[0039] Calculating the loss value of each of the model positive samples and the model negative samples corresponding to each of the original sentence groups respectively to obtain the loss value corresponding to each of the original sentence groups;

[0040] The parameters of the original language model are updated according to all the loss values ​​to obtain a target language model.

[0041] It should be understood that CLS pooling is a special type of pooling operation, which is usually used for image classification tasks in deep learning. Its core idea is to extract feature representations of images through pooling operations at the final stage of convolutional neural networks (CNNs), and then use these features for classification.

[0042] It should be understood that average pooling is a common pooling operation in convolutional neural networks (CNNs) in deep learning. Its purpose is to reduce the spatial dimensions of the data (e.g., the width and height of the image) while retaining important information as much as possible. Average pooling is achieved by calculating the average value within each pooling window in the input feature map.

[0043] Specifically, the obtained sentence pairs (i.e., the original sentence group) are input into the pre-trained language model (i.e., the original language model), and weights are assigned layer by layer to the original pre-trained language model (i.e., the original language model) using a layer-by-layer pooling method based on a multiplicative attention mechanism. A multi-layer perceptron (MLP) layer is added to the pooled layer vector by randomly initializing it to obtain a vector with the same dimension as the original pre-trained language model (i.e., the fourth sentence feature vector).

[0044] Specifically, a multi-layer perceptron (MLP) layer is added with random initialization after pooling and its random initialization is maintained. In the formula, The input of the last layer represents the vector of the [CLS] tag (i.e. the last first sentence feature vector). (i.e. the last first sentence feature vector) and the layer representation y B (i.e., target importance score) is concatenated as the input vector y of the MLP LP (i.e., the third sentence feature vector). Finally, y is represented by the MLP layer with the same dimension as the sentence representation dimension of the original language model (i.e., the fourth sentence feature vector). The formula is as follows:

[0045]

[0046] y=MLP(y LP ).

[0047] In the above embodiment, the target language model is obtained by performing model analysis on the original language model according to all the original sentence groups, which enhances the diversity and richness of the text, effectively alleviates the over-smoothing problem, provides more accurate semantic matching capabilities, and also achieves a breakthrough in the field of semantic search, so that the model can quickly and accurately obtain the semantically closest results from a large amount of text data, thereby improving the accuracy of text matching.

[0048] Optionally, as an embodiment of the present invention, the process of respectively calculating the target importance scores of the multiple first sentence feature vectors corresponding to each of the original sentence groups and the multiple second sentence feature vectors corresponding to each of the original sentence groups to obtain the target importance scores corresponding to each of the original sentence groups includes:

[0049] The target importance scores are calculated for the multiple first sentence feature vectors corresponding to each of the original sentence groups and the multiple second sentence feature vectors corresponding to each of the original sentence groups through the third formula to obtain the target importance scores corresponding to each of the original sentence groups. The third formula is:

[0050]

[0051] in,

[0052] in,

[0053] in, is the target importance score corresponding to the e-th original sentence group, L is the total number of feature vectors of the first sentence or the total number of feature vectors of the second sentence, is the initial importance score corresponding to the mth first sentence feature vector of the eth original sentence group, β e-m is the importance weight corresponding to the mth first sentence feature vector of the eth original sentence group, F c 、F x and F z are all learnable parameters. is the mth second sentence feature vector corresponding to the eth original sentence group, is the mth first sentence feature vector corresponding to the eth original sentence group.

[0054] Specifically, the performance when using [CLS] pooling (label classification pooling) varies significantly depending on the pooling layer, and when using AVG pooling, there is no significant difference in performance between layers. This can be explained by the fact that each layer can contain different information represented by the sentence, while average pooling can reduce the information gap between layers. Combining the characteristics of both, assigning different weights to each layer in the model can highlight those information layers that are more critical to downstream tasks, while suppressing those information layers that may contain noise or are less important. The formula is as follows:

[0055]

[0056] The subscript m indicates the number of layers, y v is the AVG representation, i.e., the average vector of the tokens in the sentence (i.e., the second sentence feature vector), y o is the input of each layer representing the vector of [CLS] tags (i.e., the first sentence feature vector), β m is the importance (i.e., importance weight) of the mth layer. b is the representation of each layer with importance scores (i.e., initial importance scores). B Yes bThe average vector of (i.e., target importance score) is also a representation of the correlation of all layers (L is the number of layers). z 、F x and F c are learnable parameters.

[0057] In the above embodiment, the target importance scores are calculated for the first sentence feature vector and the second sentence feature vector respectively to obtain the target importance scores, which can highlight the information layers that are more critical to the downstream tasks, while suppressing the information layers that contain noise or are less important, thereby enhancing the diversity and richness of the text, effectively alleviating the over-smoothing problem, and providing more accurate semantic matching capabilities.

[0058] Optionally, as an embodiment of the present invention, the process of respectively calculating the loss value of each of the model positive samples and the model negative samples corresponding to each of the original sentence groups to obtain the loss value corresponding to each of the original sentence groups includes:

[0059] The loss values ​​of each of the model positive samples and the model negative samples corresponding to each of the original sentence groups are calculated by the fourth formula to obtain the loss values ​​corresponding to each of the original sentence groups. The fourth formula is:

[0060]

[0061] Among them, I e is the loss value corresponding to the e-th original sentence group, is the model positive sample corresponding to the e-th original sentence group, t is the temperature coefficient, H is the total number of original sentence groups, is the model negative sample corresponding to the fth original sentence group, sim(,) is the cosine similarity function, is the model negative sample corresponding to the e-th original sentence group.

[0062] It should be understood that the obtained hierarchical dimensional representation is subjected to unsupervised contrastive learning to obtain a training target, and the obtained hidden representation of the intermediate layer is used as a negative sample (ie, a model negative sample) to optimize the unsupervised contrastive learning.

[0063] Specifically, let y e (i.e., model negative samples) and (i.e., model positive samples) represent the sentence representation of the last layer in PLMs. For a mini-batch containing H pairs, The loss function I e It is expressed as:

[0064]

[0065] in For sentence ye The positive sample (i.e., the model positive sample) is obtained by LP The same batch of sentence representations generated (y LP ), by maximizing the similarity between positive sample pairs to guide model learning, the model learns how to better capture and represent the semantic content of sentences, enhancing the richness and accuracy of sentence representation. sim(,) represents the cosine similarity function, and t is the temperature coefficient. This method focuses on mining positive samples while directly using negative samples in the batch during training.

[0066] The hidden representations of the intermediate layers of PLMs are used as negative samples (i.e., model negative samples), and the representations of the last layer are regarded as the final sentence representations that need to be optimized. Specifically, the sentence representations of the intermediate layers in PLMs are collected and regarded as negative samples of the representations of the last layer, and are named Therefore, we obtain a negative As mentioned above, the same as positive samples obtained by any data augmentation method. Then, the loss function I e It can be restated as follows:

[0067]

[0068] The first term in the denominator refers to the original in-batch negative samples, and the obtained y is used as the sentence representation of the original in-batch negative sample pair. The second term represents the intermediate negative samples. Through this method, the last layer representation of PLMs is made more distinguishable from the previous layers by simply increasing the number of negative samples, thereby alleviating the over-smoothing problem.

[0069] In the above embodiment, the loss value is calculated for each model positive sample and model negative sample respectively, and the number of negative samples is increased, so that the last layer representation of PLMs is more distinguishable from the previous layers, thereby alleviating the over-smoothing problem and enhancing the richness and accuracy of sentence representation.

[0070] Optionally, as another embodiment of the present invention, the present invention includes:

[0071] By applying an attention mechanism to each layer of a pre-trained language model (PLMs), different weights are assigned to representations at different levels, and a self-contrastive learning strategy is introduced, the representations of the middle layers of the model are used as negative samples, and the contrast and discrimination of sentence representations are enhanced for learning unsupervised sentence representations. The beneficial effects of the present invention are as follows: the present invention proposes a layer-by-layer attention pooling strategy that assigns weights to each layer, and in order to solve the over-smoothing problem, the quality of sentence representations is improved by sampling negative samples from the middle layers of PLMs, which is used for learning unsupervised sentence representations and can be easily extended to various state-of-the-art models to improve performance. Ultimately, the performance of the model on specific NLP tasks is improved, especially in sentence semantic similarity measurement and semantic search.

[0072] Optionally, as another embodiment of the present invention, the present invention includes:

[0073] S1. Generate sentence pairs from the acquired corpus;

[0074] S2, input the sentence pairs in S1 into the pre-trained language model;

[0075] S3, using a layer-by-layer pooling method based on a multiplicative attention mechanism to assign weights layer by layer to the original pre-trained language model in S2;

[0076] S4, randomly initialize the pooled layer vector in S3 and add a multi-layer perceptron (MLP) layer to obtain a vector with the same dimension as the original pre-trained language model;

[0077] S5, performing unsupervised contrastive learning on the hierarchical dimension representation obtained in S4 to obtain a training target;

[0078] S6. Use the hidden representation of the intermediate layer obtained in S4 as a negative sample to optimize the unsupervised contrastive learning in S5.

[0079] Optionally, as another embodiment of the present invention, the present invention is based on the attention mechanism and can weight the output of each layer in PLMs to highlight the information that is more critical to downstream tasks. In this way, the model can not only integrate semantic information from different levels, but also enhance the diversity and richness of sentence representation. At the same time, an unsupervised self-contrastive learning strategy is proposed, which effectively alleviates the over-smoothing problem by generating negative samples from the middle layer of PLMs. Combining these two technical inventions, the information utilization method of PLMs in sentence representation learning is improved, which not only provides the model with more accurate semantic matching capabilities in the task of measuring sentence semantic similarity, but also achieves a breakthrough in the field of semantic search, enabling the model to quickly and accurately retrieve the results that are closest to the query semantics from a large amount of text data.

[0080] Figure 2A module block diagram of a text matching device provided in an embodiment of the present invention.

[0081] Optionally, as another embodiment of the present invention, Figure 2 As shown, a text matching device comprises:

[0082] Import module, used to import multiple raw texts;

[0083] A grouping module, used for grouping all the original texts into groups of two each, thereby obtaining a plurality of original sentence groups;

[0084] A model analysis module, used for constructing an original language model, performing model analysis on the original language model according to all the original sentence groups, and obtaining a target language model;

[0085] The import module is also used to import multiple texts to be processed;

[0086] The text matching result obtaining module is used to match all the to-be-processed texts according to the target language model to obtain text matching results.

[0087] Optionally, as an embodiment of the present invention, the model analysis module is specifically used for:

[0088] Inputting each of the original sentence groups into the original language model respectively, obtaining a sentence feature vector group corresponding to each of the original sentence groups, wherein the sentence feature vector group includes a plurality of first sentence feature vectors arranged in an output order and a plurality of second sentence feature vectors arranged in an output order, wherein the plurality of first sentence feature vectors are feature vectors after multiple CLS pooling in the original language model, and the plurality of second sentence feature vectors are feature vectors after multiple average pooling in the original language model;

[0089] Calculating target importance scores for the multiple first sentence feature vectors corresponding to each of the original sentence groups and the multiple second sentence feature vectors corresponding to each of the original sentence groups, respectively, to obtain target importance scores corresponding to each of the original sentence groups;

[0090] The third sentence feature vector is calculated for the last first sentence feature vector corresponding to each of the original sentence groups and the target importance score corresponding to each of the original sentence groups through the first formula, and the third sentence feature vector corresponding to each of the original sentence groups is obtained, and the third sentence feature vector is used as a model positive sample. The first formula is:

[0091]

[0092] in, is the third sentence feature vector corresponding to the e-th original sentence group, is the last first sentence feature vector corresponding to the e-th original sentence group, is the target importance score corresponding to the e-th original sentence group;

[0093] The fourth sentence feature vector is calculated for each of the third sentence feature vectors by the second formula to obtain the fourth sentence feature vector corresponding to each of the original sentence groups, and the fourth sentence feature vector is used as a model negative sample. The second formula is:

[0094]

[0095] Among them, y e is the fourth sentence feature vector corresponding to the e-th original sentence group, MLP() is the multi-layer perceptron function, is the third sentence feature vector corresponding to the e-th original sentence group;

[0096] Calculating the loss value of each of the model positive samples and the model negative samples corresponding to each of the original sentence groups respectively to obtain the loss value corresponding to each of the original sentence groups;

[0097] The parameters of the original language model are updated according to all the loss values ​​to obtain a target language model.

[0098] Optionally, as an embodiment of the present invention, in the model analysis module, the target importance scores are calculated for the multiple first sentence feature vectors corresponding to each of the original sentence groups and the multiple second sentence feature vectors corresponding to each of the original sentence groups, and the process of obtaining the target importance scores corresponding to each of the original sentence groups includes:

[0099] The target importance scores are calculated for the multiple first sentence feature vectors corresponding to each of the original sentence groups and the multiple second sentence feature vectors corresponding to each of the original sentence groups through the third formula to obtain the target importance scores corresponding to each of the original sentence groups. The third formula is:

[0100]

[0101] in,

[0102] in,

[0103] in, is the target importance score corresponding to the e-th original sentence group, L is the total number of feature vectors of the first sentence or the total number of feature vectors of the second sentence, is the initial importance score corresponding to the mth first sentence feature vector of the eth original sentence group, β e-m is the importance weight corresponding to the mth first sentence feature vector of the eth original sentence group, F c 、F x and F z are all learnable parameters. is the mth second sentence feature vector corresponding to the eth original sentence group, is the mth first sentence feature vector corresponding to the eth original sentence group.

[0104] Optionally, as an embodiment of the present invention, in the model analysis module, the process of calculating the loss value of each of the model positive samples and the model negative samples corresponding to each of the original sentence groups, and obtaining the loss value corresponding to each of the original sentence groups includes:

[0105] The loss values ​​of each of the model positive samples and the model negative samples corresponding to each of the original sentence groups are calculated by the fourth formula to obtain the loss values ​​corresponding to each of the original sentence groups. The fourth formula is:

[0106]

[0107] Among them, I e is the loss value corresponding to the e-th original sentence group, is the model positive sample corresponding to the e-th original sentence group, t is the temperature coefficient, H is the total number of original sentence groups, is the model negative sample corresponding to the fth original sentence group, sim(,) is the cosine similarity function, is the model negative sample corresponding to the e-th original sentence group.

[0108] Optionally, another embodiment of the present invention provides a text matching system, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the text matching method described above is implemented. The system may be a computer or other system.

[0109] Optionally, another embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the text matching method as described above is implemented.

[0110] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0111] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0113] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0114] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0115] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0116] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A text matching method, characterized in that: The steps include: Importing multiple original texts, and grouping all the original texts in pairs, thereby obtaining multiple original sentence groups; Constructing an original language model, and performing model analysis on the original language model according to all the original sentence groups to obtain a target language model; Import multiple texts to be processed, match all the texts to be processed according to the target language model, and obtain text matching results.

2. The text matching method according to claim 1, characterized in that: The process of performing model analysis on the original language model according to all the original sentence groups to obtain the target language model includes: Inputting each of the original sentence groups into the original language model respectively, obtaining a sentence feature vector group corresponding to each of the original sentence groups, wherein the sentence feature vector group includes a plurality of first sentence feature vectors arranged in an output order and a plurality of second sentence feature vectors arranged in an output order, wherein the plurality of first sentence feature vectors are feature vectors after multiple CLS pooling in the original language model, and the plurality of second sentence feature vectors are feature vectors after multiple average pooling in the original language model; Calculating target importance scores for the multiple first sentence feature vectors corresponding to each of the original sentence groups and the multiple second sentence feature vectors corresponding to each of the original sentence groups, respectively, to obtain target importance scores corresponding to each of the original sentence groups; The third sentence feature vector is calculated for the last first sentence feature vector corresponding to each of the original sentence groups and the target importance score corresponding to each of the original sentence groups through the first formula, and the third sentence feature vector corresponding to each of the original sentence groups is obtained, and the third sentence feature vector is used as a model positive sample. The first formula is: in, is the third sentence feature vector corresponding to the e-th original sentence group, is the last first sentence feature vector corresponding to the e-th original sentence group, is the target importance score corresponding to the e-th original sentence group; The fourth sentence feature vector is calculated for each of the third sentence feature vectors by the second formula to obtain the fourth sentence feature vector corresponding to each of the original sentence groups, and the fourth sentence feature vector is used as a model negative sample. The second formula is: Among them, y e is the fourth sentence feature vector corresponding to the e-th original sentence group, MLP( ) is the multi-layer perceptron function, is the third sentence feature vector corresponding to the e-th original sentence group; Calculating loss values ​​for each of the model positive samples and the model negative samples corresponding to each of the original sentence groups respectively to obtain loss values ​​corresponding to each of the original sentence groups; The parameters of the original language model are updated according to all the loss values ​​to obtain a target language model.

3. The text matching method according to claim 2, characterized in that: The process of respectively calculating the target importance scores of the plurality of first sentence feature vectors corresponding to each of the original sentence groups and the plurality of second sentence feature vectors corresponding to each of the original sentence groups to obtain the target importance scores corresponding to each of the original sentence groups includes: The target importance scores are calculated for the multiple first sentence feature vectors corresponding to each of the original sentence groups and the multiple second sentence feature vectors corresponding to each of the original sentence groups through the third formula to obtain the target importance scores corresponding to each of the original sentence groups. The third formula is: in, in, in, is the target importance score corresponding to the e-th original sentence group, L is the total number of feature vectors of the first sentence or the total number of feature vectors of the second sentence, is the initial importance score corresponding to the mth first sentence feature vector of the eth original sentence group, β e-m is the importance weight corresponding to the mth first sentence feature vector of the eth original sentence group, F c 、F x and F z are all learnable parameters. is the mth second sentence feature vector corresponding to the eth original sentence group, is the mth first sentence feature vector corresponding to the eth original sentence group.

4. The text matching method according to claim 2, characterized in that: The process of respectively calculating the loss value of each of the model positive samples and the model negative samples corresponding to each of the original sentence groups to obtain the loss value corresponding to each of the original sentence groups includes: The loss values ​​of each of the model positive samples and the model negative samples corresponding to each of the original sentence groups are calculated by the fourth formula to obtain the loss values ​​corresponding to each of the original sentence groups. The fourth formula is: Among them, I e is the loss value corresponding to the e-th original sentence group, is the model positive sample corresponding to the e-th original sentence group, t is the temperature coefficient, H is the total number of original sentence groups, is the model negative sample corresponding to the fth original sentence group, sim(,) is the cosine similarity function, is the model negative sample corresponding to the e-th original sentence group.

5. A text matching device, characterized in that: include: Import module, used to import multiple raw texts; A grouping module, used for grouping all the original texts into groups of two each, thereby obtaining a plurality of original sentence groups; A model analysis module, used for constructing an original language model, performing model analysis on the original language model according to all the original sentence groups, and obtaining a target language model; The import module is also used to import multiple texts to be processed; The text matching result obtaining module is used to match all the to-be-processed texts according to the target language model to obtain text matching results.

6. The text matching device according to claim 5, characterized in that: The model analysis module is specifically used for: Inputting each of the original sentence groups into the original language model respectively, obtaining a sentence feature vector group corresponding to each of the original sentence groups, wherein the sentence feature vector group includes a plurality of first sentence feature vectors arranged in an output order and a plurality of second sentence feature vectors arranged in an output order, wherein the plurality of first sentence feature vectors are feature vectors after multiple CLS pooling in the original language model, and the plurality of second sentence feature vectors are feature vectors after multiple average pooling in the original language model; Calculating target importance scores for the multiple first sentence feature vectors corresponding to each of the original sentence groups and the multiple second sentence feature vectors corresponding to each of the original sentence groups, respectively, to obtain target importance scores corresponding to each of the original sentence groups; The third sentence feature vector is calculated for the last first sentence feature vector corresponding to each of the original sentence groups and the target importance score corresponding to each of the original sentence groups through the first formula, and the third sentence feature vector corresponding to each of the original sentence groups is obtained, and the third sentence feature vector is used as a model positive sample. The first formula is: in, is the third sentence feature vector corresponding to the e-th original sentence group, is the last first sentence feature vector corresponding to the e-th original sentence group, is the target importance score corresponding to the e-th original sentence group; The fourth sentence feature vector is calculated for each of the third sentence feature vectors by the second formula to obtain the fourth sentence feature vector corresponding to each of the original sentence groups, and the fourth sentence feature vector is used as a model negative sample. The second formula is: Among them, y e is the fourth sentence feature vector corresponding to the e-th original sentence group, MLP( ) is the multi-layer perceptron function, is the third sentence feature vector corresponding to the e-th original sentence group; Calculating the loss value of each of the model positive samples and the model negative samples corresponding to each of the original sentence groups respectively to obtain the loss value corresponding to each of the original sentence groups; The parameters of the original language model are updated according to all the loss values ​​to obtain a target language model.

7. The text matching device according to claim 6, characterized in that: In the model analysis module, the process of calculating target importance scores for the multiple first sentence feature vectors corresponding to each of the original sentence groups and the multiple second sentence feature vectors corresponding to each of the original sentence groups to obtain target importance scores corresponding to each of the original sentence groups includes: The target importance scores are calculated for the multiple first sentence feature vectors corresponding to each of the original sentence groups and the multiple second sentence feature vectors corresponding to each of the original sentence groups through the third formula to obtain the target importance scores corresponding to each of the original sentence groups. The third formula is: in, in, in, is the target importance score corresponding to the e-th original sentence group, L is the total number of feature vectors of the first sentence or the total number of feature vectors of the second sentence, is the initial importance score corresponding to the mth first sentence feature vector of the eth original sentence group, β e-m is the importance weight corresponding to the mth first sentence feature vector of the eth original sentence group, F c 、F x and F z are all learnable parameters. is the mth second sentence feature vector corresponding to the eth original sentence group, is the mth first sentence feature vector corresponding to the eth original sentence group.

8. The text matching device according to claim 6, characterized in that: In the model analysis module, the process of calculating the loss value of each model positive sample and the model negative sample corresponding to each original sentence group to obtain the loss value corresponding to each original sentence group includes: The loss values ​​of each of the model positive samples and the model negative samples corresponding to each of the original sentence groups are calculated by the fourth formula to obtain the loss values ​​corresponding to each of the original sentence groups. The fourth formula is: Among them, I e is the loss value corresponding to the e-th original sentence group, is the model positive sample corresponding to the e-th original sentence group, t is the temperature coefficient, H is the total number of original sentence groups, is the model negative sample corresponding to the fth original sentence group, sim(,) is the cosine similarity function, is the model negative sample corresponding to the e-th original sentence group.

9. A text matching system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the text matching method according to any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the text matching method according to any one of claims 1 to 4 is implemented.

Images