A text matching method, device, system and storage medium

Abstract

The application provides a text matching method, device and system and a storage medium, and belongs to the technical field of text matching, and the method comprises the steps of importing original texts, taking the original texts as a group in pairs to obtain an original sentence group, constructing an original language model, performing model analysis on the original language model according to the original sentence group to obtain a target language model, importing a text to be processed, and performing matching on the text to be processed according to the target language model to obtain a text matching result. The application enhances the diversity and richness of the text, effectively alleviates the problem of oversmoothing, provides more accurate semantic matching capability, achieves a breakthrough in the field of semantic search, enables the model to quickly and accurately obtain a result closest to semantics from a large amount of text data, and improves the accuracy of text matching.

Description

Technical Field

[0001] This invention relates primarily to the field of text matching technology, specifically to a text matching method, apparatus, system, and storage medium. Background Technology

[0002] With the development of Natural Language Processing (NLP), effective text representation learning has become a key research direction. Text representation not only needs to capture the rich semantics of language but also accurately reflect the subtle differences between sentences. With the advancement of deep learning technology, Language Models (PLMs) such as BERT and RoBERTa, through pre-training on massive amounts of text data, have been able to provide powerful text encoding capabilities. However, as model depth increases, these models face the oversmoothing problem in text representation, where the representations of different texts tend to become similar, leading to information loss, reduced discriminative power, and decreased accuracy in text matching.

[0003] Traditional language models often employ simple methods, such as using only the embeddings of the last layer or the average embeddings of all layers to generate text representations. However, this approach may overlook important information contained in other layers, while fixed pooling strategies cannot adapt to all situations, such as different domains or tasks requiring information from different layers, thus limiting performance improvements and leading to text mismatches. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a text matching method, apparatus, system and storage medium to address the shortcomings of the prior art.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: A text matching method, comprising the following steps:

[0006] Import multiple raw texts and group all the raw texts into pairs to obtain multiple raw statement groups;

[0007] Construct an original language model, and perform model analysis on the original language model based on all the original sentence groups to obtain the target language model;

[0008] Import multiple texts to be processed, and match all the texts to be processed according to the target language model to obtain the text matching results.

[0009] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: A text matching device, comprising:

[0010] The import module is used to import multiple raw text files.

[0011] The grouping module is used to group all the original text into pairs, thereby obtaining multiple groups of original sentences.

[0012] The model analysis module is used to construct the original language model and perform model analysis on the original language model based on all the original sentence groups to obtain the target language model.

[0013] The import module is also used to import multiple texts to be processed;

[0014] The text matching result acquisition module is used to match all the texts to be processed according to the target language model to obtain text matching results.

[0015] Based on the above-mentioned text matching method, the present invention also provides a text matching system.

[0016] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a text matching system, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the text matching method described above is implemented.

[0017] Based on the above-described text matching method, the present invention also provides a computer-readable storage medium.

[0018] Another technical solution of the present invention to solve the above-mentioned technical problems is as follows: a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the text matching method as described above.

[0019] The beneficial effects of this invention are as follows: by pairing all the original texts into groups of two, original sentence groups are obtained; the target language model is obtained by analyzing the original language model based on the original sentence groups; and the text matching result is obtained by matching the text to be processed based on the target language model. This enhances the diversity and richness of the text, effectively alleviates the oversmoothing problem, provides more accurate semantic matching capabilities, and also achieves a breakthrough in the field of semantic search. This enables the model to quickly and accurately obtain the semantically closest result from a large amount of text data, thereby improving the accuracy of text matching. Attached Figure Description

[0020] Figure 1 A flowchart illustrating a text matching method provided in an embodiment of the present invention;

[0021] Figure 2 This is a block diagram of a text matching device provided in an embodiment of the present invention. Detailed Implementation

[0022] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0023] Figure 1 This is a flowchart illustrating a text matching method provided in an embodiment of the present invention.

[0024] like Figure 1 As shown, a text matching method includes the following steps:

[0025] Import multiple raw texts and group all the raw texts into pairs to obtain multiple raw statement groups;

[0026] Construct an original language model, and perform model analysis on the original language model based on all the original sentence groups to obtain the target language model;

[0027] Import multiple texts to be processed, and match all the texts to be processed according to the target language model to obtain the text matching results.

[0028] It should be understood that the acquired corpus (i.e., the original text) is used to generate sentence pairs (i.e., original sentence groups).

[0029] In the above embodiments, by pairing all the original texts together to obtain the original sentence groups, the target language model is obtained by analyzing the original language model based on the original sentence groups, and the text matching result is obtained by matching the text to be processed based on the target language model. This enhances the diversity and richness of the text, effectively alleviates the oversmoothing problem, provides more accurate semantic matching capabilities, and also achieves a breakthrough in the field of semantic search. This enables the model to quickly and accurately obtain the semantically closest result from a large amount of text data, thereby improving the accuracy of text matching.

[0030] Optionally, as an embodiment of the present invention, the process of performing model analysis on the original language model based on all the original sentence groups to obtain the target language model includes:

[0031] Each of the original statement groups is input into the original language model to obtain a statement feature vector group corresponding to each of the original statement groups. The statement feature vector group includes multiple first statement feature vectors arranged in the output order and multiple second statement feature vectors arranged in the output order. The multiple first statement feature vectors are feature vectors after multiple CLS pooling in the original language model, and the multiple second statement feature vectors are feature vectors after multiple average pooling in the original language model.

[0032] Target importance scores are calculated for each of the multiple first statement feature vectors and the multiple second statement feature vectors corresponding to each of the original statement groups, respectively, to obtain the target importance score corresponding to each of the original statement groups.

[0033] The first formula is used to calculate the third statement feature vector corresponding to each of the original statement groups by calculating the feature vector of the last first statement and the target importance score corresponding to each of the original statement groups. This yields the third statement feature vector corresponding to each of the original statement groups, which is then used as the positive sample of the model.

[0034]

[0035] in, This is the feature vector of the third statement corresponding to the e-th original statement group. This is the feature vector of the last first statement corresponding to the e-th original statement group. The target importance score corresponds to the e-th original statement group;

[0036] The second formula is used to calculate the fourth statement feature vector for each of the third statement feature vectors, thereby obtaining the fourth statement feature vector corresponding to each of the original statement groups. This fourth statement feature vector is then used as a negative sample for the model. The second formula is:

[0037]

[0038] Among them, y e Let MLP() be the feature vector of the fourth statement corresponding to the e-th original statement group, and MLP() be the multilayer perceptron function. This is the feature vector of the third statement corresponding to the e-th original statement group;

[0039] Loss values ​​are calculated for each positive sample of the model and the negative sample of the model corresponding to each original statement group to obtain the loss value corresponding to each original statement group.

[0040] The original language model is updated with parameters based on all the loss values ​​to obtain the target language model.

[0041] It should be understood that CLS pooling is a special type of pooling operation, commonly used in image classification tasks in deep learning. Its core idea is to extract feature representations of an image through pooling operations in the final stage of a convolutional neural network (CNN), and then use these features for classification.

[0042] As should be understood, average pooling is a commonly used pooling operation in convolutional neural networks (CNNs) in deep learning. Its purpose is to reduce the spatial dimensionality of data (e.g., the width and height of an image) while preserving as much important information as possible. Average pooling is achieved by calculating the average value within each pooling window of the input feature map.

[0043] Specifically, the obtained sentence pairs (i.e., the original sentence groups) are input into the pre-trained language model (i.e., the original language model). Using a layer-by-layer pooling method based on the multiplicative attention mechanism, weights are assigned layer by layer to the original pre-trained language model (i.e., the original language model). The pooled layer vectors are randomly initialized and a multilayer perceptron (MLP) layer is added to obtain a vector with the same dimension as the original pre-trained language model (i.e., the fourth sentence feature vector).

[0044] Specifically, a multilayer perceptron (MLP) layer is added after pooling and randomly initialized, while maintaining its random initialization. In the formula, It is the vector of the [CLS] markers representing the input of the last layer (i.e., the feature vector of the last first statement). The input representation... (i.e., the feature vector of the last first statement) and the layer representation y B (i.e., target importance scores) are concatenated to form the input vector y of the MLP. LP (i.e., the third sentence feature vector). Finally, y is the fourth sentence feature vector, represented by the MLP layer with the same dimension as the sentence representation of the original language model. The formula is as follows:

[0045]

[0046] y = MLP(y LP ).

[0047] In the above embodiments, the target language model is obtained by performing model analysis on the original language model based on all original sentence groups, which enhances the diversity and richness of the text, effectively alleviates the oversmoothing problem, provides more accurate semantic matching capabilities, and also achieves a breakthrough in the field of semantic search. This enables the model to quickly and accurately obtain the semantically closest result from a large amount of text data, thereby improving the accuracy of text matching.

[0048] Optionally, as an embodiment of the present invention, the process of calculating the target importance score for each of the original statement groups by calculating the target importance score for each of the multiple first statement feature vectors and the multiple second statement feature vectors corresponding to each of the original statement groups includes:

[0049] The target importance score is calculated by performing a third formula on the multiple first statement feature vectors and multiple second statement feature vectors corresponding to each of the original statement groups, respectively, to obtain the target importance score corresponding to each of the original statement groups. The third formula is as follows:

[0050]

[0051] in,

[0052] in,

[0053] in, Let L be the target importance score corresponding to the e-th original statement group, and L be the total number of feature vectors of the first statement or the total number of feature vectors of the second statement. Let β be the initial importance score corresponding to the feature vector of the m-th first statement in the e-th original statement group. e-m F represents the importance weight corresponding to the feature vector of the m-th first statement in the e-th original statement group. c F x and F z All of these are learnable parameters. This is the feature vector of the m-th second statement corresponding to the e-th original statement group. This is the feature vector of the m-th first statement corresponding to the e-th original statement group.

[0054] Specifically, the performance of [CLS] pooling (label classification pooling) varies significantly depending on the pooling layer, while there is no significant performance difference between layers when using AVG pooling. This can be explained by the fact that each layer can contain different information about the sentence representation, and average pooling can reduce the information gap between layers. Combining the characteristics of both, assigning different weights to each layer in the model can highlight those information layers that are more critical to downstream tasks, while suppressing those information layers that may contain noise or are less important, as shown in the following formula:

[0055]

[0056] Where the subscript m represents the number of layers, y v It is represented by AVG, which is the average vector of markers in the sentence (i.e., the feature vector of the second sentence), y o It is the input vector representing the [CLS] markers for each layer (i.e., the first statement feature vector), β m y represents the importance (i.e., importance weight) of the m-th layer. b This represents each layer with an importance score (i.e., the initial importance score). y B It is y bThe average vector (i.e., the target importance score) is also a representation that includes the relevance of all layers (L is the number of layers). F z F x and F c These are learnable parameters.

[0057] In the above embodiments, the target importance score is obtained by calculating the target importance score of the feature vector of the first statement and the feature vector of the second statement respectively. This can highlight the information layer that is more critical to the downstream task, while suppressing the information layer containing noise or less important information, enhancing the diversity and richness of the text, effectively alleviating the oversmoothing problem, and providing more accurate semantic matching capabilities.

[0058] Optionally, as an embodiment of the present invention, the process of calculating the loss value for each of the positive samples of the model and the negative samples of the model corresponding to each of the original statement groups to obtain the loss value corresponding to each of the original statement groups includes:

[0059] The loss value corresponding to each original statement group is obtained by calculating the loss value for each positive sample of the model and the negative sample of the model corresponding to each original statement group using the fourth formula:

[0060]

[0061] Among them, I e Let e ​​be the loss value corresponding to the e-th original statement group. Let be the positive sample of the model corresponding to the e-th original statement group, t be the temperature coefficient, and H be the total number of original statement groups. Let f be the negative sample of the model corresponding to the f-th original sentence group, and sim(,) be the cosine similarity function. This is the negative sample of the model corresponding to the e-th original statement group.

[0062] It should be understood that the obtained hierarchical dimensional representations are subjected to unsupervised contrastive learning to obtain the training target, and the obtained hidden representations of the intermediate layers are used as negative samples (i.e. model negative samples) to optimize the unsupervised contrastive learning.

[0063] Specifically, let y e (i.e., negative samples of the model) and (i.e., model positive samples) represent the sentence representations of the last layer in PLMs. For a mini-batch containing H pairs, loss function I e Represented as:

[0064]

[0065] in For sentence ye The positive samples (i.e., the model positive samples) are obtained from y. LP The same batch of sentences generated (y LP This approach guides model learning by maximizing the similarity between positive sample pairs, enabling the model to better capture and represent the semantic content of sentences, thus enhancing the richness and accuracy of sentence representation. `sim(,)` represents the cosine similarity function, and `t` is the temperature coefficient. This method focuses on mining positive samples while directly using in-batch negative samples during training.

[0066] The hidden representations of the intermediate layers of PLMs are used as negative samples (i.e., negative samples of the model), and the representation of the last layer is regarded as the final sentence representation that needs to be optimized. Specifically, the sentence representations of the intermediate layers in PLMs are collected, treated as negative samples of the last layer representation, and named as follows: Therefore, a negative pair was obtained. As mentioned earlier, the same process is also handled. As positive samples obtained through any data augmentation method. Subsequently, the loss function I e It can be rephrased as follows:

[0067]

[0068] The first term in the denominator refers to the original in-batch negative samples, with the obtained y serving as the sentence representation of the original in-batch negative sample pairs. The second term represents intermediate negative samples. This method, by simply increasing the number of negative samples, makes the final layer representation of PLMs more distinguishable from the preceding layers, thus mitigating the oversmoothing problem.

[0069] In the above embodiments, loss values ​​are calculated for each model's positive and negative samples respectively, increasing the number of negative samples and making the last layer of PLMs more distinguishable from the previous layers, thereby alleviating the oversmoothing problem and enhancing the richness and accuracy of sentence representation.

[0070] Optionally, as another embodiment of the present invention, the present invention includes:

[0071] By applying an attention mechanism to each layer of a pre-trained language model (PLM), different weights are assigned to representations at different levels. Simultaneously, a self-contradictory learning strategy is introduced, utilizing representations from intermediate layers of the model as negative samples to enhance the contrast and discriminative power of sentence representations for learning unsupervised sentence representations. The beneficial effects of this invention are: it proposes a layer-wise attention pooling strategy that assigns weights to each layer, and to address the oversmoothing problem, it improves the quality of sentence representations by sampling negative samples from intermediate layers of PLMs for learning unsupervised sentence representations. This approach can be easily extended to various state-of-the-art models to improve performance. Ultimately, it enhances the model's performance on specific NLP tasks, particularly in sentence semantic similarity measurement and semantic search.

[0072] Optionally, as another embodiment of the present invention, the present invention includes:

[0073] S1. Generate sentence pairs from the acquired corpus;

[0074] S2. Input the statements in S1 into the pre-trained language model;

[0075] S3. Using a layer-by-layer pooling method based on multiplicative attention mechanism, weights are assigned layer by layer to the original pre-trained language model in S2.

[0076] S4. Randomly initialize the pooled layer vectors in S3 and add a multilayer perceptron (MLP) layer to obtain a vector with the same dimensions as the original pre-trained language model.

[0077] S5. Take the hierarchical dimension representation obtained in S4 and perform unsupervised contrastive learning to obtain the training target.

[0078] S6. Use the hidden representation of the intermediate layer obtained in S4 as a negative sample to optimize the unsupervised contrastive learning in S5.

[0079] Optionally, as another embodiment of the present invention, this invention is based on an attention mechanism, which can weight the output of each layer in PLMs, highlighting information that is more critical to downstream tasks. In this way, the model can not only integrate semantic information from different levels, but also enhance the diversity and richness of sentence representations. At the same time, an unsupervised self-comparative learning strategy is proposed, which effectively alleviates the oversmoothing problem by generating negative samples from the intermediate layers of PLMs. Combining these two technological inventions improves the way PLMs utilize information in sentence representation learning. This not only provides the model with more accurate semantic matching capabilities in sentence semantic similarity measurement tasks, but also achieves a breakthrough in the field of semantic search, enabling the model to quickly and accurately retrieve results that are semantically closest to the query from a large amount of text data.

[0080] Figure 2This is a block diagram of a text matching device provided in an embodiment of the present invention.

[0081] Alternatively, as another embodiment of the present invention, such as Figure 2 As shown, a text matching device includes:

[0082] The import module is used to import multiple raw text files.

[0083] The grouping module is used to group all the original text into pairs, thereby obtaining multiple groups of original sentences.

[0084] The model analysis module is used to construct the original language model and perform model analysis on the original language model based on all the original sentence groups to obtain the target language model.

[0085] The import module is also used to import multiple texts to be processed;

[0086] The text matching result acquisition module is used to match all the texts to be processed according to the target language model to obtain text matching results.

[0087] Optionally, as an embodiment of the present invention, the model analysis module is specifically used for:

[0088] Each of the original statement groups is input into the original language model to obtain a statement feature vector group corresponding to each of the original statement groups. The statement feature vector group includes multiple first statement feature vectors arranged in the output order and multiple second statement feature vectors arranged in the output order. The multiple first statement feature vectors are feature vectors after multiple CLS pooling in the original language model, and the multiple second statement feature vectors are feature vectors after multiple average pooling in the original language model.

[0089] Target importance scores are calculated for each of the multiple first statement feature vectors and the multiple second statement feature vectors corresponding to each of the original statement groups, respectively, to obtain the target importance score corresponding to each of the original statement groups.

[0090] The first formula is used to calculate the third statement feature vector corresponding to each of the original statement groups by calculating the feature vector of the last first statement and the target importance score corresponding to each of the original statement groups. This yields the third statement feature vector corresponding to each of the original statement groups, which is then used as the positive sample of the model.

[0091]

[0092] in, This is the feature vector of the third statement corresponding to the e-th original statement group. This is the feature vector of the last first statement corresponding to the e-th original statement group. The target importance score corresponds to the e-th original statement group;

[0093] The second formula is used to calculate the fourth statement feature vector for each of the third statement feature vectors, thereby obtaining the fourth statement feature vector corresponding to each of the original statement groups. This fourth statement feature vector is then used as a negative sample for the model. The second formula is:

[0094]

[0095] Among them, y e Let MLP() be the feature vector of the fourth statement corresponding to the e-th original statement group, and MLP() be the multilayer perceptron function. This is the feature vector of the third statement corresponding to the e-th original statement group;

[0096] Loss values ​​are calculated for each positive sample of the model and the negative sample of the model corresponding to each original statement group to obtain the loss value corresponding to each original statement group.

[0097] The original language model is updated with parameters based on all the loss values ​​to obtain the target language model.

[0098] Optionally, as an embodiment of the present invention, the process of calculating the target importance score for each original statement group by the model analysis module, and calculating the target importance score for each original statement group, includes:

[0099] The target importance score is calculated by performing a third formula on the multiple first statement feature vectors and multiple second statement feature vectors corresponding to each of the original statement groups, respectively, to obtain the target importance score corresponding to each of the original statement groups. The third formula is as follows:

[0100]

[0101] in,

[0102] in,

[0103] in, Let L be the target importance score corresponding to the e-th original statement group, and L be the total number of feature vectors of the first statement or the total number of feature vectors of the second statement. Let β be the initial importance score corresponding to the feature vector of the m-th first statement in the e-th original statement group. e-m F represents the importance weight corresponding to the feature vector of the m-th first statement in the e-th original statement group. c F x and F z All of these are learnable parameters. This is the feature vector of the m-th second statement corresponding to the e-th original statement group. This is the feature vector of the m-th first statement corresponding to the e-th original statement group.

[0104] Optionally, as an embodiment of the present invention, the process of calculating loss values ​​for each positive model sample and each negative model sample corresponding to each original statement group in the model analysis module to obtain the loss value corresponding to each original statement group includes:

[0105] The loss value corresponding to each original statement group is obtained by calculating the loss value for each positive sample of the model and the negative sample of the model corresponding to each original statement group using the fourth formula:

[0106]

[0107] Among them, I e Let e ​​be the loss value corresponding to the e-th original statement group. Let be the positive sample of the model corresponding to the e-th original statement group, t be the temperature coefficient, and H be the total number of original statement groups. Let f be the negative sample of the model corresponding to the f-th original sentence group, and sim(,) be the cosine similarity function. This is the negative sample of the model corresponding to the e-th original statement group.

[0108] Optionally, another embodiment of the present invention provides a text matching system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the text matching method as described above. This system can be a computer or similar system.

[0109] Optionally, another embodiment of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the text matching method as described above.

[0110] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0111] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0112] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0113] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0114] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0115] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0116] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A text matching method, characterized in that, Includes the following steps: Import multiple raw texts and group all the raw texts into pairs to obtain multiple raw statement groups; Construct an original language model, and perform model analysis on the original language model based on all the original sentence groups to obtain the target language model; Import multiple texts to be processed, and match all the texts to be processed according to the target language model to obtain the text matching results; The process of performing model analysis on the original language model based on all the original sentence groups to obtain the target language model includes: Each of the original statement groups is input into the original language model to obtain a statement feature vector group corresponding to each of the original statement groups. The statement feature vector group includes multiple first statement feature vectors arranged in the output order and multiple second statement feature vectors arranged in the output order. The multiple first statement feature vectors are feature vectors after multiple CLS pooling in the original language model, and the multiple second statement feature vectors are feature vectors after multiple average pooling in the original language model. Target importance scores are calculated for each of the multiple first statement feature vectors and the multiple second statement feature vectors corresponding to each of the original statement groups, respectively, to obtain the target importance score corresponding to each of the original statement groups. The first formula is used to calculate the third statement feature vector corresponding to each of the original statement groups by calculating the feature vector of the last first statement and the target importance score corresponding to each of the original statement groups. This yields the third statement feature vector corresponding to each of the original statement groups, which is then used as the positive sample of the model. ， in, For the first The feature vector of the third statement corresponding to each original statement group For the first The feature vector of the last first statement corresponding to each original statement group. For the first The target importance score corresponding to each original statement group; The second formula is used to calculate the fourth statement feature vector for each of the third statement feature vectors, thereby obtaining the fourth statement feature vector corresponding to each of the original statement groups. This fourth statement feature vector is then used as a negative sample for the model. The second formula is: ， in, For the first The feature vector of the fourth statement corresponding to each original statement group For multilayer perceptron functions, For the first The feature vector of the third statement corresponding to each original statement group; Loss values ​​are calculated for each positive sample of the model and the negative sample of the model corresponding to each original statement group to obtain the loss value corresponding to each original statement group. The original language model is updated with parameters based on all the loss values ​​to obtain the target language model; The process of calculating the target importance score for each of the original statement groups by respectively calculating the multiple first statement feature vectors and the multiple second statement feature vectors corresponding to each of the original statement groups, and obtaining the target importance score corresponding to each of the original statement groups, includes: The target importance score is calculated by performing a third formula on the multiple first statement feature vectors and multiple second statement feature vectors corresponding to each of the original statement groups, respectively, to obtain the target importance score corresponding to each of the original statement groups. The third formula is as follows: ， in, , in, , in, For the first The target importance score corresponding to each original statement group This refers to the total number of feature vectors for the first statement or the total number of feature vectors for the second statement. For the first The first of the original statement groups The initial importance scores corresponding to the feature vectors of the first statement. For the first The first of the original statement groups The importance weights corresponding to the feature vectors of each first statement. , and All of these are learnable parameters. For the first The first original statement group corresponding to the first The second statement feature vector, For the first The first original statement group corresponding to the first The first statement feature vector.

2. The text matching method according to claim 1, characterized in that, The process of calculating the loss value for each of the positive samples of the model and the negative samples of the model corresponding to each of the original statement groups to obtain the loss value corresponding to each of the original statement groups includes: The loss value corresponding to each original statement group is obtained by calculating the loss value for each positive sample of the model and the negative sample of the model corresponding to each original statement group using the fourth formula: ， in, For the first The loss value corresponding to each original statement group For the first The model's positive samples corresponding to each original statement group For temperature coefficient, This represents the total number of the original statement groups. For the first The negative samples of the model corresponding to each original statement group The cosine similarity function is used. For the first The negative samples of the model corresponding to each original statement group.

3. A text matching device, characterized in that, include: The import module is used to import multiple raw text files. The grouping module is used to group all the original text into pairs, thereby obtaining multiple groups of original sentences. The model analysis module is used to construct the original language model and perform model analysis on the original language model based on all the original sentence groups to obtain the target language model. The import module is also used to import multiple texts to be processed; The text matching result acquisition module is used to match all the texts to be processed according to the target language model to obtain text matching results; The model analysis module is specifically used for: Each of the original statement groups is input into the original language model to obtain a statement feature vector group corresponding to each of the original statement groups. The statement feature vector group includes multiple first statement feature vectors arranged in the output order and multiple second statement feature vectors arranged in the output order. The multiple first statement feature vectors are feature vectors after multiple CLS pooling in the original language model, and the multiple second statement feature vectors are feature vectors after multiple average pooling in the original language model. Target importance scores are calculated for each of the multiple first statement feature vectors and the multiple second statement feature vectors corresponding to each of the original statement groups, respectively, to obtain the target importance score corresponding to each of the original statement groups. The first formula is used to calculate the third statement feature vector corresponding to each of the original statement groups by calculating the feature vector of the last first statement and the target importance score corresponding to each of the original statement groups. This yields the third statement feature vector corresponding to each of the original statement groups, which is then used as the positive sample of the model. ， in, For the first The feature vector of the third statement corresponding to each original statement group For the first The feature vector of the last first statement corresponding to each original statement group. For the first The target importance score corresponding to each original statement group; The second formula is used to calculate the fourth statement feature vector for each of the third statement feature vectors, thereby obtaining the fourth statement feature vector corresponding to each of the original statement groups. This fourth statement feature vector is then used as a negative sample for the model. The second formula is: ， in, For the first The feature vector of the fourth statement corresponding to each original statement group For multilayer perceptron functions, For the first The feature vector of the third statement corresponding to each original statement group; Loss values ​​are calculated for each positive sample of the model and the negative sample of the model corresponding to each original statement group to obtain the loss value corresponding to each original statement group. The original language model is updated with parameters based on all the loss values ​​to obtain the target language model; In the model analysis module, the process of calculating the target importance score for each original statement group's multiple first statement feature vectors and multiple second statement feature vectors, and obtaining the target importance score for each original statement group, includes: The target importance score is calculated by performing a third formula on the multiple first statement feature vectors and multiple second statement feature vectors corresponding to each of the original statement groups, respectively, to obtain the target importance score corresponding to each of the original statement groups. The third formula is as follows: ， in, , in, , in, For the first The target importance score corresponding to each original statement group This refers to the total number of feature vectors for the first statement or the total number of feature vectors for the second statement. For the first The first of the original statement groups The initial importance scores corresponding to the feature vectors of the first statement. For the first The first of the original statement groups The importance weights corresponding to the feature vectors of each first statement. , and All of these are learnable parameters. For the first The first original statement group corresponding to the first The second statement feature vector, For the first The first original statement group corresponding to the first The first statement feature vector.

4. The text matching device according to claim 3, characterized in that, In the model analysis module, the process of calculating the loss value for each positive sample of the model and the negative sample of the model corresponding to each original statement group, and obtaining the loss value corresponding to each original statement group, includes: The loss value corresponding to each original statement group is obtained by calculating the loss value for each positive sample of the model and the negative sample of the model corresponding to each original statement group using the fourth formula: ， in, For the first The loss value corresponding to each original statement group For the first The model's positive samples corresponding to each original statement group For temperature coefficient, This represents the total number of the original statement groups. For the first The negative samples of the model corresponding to each original statement group The cosine similarity function is used. For the first The negative samples of the model corresponding to each original statement group.

5. A text matching system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the text matching method as described in any one of claims 1 to 2.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the text matching method as described in any one of claims 1 to 2.

Images