Text similarity determination method, apparatus, computer equipment and storage medium

By optimizing the similarity fusion calculation model and the BERT model through pre-training, the problem of low accuracy in existing text similarity determination methods has been solved, achieving more accurate text similarity assessment and improving the automatic scoring capability of online examination systems.

CN116263807BActive Publication Date: 2026-11-17CHINA MOBILE (SUZHOU) SOFTWARE TECH CO LTD +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211372256.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-11-03
Publication Date
2026-11-17
Estimated Expiration
2042-11-03

AI Technical Summary

Technical Problem

Existing methods for determining text similarity rely on human-defined features and rules, which are difficult to cover all knowledge areas, and existing vector models cannot fully represent contextual semantic relationships, resulting in low accuracy.

Method used

A similarity fusion calculation model is adopted, including a text similarity prediction model, a word similarity calculation module, and a text edit distance similarity calculation module. It is pre-trained by combining the BERT model and the SOP task, extracts matching information through a convolutional neural network, optimizes model training using a smooth loss function, and calculates similarity by fusing multiple features.

Benefits of technology

It improves the accuracy of text similarity determination, enabling a more comprehensive assessment of the similarity between subjective question answers and reference answers, reducing the influence of human bias, and improving the fairness of scoring.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116263807B_ABST
    Figure CN116263807B_ABST
Patent Text Reader

Abstract

This invention relates to the field of text processing technology and discloses a method for determining text similarity. The method includes: acquiring a first text and a second text to be identified; inputting the first text and the second text into a similarity fusion calculation model for calculation to obtain a similarity result between the first text and the second text; wherein the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module, and a text edit distance similarity calculation module; text edit distance similarity represents the minimum number of edits required to convert one string into another. Through the above method, this invention achieves the effect of accurately determining the similarity between two texts.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of text processing technology, specifically to a text similarity determination method, a text similarity determination device, a computer device, and a computer-readable storage medium. Background Technology

[0002] Currently, with the rapid development of information technology, text similarity is being applied more and more widely in various industries. For example, in the grading process of online examination systems, automatic scoring can be performed based on text similarity.

[0003] However, the inventors of this application have found that existing text similarity calculations, which are generally based on traditional rules and knowledge, largely rely on human-defined features and rule formulation. Different language experts customize rules differently and it is difficult to cover all knowledge areas. On the other hand, the vectors used for similarity calculations between different levels can only represent the features of the language itself and cannot contain the semantic relationship of the context. The semantic extraction of the whole sentence is not sufficient, which makes the accuracy of existing text similarity determination methods low. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention provide a text similarity determination method, a text similarity determination device, a computer device, and a computer-readable storage medium to solve the problem of low accuracy in existing text similarity determination methods in the prior art.

[0005] According to one aspect of the present invention, a text similarity determination method is provided, the method comprising:

[0006] Obtain the first and second texts to be identified;

[0007] The first text and the second text are respectively input into the similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text; wherein the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module and a text edit distance similarity calculation module; the text edit distance similarity represents the minimum number of edits required to convert one string into another.

[0008] In an optional approach, the step of inputting the first text and the second text into a similarity calculation model for calculation to obtain a similarity result further includes:

[0009] Input the first text and the second text into the text similarity prediction model to obtain the predicted text similarity between the first text and the second text;

[0010] Input the first text and the second text into the word similarity calculation module to obtain the word similarity between each word in the first text and the second text;

[0011] The first text and the second text are input into the text edit distance similarity calculation module to obtain the text edit distance similarity.

[0012] The similarity result is obtained by weighted fusion calculation based on the predicted text similarity, the word similarity, and the text edit distance similarity.

[0013] In one optional embodiment, the text similarity prediction model includes an input layer, an encoding layer, a matching layer, a matching information extraction layer, and an output layer; the step of inputting the first text and the second text into the text similarity prediction model to obtain the predicted text similarity between the first text and the second text includes:

[0014] The first text and the second text are respectively input into the encoding layer through the input layer for encoding to obtain the first context vector of the first text and the second up context vector of the second text.

[0015] The matching layer calculates a matching matrix by analyzing the first context vector and the second context vector; the matching matrix represents the similarity between each word in the first text and each word in the second text.

[0016] Based on the matching matrix, the matching information extraction layer extracts matching information between the first text and the second text to obtain sentence-level matching information.

[0017] Based on the sentence-level matching information, the text prediction similarity between the first text and the second text is predicted.

[0018] In one optional approach, the step of extracting matching information between the first text and the second text using the matching information extraction layer based on the matching matrix to obtain sentence-level matching information includes:

[0019] Using the similarity between words in the matching matrix as local features, a convolutional neural network is used to extract global features from the matching matrix, and after pooling, it becomes a phrase matching matrix.

[0020] The phrase matching matrix is ​​convolved and pooled to obtain sentence-level matching information.

[0021] In an optional manner, before inputting the first text and the second text into a similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text, the method further includes:

[0022] The initial text similarity prediction model is pre-trained using the SOP task to obtain the pre-trained text similarity prediction model; the SOP task uses two consecutive sentences as positive samples and two consecutive sentences after being reversed in order as negative samples.

[0023] The similar text sample set is input into the pre-trained text similarity prediction model for training, and the prediction results are obtained.

[0024] Based on the predicted and expected results, a preset loss function is used to calculate the loss of the pre-trained text similarity prediction model. The parameters of the pre-trained text similarity prediction model are adjusted according to the loss. The process of inputting a set of similar text samples into the pre-trained text similarity prediction model for training, obtaining prediction results, and calculating the loss of the pre-trained text similarity prediction model based on the predicted and expected results using a preset loss function, and adjusting the parameters of the pre-trained text similarity prediction model according to the loss, continues until the loss reaches a preset loss threshold or a preset number of iterations is reached, resulting in a trained text similarity prediction model.

[0025] In one optional approach, the step of calculating the loss of the pre-trained text similarity prediction model using a preset loss function based on the predicted and expected results includes:

[0026] The loss of the pre-trained text similarity prediction model is calculated using a smoothing loss function as the preset loss function. The smoothing loss function characterizes the difference between the true class and the predicted class of the sample, plus the degree of deviation between the predicted distribution and the prior distribution.

[0027] In one optional approach, the first text is the answer text to the test question, and the second text is the answer text to the test question; after inputting the first text and the second text into a similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text, the method further includes:

[0028] Determine the score for each question in the test;

[0029] Based on the similarity results and the question scores, the score corresponding to the answer text is calculated.

[0030] According to another aspect of the present invention, a text similarity determination apparatus is provided, comprising:

[0031] The acquisition module is used to acquire the first and second texts to be recognized;

[0032] The calculation module is used to input the first text and the second text into the similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text; wherein the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module and a text edit distance similarity calculation module; the text edit distance similarity represents the minimum number of edits required to convert one string into another.

[0033] According to another aspect of the present invention, a computer device is provided, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus;

[0034] The memory is used to store at least one executable instruction that causes the processor to perform the operation of the text similarity determination method.

[0035] According to another aspect of the present invention, a computer-readable storage medium is provided, the storage medium storing at least one executable instruction, which, when executed on a computer device, causes the computer device to perform the operation of the text similarity determination method.

[0036] This invention provides an embodiment of the invention that obtains a first text and a second text to be identified; inputs the first text and the second text into a similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text; wherein the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module and a text edit distance similarity calculation module; text edit distance similarity represents the minimum number of edits required to convert one string into another, and can accurately determine the similarity between two texts.

[0037] The above description is merely an overview of the technical solutions of the embodiments of the present invention. In order to better understand the technical means of the embodiments of the present invention and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0038] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0039] Figure 1A flowchart illustrating the text similarity determination method provided in an embodiment of the present invention is shown.

[0040] Figure 2 This diagram illustrates the application environment of the text similarity determination method provided in this embodiment of the invention.

[0041] Figure 3 This diagram illustrates the structure of the text similarity prediction model in the text similarity determination method provided in this embodiment of the invention.

[0042] Figure 4 A schematic diagram of the text similarity determination device provided in an embodiment of the present invention is shown;

[0043] Figure 5 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention is shown. Detailed Implementation

[0044] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0045] First, the problems with the related technologies of the embodiments of the present invention and their corresponding application scenarios will be further explained:

[0046] With the rapid development of information technology, online examination systems are increasingly widely used in education and various industries. Their flexibility and convenience, requiring only an internet connection, eliminate the complex procedures of paper-based examinations, and the same grading strategies make the exams fairer. Objective questions can be precisely scored by programming based on the answers, but subjective questions, due to the non-uniqueness of their answers and the semantic complexity of Chinese, cannot be accurately judged for similarity even when provided with standard answers. Currently, there are no widely used automatic scoring methods for subjective questions in online examination systems, and teachers still grade them manually. This task is very demanding and easily affected by the grader's subjective preferences, psychological state, and physical condition, thus reducing the fairness of the grading.

[0047] Automatic scoring of subjective questions is mainly determined by calculating the text similarity between the reference answer and the candidate's answer and combining the score. Currently, it is mainly divided into two categories: natural language processing methods based on traditional rules, knowledge and vector representation, and methods based on deep learning models.

[0048] Rule-based and knowledge-based methods construct the grammatical structure, semantic concepts, dictionaries, and corpora of texts using rules and usage norms developed by linguists, and describe the attribute relationships between concepts based on language-inherent information. Common methods include segmenting the text into words, sentences, and tagging parts of speech, and then calculating the similarity between words and sentences in the test taker's answers based on key scoring points from the reference answers, combined with large knowledge bases such as synonym forests or statistical models such as LDA. Alternatively, the overall similarity can be calculated by analyzing the structure of answer text pairs and combining factors such as dependency trees, grammatical rules, word forms and word order, and domain ontology.

[0049] Deep learning-based methods train deep neural networks on large datasets to automatically learn distributed word vectors with controllable dimensions and contextual semantics. These vectors are then combined with relevant algorithms to calculate distances in the vector space, or network structures are designed to achieve end-to-end computation. Common methods include using word2vec to train word vectors and then combining them with TF-IDF, SIF, etc., to calculate similarity at different levels; or using LSTM, CNN, etc., to directly extract text context and local semantic features and then classify them to obtain similarity probabilities. With the release of pre-trained models such as BERT and XLNET, some methods directly input answer text pairs to predict similarity.

[0050] Therefore, existing similarity determination methods suffer from the following problems:

[0051] 1. Traditional text similarity calculation methods based on rules and knowledge largely rely on human-defined features and rule formulation. Different language experts customize rules differently, and it is difficult to cover all knowledge areas. For example, the construction and improvement of the "Thesaurus" has taken more than 30 years. On the other hand, the vectors used for similarity calculation between different levels can only represent the features of the language itself and cannot contain the semantic relationship of the context, resulting in insufficient semantic extraction of the entire sentence.

[0052] 2. Distributed word vectors can extract contextual semantics, and recurrent neural networks can store them, but they cannot solve the problem of polysemy. The BERT pre-trained model obtains dynamic word vectors through a multi-head self-attention mechanism and a bidirectional encoder structure to improve polysemy. However, directly using the BERT pre-trained model to calculate similarity and score answer texts after word embeddings yields poor results. Firstly, the general corpus used for pre-training is irrelevant to the task domain. When predicting exam texts in a specific domain, word embeddings cannot fully represent semantic information. Furthermore, the resulting high-frequency word vectors are densely distributed and close to the origin, while low-frequency word vectors are sparsely distributed and far from the origin, leading to excessively high similarity between semantically similar texts. Directly using BERT to connect to downstream tasks for fine-tuning word embeddings at the character level, however, can cause some Chinese words to lose their semantic meaning after splitting, requiring improvement.

[0053] 3. In a real subjective question marking environment, not only the overall semantic similarity of the answer text pairs will be considered, but also whether the key knowledge points have been answered, as well as factors such as language logic, writing standard and fluency. Therefore, multiple features need to be considered to obtain better results.

[0054] Figure 1 A flowchart of a text similarity determination method provided in an embodiment of the present invention is shown, which is executed by a computer device. Figure 1 As shown, the method includes the following steps:

[0055] Step 110: Obtain the first and second texts to be identified.

[0056] The first text can be the answer text of the test question, and the second text can be the answer text of the test question. Specifically, the test question can be a subjective question. The first text can be the answer text obtained by scanning the candidate's offline test paper, or it can be the answer text obtained from the candidate's online answers in the online examination system.

[0057] Step 120: Input the first text and the second text into the similarity fusion calculation model respectively for calculation to obtain the similarity result between the first text and the second text; wherein the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module and a text edit distance similarity calculation module; the text edit distance similarity represents the minimum number of edits required to convert one string into another.

[0058] The first text and the second text are input into a similarity fusion calculation model to calculate the similarity between the first text and the second text.

[0059] like Figure 2 The diagram shown illustrates the application environment of an embodiment of the present invention. In this embodiment, the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module, and a text edit distance similarity calculation module. Therefore, the first text and the second text are respectively input into the similarity calculation model for calculation to obtain similarity results, specifically including:

[0060] Step 1201: Input the first text and the second text into the text similarity prediction model to obtain the text prediction similarity between the first text and the second text.

[0061] Among them, such as Figure 3As shown, the text similarity prediction model includes an input layer, an encoding layer, a matching layer, a matching information extraction layer, and an output layer. The encoding layer is a BERT model. The text similarity prediction model uses a convolutional neural network to extract single-character-level word embedding matching information from the first and second texts to obtain a more complete sentence relationship feature vector. The step of inputting the first and second texts into the text similarity prediction model to obtain the predicted text similarity between the first and second texts includes:

[0062] First, the first text and the second text are respectively input into the encoding layer through the input layer for encoding, to obtain the first context vector of the first text and the second up-context vector of the second text.

[0063] Then, the matching layer calculates the first context vector and the second context vector to obtain a matching matrix. Specifically, the matching layer calculates the first context vector and the second context vector to obtain a matching matrix E, where the element in the i-th row and j-th column of the matching matrix E is... This represents the similarity between the i-th word in the first text and the j-th word in the second text.

[0064] Subsequently, based on the matching matrix, the matching information extraction layer extracts matching information between the first text and the second text to obtain sentence-level matching information. Specifically, using the similarity between words in the matching matrix as local features, a convolutional neural network is used to perform global feature extraction and pooling on the matching matrix to obtain a phrase-level matching matrix; convolution and pooling are then performed on the phrase-level matching matrix to obtain sentence-level matching information. Here, feature extraction is performed twice based on the matching matrix through the matching information extraction layer. The first extraction extracts phrase-level matching information from the matching matrix. The second extraction extracts sentence-level matching information from the phrase-level matching information. Specifically, global feature extraction and pooling of the matching matrix using a convolutional neural network yields phrase-level matching information; global feature extraction and pooling of the phrase-level matching information using a convolutional neural network yields sentence-level matching information. The matching matrix represents the similarity between each word in the first text and each word in the second text. Treating word-to-word matching information as local features, a convolutional neural network is used to extract global features from the pairwise matching matrix E. The resulting pooled value can then be used as the next level, i.e., phrase-to-phrase matching information. Further convolution and pooling of the phrase matching matrix yields sentence-level matching information. The convolution calculation is as follows:

[0065]

[0066] The matching information at different levels is extracted using a multi-layer convolutional neural network. Global max pooling is then applied after the last convolutional layer to generate a matching information vector r2 with a length equal to the number of channels in the model.

[0067] Finally, based on the sentence-level matching information, the predicted text similarity between the first text and the second text is obtained. This matching information vector and text tag vector are concatenated and input into a fully connected network. The probability of each tag is predicted using the softmax function to obtain the predicted text similarity between the first text and the second text. The text tag vector is an encoding vector that uses a special tag [CLS] on the first and second texts. This special tag encoding vector is obtained by embedding all individual characters in the first and second texts and then passing them through a 12-layer bidirectional encoder, resulting in an encoding vector containing rich contextual semantics and positional information.

[0068] In this embodiment of the invention, the text similarity prediction model is also pre-trained and fine-tuned. Specifically, this includes:

[0069] First, the initial text similarity prediction model is pre-trained to obtain a pre-trained text similarity prediction model. The encoding layer of this initial text similarity prediction model is a BERT model. During pre-training, the SOP task is also used to pre-train the initial text similarity prediction model for text prediction. The SOP task uses two consecutive sentences as positive samples and reverses the order of two consecutive sentences as negative samples. Existing BERT models use the Next Sentence Prediction (NSP) task to predict the association between sentences, selecting the second sentence in the sample sentences with a 50% probability that it is a continuous segment of the first sentence and a 50% probability that it is a random segment in the corpus. This can be understood as a topic classification of sentence pairs plus a coherence judgment. However, negative samples are selected from irrelevant corpora and are unidirectional predictions, making the task too simple and unable to fully learn contextual information. Therefore, this embodiment of the invention uses the Sentence Order Prediction (SOP) task, extracting two consecutive sentences A and B as positive samples, and reversing the order of A and B as negative samples before training the model, allowing the model to more fully learn contextual semantic information. During pre-training, the sample text pairs are concatenated and labeled with similarity before being input into the initial text similarity prediction model for pre-training.

[0070] Next, the similar text sample set is input into the pre-trained text similarity prediction model for training, and the prediction results are obtained. This similar text sample set can include teacher-graded test paper datasets and existing test paper datasets.

[0071] Finally, based on the predicted and expected results, a preset loss function is used to calculate the loss of the pre-trained text similarity prediction model. The parameters of the pre-trained text similarity prediction model are adjusted according to the loss. The process of inputting a similar text sample set into the pre-trained text similarity prediction model for training, obtaining prediction results, and calculating the loss of the pre-trained text similarity prediction model based on the predicted and expected results using the preset loss function, and adjusting the parameters of the pre-trained text similarity prediction model according to the loss, continues until the loss reaches a preset loss threshold or a preset number of iterations, resulting in a trained text similarity prediction model. A smoothing loss function is used as the preset loss function to calculate the loss of the pre-trained text similarity prediction model. The smoothing loss function characterizes the difference between the true and predicted categories of the samples, plus the deviation between the predicted distribution and the prior distribution. Specifically, this embodiment of the invention improves upon the use of minimizing the negative log-likelihood function to obtain a smoothing loss function. The ERT model loss function uses minimizing the negative log-likelihood function and improves upon it to obtain a smoothing loss function. The function that minimizes the negative log-likelihood is essentially the same as the cross-entropy function. For each training sample i, the probability of it belonging to each class label k∈{1…K} is predicted using softmax as follows:

[0072]

[0073] in This is a logit transformation, where the score of the i-th sample belonging to the k-th class is given by the logit transformation. The probability that each sample i truly belongs to the k-th class is given by... Normalization of it Generally, the probability of a sample belonging to class k is represented by one-hot encoding, meaning it belongs to only one class. If the number of samples is m, then the cross-entropy loss function is expressed as:

[0074]

[0075] Wherein, the probability that each sample i truly belongs to the k-th category is: This represents the predicted probability that sample i belongs to the k-th class. The loss statement is related to... It is differentiable, therefore, when using gradient descent for parameter tuning, the simplified form is:

[0076]

[0077] As shown in the above equation, in order to minimize the loss, it will... and As close as possible, only when The score for the correct category label must be significantly higher than that for other incorrect categories. This means that during training, the model will prioritize the loss for correct classification over the loss for incorrect label positions, leading to overconfidence in its predictions, poor adjustment capabilities, and ultimately overfitting.

[0078] To address the above issues, a label smoothing regularization (LSR) method is introduced to improve the loss function. Noise is added to the true distribution of sample class labels, replacing it with:

[0079]

[0080] Where ∈ is a hyperparameter, and u(k) is the prior distribution of class k, which is usually taken as a uniform distribution. Then, after adding the LSR smoothing factor, the loss function becomes:

[0081]

[0082] The above formula shows that the smoothing loss function is the difference between the true class and the predicted class of the sample plus the degree of deviation between the predicted distribution and the prior distribution. This improves upon the shortcomings of the original method, which only focuses on approximating the true classification of the label, and enhances its generalization ability.

[0083] Step 1202: Input the first text and the second text into the word similarity calculation module to obtain the word similarity between each word in the first text and the second text.

[0084] In this embodiment of the invention, after obtaining the first text and the second text, text preprocessing is required to obtain preprocessed first text and preprocessed second text. The text preprocessing includes: segmenting the first text and the second text using natural language processing methods, word segmentation, stop word removal, and keyword extraction. Specifically, the HanLP natural language processing platform tool is used to remove stop words, clean punctuation and whitespace characters, and then perform word segmentation to obtain the vocabulary sequence S1 = {w1, w1, ... w1} of the first text. m The word sequence of the second text is S2 = {w1, w1, ... w1}. n}. Where the first text can be the answer text to the test question, and the second text can be the answer text to the test question, the number of keywords extracted from the answer text S1 is determined based on the number of scoring point keywords in the answer text. If the reference answer does not set any scoring points, the TextRank algorithm is used to extract m and n keywords from S1 and S2 respectively, resulting in the word sequence S1 = {w1, w1, ... w1} of the first text. m The word sequence of the second text is S2 = {w1, w1, ... w1}. n}. Specifically, the first text and the second text are input into the word similarity calculation module to obtain the word similarity between each word in the first text and the second text. This is achieved by inputting the word sequence S1 = {w1, w1, ... w1} of the first text. m The word sequence of the second text is S2 = {w1, w1, ... w1}. n Input the word similarity calculation module to obtain the word similarity between each word in the first text and the second text.

[0085] The method involves calculating the similarity between words using a pre-defined concept similarity database. This database describes the similarity between concepts of words. The similarity between m words in the first text S1 and n words in the second text S2 is calculated to obtain an m×n word matching matrix. The sum of the maximum values ​​in each row of the word matching matrix is ​​then taken as the arithmetic mean to obtain the first word similarity between the first text S1 and the second text S2. The calculation formula is as follows:

[0086]

[0087] Among them, w 1i Let w represent the similarity between the i-th word in the first text S1 and the n-th words in the second text S2. 2j This represents the similarity between the j-th word in the second text S2 and the m words in the first text S1.

[0088] The second word similarity between the second text S2 and the first text S1 is obtained using the same method. Then, the average of the first word similarity and the second word similarity is taken as the word similarity between the first text and the second text.

[0089] Step 1203: Input the first text and the second text into the text edit distance similarity calculation module to obtain the text edit distance similarity.

[0090] In this embodiment of the invention, the word sequence S1 = {w1, w1, ... w1} of the first text is... m The word sequence of the second text is S2 = {w1, w1, ... w1}. n Input the text edit distance similarity calculation module to obtain the text edit distance similarity.

[0091] The edit distance between the first text S1 and the second text S2 is represented by the following formula, which represents the minimum number of edits required to transform one string into the other.

[0092]

[0093] Where a and b are the two strings S1 and S2, i and j are their lengths, and lev is the length of the string. a,b (i,j) is defined as the number of operations required to transform the substring a[1,2,…i] represented by the first i characters of string a into the substring b[1,2,…j] represented by the first j characters of string b through insertion, deletion, and replacement operations. max(i,j) represents the number of operations required if i or j is 0, then only characters of length k and i need to be added. lev a,b (i-1,j)+1 represents the minimum number of times substring a needs to be matched against substring b by deleting one byte, lev a,b (i,j-1)+1 and lev a,b (i-1, j-1+1 represent the number of times the add and replace operations are performed, respectively.)

[0094] Calculate the edit distance similarity of each word between the first text S1 and the second text S2, and take the average value to represent the shallow feature similarity between the answer texts, i.e., the text edit distance similarity. The formula for the edit distance between words is:

[0095] Len(a,b) represents the maximum length of the string between a and b.

[0096] Step 1204: Calculate the similarity result by weighted fusion based on the predicted text similarity, the word similarity, and the text edit distance similarity.

[0097] Specifically, after obtaining the text prediction similarity, the word similarity, and the text edit distance similarity, the similarities are linearly weighted to calculate the fusion similarity, which is the similarity result.

[0098] When the first text is the answer text to a test question, and the second text is the answer text to the test question, the test question score is determined; based on the similarity result and the test question score, the score corresponding to the answer text is calculated, and then multiplied by the test question score to calculate the final score. Wherein, if the test question is a subjective question, the calculation formula for the subjective question scoring method is:

[0099] Score=[α·TextSim(S1,S2)+β·WordSim(S1,S2)+γ·EditSim(S1,S2)]·MaxScore

[0100] Wherein, MaxScore is the test score, TextSim(S1,S2) is the predicted similarity of the text, WordSim(S1,S2) is the word similarity, EditSim(S1,S2) is the edit distance similarity of the text, α, β, γ are hyperparameters and their sum is 1. In this embodiment of the invention, the proportion can be set as α = 0.6, β = 0.3, γ = 0.1.

[0101] This invention provides an embodiment of the invention that obtains a first text and a second text to be identified; inputs the first text and the second text into a similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text; wherein the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module and a text edit distance similarity calculation module; text edit distance similarity represents the minimum number of edits required to convert one string into another, and can accurately determine the similarity between two texts.

[0102] Figure 4 A schematic diagram of the text similarity determination device provided in an embodiment of the present invention is shown. Figure 4 As shown, the device 300 includes:

[0103] The acquisition module 310 is used to acquire the first text and the second text to be recognized;

[0104] The calculation module 320 is used to input the first text and the second text into the similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text; wherein the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module and a text edit distance similarity calculation module; the text edit distance similarity represents the minimum number of edits required to convert one string into another.

[0105] In an optional approach, the step of inputting the first text and the second text into a similarity calculation model for calculation to obtain a similarity result further includes:

[0106] Input the first text and the second text into the text similarity prediction model to obtain the predicted text similarity between the first text and the second text;

[0107] Input the first text and the second text into the word similarity calculation module to obtain the word similarity between each word in the first text and the second text;

[0108] The first text and the second text are input into the text edit distance similarity calculation module to obtain the text edit distance similarity.

[0109] The similarity result is obtained by weighted fusion calculation based on the predicted text similarity, the word similarity, and the text edit distance similarity.

[0110] In one optional embodiment, the text similarity prediction model includes an input layer, an encoding layer, a matching layer, a matching information extraction layer, and an output layer; the step of inputting the first text and the second text into the text similarity prediction model to obtain the predicted text similarity between the first text and the second text includes:

[0111] The first text and the second text are respectively input into the encoding layer through the input layer for encoding to obtain the first context vector of the first text and the second up context vector of the second text.

[0112] The matching layer calculates a matching matrix by analyzing the first context vector and the second context vector; the matching matrix represents the similarity between each word in the first text and each word in the second text.

[0113] Based on the matching matrix, the matching information extraction layer extracts matching information between the first text and the second text to obtain sentence-level matching information.

[0114] Based on the sentence-level matching information, the text prediction similarity between the first text and the second text is predicted.

[0115] In one optional approach, the step of extracting matching information between the first text and the second text using the matching information extraction layer based on the matching matrix to obtain sentence-level matching information includes:

[0116] Using the similarity between words in the matching matrix as local features, a convolutional neural network is used to extract global features from the matching matrix, and after pooling, it becomes a phrase matching matrix.

[0117] The phrase matching matrix is ​​convolved and pooled to obtain sentence-level matching information.

[0118] In an optional manner, before inputting the first text and the second text into a similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text, the method further includes:

[0119] The initial text similarity prediction model was pre-trained using the SOP task to obtain the pre-trained text similarity prediction model.

[0120] The similar text sample set is input into the pre-trained text similarity prediction model for training, and the prediction results are obtained.

[0121] Based on the predicted and expected results, a preset loss function is used to calculate the loss of the pre-trained text similarity prediction model. The parameters of the pre-trained text similarity prediction model are adjusted according to the loss. The process of inputting a set of similar text samples into the pre-trained text similarity prediction model for training, obtaining prediction results, and calculating the loss of the pre-trained text similarity prediction model based on the predicted and expected results using a preset loss function, and adjusting the parameters of the pre-trained text similarity prediction model according to the loss, continues until the loss reaches a preset loss threshold or a preset number of iterations is reached, resulting in a trained text similarity prediction model.

[0122] In one optional approach, the step of calculating the loss of the pre-trained text similarity prediction model using a preset loss function based on the predicted and expected results includes:

[0123] The loss of the pre-trained text similarity prediction model is calculated using a smoothing loss function as the preset loss function. The smoothing loss function characterizes the difference between the true class and the predicted class of the sample, plus the degree of deviation between the predicted distribution and the prior distribution.

[0124] In one optional approach, the first text is the answer text to the test question, and the second text is the answer text to the test question; after inputting the first text and the second text into a similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text, the method further includes:

[0125] Determine the score for each question in the test;

[0126] Based on the similarity results and the question scores, the score corresponding to the answer text is calculated.

[0127] The working process of the text similarity determination device in this embodiment of the invention is largely the same as the specific implementation steps in the above method embodiments, and will not be repeated here.

[0128] This invention provides an embodiment of the invention that obtains a first text and a second text to be identified; inputs the first text and the second text into a similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text; wherein the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module and a text edit distance similarity calculation module; text edit distance similarity represents the minimum number of edits required to convert one string into another, and can accurately determine the similarity between two texts.

[0129] Figure 5 The diagram shows a structural schematic of a computer device provided in an embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the computer device.

[0130] like Figure 5 As shown, the computer device may include: a processor 402, a communications interface 404, a memory 406, and a communications bus 408.

[0131] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408. Communication interface 404 is used to communicate with other network elements such as clients or other servers. The processor 402 executes program 410, specifically performing the relevant steps described above in the text similarity determination method embodiment.

[0132] Specifically, program 410 may include program code, which includes computer-executable instructions.

[0133] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The computer device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0134] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0135] Specifically, program 410 can be called by processor 402 to cause the computer device to perform the following operations:

[0136] Obtain the first and second texts to be identified;

[0137] The first text and the second text are respectively input into the similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text; wherein the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module and a text edit distance similarity calculation module; the text edit distance similarity represents the minimum number of edits required to convert one string into another.

[0138] In an optional approach, the step of inputting the first text and the second text into a similarity calculation model for calculation to obtain a similarity result further includes:

[0139] Input the first text and the second text into the text similarity prediction model to obtain the predicted text similarity between the first text and the second text;

[0140] Input the first text and the second text into the word similarity calculation module to obtain the word similarity between each word in the first text and the second text;

[0141] The first text and the second text are input into the text edit distance similarity calculation module to obtain the text edit distance similarity.

[0142] The similarity result is obtained by weighted fusion calculation based on the predicted text similarity, the word similarity, and the text edit distance similarity.

[0143] In one optional embodiment, the text similarity prediction model includes an input layer, an encoding layer, a matching layer, a matching information extraction layer, and an output layer; the step of inputting the first text and the second text into the text similarity prediction model to obtain the predicted text similarity between the first text and the second text includes:

[0144] The first text and the second text are respectively input into the encoding layer through the input layer for encoding to obtain the first context vector of the first text and the second up context vector of the second text.

[0145] The matching layer calculates a matching matrix by analyzing the first context vector and the second context vector; the matching matrix represents the similarity between each word in the first text and each word in the second text.

[0146] Based on the matching matrix, the matching information extraction layer extracts matching information between the first text and the second text to obtain sentence-level matching information.

[0147] Based on the sentence-level matching information, the text prediction similarity between the first text and the second text is predicted.

[0148] In one optional approach, the step of extracting matching information between the first text and the second text using the matching information extraction layer based on the matching matrix to obtain sentence-level matching information includes:

[0149] Using the similarity between words in the matching matrix as local features, a convolutional neural network is used to extract global features from the matching matrix, and after pooling, it becomes a phrase matching matrix.

[0150] The phrase matching matrix is ​​convolved and pooled to obtain sentence-level matching information.

[0151] In an optional manner, before inputting the first text and the second text into a similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text, the method further includes:

[0152] The initial text similarity prediction model was pre-trained using the SOP task to obtain the pre-trained text similarity prediction model.

[0153] The similar text sample set is input into the pre-trained text similarity prediction model for training, and the prediction results are obtained.

[0154] Based on the predicted and expected results, a preset loss function is used to calculate the loss of the pre-trained text similarity prediction model. The parameters of the pre-trained text similarity prediction model are adjusted according to the loss. The process of inputting a set of similar text samples into the pre-trained text similarity prediction model for training, obtaining prediction results, and calculating the loss of the pre-trained text similarity prediction model based on the predicted and expected results using a preset loss function, and adjusting the parameters of the pre-trained text similarity prediction model according to the loss, continues until the loss reaches a preset loss threshold or a preset number of iterations is reached, resulting in a trained text similarity prediction model.

[0155] In one optional approach, the step of calculating the loss of the pre-trained text similarity prediction model using a preset loss function based on the predicted and expected results includes:

[0156] The loss of the pre-trained text similarity prediction model is calculated using a smoothing loss function as the preset loss function. The smoothing loss function characterizes the difference between the true class and the predicted class of the sample, plus the degree of deviation between the predicted distribution and the prior distribution.

[0157] In one optional approach, the first text is the answer text to the test question, and the second text is the answer text to the test question; after inputting the first text and the second text into a similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text, the method further includes:

[0158] Determine the score for each question in the test;

[0159] Based on the similarity results and the question scores, the score corresponding to the answer text is calculated.

[0160] This invention provides an embodiment of the invention that obtains a first text and a second text to be identified; inputs the first text and the second text into a similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text; wherein the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module and a text edit distance similarity calculation module; text edit distance similarity represents the minimum number of edits required to convert one string into another, and can accurately determine the similarity between two texts.

[0161] This invention provides a computer-readable storage medium storing at least one executable instruction that, when executed on a computer device, causes the computer device to perform the text similarity determination method in any of the above method embodiments.

[0162] Executable instructions can be used to cause computer devices to perform the following operations:

[0163] Obtain the first and second texts to be identified;

[0164] The first text and the second text are respectively input into the similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text; wherein the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module and a text edit distance similarity calculation module; the text edit distance similarity represents the minimum number of edits required to convert one string into another.

[0165] In an optional approach, the step of inputting the first text and the second text into a similarity calculation model for calculation to obtain a similarity result further includes:

[0166] Input the first text and the second text into the text similarity prediction model to obtain the predicted text similarity between the first text and the second text;

[0167] Input the first text and the second text into the word similarity calculation module to obtain the word similarity between each word in the first text and the second text;

[0168] The first text and the second text are input into the text edit distance similarity calculation module to obtain the text edit distance similarity.

[0169] The similarity result is obtained by weighted fusion calculation based on the predicted text similarity, the word similarity, and the text edit distance similarity.

[0170] In one optional embodiment, the text similarity prediction model includes an input layer, an encoding layer, a matching layer, a matching information extraction layer, and an output layer; the step of inputting the first text and the second text into the text similarity prediction model to obtain the predicted text similarity between the first text and the second text includes:

[0171] The first text and the second text are respectively input into the encoding layer through the input layer for encoding to obtain the first context vector of the first text and the second up context vector of the second text.

[0172] The matching layer calculates a matching matrix by analyzing the first context vector and the second context vector; the matching matrix represents the similarity between each word in the first text and each word in the second text.

[0173] Based on the matching matrix, the matching information extraction layer extracts matching information between the first text and the second text to obtain sentence-level matching information.

[0174] Based on the sentence-level matching information, the text prediction similarity between the first text and the second text is predicted.

[0175] In one optional approach, the step of extracting matching information between the first text and the second text using the matching information extraction layer based on the matching matrix to obtain sentence-level matching information includes:

[0176] Using the similarity between words in the matching matrix as local features, a convolutional neural network is used to extract global features from the matching matrix, and after pooling, it becomes a phrase matching matrix.

[0177] The phrase matching matrix is ​​convolved and pooled to obtain sentence-level matching information.

[0178] In an optional manner, before inputting the first text and the second text into a similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text, the method further includes:

[0179] The initial text similarity prediction model was pre-trained using the SOP task to obtain the pre-trained text similarity prediction model.

[0180] The similar text sample set is input into the pre-trained text similarity prediction model for training, and the prediction results are obtained.

[0181] Based on the predicted and expected results, a preset loss function is used to calculate the loss of the pre-trained text similarity prediction model. The parameters of the pre-trained text similarity prediction model are adjusted according to the loss. The process of inputting a set of similar text samples into the pre-trained text similarity prediction model for training, obtaining prediction results, and calculating the loss of the pre-trained text similarity prediction model based on the predicted and expected results using a preset loss function, and adjusting the parameters of the pre-trained text similarity prediction model according to the loss, continues until the loss reaches a preset loss threshold or a preset number of iterations is reached, resulting in a trained text similarity prediction model.

[0182] In one optional approach, the step of calculating the loss of the pre-trained text similarity prediction model using a preset loss function based on the predicted and expected results includes:

[0183] The loss of the pre-trained text similarity prediction model is calculated using a smoothing loss function as the preset loss function. The smoothing loss function characterizes the difference between the true class and the predicted class of the sample, plus the degree of deviation between the predicted distribution and the prior distribution.

[0184] In one optional approach, the first text is the answer text to the test question, and the second text is the answer text to the test question; after inputting the first text and the second text into a similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text, the method further includes:

[0185] Determine the score for each question in the test;

[0186] Based on the similarity results and the question scores, the score corresponding to the answer text is calculated.

[0187] This invention provides an embodiment of the invention that obtains a first text and a second text to be identified; inputs the first text and the second text into a similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text; wherein the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module and a text edit distance similarity calculation module; text edit distance similarity represents the minimum number of edits required to convert one string into another, and can accurately determine the similarity between two texts.

[0188] This invention provides a text similarity determination apparatus for performing the above-described text similarity determination method.

[0189] This invention provides a computer program that can be called by a processor to cause a computer device to execute the text similarity determination method in any of the above method embodiments.

[0190] This invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed on a computer, cause the computer to perform the text similarity determination method in any of the above method embodiments.

[0191] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of the present invention are not directed to any particular programming language. It should be understood that the content of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0192] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0193] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the embodiments of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.

[0194] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0195] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

Claims

1. A method for determining text similarity, characterized in that, The method includes: Obtain the first and second texts to be identified; The first text and the second text are respectively input into the similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text; wherein the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module and a text edit distance similarity calculation module; the text edit distance similarity represents the minimum number of edits required to convert one string into another; The text similarity prediction model includes an input layer, an encoding layer, a matching layer, a matching information extraction layer, and an output layer; the method specifically includes: The first text and the second text are respectively input into the encoding layer through the input layer for encoding to obtain the first context vector of the first text and the second context vector of the second text; wherein, the encoding layer is a BERT model; The matching layer calculates a matching matrix by analyzing the first context vector and the second context vector; the matching matrix represents the similarity between each word in the first text and each word in the second text. Based on the matching matrix, the matching information extraction layer extracts matching information between the first text and the second text to obtain sentence-level matching information; wherein, the similarity between words in the matching matrix is ​​used as a local feature, a convolutional neural network is used to extract global features from the matching matrix, and after pooling, it is used as a phrase matching matrix; convolution and pooling are performed on the phrase matching matrix to obtain sentence-level matching information; Based on the sentence-level matching information, the text prediction similarity between the first text and the second text is predicted.

2. The method according to claim 1, characterized in that, Before inputting the first text and the second text into the similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text, the method further includes: The initial text similarity prediction model is pre-trained using the SOP task to obtain the pre-trained text similarity prediction model; the SOP task uses two consecutive sentences as positive samples and two consecutive sentences after being reversed in order as negative samples. The similar text sample set is input into the pre-trained text similarity prediction model for training, and the prediction results are obtained. Based on the predicted and expected results, a preset loss function is used to calculate the loss of the pre-trained text similarity prediction model. The parameters of the pre-trained text similarity prediction model are adjusted according to the loss. The process of inputting a set of similar text samples into the pre-trained text similarity prediction model for training, obtaining prediction results, and calculating the loss of the pre-trained text similarity prediction model based on the predicted and expected results using a preset loss function, and adjusting the parameters of the pre-trained text similarity prediction model according to the loss, continues until the loss reaches a preset loss threshold or a preset number of iterations is reached, resulting in a trained text similarity prediction model.

3. The method according to claim 2, characterized in that, The step of calculating the loss of the pre-trained text similarity prediction model using a preset loss function based on the predicted and expected results includes: The loss of the pre-trained text similarity prediction model is calculated using a smoothing loss function as the preset loss function. The smoothing loss function characterizes the difference between the true class and the predicted class of the sample, plus the degree of deviation between the predicted distribution and the prior distribution.

4. The method according to claim 1, characterized in that, The step of inputting the first text and the second text into a similarity calculation model for calculation to obtain similarity results further includes: Input the first text and the second text into the text similarity prediction model to obtain the predicted text similarity between the first text and the second text; Input the first text and the second text into the word similarity calculation module to obtain the word similarity between each word in the first text and the second text; The first text and the second text are input into the text edit distance similarity calculation module to obtain the text edit distance similarity. The similarity result is obtained by weighted fusion calculation based on the predicted text similarity, the word similarity, and the text edit distance similarity.

5. The method according to any one of claims 1-4, characterized in that, The first text is the answer text to the test question, and the second text is the answer text to the test question; after inputting the first text and the second text into the similarity fusion calculation model for calculation to obtain the similarity result between the first text and the second text, the method further includes: Determine the score for each question in the test; Based on the similarity results and the question scores, the score corresponding to the answer text is calculated.

6. A text similarity determination device, characterized in that, The device includes: The acquisition module is used to acquire the first and second texts to be recognized; The calculation module is used to input the first text and the second text into the similarity fusion calculation model for calculation, and obtain the similarity result between the first text and the second text; wherein the similarity fusion calculation model includes a text similarity prediction model, a word similarity calculation module and a text edit distance similarity calculation module; the text edit distance similarity represents the minimum number of edits required to convert one string into another; The text similarity prediction model includes an input layer, an encoding layer, a matching layer, a matching information extraction layer, and an output layer; the calculation module is specifically used for: The first text and the second text are respectively input into the encoding layer through the input layer for encoding to obtain the first context vector of the first text and the second context vector of the second text; wherein, the encoding layer is a BERT model; The matching layer calculates a matching matrix by analyzing the first context vector and the second context vector; the matching matrix represents the similarity between each word in the first text and each word in the second text. Based on the matching matrix, the matching information extraction layer extracts matching information between the first text and the second text to obtain sentence-level matching information; wherein, the similarity between words in the matching matrix is ​​used as a local feature, a convolutional neural network is used to extract global features from the matching matrix, and after pooling, it is used as a phrase matching matrix; convolution and pooling are performed on the phrase matching matrix to obtain sentence-level matching information; Based on the sentence-level matching information, the text prediction similarity between the first text and the second text is predicted.

7. A computer device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation of the text similarity determination method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The storage medium stores at least one executable instruction, which, when executed on a computer device, causes the computer device to perform the operation of the text similarity determination method as described in any one of claims 1-5.

Citation Information

Patent Citations

  • User theme recommendation method and device, computer equipment and storage medium

    CN111651666A

  • Text similarity calculation method and device, computer equipment and storage medium

    CN114358210A