Text recommendation method, device, electronic device and storage medium

Abstract

The present invention provides a text recommendation method, apparatus, electronic device, and storage medium. The method includes: inputting an original text into a contrastive learning model to obtain target feature information of the original text, where the contrastive learning model is obtained by training an initial contrastive learning model based on multiple batches of sample texts. Each batch of sample texts includes at least one first triple and at least one second triple. The first triple includes an original sample text, a first positive example sample of the original sample text, and a negative example sample. The second triple includes an original sample text, a second positive example sample of the original sample text, and a negative example sample. The first positive example sample includes a text with a similarity greater than a preset value to the original sample text, and the second positive example sample includes a partial text of the original sample text. Determine the similarity between the target feature information and the preset feature information corresponding to multiple preset texts to determine target similar texts. The present invention can improve the accuracy of text recommendation.

Description

Technical Field

[0001] The present invention relates to the technical field of text recommendation, and in particular, to a text recommendation method, device, electronic device, and storage medium. Background Art

[0002] With the popularization and development of personalized education, teaching students in accordance with their aptitude has become an important goal pursued in the field of education. For example, in teaching evaluation, if it is possible to recommend corresponding exercise texts for students' weak links for practice, it plays a crucial role in improving students' learning.

[0003] Most of the existing text recommendation methods adopt static language models such as Glove (Global Vectors). The embedding representation of each word vector extracted by it is independent of the context. Therefore, when recommending more complex discourse - type test questions, the model will lead to the anisotropy of the representation space, making most of the representation vectors concentrated in a conical region of the representation space, resulting in a very high similarity between any two texts, that is, the similarity is concentrated above 0.9. A large number of texts are concentrated in the same similarity interval, which poses a great risk to both the threshold setting and the model robustness. This is extremely disadvantageous for the recall scheme of similar texts, that is, it cannot meet the existing recall requirements.

[0004] To solve the above - mentioned technical problems, a dynamic language model, namely the SimCSE (Simple Contrastive Learning of Sentence Embeddings) model, is proposed in the prior art. This model adopts the framework of BERT (Bidirectional Encoder Representation from Transformers). It can achieve SOTA (State Of The Art) results in the text similarity task through simple contrastive learning. In addition, the self - attention mechanism and the internal complex interaction method in this model have great advantages compared with the Glove static model, and there is no significant change to the original recall logic, that is, the original recall logic can be inherited. Thus, it can be seen that the SimCSE dynamic model only changes the vector representation of the text, that is, it has achieved a great improvement in the extraction of text semantic information and realized text recommendation with higher accuracy.

[0005] However, when recommending texts similar to the original text through the SimCSE model, it may occur that the recommended similar text is a partial text of the original text, reducing the accuracy of text recommendation. Summary of the Invention

[0006] The present invention provides a text recommendation method, apparatus, electronic device and storage medium, which are used to solve the defect of inaccurate text recommendation in the prior art and achieve more accurate text recommendation.

[0007] The present invention provides a text recommendation method, including:

[0008] Obtain the original text;

[0009] Input the original text into a contrastive learning model to obtain the target feature information of the original text. The contrastive learning model is obtained by training an initial contrastive learning model based on multiple batches of sample texts. Each batch of sample texts includes at least one first triple and at least one second triple. The first triple includes the original sample text, the first positive example sample of the original sample text, and a negative example sample. The second triple includes the original sample text, the second positive example sample of the original sample text, and the negative example sample. The first positive example sample includes a text with a similarity greater than a preset value to the original sample text. The second positive example sample includes a partial text of the original sample text;

[0010] Determine the similarity between the target feature information and the preset feature information corresponding to multiple preset texts, and determine the target preset feature information corresponding to the similarity between the recall threshold and the deduplication threshold. The recall threshold is less than the deduplication threshold;

[0011] Determine the preset text corresponding to the target preset feature information as the target similar text.

[0012] According to the text recommendation method provided by the present invention, at least one first triple included in each batch of sample texts is obtained based on the following method:

[0013] Obtain at least one original sample text;

[0014] In the case that there is no target positive example sample of the first original sample text in the sample text set, determine the first original sample text as the first positive example sample of the first original sample text. The first original sample text is one of the at least one original sample text;

[0015] Based on the first original sample text, the first positive example sample of the first original sample text, and the negative example sample of the first original sample text, determine the first triple.

[0016] According to the text recommendation method provided by the present invention, at least one second triple included in each batch of sample texts is obtained based on the following method:

[0017] For each original sample text, determine a part of the text in the original sample text as the second positive example sample of the original sample text;

[0018] Based on the original sample text, the second positive example sample of the original sample text, and the negative example sample of the original sample text, determine the second triple.

[0019] According to a text recommendation method provided by the present invention, the negative example sample of the original sample text is obtained based on the following method:

[0020] In the case where there is no target negative example sample of the second original sample text in the sample text set, based on the other original sample texts in the multiple original sample texts except the second original sample text, determine the negative example sample of the second original sample text, where the second original sample text is one of the at least one original sample text.

[0021] According to a text recommendation method provided by the present invention, determining the negative example sample of the second original sample text based on the other original sample texts in the multiple original sample texts except the second original sample text includes:

[0022] Determine the similarity between the second original sample text and each other original sample text respectively;

[0023] Based on the similarity, determine the negative example sample from the other original sample texts.

[0024] According to a text recommendation method provided by the present invention, determining the negative example sample of the second original sample text based on the other original sample texts in the multiple original sample texts except the second original sample text includes:

[0025] Randomly select a target original sample text from the other original sample texts, and determine the target original sample text as the negative example sample of the second original sample text.

[0026] According to a text recommendation method provided by the present invention, the number of the first triples is at least two, and the original sample texts included in each first triple are different.

[0027] The present invention also provides a text recommendation device, including:

[0028] An acquisition module, configured to acquire original text;

[0029] An input module for inputting the original text into a contrastive learning model to obtain target feature information of the original text. The contrastive learning model is obtained by training an initial contrastive learning model based on multiple batches of sample texts. Each batch of sample texts includes at least one first triple and at least one second triple. The first triple includes an original sample text, a first positive example sample of the original sample text, and a negative example sample. The second triple includes the original sample text, a second positive example sample of the original sample text, and the negative example sample. The first positive example sample includes a text with a similarity greater than a preset value to the original sample text. The second positive example sample includes a partial text of the original sample text;

[0030] A determination module for determining the similarity between the target feature information and preset feature information corresponding to multiple preset texts, and determining the target preset feature information corresponding to the similarity between the recall threshold and the deduplication threshold. The recall threshold is less than the deduplication threshold;

[0031] The determination module is further configured to determine the preset text corresponding to the target preset feature information as the target similar text.

[0032] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the text recommendation method described in any one of the above is implemented.

[0033] The present invention also provides an electronic device, including a display, a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the text recommendation method described in any one of the above is implemented;

[0034] The display is configured to display the target similar text.

[0035] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the text recommendation method described in any one of the above is implemented.

[0036] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the text recommendation method described in any one of the above is implemented.

[0037] The text recommendation method, device, electronic device, and storage medium provided by the present invention input the obtained original text into a contrastive learning model to obtain the target feature information of the original text, determine the similarity between the target feature information and the preset feature information corresponding to multiple preset texts, and determine the target preset feature information corresponding to the similarity between the recall threshold and the deduplication threshold, so as to determine the preset text corresponding to the target preset feature information as the target similar text. Among them, each batch of sample texts for training the contrastive learning model includes at least one first triple and at least one second triple. The first triple includes the original sample text, the first positive example sample of the original sample text, and the negative example sample. The second triple includes the original sample text, the second positive example sample of the original sample text, and the negative example sample. The first positive example sample includes a text with a similarity greater than a preset value to the original sample text, and the second positive example sample includes a partial text of the original sample text. Since the second triple including the second positive example sample is added to each batch of sample texts, the target feature information of the original text extracted by the trained contrastive learning model can match or be closer to the feature information corresponding to the partial text of the original text. In this way, when there is a partial text of the original text in the preset text, the similarity between the target feature information and the preset feature information corresponding to the partial text will be relatively high. Therefore, based on the deduplication threshold, the partial text of the original text can be screened out, and based on the recall threshold, the target similar text similar to the original text can be determined, thereby improving the accuracy of similar text recommendation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0039] Figure 1 is one of the process schematic diagrams of text recommendation in the prior art;

[0040] Figure 2 is another process schematic diagram of text recommendation in the prior art;

[0041] Figure 3 is the process schematic diagram of the text recommendation method provided by the embodiment of the present invention;

[0042] Figure 4 is the length distribution schematic diagram of the sample text provided by the embodiment of the present invention;

[0043] Figure 5 is the precision ratio comparison diagram of the model under different parameters in the embodiment of the present invention;

[0044] Figure 6 It is a comparison graph of the recall rates of models under different parameters in the embodiments of the present invention;

[0045] Figure 7 It is a schematic diagram of the similarity distribution of different labels in the embodiments of the present invention;

[0046] Figure 8 It is a schematic diagram of the similarity distribution of different labels in the prior art;

[0047] Figure 9 It is a schematic diagram of the distribution of the parameter indicators of SimCSE8_MoCo in the embodiments of the present invention;

[0048] Figure 10 It is a comparison graph of the parameter indicators of SimCSE8 and SimCSE8_MoCo in the embodiments of the present invention;

[0049] Figure 11 It is one of the schematic diagrams of the distribution of the parameter indicators of SimSmall on the development set in the embodiments of the present invention;

[0050] Figure 12 It is one of the schematic diagrams of the distribution of the parameter indicators of SimSmall on the test set in the embodiments of the present invention;

[0051] Figure 13 It is the second schematic diagram of the distribution of the parameter indicators of SimSmall on the development set in the embodiments of the present invention;

[0052] Figure 14 It is the second schematic diagram of the distribution of the parameter indicators of SimSmall on the test set in the embodiments of the present invention;

[0053] Figure 15 It is a schematic diagram of the similarity distribution of the SimSmall model for random distribution in the embodiments of the present invention;

[0054] Figure 16 It is a schematic diagram of the rejection rate distribution of the SimSmall model for random distribution in the embodiments of the present invention;

[0055] Figure 17 It is a schematic diagram of the structure of the text recommendation device provided in the embodiments of the present invention;

[0056] Figure 18 It is one of the schematic diagrams of the structure of the electronic device provided by the present invention;

[0057] Figure 19 It is the second schematic diagram of the structure of the electronic device provided by the present invention. Detailed implementation manners

[0058] To make the objectives, technical solutions, and advantages of the present invention more clear, the following will, in conjunction with the accompanying drawings in the present invention, clearly and completely describe the technical solutions in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts fall within the scope of protection of the present invention.

[0059] Text recommendation plays a huge role in certain specific fields. For example, in the field of teaching, the recent learning effects of students can be detected through the recommended practice of similar question types, so as to achieve the purpose of assisting teachers in teaching.

[0060] Currently, in addition to text recommendation through manual means, text recommendation in the prior art is mainly divided into two types: one is text recommendation implemented through static language models such as Glove, and the other is text recommendation implemented through dynamic language models such as SimCSE.

[0061] Below, taking the first static language model Glove as an example, the text recommendation process for similar discourse type questions is introduced.

[0062] Figure 1 As one of the process schematic diagrams of text recommendation in the prior art, as Figure 1 shown, the text corresponding to the student's wrong questions is input into the similarity prediction network (Glove model) to obtain the intermediate vector of the text. Specifically, the similarity prediction network will, through the vocabulary table, obtain the vector representation embedding of each word in the text, and according to the manually screened stop word table, screen and eliminate some words without real meaning, such as that, the, and punctuation marks, as well as some extremely frequent words, such as I, he, is, etc., one by one, and take the average value of the vector representations of the remaining words to obtain the semantic vector representation of the text corresponding to the student's wrong questions, that is, the intermediate vector h(t). Then, according to all the vector features in the preset question bank, determine the similarity between the intermediate vector h(t) and each vector feature in the preset question bank, so as to recommend the question corresponding to the vector feature with the highest similarity in the question bank as the similarity corresponding to the student's wrong questions.

[0063] In addition, as Figure 1 shown, when training the similarity prediction network, the loss information can be determined by judging the similarity between sample data, so as to iteratively train the similarity prediction network.

[0064] As can be seen from the above, the focus of the prior art lies in the construction of the similarity prediction network. That is, for discourse-based test questions, the similarity prediction network will recommend other test questions with semantic content similar to the input test question stem according to the input test question stem. Exemplarily, the text recommendation process in the prior art includes: (1) segmenting the text into individual words, for example, the text can be segmented by the tokenization tool NLTK (Natural Language ToolKit); (2) deleting the words that appear in the stop word list according to the manually set stop word list; (3) determining the average value of the vectors of the remaining words according to the word vector query in the Glove word list, and taking this average value as the text vector; (4) determining the test question with the highest similarity to the text vector among all the preset text vectors in the sample text set, and taking it as the output result of the similar text.

[0065] Since the static model adopted in the above prior art does not pay attention to context information and only extracts the vector representations of individual words, it cannot solve problems such as polysemy and insufficient sensitivity to key words. Especially in the question types at the high school discourse level, factors such as the difficulty of text understanding and the length of the text will make the effect of this static model worse, thus affecting the final recommendation result.

[0066] For the above-mentioned static model, from the perspective of the finally presented vector distribution, most of the text representation vectors are concentrated in a conical range in the representation space, so that even two completely unrelated texts may have a cosine similarity score of at least 0.8. This is extremely disadvantageous for setting the threshold during test question recall, because slightly increasing the threshold will cause the number of recalled test questions to drop sharply, while slightly decreasing the threshold will result in the recall of completely unrelated test questions.

[0067] To solve the above problems, a dynamic language model has been proposed in the prior art. Taking the same similar discourse-based test questions as an example below, this text recommendation method will be described.

[0068] Figure 2 It is the second schematic diagram of the text recommendation process in the prior art, as Figure 2As shown in the figure, before inputting the original test question text into the SimCSE model, it is necessary to perform word segmentation on the text first. Here, the BERT model can be used for word segmentation. Among them, for test questions with underlines or other symbols, the underlines or other symbols can be discarded first. Then, the processed text is input into the SimCSE model to obtain the vector representation of the text. Based on this, each preset text in the preset question bank is processed by the SimCSE model. After obtaining the representation vectors of all preset texts, the similarity between the representation vector of the preset text and the vector of the original test question text is calculated one by one, and sorted according to the similarity, so as to output the final recommended text, that is, the preset text that is more similar to the original test question text.

[0069] Although the above dynamic model can avoid some inherent drawbacks of the static model, since the training data used in the supervised training of this model is a triple-form sentence group from the NLI (Natural Language Inference) dataset That is, each sentence corresponds to a positive example sample and a negative example sample. Among them, the positive example sample is a text with a high similarity to the original text. The target feature information of the original text extracted by the SimCSE model cannot match the feature information corresponding to part of the original text highly, so part of the original text will be recommended as a similar text to the original text, reducing the accuracy of text recommendation.

[0070] Based on this, the embodiment of the present invention proposes a text recommendation method. In each batch of sample texts for training the contrastive learning model, a second triple is added. Among them, the second triple includes the original sample text, the second positive example sample of the original sample text, and the negative example sample, and the second positive example sample includes part of the original sample text. Since part of the original sample text is used as the positive example sample of the original sample to train the contrastive learning model, in this way, the target feature information of the original text extracted by the contrastive learning model can match the feature information corresponding to part of the original text highly. In this way, when there is part of the original text in the preset text, the similarity between the target feature information and the preset feature information corresponding to the part of the text will be relatively high. Thus, based on the deduplication threshold, part of the original text can be screened out, and based on the recall threshold, the target similar text similar to the original text can be determined, avoiding the phenomenon in the prior art of recommending part of the original text as a similar text to the original text, and improving the accuracy of text recommendation.

[0071] The following combines Figures 3 - 16The text recommendation method provided by the embodiments of the present invention will be described. The text recommendation method provided by the embodiments of the present invention can be applied to text recommendation scenarios, especially similar text recommendation scenarios. The execution subject of this method can be electronic devices such as mobile phones, computers, and computers, or any other device capable of performing text recommendation. No specific limitation is made in the present invention.

[0072] Figure 3 It is a schematic flowchart of the text recommendation method provided by the embodiments of the present invention. As Figure 3 shown, this method includes:

[0073] Step 301: Obtain the original text.

[0074] Among them, the original text can be text composed of any language. Specifically, the original text can be obtained by means of downloading through web crawlers, or by means of text scanning, text shooting, etc. No specific limitation is made in this regard.

[0075] Step 302: Input the original text into the contrast learning model to obtain the target feature information of the original text.

[0076] Among them, the contrast learning model is obtained by training the initial contrast learning model based on multiple batches of sample texts. Each batch of sample texts includes at least one first triple and at least one second triple. The first triple includes the original sample text, the first positive example sample of the original sample text, and the negative example sample. The second triple includes the original sample text, the second positive example sample of the original sample text, and the negative example sample. The first positive example sample includes text with a similarity greater than a preset value to the original sample text. The second positive example sample includes part of the text of the original sample text.

[0077] Specifically, before inputting the obtained original text into the pre-trained contrast learning model, the original text can be tokenized first. Further, when the original text includes symbols such as underlines or parentheses, these symbols can also be deleted, and the text after deleting the symbols is input into the pre-trained contrast learning model to obtain the target feature information of the original text, that is, the vector representation of the original text. Among them, the contrast learning model can be, for example, the SimCSE model. Exemplarily, the target feature information of the original text can be generated in various ways, such as averaging the last layer vectors of the contrast learning model, averaging the first layer and the last layer vectors, or using the pooling layer of the [CLS] vector, etc. It is found through subsequent ablation experiments that more accurate results are obtained after processing using the pooling layer of the [CLS] vector.

[0078] In addition, before inputting the original text into the contrastive learning model, it is necessary to first train the initial contrastive learning model according to multiple batches of sample texts composed of a first triple including the original sample text, the first positive example sample and the negative example sample of the original sample text, and a second triple including the original sample text, the second positive example sample and the negative example sample of the original sample text. Among them, the second positive example sample is a partial text of the original sample text, which can be understood as a sub-text of the original sample text. Exemplarily, the partial text of the original sample text can be obtained by randomly deleting part of the text starting from the beginning or the end of the original sample text, or can be obtained by deleting according to a preset rule starting from any position of the original sample text. For example, 1 / 3 of the text can be deleted from the beginning of the original sample text to obtain the above partial text.

[0079] Exemplarily, the initial contrastive learning model can be trained with the following parameters: lr = 3e-5, batch-size = 8, optimizer = Adam, the model framework is bert-base-uncased, and the loss function is the cross-entropy loss function of the contrastive loss. Among them, lr represents the learning rate, batch-size represents the number of sample texts processed in each batch, and optimizer represents the optimizer used during model training.

[0080] Since, based on the first positive example sample, the second positive example sample is added to each batch of sample texts to train the initial contrastive learning model to obtain the contrastive learning model, after inputting the original text into the above contrastive learning model, the similarity or matching degree between the obtained target feature information and the feature information corresponding to the partial text of the original text will be very high. Based on this similarity or matching degree, the partial text of the original text can be screened out.

[0081] Step 303: Determine the similarity between the target feature information and the preset feature information corresponding to multiple preset texts.

[0082] Specifically, the preset text library includes the preset feature information corresponding to multiple preset texts, and the preset feature information corresponding to each preset text can be obtained in the manner described in the above steps. On this basis, the cosine similarity between the preset feature information corresponding to each preset text and the target feature information of the original text can be calculated, so that the similarity between the original text and each preset text can be determined based on this cosine similarity.

[0083] Step 304: Determine the target preset feature information corresponding to the similarity between the recall threshold and the deduplication threshold, where the recall threshold is less than the deduplication threshold.

[0084] Among them, the recall threshold refers to the lower bound value of the similarity used to distinguish target similar texts, and the deduplication threshold refers to the upper bound value of the similarity used to distinguish target similar texts. Specifically, according to the recall threshold and the deduplication threshold, all preset feature information with similarity values between the two can be queried. Among them, the recall threshold can be, for example, 0.834, and the deduplication threshold can be, for example, 0.978.

[0085] Specifically, after training the contrastive learning model, the recall threshold and the deduplication threshold can be determined according to the validation set. That is, after calculating the similarity of each preset text in the sample text set, these thresholds will be used to decide whether to recall or not. For example, in the second-round recall of text recommendation, assuming that the accurate recall threshold for the first round is 0.834, the fallback recall threshold for the second round is 0.652, and the deduplication threshold is 0.978. Then, when making text recommendations, all preset texts with similarity values between 0.834 and 0.978 will be preferentially output as accurate similar texts. If the number of texts output at this time is insufficient or cannot meet the user's needs, all preset texts with similarity values between 0.652 and 0.834 will be further output as supplementary texts to meet the text recommendation requirements. For example, if 50 texts similar to the original text need to be recommended, after determining the similarity between the target feature information of the original text and the preset features of each preset text in the question bank, 50 preset texts can be selected from all texts with similarity values between 0.834 and 0.978 in the first round. If the number is less than 50, texts need to be continuously selected from the range of 0.652 to 0.834 until the number reaches 50, thus completing the recommendation requirements.

[0086] Of course, in order to control the quantity and quality of the output of target similar texts, according to preset rules, such as fixed quantity, similarity sorting, etc., the recommended texts selected in each round can be output according to the similarity size, so as to achieve more accurate text recommendation.

[0087] Step 305: Determine the preset text corresponding to the target preset feature information as the target similar text.

[0088] Specifically, the preset text corresponding to the target preset feature information with similarity values between the recall threshold and the deduplication threshold can be determined as the target similar text. In this way, dissimilar preset texts and preset texts that are highly similar to the original text can be excluded, thus achieving more accurate text recommendation. The preset texts that are highly similar to the original text can include texts that are exactly the same as the original text, texts with only a small number of word replacements, or partial texts of the original text.

[0089] The text recommendation method provided by the embodiments of the present invention inputs the obtained original text into a contrastive learning model to obtain the target feature information of the original text, determines the similarity between the target feature information and the preset feature information corresponding to multiple preset texts, and determines the target preset feature information corresponding to the similarity between the recall threshold and the deduplication threshold, so as to determine the preset text corresponding to the target preset feature information as the target similar text. Among them, each batch of sample texts for training the contrastive learning model includes at least one first triple and at least one second triple. The first triple includes the original sample text, the first positive example sample of the original sample text, and the negative example sample. The second triple includes the original sample text, the second positive example sample of the original sample text, and the negative example sample. The first positive example sample includes a text with a similarity greater than a preset value to the original sample text, and the second positive example sample includes a partial text of the original sample text. Since the second triple containing the second positive example sample is added to each batch of sample texts, the target feature information of the original text extracted by the trained contrastive learning model can match or be closer to the feature information corresponding to the partial text of the original text. In this way, when there is a partial text of the original text in the preset text, the similarity between the target feature information and the preset feature information corresponding to the partial text will be relatively high. Thus, based on the deduplication threshold, the partial text of the original text can be screened out, and based on the recall threshold, the target similar text similar to the original text can be determined, thereby improving the accuracy of similar text recommendation.

[0090] Based on the above embodiments, when training the initial contrastive learning model, multiple batches of sample texts need to be obtained. Among them, at least one first triple included in each batch of sample texts can be obtained in the following manner: obtain at least one original sample text; in the case that there is no target positive example sample of the first original sample text in the sample text set, determine the first original sample text as the first positive example sample of the first original sample text, where the first original sample text is one of the at least one original sample text; based on the first original sample text, the first positive example sample of the first original sample text, and the negative example sample of the first original sample text, determine the first triple.

[0091] Specifically, in the embodiments of the present invention, the contrastive learning model is taken as the SimCSE model as an example for illustration. For other contrastive learning models, the determination method of their sample texts is similar to that of the SimCSE model, and will not be elaborated here.

[0092] Among them, for the supervised learning of the SimCSE model, its training data mainly comes from the triple-form sentence groups of the NLI dataset That is, each sentence corresponds to a positive example sample and a negative example sample. In the embodiments of the present invention, the form of this training data is Therefore, it is necessary to first convert this form into m * n triples that meet the requirements of the SimCSE model. Among them, and are the positive example sample and negative example sample of the original sample text selected from the sample text set. The positive example sample can be understood as a sample with a similarity greater than the first preset value to the original sample text, and the negative example sample can be understood as a sample with a similarity less than the second preset value to the original sample text, where the first preset value is greater than or equal to the second preset value.

[0093] For the first original sample text in at least one obtained original sample text, if there is no corresponding target positive example sample in the sample text set, that is, when there is no sample in the sample text set with a similarity greater than the first preset value to the first original sample text, positive example samples need to be supplemented. Exemplarily, the first original sample text itself can be determined as the first positive example sample. For example, assume h1, h2,..., h m are the original sample texts, and h1 is the first original sample text. When the target positive example sample corresponding to h1 is not found in the preset sample text library, that is, all the samples in the preset sample text set are negative example samples with a similarity less than the second preset value, h1 itself can be used as the first positive example sample of h1. Similarly, when any one of h2,..., h n is the first original sample text, its corresponding first positive example sample can be constructed in the above way. By analogy, the first positive example samples of the first original sample texts for which no target positive example samples are found in each batch of sample texts can be obtained. Thus, the first triple can be constructed based on the first original sample text, the first positive example sample of the first original sample text, and the negative example sample of the first original sample text.

[0094] It should be noted that if there is a target positive example sample of the first original sample text in the sample text set, the target positive example sample can be directly used as the first positive example sample of the first original sample text.

[0095] In this embodiment, in the case where there is no target positive example sample of the first original sample text in the sample text set, the first original sample text itself is determined as the first positive example sample of the first original sample text, so that the first triple can be determined. In this way, it can be ensured that the contrast learning model will not have problems with incorrect model training due to the absence of the first positive example sample of the first original sample text in the sample text set. In addition, determining the first original sample text itself as the first positive example sample of the first original sample text makes the similarity between the first positive example sample and the first original sample text higher, thereby improving the accuracy of model training and providing a basis for subsequent text recommendation.

[0096] Further, at least one second triple included in each batch of sample texts can be obtained in the following manner: for each original sample text, determine a part of the text in the original sample text as the second positive example sample of the original sample text; based on the original sample text, the second positive example sample of the original sample text, and the negative example sample of the original sample text, determine the second triple.

[0097] Specifically, first, determine a part of the text of each original sample text in each batch of sample texts as the second positive example sample of each original sample text. Among them, part of the text can be randomly deleted from the beginning or the end of the original sample text, and the remaining text is determined as the second positive example sample. According to the obtained original sample text, the second positive example sample of the original sample text, and the negative example sample of the original sample text, construct the corresponding second triple of the original sample text, so as to obtain multiple second triples.

[0098] Further, by determining a part of the text of each original sample text in each batch of sample texts as the second positive example sample of each original sample text, at least one second triple can be constructed by using the obtained original sample text, the second positive example sample of the original sample text, and the negative example sample of the original sample text. The target feature information of the original text extracted by the contrast learning model trained based on the second triple can match or be closer to the feature information corresponding to a part of the text of the original text. In this way, when there is a part of the text of the original text in the preset text, the similarity between the target feature information and the preset feature information corresponding to the part of the text will be relatively high, so as to achieve the purpose of deduplication, that is, the part of the text of the original text can be screened out, thus improving the accuracy of the contrast learning model and further improving the accuracy of text recommendation.

[0099] Further, on the basis of any of the above embodiments, the negative example sample of the original sample text can be obtained in the following manner: when there is no target negative example sample of the second original sample text in the sample text set, based on the other original sample texts except the second original sample text among the multiple original sample texts, determine the negative example sample of the second original sample text, and the second original sample text is one of at least one original sample text.

[0100] Specifically, if the similarity between the second original sample text and all the sample texts in the preset sample text set is greater than the first preset value, that is, they are all target positive example samples of the second original sample text, it means that there is no target negative example sample of the second original sample text. At this time, negative example samples need to be supplemented. By supplementing negative example samples, the contrast learning model will not have the problem of inaccurate model training due to the absence of negative example samples of the second original sample text in the sample text set, achieving the purpose of improving the accuracy of model training and further improving the accuracy of text recommendation.

[0101] When supplementing negative example samples, in one possible implementation, the similarity between the second original sample text and each other original sample text can be determined respectively; and based on the similarity, negative example samples can be determined from the other original sample texts.

[0102] Specifically, the similarity between the second original sample text and each other original sample text can be calculated first, and then the other original sample texts can be sorted according to the similarity. The first preset number of other original sample texts can be determined as the negative example samples of the second original sample text in ascending order of similarity, or the other original sample texts with similarity less than the third preset value can be determined as the negative example samples of the second original sample text. Since negative example samples are determined from other original sample texts based on the similarity, the accuracy of the determined negative example samples can be improved, and thus the accuracy of the contrastive learning model training can be improved.

[0103] In another possible implementation, the target original sample text can be randomly selected from the other original sample texts, and the target original sample text can be determined as the negative example sample of the second original sample text.

[0104] Specifically, at least one target original sample text can be randomly selected from the other original sample texts as the negative example sample of the second original sample text through a random algorithm, thereby improving the efficiency of negative example sample determination. For example, when h1, h2,..., h n are the original sample texts and h1 is the second original sample text, if the target negative example sample cannot be found in its preset sample text set, any one of the remaining h2,..., h n can be used as the negative example sample of h1. Similarly, when any one of h2,..., h n is used as the second original sample text, its negative example sample can also be constructed in the above manner.

[0105] Furthermore, when the number of the first triples is at least two, when training the contrastive learning model, the original sample texts included in each of the first triples in each batch of sample texts are different. Similarly, when the number of the second triples is at least two, the original sample texts included in each of the second triples in each batch of sample texts can also be different.

[0106] Specifically, after obtaining multiple first triples and second triples, when training the contrastive learning model, the multiple first triples and second triples can be shuffled and sorted to avoid the problem of having the same original samples among different triples within the same batch of samples in the subsequent training process. This can avoid the problem that the training error of the contrastive learning model decreases slowly or even the inference is poor due to the existence of similar sample texts in the training data, and improve the speed and accuracy of training the contrastive learning model.

[0107] Next, the implementation process of the above text recommendation method will be specifically described by analyzing and comparing the parameter indicators of different parameters and methods in combination with specific experimental results.

[0108] Specifically, for example, training data with a positive-negative example ratio of approximately 2:3 can be used to train the contrastive learning model. Suppose there are 950 groups of original training data. Among them, there are 102 groups of first original sample texts without target positive example samples, and 28 groups of second original sample texts without target negative example samples. According to the methods described in the foregoing embodiments, positive example samples and negative example samples can be supplemented. For example, the original 950 groups of training data can be expanded to 22969 groups of training data, and the expanded training data can be shuffled, sorted, etc. to avoid the existence of the same text among different triples within the same batch of sample texts in the subsequent training process.

[0109] Among them, when performing supervised training on the contrastive learning model, the parameter of the maximum text length of the contrastive learning model also needs to be set according to the actual sample text length. Specifically, Figure 4 is a schematic diagram of the length distribution of the sample text provided by the embodiment of the present invention. As Figure 4 shown, the lengths of most sample texts do not exceed 512, which is also the default maximum sentence length of the BERT model. Therefore, in the embodiment of the present invention, there is no need to change the parameter of the maximum text length in the contrastive learning model.

[0110] In addition, to avoid the problem that the test results of the trained contrastive learning model are not very accurate due to the existence of the same text in the training set and the test set, in the embodiments of the present invention, before testing the contrastive learning model, it is also necessary to remove duplicates in the test set relative to the training set. Here, an example is given where the training set has 50 original sample texts, the sample text set has 12 preset texts, and the test set has 253 test texts. First, it is necessary to calculate the similarity between each original sample text and the 12 preset texts, so as to select a suitable similarity threshold P according to parameter indicators such as recall rate and precision rate on these 50 * 12 text pairs. Then, the Glove model is used to calculate the similarity of all texts in the training set and the test set (the texts in the training set and the test set are marked with different IDs), and these similarities are sorted by size. In this way, N texts with the same similarity but different IDs can be obtained, that is, there are N texts in the test set that are repeated with those in the training set. Suppose there are a total of 11 repeated test texts, that is, there are actually only 242 valid test samples. Specifically, here, these 11 repeated texts can be removed from the test set through this same similarity, so as to ensure the validity of the test set.

[0111] Further, if the model calculates the similarity between each original sample text and 5 preset texts during testing. Then, according to the parameter P selected on the training set, the recall rate, precision rate and other parameters are determined on these 242 * 5 texts, so as to obtain the following four prediction results:

[0112] TP = #{similarity > P, Label is positive}, TN = #{similarity ≤ P, Label is negative}

[0113] FP = #{similarity > P, Label is negative}, FN = #{similarity ≤ P, Label is positive}

[0114] Among them, TP represents the number of positive example samples predicted as positive by the model, that is, true positive examples; TN represents the number of negative example samples predicted as negative by the model, that is, true negative examples; FP represents the number of negative example samples predicted as positive by the model, that is, false positive examples; FN represents the number of positive example samples predicted as negative by the model, that is, false negative examples.

[0115] Specifically, the above four prediction results can be substituted into the following formulas (1)-(3) to obtain the following three parameter indicators of the model:

[0116] precision = TP / (TP + FP) (1)

[0117] recall = TP / (TP + FN) (2)

[0118] F1 = 2 * (precision * recall) / (precision + recall) (3)

[0119] Among them, precision represents the precision rate of the model, recall represents the recall rate of the model, and F1 represents the comprehensive evaluation index, that is, the harmonic mean of the precision rate and the recall rate.

[0120] Similarly, the various parameter indicators of the model under different parameters can be calculated through the above formulas (1)-(3) to compare and obtain the influence of the parameter on the effect of the model. Exemplarily, the threshold P on the training set and the test set can be set from 0.000 to 1.000, and thus the precision rate, recall rate, and various parameter indicators of the comprehensive average index of each threshold P can be calculated, where the threshold P can be understood as the recall threshold in the foregoing embodiments. In addition, in order to ensure a fair comparison between the models under different parameters, the same precision rate also needs to be set on the training set, that is, by sampling their respective P values to compare the various parameter indicators of the models on the test set according to the same P value. Specifically, taking the multiple precision rates (0.6, 0.7, 0.8, 0.9) set on the training set as the division criteria, record the various parameter indicators of each model with different parameters under the same precision rate. The specific data is shown in Table 1 below:

[0121] Table 1

[0122]

[0123] Among them, SimCSE represents the SimCSE model with a batch size of 512; SimCSE8 represents the SimCSE model with a batch size of 8; SimCSE16 represents the SimCSE model with a batch size of 16.

[0124] In order to more concisely and intuitively compare the experimental data in Table 1, the above experimental data can be converted into a graphical form. Specifically, Figure 5 is the precision rate comparison graph of the models under different parameters in the embodiment of the present invention, Figure 6 is the recall rate comparison graph of the models under different parameters in the embodiment of the present invention. As Figure 5 shown, curve 51 represents the precision rate of the Glove model, curve 52 represents the precision rate of the SimCSE model, curve 53 represents the precision rate of SimCSE8, and curve 54 represents the precision rate of SimCSE16. As Figure 6 shown, curve 61 represents the recall rate of the Glove model, curve 62 represents the recall rate of the SimCSE model, curve 63 represents the recall rate of SimCSE8, and curve 64 represents the recall rate of SimCSE16.

[0125] It should be understood that in different demand scenarios, the parameter indicators emphasized by the contrast learning model are also inconsistent. For example, in the recommendation scenario of similar discourse type questions, in order for the contrast learning model to find questions with similar semantics to the original question from the question bank, the requirement for precision is usually much higher than the recall rate. Therefore, the contrast learning model should focus more on precision.

[0126] From Figure 5 it can be seen that compared with the Glove model, the similarity distribution of SimCSE8 (the model after fine-tuning (FT)) is more dispersed, which is more conducive to setting the recall threshold to achieve the effect of distinguishing positive and negative examples. And from Figure 6 it can be seen that under the same recall rate, the precision of SimCSE8 is much higher than that of the Glove model. Thus, it can be seen that SimCSE8 has improved the quality of the recalled text.

[0127] Furthermore, it is also necessary to analyze the test text situation of FPs in the test set to better improve the quality of the recalled text. For the convenience of searching, in the embodiments of the present invention, negative examples with common high scores of SimCSE, SimCSE8, and SimCSE16 are preferentially selected for analysis. Here, only 6 combinations with relatively high scores on SimCSE8 are shown, as shown in Table 2 below.

[0128] Table 2

[0129]

[0130]

[0131] It can be seen from Table 2 that most of the text pairs are of obviously the same theme and should be marked as accepted or repeated. Thus, it can be known that the annotation quality of the test set itself will affect the evaluation of the effect of the contrast learning model. Therefore, the same operation also needs to be performed on the training set in the follow-up, that is, the labels of some high-score FPs are evaluated and corrected. Among them, the specific correction method is similar to the self-iteration scheme of the machine. The FPs of the model are handed over to the resource department for re-annotation from low to high, so as to reduce the workload of the resource department and increase the re-annotation efficiency. In addition, for low-score FPs (0.825 - 0.860), after analysis and verification, most of the texts should also be accepted. Thus, it can be seen that the labels of these data are also too strict.

[0132] In addition to the above-mentioned accepted and unaccepted tags, there are also duplicate tags as mentioned above, that is, the two texts are considered too similar to be recommended. Therefore, the positive example samples in the training set and the test set need to be divided into two categories according to the duplicate tag repeat, namely accept and repeat. Since the sample text already contains two tags, here only need to distinguish repeat from accept.

[0133] Specifically, Figure 7 is a schematic diagram of the similarity distribution of different tags in the embodiment of the present invention. Figure 8 is a schematic diagram of the similarity distribution of different tags in the prior art. Among them, Figure 7 in the curve 71 represents the tag reject, the curve 72 represents the tag accept, and the curve 73 represents the tag repeat; Figure 8 in the curve 81 represents the tag reject, the curve 82 represents the tag accept, and the curve 83 represents the tag repeat.

[0134] From Figure 7 it can be seen that the overlapping regions of the similarity distributions of the tags reject, accept, and repeat are relatively small. Therefore, SimCSE8 can not only separate accept from reject, but also separate the high-score similarity repeat from accept, that is, the expected effect is achieved. On the contrary, looking at the similarity distributions of the three tags of Glove in Figure 8 the overlapping regions are relatively large. Therefore, this model can neither separate repeat from accept nor separate accept from reject well.

[0135] Furthermore, for the situation that SimCSE8 cannot separate the low-score similarity repeat from accept, it can be specifically divided into three categories: (1) The themes are relatively similar; (2) Obvious annotation errors; (3) The text content is large wrapping small, where large wrapping small can be understood as the recommended similar text is part of the original text or part of the fragment. 17

[0136] Therefore, here, through data augmentation, a positive example sample can be added to each original sample text to detect the sensitivity of SimCSE8 to the large-wrapping-small text pair. Specifically, here, randomly select from the beginning or the end and delete 1 / 3 of the article (in units of sentences), so that the number of the training set can be increased from 22969 to 28688. Thus, a new model SimCSE8c can be trained.

[0137] Furthermore, in order to improve the discrimination effect of SimCSE8 on text labels, the momentum contrast learning method proposed by MoCo for increasing negative example samples, i.e., the MoCo method, is applied to SimCSE8. Specifically, the momentum contrast learning method can be understood as for each batch of sample texts, in addition to the other sample texts in this batch of sample texts, a dictionary queue queue is constructed as an additional negative example. In the loss function, the word vectors embedding in the dictionary queue queue do not propagate gradients. The update in the dictionary queue queue is completed in two steps after each batch ends. One is to add all the word vectors embedding in this batch of sample texts to the dictionary queue queue and delete the oldest word vector embedding. The other is that for the part of the word vectors embedding retained in the dictionary queue queue, they can be slowly updated through the following formula (4) to obtain the updated word vector θ m+1 。

[0138] θ m+1 =λθ m +(1 - λ)θ e (4)

[0139] where θ m+1 represents the word vector between θ m and θ e , θ m represents the word vector embedding before update in the dictionary queue queue, θ e is the other word vector embedding in the dictionary queue queue, and λ represents the update parameter.

[0140] Exemplarily, for the above momentum contrast learning method, it can be described from the following execution perspective. First, a dynamic database is constructed during training, so that the input data of the dynamic database is continuously updated during training. The update method is specifically to delete the oldest 8 data and supplement the latest 8 data. Then, through the above formula (4), each data output by the dynamic database is continuously updated, but it is necessary to ensure that its update does not backpropagate gradients. During this process, the dynamic database needs to participate in the calculation of the loss function in each training step. Thus, this method can increase the number of negative example samples without increasing the GPU capacity pressure, and therefore can make the training of the model more accurate.

[0141] Specifically, here SimCSE8 that applies the momentum contrast learning method but the other parameters are the same as SimCSE8 is called SimCSE8_MoCo. Exemplarily, Figure 9It is a schematic diagram showing the distribution of the parameter indicators of SimCSE8_MoCo in the embodiments of the present invention. Among them, Figure 9 in the figure, curve 91 represents the precision rate of SimCSE8_MoCo, curve 92 represents the recall rate of SimCSE8_MoCo, and curve 93 represents the harmonic mean of the precision rate and the recall rate of SimCSE8_MoCo. As Figure 9 shown, the similarity distribution of SimCSE8_MoCo is too concentrated around 0.9. For example, when P increases from 0.937 to 0.938, its corresponding precision rate increases from 0.8 to 0.9. Therefore, it is difficult for SimCSE8_MoCo to find a suitable P to determine the value of the precision rate.

[0142] In order to better compare the effects of SimCSE8_MoCo and SimCSE8, the parameter indicators of the two can be placed in the same figure for representation. Specifically, Figure 10 It is a comparison diagram of the parameter indicators of SimCSE8 and SimCSE8_MoCo in the embodiments of the present invention. Among them, Figure 10 in the figure, curve 101 represents the precision rate of SimCSE8, and curve 102 represents the precision rate of SimCSE8_MoCo.

[0143] As Figure 10 shown, at the same threshold p of precision rate = 0.8, although both models can ensure a precision rate above 0.8, the recall rate of SimCSE8_MoCo, 0.1487, is much lower than the recall rate of SimCSE8, 0.2425. Thus, it can be seen that the application of the momentum contrast learning method not only does not improve the model effect, but also reduces the recall rate of the model.

[0144] It is easy to think from this that the method of using other samples in the same batch of sample texts as negative example samples may generate pseudo-negative examples, that is, different samples within the same batch of samples may also have similar semantics. Therefore, in order to reduce the influence brought by such samples, when training, the model will refer to the similarity score of another model and compare it with the threshold Φ to determine whether there are pseudo-negative examples.

[0145] Specifically, when training the SimCSE model on the training set, an auxiliary SimCSE model that does not participate in the update can also be used to determine whether pseudo-negative examples are generated. Among them, when calculating the contrast loss function, it is necessary to use the auxiliary model SimCSE to calculate the similarity of each text to be compared, so that when its similarity is higher than the pre-set threshold Φ, this text will not be sent into the loss function as a negative example sample.

[0146] Exemplarily, Table 3 shows the various parameter indicators of the model at different thresholds Φ. Here, the precision of the fixed development set can be used to compare the indicators of three groups of models with relatively large differences, namely SimCSE8, SimCSE8-0.75, and SimCSE8-0.65, on the test set. Among them, the threshold Φ of SimCSE8 is 1.0, the threshold Φ of SimCSE8-0.75 is 0.75, and the threshold Φ of SimCSE8-0.65 is 0.65.

[0147] Table 3

[0148]

[0149] As can be seen from Table 3, when the weight threshold of the introduced negative example samples is Φ = 0.75, the various parameter indicators of the model have improved but the effect is not obvious. This shows that the existence of pseudo-negative examples will have a slight impact on the training of the model. However, there is another possible influencing factor, that is, the training set has been sorted before the model training, reducing the possibility of the same text appearing in the same batch of sample texts. Therefore, the effect after removing the pseudo-negative examples is weakened a lot.

[0150] In addition, in order to improve the applicability of the SimCSE8 model, it is necessary to compress the BERT structure used by this model. Exemplarily, a relatively cost-effective method of distilling BERT can be adopted here, that is, the TinyBERT distillation method. The error function of this method consists of the mean square error of the embedding layer, the attention of the hidden layer, and the embedding. Among them, through the comparison of experimental results, it is found that distilling all the hidden layers has a better effect than only distilling the embedding of [CLS]. Usually, after distilling the BERT structure for about 20 epochs (an epoch means traversing all samples in the training set once), supervised training can be carried out on the fine-labeled training set to obtain the compressed SimCSE8 model. For the convenience of description, it is hereinafter referred to as the SimSmall model. Among them, the parameter initialization of this model consists of the parameters of the first six layers of BERT and the Pooler layer of SimCSE8.

[0151] In order to verify the effect of the SimSmall model, the various parameter indicators of it on the development set and the test set will be specifically compared and analyzed below.

[0152] Specifically, Figure 11 This is one of the distribution diagrams of the parameter indicators of SimSmall in the development set in the embodiments of the present invention. Figure 12 This is one of the distribution diagrams of the parameter indicators of SimSmall in the test set in the embodiments of the present invention. Figure 13This is the second schematic diagram of the parameter indicators of SimSmall on the development set in the embodiments of the present invention. Figure 14 This is the second schematic diagram of the parameter indicators of SimSmall on the test set in the embodiments of the present invention. Among them, Figure 11 In it, curve 111 represents the precision of SimSmall, curve 112 represents the recall rate of SimSmall, and curve 113 represents the comprehensive evaluation index of SimSmall; Figure 12 In it, curve 121 represents the precision of SimSmall, curve 122 represents the recall rate of SimSmall, and curve 123 represents the comprehensive evaluation index of SimSmall; Figure 14 In it, curve 131 represents the parameter indicators of SimSmall, and curve 132 represents the parameter indicators of SimCSE8; Figure 14 In it, curve 141 represents the parameter indicators of SimSmall, and curve 142 represents the parameter indicators of SimCSE8.

[0153] From Figures 11 - 14 it can be seen that when the model has a high precision, the judgment effect on individual negative example text pairs is poor. However, through statistical analysis, it is found that judging these text pairs as positive examples is not unacceptable. In addition, the performance of the model SimSmall on the test set is even better than that of SimCSE8, which indicates that this distillation experiment is relatively successful.

[0154] In order to further verify the effect of the SimSmall model, the similarity distribution and rejection rate curve of the SimSmall model for random distribution are also analyzed below. Specifically, Figure 15 This is the schematic diagram of the similarity distribution of the SimSmall model for random distribution in the embodiments of the present invention, Figure 16 This is the schematic diagram of the rejection rate distribution of the SimSmall model for random distribution in the embodiments of the present invention.

[0155] From Figures 15 - 16 it can be seen that on the test set, the threshold P when the precision reaches 0.8 is 0.867 (the curve on the development set is poor), and the rejection rate is 99.83%. Therefore, the model SimSmall will not recommend randomly irrelevant texts.

[0156] On this basis, in order to obtain the recall threshold P of the SimSmall model that is higher than the Glove model in terms of precision and recall rate indicators, the precision and recall rate indicators of the above two models in each test set are statistically counted in Table 4 below according to the recall thresholds of the Glove model being 0.9, 0.95, and 0.99 respectively.

[0157] Table 4

[0158]

[0159]

[0160] Among them, the ones marked as "meaningless" in the table indicate that the parameter index gap between Glove and SimSmall is too large and has no reference significance. And the three numbers in the table respectively represent the recall threshold P, precision, and recall rate of SimSmall. In addition, the first two rows in Table 4 only serve as a reference for important nodes, and the calculation of the recall threshold here is determined by the recall threshold of SimSmall for the online dataset corresponding to Glove.

[0161] As can be seen from the foregoing description, on any dataset, when the precision of Glove and SimSmall is the same, the recall rate of SimSmall is always greater than that of Glove, and vice versa. Therefore, when the recall threshold of Glove is 0.9, on the "online_dev" dataset, the two thresholds 0.592 and 0.652 of SimSmall correspond to the same precision and the same recall rate of Glove. In other words, the precision and recall rate of SimSmall at any recall threshold between 0.592 and 0.652 are both greater than the precision and recall rate of Glove when the recall threshold is 0.9. Therefore, only the average calculation of these threshold nodes is required, but for those thresholds whose mean exceeds the interval, they can be used as the lower bound of the interval to ensure that the recall rate is not lower than that of Glove.

[0162] Through the above calculation method, the following three recall thresholds of SimSmall can be obtained: 0.652, 0.834, and 0.978, which respectively correspond to 0.9, 0.95, and 0.99 of Glove. The precision / recall rate of these thresholds on the fine-label test set is: 0.4793 / 0.9628, 0.6921 / 0.5611, 1.0 / 0.0106. On this basis, the rejection rate of random matching for the online dataset is calculated to obtain the rejection rates of the two models Glove and SimSmall at the two groups of recall thresholds, namely 85.33%, 99.12%, 99.96% and 95.82%, 99.68%, 99.96%. Among them, the recall rate of SimSmall on the four datasets is higher than that of Glove. Therefore, the increase in this rejection rate also shows the extremely high accuracy of SimSmall. It can be seen that compared with the traditional static model, the text recommendation method provided by the embodiment of the present invention has obvious improvements in terms of recall quantity, recall quality, threshold division, rejection rate and other indicators, that is, it has great superiority.

[0163] The text recommendation device provided by the embodiments of the present invention will be described below. The text recommendation device described below can be correspondingly referred to the text recommendation method described above.

[0164] Figure 17 It is a schematic structural diagram of the text recommendation device provided by the embodiments of the present invention. As Figure 17 shown, the device includes:

[0165] An acquisition module 1701, configured to acquire the original text;

[0166] An input module 1702, configured to input the original text into the contrast learning model to obtain the target feature information of the original text. The contrast learning model is obtained by training the initial contrast learning model based on multiple batches of sample texts. Each batch of sample texts includes at least one first triple and at least one second triple. The first triple includes the original sample text, the first positive example sample of the original sample text, and the negative example sample. The second triple includes the original sample text, the second positive example sample of the original sample text, and the negative example sample. The first positive example sample includes texts with a similarity greater than a preset value to the original sample text. The second positive example sample includes partial texts of the original sample text;

[0167] A determination module 1703, configured to determine the similarity between the target feature information and the preset feature information corresponding to multiple preset texts, and determine the target preset feature information corresponding to the similarity between the recall threshold and the deduplication threshold, where the recall threshold is less than the deduplication threshold;

[0168] The determination module 1703 is further configured to determine the preset text corresponding to the target preset feature information as the target similar text.

[0169] The text recommendation device provided by the embodiment of the present invention inputs the original text obtained by the acquisition module 1701 into the contrastive learning model through the input module 1702 to obtain the target feature information of the original text, determines the similarity between the target feature information and the preset feature information corresponding to multiple preset texts, and determines the target preset feature information corresponding to the similarity between the recall threshold and the deduplication threshold, so as to determine the preset text corresponding to the target preset feature information as the target similar text. Among them, each batch of sample texts for training the contrastive learning model includes at least one first triple and at least one second triple. The first triple includes the original sample text, the first positive example sample of the original sample text, and the negative example sample. The second triple includes the original sample text, the second positive example sample of the original sample text, and the negative example sample. The first positive example sample includes a text with a similarity greater than a preset value to the original sample text. The second positive example sample includes a partial text of the original sample text. Since the second triple containing the second positive example sample is added to each batch of sample texts, the target feature information of the original text extracted by the trained contrastive learning model can match or be closer to the feature information corresponding to the partial text of the original text. In this way, when there is a partial text of the original text in the preset text, the similarity between the target feature information and the preset feature information corresponding to the partial text will be relatively high. Thus, based on the deduplication threshold, the partial text of the original text can be screened out, and based on the recall threshold, the determination module 1703 can determine the target similar text similar to the original text, thereby improving the accuracy of similar text recommendation.

[0170] Optionally, the acquisition module 1701 is further configured to acquire at least one original sample text;

[0171] The determination module 1703 is further configured to, when there is no target positive example sample of the first original sample text in the sample text set, determine the first original sample text as the first positive example sample of the first original sample text, where the first original sample text is one of the at least one original sample text;

[0172] The determination module 1703 is further configured to determine a first triple based on the first original sample text, the first positive example sample of the first original sample text, and the negative example sample of the first original sample text.

[0173] Optionally, the determination module 1703 is further configured to, for each original sample text, determine a partial text in the original sample text as the second positive example sample of the original sample text;

[0174] The determination module 1703 is further configured to determine a second triple based on the original sample text, the second positive example sample of the original sample text, and the negative example sample of the original sample text.

[0175] Optionally, when there is no target negative sample of the second original sample text in the sample text set, the determining module 1703 is further configured to determine a negative sample of the second original sample text based on other original sample texts except the second original sample text among the multiple original sample texts, where the second original sample text is one of at least one original sample text.

[0176] In a possible implementation, the determining module 1703 is specifically configured to:

[0177] Determine the similarity between the second original sample text and each other original sample text respectively;

[0178] Based on the similarity, determine negative samples from other original sample texts.

[0179] In another possible implementation, the determining module 1703 is specifically configured to:

[0180] Randomly select a target original sample text from other original sample texts, and determine the target original sample text as a negative sample of the second original sample text.

[0181] Optionally, the number of the first triples is at least two, and the original sample texts included in each first triple are different.

[0182] The device in this embodiment can be used to execute the method in any one of the method embodiments on the text recommendation device side. The specific implementation process and technical effects are similar to those in the method embodiments on the text recommendation device side. For details, reference can be made to the detailed introduction in the method embodiments on the text recommendation device side above, and details are not described herein again.

[0183] Figure 18 An example of a schematic diagram of the physical structure of an electronic device is shown in Figure 18As shown, the electronic device may include: a processor 1801, a communications interface 1802, a memory 1803, and a communication bus 1804. Among them, the processor 1801, the communications interface 1802, and the memory 1803 communicate with each other through the communication bus 1804. The processor 1801 may call logic instructions in the memory 1803 to execute a text recommendation method, including: obtaining an original text; inputting the original text into a contrastive learning model to obtain target feature information of the original text. The contrastive learning model is obtained by training an initial contrastive learning model based on multiple batches of sample texts. Each batch of sample texts includes at least one first triple and at least one second triple. The first triple includes an original sample text, a first positive example sample of the original sample text, and a negative example sample. The second triple includes an original sample text, a second positive example sample of the original sample text, and a negative example sample. The first positive example sample includes a text with a similarity greater than a preset value to the original sample text. The second positive example sample includes a partial text of the original sample text; determining the similarity between the target feature information and preset feature information corresponding to multiple preset texts, and determining the target preset feature information corresponding to the similarity between the recall threshold and the deduplication threshold, where the recall threshold is less than the deduplication threshold; determining the preset text corresponding to the target preset feature information as the target similar text.

[0184] In addition, when the logic instructions in the above-mentioned memory 1803 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0185] Figure 19 Illustrates a second schematic diagram of the physical structure of an electronic device, such as Figure 19As shown in the figure, the electronic device may include: a processor 1901, a communications interface 1902, a memory 1903, and a communication bus 1904. It also includes a display 1905. Among them, the processor 1901, the communications interface 1902, the memory 1903, and the display 1905 complete mutual communication through the communication bus 1904. The display 1905 is used to display the target similar text. The processor 1901 can call the logical instructions in the memory 1903 to execute a text recommendation method, including: obtaining an original text; inputting the original text into a contrast learning model to obtain the target feature information of the original text. The contrast learning model is obtained by training an initial contrast learning model based on multiple batches of sample texts. Each batch of sample texts includes at least one first triple and at least one second triple. The first triple includes an original sample text, a first positive example sample of the original sample text, and a negative example sample. The second triple includes an original sample text, a second positive example sample of the original sample text, and a negative example sample. The first positive example sample includes a text with a similarity greater than a preset value to the original sample text. The second positive example sample includes a partial text of the original sample text; determining the similarity between the target feature information and the preset feature information corresponding to multiple preset texts, and determining the target preset feature information corresponding to the similarity between the recall threshold and the deduplication threshold. The recall threshold is less than the deduplication threshold; determining the preset text corresponding to the target preset feature information as the target similar text.

[0186] In addition, when the logical instructions in the above-mentioned memory 1903 can be implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0187] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the text recommendation method provided by each of the above methods, including: obtaining an original text; inputting the original text into a contrastive learning model to obtain target feature information of the original text. The contrastive learning model is obtained by training an initial contrastive learning model based on multiple batches of sample texts. Each batch of sample texts includes at least one first triple and at least one second triple. The first triple includes an original sample text, a first positive example sample of the original sample text, and a negative example sample. The second triple includes the original sample text, a second positive example sample of the original sample text, and a negative example sample. The first positive example sample includes a text with a similarity greater than a preset value to the original sample text. The second positive example sample includes a partial text of the original sample text; determining the similarity between the target feature information and the preset feature information corresponding to multiple preset texts, and determining the target preset feature information corresponding to the similarity between the recall threshold and the deduplication threshold, where the recall threshold is less than the deduplication threshold; determining the preset text corresponding to the target preset feature information as the target similar text.

[0188] On another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the text recommendation method provided by each of the above methods, including: obtaining an original text; inputting the original text into a contrastive learning model to obtain target feature information of the original text. The contrastive learning model is obtained by training an initial contrastive learning model based on multiple batches of sample texts. Each batch of sample texts includes at least one first triple and at least one second triple. The first triple includes an original sample text, a first positive example sample of the original sample text, and a negative example sample. The second triple includes the original sample text, a second positive example sample of the original sample text, and a negative example sample. The first positive example sample includes a text with a similarity greater than a preset value to the original sample text. The second positive example sample includes a partial text of the original sample text; determining the similarity between the target feature information and the preset feature information corresponding to multiple preset texts, and determining the target preset feature information corresponding to the similarity between the recall threshold and the deduplication threshold, where the recall threshold is less than the deduplication threshold; determining the preset text corresponding to the target preset feature information as the target similar text.

[0189] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative work.

[0190] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0191] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or equivalently replace some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A text recommendation method, characterized in that, Including: Obtain the original text; Input the original text into a contrastive learning model to obtain the target feature information of the original text. The contrastive learning model is obtained by training an initial contrastive learning model based on multiple batches of sample texts. Each batch of sample texts includes at least one first triple and at least one second triple. The first triple includes an original sample text, a first positive example sample of the original sample text, and a negative example sample. The second triple includes the original sample text, a second positive example sample of the original sample text, and the negative example sample. The first positive example sample includes a text with a similarity greater than a preset value to the original sample text. The second positive example sample includes a partial text of the original sample text; Determine the similarity between the target feature information and the preset feature information corresponding to multiple preset texts, and determine the target preset feature information corresponding to the similarity between the recall threshold and the deduplication threshold. The recall threshold is less than the deduplication threshold; Determine the preset text corresponding to the target preset feature information as the target similar text.

2. The text recommendation method according to claim 1, wherein At least one first triple included in each batch of sample texts is obtained based on the following method: Obtain at least one original sample text; In the case where there is no target positive example sample of the first original sample text in the sample text set, determine the first original sample text as the first positive example sample of the first original sample text. The first original sample text is one of the at least one original sample texts; Based on the first original sample text, the first positive example sample of the first original sample text, and the negative example sample of the first original sample text, determine the first triple.

3. The text recommendation method according to claim 1, characterized in that At least one second triple included in each batch of sample texts is obtained based on the following method: For each original sample text, determine a partial text of the original sample text as the second positive example sample of the original sample text; Based on the original sample text, the second positive example sample of the original sample text, and the negative example sample of the original sample text, determine the second triple.

4. The text recommendation method according to any one of claims 1-3, characterized in that The negative example sample of the original sample text is obtained based on the following method: In the case where there is no target negative example sample of the second original sample text in the sample text set, based on the other original sample texts except the second original sample text among the multiple original sample texts, determine the negative example sample of the second original sample text. The second original sample text is one of the at least one original sample texts.

5. The text recommendation method according to claim 4, wherein The determining the negative example sample of the second original sample text based on the other original sample texts except the second original sample text among the multiple original sample texts includes: Respectively determine the similarity between the second original sample text and each other original sample text; Based on the similarity, determine the negative example sample from the other original sample texts.

6. The text recommendation method according to claim 4, wherein The determining the negative example sample of the second original sample text based on the other original sample texts except the second original sample text among the multiple original sample texts includes: Randomly select a target original sample text from the other original sample texts, and determine the target original sample text as a negative sample of the second original sample text.

7. A text recommendation device, characterized in that, Including: An acquisition module, configured to acquire original texts; An input module, configured to input the original texts into a contrastive learning model to obtain target feature information of the original texts, where the contrastive learning model is obtained by training an initial contrastive learning model based on multiple batches of sample texts, and each batch of sample texts includes at least one first triple and at least one second triple. The first triple includes an original sample text, a first positive sample of the original sample text, and a negative sample. The second triple includes the original sample text, a second positive sample of the original sample text, and the negative sample. The first positive sample includes texts with a similarity greater than a preset value to the original sample text, and the second positive sample includes partial texts of the original sample text; A determination module, configured to determine the similarity between the target feature information and preset feature information corresponding to multiple preset texts, and determine the target preset feature information corresponding to the similarity between the recall threshold and the deduplication threshold, where the recall threshold is less than the deduplication threshold; The determination module is further configured to determine the preset text corresponding to the target preset feature information as a target similar text.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the text recommendation method according to any one of claims 1 to 6.

9. An electronic device, comprising a display, a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the text recommendation method according to any one of claims 1 to 6; The display is configured to display the target similar text.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the text recommendation method according to any one of claims 1 to 6.

Images