Text matching training method and related equipment based on reinforcement learning

Abstract

The present application relates to the field of artificial intelligence, and discloses a text matching training method based on reinforcement learning and related equipment, the method comprising: obtaining sample data, dividing the sample data into first sample data and second sample data; performing word segmentation on the first sample data and the second sample data to obtain a first latent vector and a second latent vector; inputting the first latent vector and the second latent vector into a preset strategy network for calculation, respectively, to obtain a first word vector and a second word vector; inputting the first word vector and the second word vector into a preset text matching model at the same time, obtaining text accuracy, adjusting the strategy network parameters according to the text accuracy, and training the next set of sample data until the text accuracy is greater than the preset text accuracy threshold. By implementing text matching training through the present application, irrelevant information can be screened for long texts and texts in speech conversion, thereby improving the efficiency of text matching.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a text matching training method based on reinforcement learning and related equipment. Background Art

[0002] Text matching is one of the important tasks in NLP (Neuro-Linguistic Programming) and is widely used in search and dialogue systems. The current text matching algorithms mainly use two types of models. One is a model based on sentence expression; the second is a sentence interaction model. In the sentence expression model, a twin network is generally used to represent two sentences as vectors, and then the distance between the two vectors is calculated to determine whether the sentences are similar. In the interaction model, the similarity between words / phrases in the sentence is calculated separately, and then the sentence is judged by the similarity features between words and phrases. In order to train these two types of models, it is generally necessary to mark the corpus. Specifically, it is to collect some similar sentences and dissimilar sentences. These two types of models have good results when processing short written texts.

[0003] However, in some scenarios, the input text may be relatively long and contain a lot of irrelevant information. For example, with the development of voice technology, many businesses now support voice input. Generally speaking, voice input is first converted into text through the ASR (Automatic Speech Recognition) module, and then the text is further processed. Compared with text input, the text generated by voice input is quite different: in voice input, the user's input generally contains a lot of colloquial words and irrelevant sentences. For this type of input, the current text similarity model is not very effective. The conventional solution is to first train a neural network model to remove colloquial words and irrelevant sentences from the input text. However, this method requires a lot of labeled data and is relatively costly. Summary of the invention

[0004] The purpose of this application is to address the deficiencies in the prior art and provide a text matching training method and related equipment based on reinforcement learning. The method segmentes two training samples into long sentences and short sentences, and inputs the vectors obtained after segmentation into a strategy network for calculation. The sentences and words in the samples are deleted and retained to obtain new vectors. The new vectors are then input into a text matching model to obtain the sample accuracy. The sample accuracy is fed back to the strategy network for parameter adjustment, and the next round of sample training is performed until the sample accuracy reaches the predetermined requirement. By implementing text matching training in this way, irrelevant information can be screened for long texts and texts in speech conversion, thereby improving the efficiency of text matching.

[0005] In order to achieve the above-mentioned purpose, the technical solution of the present application provides a text matching training method based on reinforcement learning and related equipment.

[0006] The present application discloses a text matching training method based on reinforcement learning, comprising the following steps:

[0007] Acquire sample data, divide the sample data into multiple groups, extract a first group of sample data, and divide the first group of sample data into first sample data and second sample data;

[0008] Performing word segmentation on the first sample data and the second sample data, and inputting the first sample data and the second sample data after word segmentation into a preset neural network respectively to obtain a first latent vector and a second latent vector;

[0009] Inputting the first latent vector and the second latent vector into a preset strategy network for calculation respectively to obtain a first word vector and a second word vector;

[0010] The first word vector and the second word vector are simultaneously input into a preset text matching model to obtain a text accuracy rate, and the text accuracy rate is compared with a preset text accuracy rate threshold. If the text accuracy rate is greater than the preset text accuracy rate threshold, the training is terminated; otherwise, the policy network parameters are adjusted according to the current text accuracy rate to obtain a new policy network, and the next set of sample data is trained on the new policy network until the text accuracy rate is greater than the preset text accuracy rate threshold.

[0011] Preferably, the word segmentation of the first sample data and the second sample data includes:

[0012] Separate the first sample data and the second sample data into sentences to obtain sentence data of the first sample data and sentence data of the second sample data;

[0013] The sentence data of the first sample data and the sentence data of the second sample data are respectively segmented.

[0014] Preferably, said respectively performing sentence segmentation on the first sample data and the second sample data to obtain sentence data of the first sample data and sentence data of the second sample data comprises:

[0015] Separate the first sample data and the second sample data into sentences according to preset separators to obtain sentence data of the first sample data and sentence data of the second sample data, wherein the separators include commas and semicolons;

[0016] The sentence data of the first sample data is associated with the first sample data, and the sentence data of the second sample data is associated with the second sample data.

[0017] Preferably, the performing word segmentation on the first sample data and the second sample data, respectively inputting the first sample data and the second sample data after word segmentation into a preset neural network, and obtaining the first latent vector and the second latent vector comprises:

[0018] Copying the first sample data and the second sample data to obtain a first sample data copy and a second sample data copy;

[0019] Performing word segmentation on the first sample data and the second sample data, and inputting the first sample data and the second sample data after word segmentation into a preset neural network respectively to obtain a latent vector of the first sample data and a latent vector of the second sample data;

[0020] Sentence the first sample data copy and the second sample data copy to obtain first sample data copy sentence data and second sample data copy sentence data, segment the first sample data copy sentence data and the second sample data copy sentence data, and input the segmented first sample data copy sentence data and the second sample data copy sentence data into a preset neural network, respectively, to obtain a first sample data copy latent vector and a second sample data copy latent vector;

[0021] A first latent vector and a second latent vector are obtained according to the first sample data latent vector, the second sample data latent vector, the first sample data copy latent vector and the second sample data copy latent vector.

[0022] Preferably, obtaining the first latent vector and the second latent vector according to the first sample data latent vector, the second sample data latent vector, the first sample data copy latent vector and the second sample data copy latent vector comprises:

[0023] Concatenate the first sample data latent vector with the first sample data copy latent vector to obtain a first latent vector;

[0024] The second sample data latent vector is concatenated with the second sample data copy latent vector to obtain a second latent vector.

[0025] Preferably, the step of simultaneously inputting the first word vector and the second word vector into a preset text matching model to obtain the text accuracy comprises:

[0026] Inputting the first word vector and the second word vector into a preset text matching model in sequence to obtain text similarity;

[0027] If the text similarity matches the preset similarity tag, the number of text accuracy times is accumulated;

[0028] The text accuracy is obtained according to the cumulative number of text accuracy times, the first word vector and the total number of the second word vector.

[0029] Preferably, adjusting the policy network parameters according to the accuracy of the current text to obtain a new policy network includes:

[0030] According to the formula Adjust the policy network parameters θ to obtain a new policy network, where N is the number of samples, t is the step count, π θ (a t |s t ) is the output of the policy network, a t is the action at time t, s t is the latent vector at time t, γ is the attenuation factor, and r is the text accuracy.

[0031] The present application also discloses a text matching training device based on reinforcement learning, the device comprising:

[0032] An acquisition module, used for acquiring sample data, dividing the sample data into multiple groups, extracting a first group of sample data, and dividing the first group of sample data into first sample data and second sample data;

[0033] A word segmentation module, used to perform word segmentation on the first sample data and the second sample data, and input the first sample data and the second sample data after word segmentation into a preset neural network respectively to obtain a first latent vector and a second latent vector;

[0034] A calculation module, used to input the first latent vector and the second latent vector into a preset strategy network for calculation to obtain a first word vector and a second word vector;

[0035] A training module is used to input the first word vector and the second word vector into a preset text matching model at the same time to obtain a text accuracy, compare the text accuracy with a preset text accuracy threshold, and terminate this training if the text accuracy is greater than the preset text accuracy threshold; otherwise, adjust the policy network parameters according to this text accuracy to obtain a new policy network, and perform training on the next set of sample data on the new policy network until the text accuracy is greater than the preset text accuracy threshold.

[0036] The present application also discloses a reinforcement learning-based text matching training device, which includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by one or more of the processors, the one or more processors execute the above-mentioned reinforcement learning-based text matching training method.

[0037] The present application also discloses a computer-readable storage medium, which can be read and written by a processor, and the storage medium stores computer instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the above-mentioned reinforcement learning-based text matching training method.

[0038] The beneficial effects of the present application are as follows: the present application performs word segmentation on long sentences and short sentences of two training samples, and inputs the vectors obtained after word segmentation into the strategy network for calculation, deletes and retains the sentences and words in the samples, obtains new vectors, and then inputs the new vectors into the text matching model to obtain the sample accuracy, and feeds the sample accuracy back to the strategy network for parameter adjustment, and conducts the next round of sample training until the sample accuracy reaches the predetermined requirements. By implementing text matching training in this way, irrelevant information can be screened for long texts and texts in speech conversion, thereby improving the efficiency of text matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A flowchart of a text matching training method based on reinforcement learning according to the first embodiment of the present application;

[0040] Figure 2 A flowchart of a text matching training method based on reinforcement learning according to the second embodiment of the present application;

[0041] Figure 3 A flowchart of a text matching training method based on reinforcement learning according to the third embodiment of the present application;

[0042] Figure 4 This is a flowchart of a text matching training method based on reinforcement learning according to the fourth embodiment of the present application;

[0043] Figure 5 This is a flowchart of a text matching training method based on reinforcement learning according to the fifth embodiment of the present application;

[0044] Figure 6 This is a flowchart of a text matching training method based on reinforcement learning according to the sixth embodiment of the present application;

[0045] Figure 7 This is a structural schematic diagram of a text matching training device based on reinforcement learning according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0048] The process of a text matching training method based on reinforcement learning in the first embodiment of the present application is as follows: Figure 1 As shown, this embodiment includes the following steps:

[0049] Step s101, acquiring sample data, dividing the sample data into multiple groups, extracting a first group of sample data, and dividing the first group of sample data into first sample data and second sample data;

[0050] Specifically, in this step, the sample data includes a set of long sentences, in which each two sentences form a pair of combinations, wherein the relationship between the two sentences in a pair of combinations includes similar or dissimilar labels, and the set of long sentences can be obtained from websites, microblogs, and forums. After the set of long sentences is obtained, each pair of combinations is labeled with similar or dissimilar labels. Then the set of long sentences is divided into several groups, each group of data is used for a round of training, so that the parameters of the neural network can be adjusted after each round of training. For example, assuming that there are currently 10,000 sample data, divided into 100 groups, then each group has 100 data, which means that 100 rounds of training can be performed, and in any group of sample data, each data contains a pair of sentences, that is, contains 2 sentences, some of which are similar, some are dissimilar, and are labeled, and the labels are used to verify the accuracy of this round of training.

[0051] Specifically, in this step, after the sample data is obtained and grouped, the first round of training begins. The first round of training uses the first group of data, which is arbitrarily extracted from the sample data. After the first group of data is obtained, each pair of combinations in the first group of data is divided into two parts, namely the first sample data and the second sample data, wherein a sentence in the first sample data and a sentence in the second sample data are a pair of combinations. For example, sentence A in the first sample data and sentence B in the second sample data are a pair of combinations. Therefore, when training, sentence A and sentence B need to be processed at the same time. The relationship between the first sample data and the second sample data is similar or dissimilar.

[0052] Step s102, performing word segmentation on the first sample data and the second sample data, and inputting the first sample data and the second sample data after word segmentation into a preset neural network respectively to obtain a first latent vector and a second latent vector;

[0053] Specifically, in this step, after obtaining the first sample data and the second sample data, the first sample data and the second sample data can be segmented respectively, and the segmentation can be performed through the jieba tool. After the first sample data and the second sample data are segmented, the segmentation of the first sample data and the segmentation of the second sample data can be obtained respectively; then the segmentation of the first sample data and the segmentation of the second sample data can be input into the neural network respectively, and the neural network can be an embedding network, and the embedding network can be composed of a Bi-LSTM (Bi-directional Long Short-Term Memory) or a transformer model to obtain a first latent vector and a second latent vector, wherein, in the process of generating the latent vector, when each segmentation is input into the neural network one by one in sequence, the currently generated latent vector will refer to the previous latent vector information, so the last latent vector is the latent vector of the entire sentence.

[0054] Step s103, inputting the first latent vector and the second latent vector into a preset strategy network for calculation, respectively, to obtain a first word vector and a second word vector;

[0055] Specifically, in this step, after obtaining the first latent vector and the second latent vector, the first latent vector and the second latent vector are input into the policy network for calculation. The policy network is a fully connected neural network with n layers and an activation function of Relu. Its network parameters are θ, and the activation function of the last layer is the SIGMOD function. After the policy network is calculated, it can be determined whether to retain or delete the sentence. In a specific application, the latent vector is calculated by the neural network and outputs 0 or 1. 0 means deletion and 1 means retention. If it is deleted, all words in the sentence are deleted; if the sentence is determined to be retained, all words in the sentence are retained. After the calculation of the policy network, the first word vector and the second word vector are obtained respectively.

[0056] Step s104, input the first word vector and the second word vector into the preset text matching model at the same time, obtain the text accuracy, compare the text accuracy with the preset text accuracy threshold, if the text accuracy is greater than the preset text accuracy threshold, then end the training; otherwise, adjust the policy network parameters according to the current text accuracy, obtain a new policy network, and train the next set of sample data on the new policy network until the text accuracy is greater than the preset text accuracy threshold.

[0057] Specifically, in this step, after obtaining the first word vector and the second word vector, the first word vector and the second word vector are simultaneously input into a text matching model, which is also composed of a neural network and is used to determine whether two texts are similar. The neural network can be a twin network or Bert (Bidirectional Encoder Representation from Transformers). After the text matching model is calculated, the similarity between the current first word vector and the second word vector is obtained, and the text accuracy of this round of training is calculated.

[0058] Specifically, in this step, a text accuracy threshold can also be preset, and the text accuracy threshold is used to determine whether to continue training, that is, whether the current model meets the requirements. For example, the preset text accuracy threshold is 90%. If the current text accuracy is 80%, the next round of data training is required, otherwise the training is terminated.

[0059] Specifically, in this step, if the next round of training is required, the next set of data can be obtained from the sample data, and the text accuracy can be fed back to the policy network so that the policy network can adjust its parameters. After adjusting the parameters of the policy network, a new policy network can be obtained, and the training of the next set of sample data can be continued. When training the next set of samples, that is, training the second set of sample data, a set of data is also randomly extracted from the sample data, but the second set of sample data is different from the first set of data, that is, a set of sample data will no longer be trained after training. Similarly, when the sample data is subsequently trained, the sample data trained later must be different from any of the previous sets of sample data. When training the second set of sample data, step s101 is also performed to divide the second set of sample data into The first sample data and the second sample data, and step s102, i.e., segmenting the first sample data and the second sample data in the second group of sample data, and inputting the preset neural network to obtain the first latent vector and the second latent vector respectively, and step s103, i.e., inputting the first latent vector and the second latent vector into the new strategy network to obtain the first word vector and the second word vector, and step s104, i.e., inputting the first word vector and the second word vector into the text matching network to obtain the text accuracy again. If the text accuracy of this round reaches the threshold, the training is terminated, otherwise the text accuracy is fed back to the strategy network for parameter adjustment, and the training of the next group of sample data is continued until the text accuracy reaches the threshold. The training of subsequent sample data is also carried out according to this step until the sample data is trained.

[0060] In this embodiment, two training samples are segmented into long sentences and short sentences, and the vectors obtained after segmentation are respectively input into the strategy network for calculation, the sentences and words in the samples are deleted and retained to obtain new vectors, and then the new vectors are input into the text matching model to obtain the sample accuracy, and the sample accuracy is fed back to the strategy network for parameter adjustment, and the next round of sample training is carried out until the sample accuracy reaches the predetermined requirement. By implementing text matching training in this way, irrelevant information can be screened for long texts and texts in speech conversion, thereby improving the efficiency of text matching.

[0061] Figure 2 This is a flowchart of a text matching training method based on reinforcement learning according to the second embodiment of the present application. As shown in the figure, step s102, segmenting the first sample data and the second sample data, includes:

[0062] Step s201, respectively performing sentence segmentation on the first sample data and the second sample data to obtain sentence segmentation data of the first sample data and sentence segmentation data of the second sample data;

[0063] Specifically, the sentences in the first sample data and the second sample data are usually long sentences, and the long sentence refers to a sentence containing multiple clauses, and a sentence usually ending with a period is a complete sentence, and a sentence within a period usually contains multiple clauses. Therefore, the sentences in the first sample data and the second sample data can be input into a neural network recognition model to identify each clause in the first sample data and the second sample data.

[0064] Step s202 , segmenting the sentence data of the first sample data and the sentence data of the second sample data respectively.

[0065] Specifically, after the sentences in the first sample data and the second sample data are segmented, each sentence in the first sample data and the second sample data is then segmented.

[0066] In this embodiment, by first segmenting the sample data into sentences and then segmenting the words, the meaning of the sentences can be better grasped and the training efficiency of the text accuracy can be improved.

[0067] Figure 3 This is a flowchart of a text matching training method based on reinforcement learning according to the third embodiment of the present application. As shown in the figure, the step s201, respectively sentence-segmenting the first sample data and the second sample data to obtain sentence data of the first sample data and sentence data of the second sample data, includes:

[0068] Step s301, respectively dividing the first sample data and the second sample data into sentences according to preset separators to obtain sentence data of the first sample data and sentence data of the second sample data, wherein the separators include commas and semicolons;

[0069] Specifically, when the long sentences in the first sample data and the second sample data are segmented, the segmentation can be performed according to the separators, and the separators include commas and semicolons, that is, one comma or one semicolon corresponds to one sentence. Therefore, in the neural network recognition model, the long sentences can be segmented by identifying commas and semicolons. For example, a long sentence can be segmented into several sentences according to the number of commas in it, or a long sentence can be segmented into several sentences according to the number of semicolons in it. Take two long sentences as an example: long sentence A consists of 3 sentences, and the format is as follows: "aaaa, bbbb, cccc.", and long sentence B consists of 2 sentences, and the format is as follows: "xxxx; yyyy.". After the segmentation, long sentence A will be divided into 3 sentences, namely "aaaa", "bbbb" and "cccc", and long sentence B will become "xxxx" and "yyyy", and then the three sentences of long sentence A and the three sentences of long sentence B are segmented. .

[0070] Step s302: Associating the sentence data of the first sample data with the first sample data, and associating the sentence data of the second sample data with the second sample data.

[0071] Specifically, after the long sentences in the first sample data and the second sample data are segmented respectively, in order to avoid confusion between the sentences of the first sample data and the sentences of the second sample data, it is necessary to associate the sentences of the first sample data with the first sample, and to associate the sentences of the second sample data with the second sample data.

[0072] In this embodiment, the sample data is segmented into sentences by using preset separators, which can effectively improve the efficiency of sentence segmentation.

[0073] Figure 4 This is a flowchart of a text matching training method based on reinforcement learning according to the fourth embodiment of the present application. As shown in the figure, the step s102, segmenting the first sample data and the second sample data, inputting the segmented first sample data and the second sample data into a preset neural network respectively, to obtain a first latent vector and a second latent vector, includes:

[0074] Step s401, copying the first sample data and the second sample data to obtain a first sample data copy and a second sample data copy;

[0075] Specifically, before word segmentation is performed on the first sample data and the second sample data, the first sample data and the second sample data may be copied to obtain a first sample data copy and a second sample data copy. The first sample data and the second sample data may be directly used for word segmentation, and the first sample data copy and the second sample data copy may be further processed.

[0076] Step s402, performing word segmentation on the first sample data and the second sample data, and inputting the first sample data and the second sample data after word segmentation into a preset neural network respectively to obtain a latent vector of the first sample data and a latent vector of the second sample data;

[0077] Specifically, this step is consistent with step s102, that is, the first sample data and the second sample data are segmented, and the segmented first sample data and the second sample data are respectively input into a preset neural network to obtain a latent vector of the first sample data and a latent vector of the second sample data.

[0078] Step s403, performing sentence segmentation on the first sample data copy and the second sample data copy to obtain first sample data copy sentence data and second sample data copy sentence data, performing word segmentation on the first sample data copy sentence data and the second sample data copy sentence data, and inputting the word segmented first sample data copy sentence data and the second sample data copy sentence data into a preset neural network respectively to obtain a first sample data copy latent vector and a second sample data copy latent vector;

[0079] Specifically, before word segmentation is performed on the first sample data copy and the second sample data copy, sentence segmentation may also be performed. The sentence segmentation may be performed according to step s301 to obtain first sample data copy sentence data and second sample data copy sentence data, and then the first sample data copy sentence data and the second sample data copy sentence data are word segmented again, and the first sample data copy sentence data and the second sample data copy sentence data after word segmentation are respectively input into a preset neural network to obtain a first sample data copy latent vector and a second sample data copy latent vector.

[0080] Step s404, obtaining a first latent vector and a second latent vector according to the first sample data latent vector, the second sample data latent vector, the first sample data copy latent vector and the second sample data copy latent vector.

[0081] Specifically, after word segmentation of the first sample data and the second sample data, sentence segmentation and word segmentation of the first sample data copy and the second sample data copy, and obtaining the first sample data latent vector, the second sample data latent vector, the first sample data copy latent vector and the second sample data copy latent vector, the first sample data latent vector, the second sample data latent vector, the first sample data copy latent vector and the second sample data copy latent vector can be merged to obtain the first latent vector and the second latent vector.

[0082] In this embodiment, by creating a copy of the sample data, and performing word segmentation on the sample data and sentence segmentation and word segmentation on the copy of the sample data, a first latent vector and a second latent vector are obtained, so that the meaning of the sentence can be obtained more accurately, the recognition efficiency of the sentence can be improved, and the text training efficiency can be improved.

[0083] Figure 5 This is a flowchart of a text matching training method based on reinforcement learning according to the fifth embodiment of the present application. As shown in the figure, the step s404, obtaining a first latent vector and a second latent vector according to the first sample data latent vector, the second sample data latent vector, the first sample data copy latent vector and the second sample data copy latent vector, includes:

[0084] Step s501, concatenating the first sample data latent vector with the first sample data copy latent vector to obtain a first latent vector;

[0085] Specifically, in practical applications, when the first sample data is segmented and processed by a neural network, a first sample data latent vector is obtained, and when the first sample data copy is sentence-segmented and segmented, and processed by a neural network, a first sample data copy latent vector is obtained; then the first sample data latent vector and the first sample data copy latent vector can be spliced ​​to obtain a first latent vector, and the splicing can be performed in any order.

[0086] Step s502: concatenate the second sample data latent vector with the second sample data copy latent vector to obtain a second latent vector.

[0087] Specifically, similarly, when the second sample data is word segmented and processed by a neural network, a second sample data latent vector is obtained, and the second sample data copy is sentence segmented and word segmented, and processed by a neural network, a second sample data copy latent vector is obtained; then the second sample data latent vector and the second sample data copy latent vector can be spliced ​​to obtain a second latent vector, and the splicing can be performed in any order.

[0088] In this embodiment, the final latent vector is obtained by concatenating the latent vector of the sample data and the latent vector of the copy data, and the word vector is obtained from the latent vector, which can improve the extraction of sentence meaning and improve the recognition accuracy of the sentence.

[0089] Figure 6 This is a flowchart of a text matching training method based on reinforcement learning according to the sixth embodiment of the present application. As shown in the figure, the step s104, inputting the first word vector and the second word vector into a preset text matching model at the same time to obtain the text accuracy, includes:

[0090] Step s601, inputting the first word vector and the second word vector into a preset text matching model in sequence to obtain text similarity;

[0091] Specifically, the first word vector and the second word vector are obtained from a pair of long sentences with similarity labels, and the first word vector and the second word vector are a combination of multiple pairs of word vectors. For example, the first word vector is a vector set including word vectors A, B, and C, and the second word vector is a vector set including word vectors a, b, and c, wherein A and a are a pair of combined vectors with similarity labels, B and b are a pair of combined vectors with similarity labels, and C and c are a pair of combined vectors with similarity labels. Thus, the three pairs of vectors can be input into the preset text matching model in sequence to obtain the corresponding text similarities. When A and a are input into the preset text matching model, the text similarity of A and a can be obtained; similarly, when B and b are input into the preset text matching model, the text similarity of B and b can be obtained, and when C and c are input into the preset text matching model, the text similarity of C and c can be obtained.

[0092] Step s602, if the text similarity matches the preset similarity tag, the text accuracy times are accumulated;

[0093] Specifically, whenever a pair of combined vectors is input into the text matching model to obtain text similarity, the obtained similarity can be compared with the preset similarity label. If the results of the two are consistent, the accurate result is recorded as 1, otherwise the inaccurate result is recorded as 1 or no record is made, and then the results of the next pair of combined vectors are compared. After each pair of combined vectors in the first group of sample data is calculated, the comparison results of each pair of data are obtained and accumulated.

[0094] Step s603, obtaining the text accuracy rate according to the cumulative number of accurate texts, the first word vectors and the total number of the second word vectors.

[0095] Specifically, after the cumulative number of text accuracy is counted, the text accuracy is obtained based on the cumulative number of text accuracy and the total number of the first word vector and the second word vector. Since the text accuracy is the cumulative number of text accuracy divided by the total number of first word vectors or the total number of second word vectors, where the first word vector or the second word vector is a pair, it is only necessary to know the total number of first word vectors or the total number of second word vectors. For example, the first group of sample data has 100 pairs of sentences, that is, the total number of first word vectors and the total number of second word vectors are both 100. If 80 pairs of sentences are accurate, that is, the cumulative number of text accuracy is 80, then the text accuracy of this round is 80 / 100=80%.

[0096] In this embodiment, by using threshold similarity labels and calculating the similarity of vectors in sequence, the text accuracy is statistically calculated, thereby improving the efficiency and accuracy of text matching training.

[0097] In one embodiment, the step s104, adjusting the policy network parameters according to the text accuracy to obtain a new policy network, includes:

[0098] According to the formula Adjust the policy network parameters θ to obtain a new policy network, where N is the number of samples, t is the step count, π θ (a t |s t ) is the output of the policy network, a t is the action at time t, s t is the latent vector at time t, γ is the attenuation factor, and r is the text accuracy.

[0099] Specifically, when the policy network receives the text accuracy, the policy network parameters can be updated through the policy gradient method to obtain a new policy network. First, the gradient can be calculated. The formula is as follows:

[0100] Among them, θ is the parameter that the policy network needs to adjust, N is the number of samples, t is the step count, and π θ (a t |s t ) is the output of the policy network, a t is the action at time t, s t is the latent vector at time t, γ is the attenuation factor, and r is the text accuracy; when the gradient is calculated After that, the gradient update algorithm can be used to adjust the policy network parameter θ. The formula is as follows:

[0101] Where θ is the current policy network parameter, θ' is the updated policy network parameter, and α is the step size parameter. α can be set as a constant and can be pre-set so that it will not change in subsequent training processes; it can also be adjusted with the results of each round of training, for example, continuously reducing α to continuously optimize the training results.

[0102] In this embodiment, the parameters of the policy network are updated by the policy gradient method and are updated in each round of training until the text accuracy reaches a threshold, which can improve the efficiency of text matching training.

[0103] The structure of a text matching training device based on reinforcement learning in the embodiment of the present application is as follows: Figure 7 As shown, including:

[0104] an acquisition module 701, a word segmentation module 702, a calculation module 703 and a training module 704; wherein the acquisition module 701 is connected to the word segmentation module 702, the word segmentation module 702 is connected to the calculation module 703, and the calculation module 703 is connected to the training module 704; the acquisition module 701 is used to acquire sample data, divide the sample data into multiple groups, extract a first group of sample data, and divide the first group of sample data into first sample data and second sample data; the word segmentation module 702 is used to segment the first sample data and the second sample data, and input the segmented first sample data and the second sample data into a preset neural network respectively to obtain a first latent vector and a second latent vector The calculation module 703 is used to input the first latent vector and the second latent vector into the preset strategy network for calculation, so as to obtain the first word vector and the second word vector; the training module 704 is used to input the first word vector and the second word vector into the preset text matching model at the same time, so as to obtain the text accuracy, and compare the text accuracy with the preset text accuracy threshold. If the text accuracy is greater than the preset text accuracy threshold, the training is terminated; otherwise, the policy network parameters are adjusted according to the text accuracy, a new policy network is obtained, and the next set of sample data is trained on the new policy network until the text accuracy is greater than the preset text accuracy threshold.

[0105] An embodiment of the present application also discloses a text matching training device based on reinforcement learning, which includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by one or more of the processors, the one or more processors execute the text matching training method based on reinforcement learning described in the above embodiments.

[0106] An embodiment of the present application also discloses a storage medium, which can be read and written by a processor, and the memory stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the reinforcement learning-based text matching training method described in the above embodiments.

[0107] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0108] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0109] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A text matching training method based on reinforcement learning, characterized in that: The text matching training method based on reinforcement learning includes: Acquire sample data, divide the sample data into multiple groups, extract a first group of sample data, and divide the first group of sample data into first sample data and second sample data; Performing word segmentation on the first sample data and the second sample data, and inputting the first sample data and the second sample data after word segmentation into a preset neural network respectively to obtain a first latent vector and a second latent vector; Inputting the first latent vector and the second latent vector into a preset strategy network for calculation respectively to obtain a first word vector and a second word vector; Input the first word vector and the second word vector into the preset text matching model at the same time to obtain the text accuracy, compare the text accuracy with the preset text accuracy threshold, and if the text accuracy is greater than the preset text accuracy threshold, end this training; otherwise, adjust the policy network parameters according to this text accuracy to obtain a new policy network, and perform training on the next set of sample data on the new policy network until the text accuracy is greater than the preset text accuracy threshold; The word segmentation of the first sample data and the second sample data, inputting the first sample data and the second sample data after word segmentation into a preset neural network respectively, and obtaining the first latent vector and the second latent vector comprises: copying the first sample data and the second sample data to obtain a first sample data copy and a second sample data copy; word segmentation of the first sample data and the second sample data, inputting the first sample data and the second sample data after word segmentation into a preset neural network respectively, and obtaining the first sample data latent vector and the second sample data latent vector; sentence segmentation of the first sample data copy and the second sample data copy to obtain first sample data copy sentence data and second sample data copy sentence data, word segmentation of the first sample data copy sentence data and the second sample data copy sentence data, and inputting the first sample data copy sentence data and the second sample data copy sentence data after word segmentation into a preset neural network respectively, and obtaining the first sample data copy latent vector and the second sample data copy latent vector; obtaining the first latent vector and the second latent vector according to the first sample data latent vector, the second sample data latent vector, the first sample data copy latent vector, and the second sample data copy latent vector; The obtaining of the first latent vector and the second latent vector according to the first sample data latent vector, the second sample data latent vector, the first sample data copy latent vector and the second sample data copy latent vector comprises: concatenating the first sample data latent vector with the first sample data copy latent vector to obtain the first latent vector; and concatenating the second sample data latent vector with the second sample data copy latent vector to obtain the second latent vector.

2. The text matching training method based on reinforcement learning as claimed in claim 1, characterized in that: The word segmentation of the first sample data and the second sample data includes: Separate the first sample data and the second sample data into sentences to obtain sentence data of the first sample data and sentence data of the second sample data; The sentence data of the first sample data and the sentence data of the second sample data are respectively segmented.

3. The text matching training method based on reinforcement learning as claimed in claim 2, characterized in that: The sentence segmenting the first sample data and the second sample data respectively to obtain the sentence data of the first sample data and the sentence data of the second sample data comprises: Separate the first sample data and the second sample data into sentences according to preset separators to obtain sentence data of the first sample data and sentence data of the second sample data, wherein the separators include commas and semicolons; The sentence data of the first sample data is associated with the first sample data, and the sentence data of the second sample data is associated with the second sample data.

4. The text matching training method based on reinforcement learning as claimed in claim 1, characterized in that: The step of simultaneously inputting the first word vector and the second word vector into a preset text matching model to obtain the text accuracy includes: Inputting the first word vector and the second word vector into a preset text matching model in sequence to obtain text similarity; If the text similarity matches the preset similarity tag, the number of text accuracy times is accumulated; The text accuracy is obtained according to the cumulative number of text accuracy times, the first word vector and the total number of the second word vector.

5. The text matching training method based on reinforcement learning as claimed in claim 4, characterized in that: The adjusting of the policy network parameters according to the accuracy of the current text to obtain a new policy network includes: According to the formula Adjusting policy network parameters , obtain a new policy network, where , are the updated policy network parameters, is the step size parameter, N is the number of samples, t is the step count, is the output of the policy network, is the action at time t, is the latent vector at time t, γ is the attenuation factor, and r is the text accuracy.

6. A text matching training device based on reinforcement learning, characterized in that: The text matching training device based on reinforcement learning includes: An acquisition module, used for acquiring sample data, dividing the sample data into multiple groups, extracting a first group of sample data, and dividing the first group of sample data into first sample data and second sample data; A word segmentation module, used to perform word segmentation on the first sample data and the second sample data, and input the first sample data and the second sample data after word segmentation into a preset neural network respectively to obtain a first latent vector and a second latent vector; A calculation module, used to input the first latent vector and the second latent vector into a preset strategy network for calculation to obtain a first word vector and a second word vector; A training module is used to input the first word vector and the second word vector into a preset text matching model at the same time, obtain a text accuracy, compare the text accuracy with a preset text accuracy threshold, and end this training if the text accuracy is greater than the preset text accuracy threshold; otherwise, adjust the policy network parameters according to this text accuracy to obtain a new policy network, and perform training on the next set of sample data on the new policy network until the text accuracy is greater than the preset text accuracy threshold; The word segmentation of the first sample data and the second sample data, inputting the first sample data and the second sample data after word segmentation into a preset neural network respectively, and obtaining the first latent vector and the second latent vector comprises: copying the first sample data and the second sample data to obtain a first sample data copy and a second sample data copy; word segmentation of the first sample data and the second sample data, inputting the first sample data and the second sample data after word segmentation into a preset neural network respectively, and obtaining the first sample data latent vector and the second sample data latent vector; sentence segmentation of the first sample data copy and the second sample data copy to obtain first sample data copy sentence data and second sample data copy sentence data, word segmentation of the first sample data copy sentence data and the second sample data copy sentence data, and inputting the first sample data copy sentence data and the second sample data copy sentence data after word segmentation into a preset neural network respectively, and obtaining the first sample data copy latent vector and the second sample data copy latent vector; obtaining the first latent vector and the second latent vector according to the first sample data latent vector, the second sample data latent vector, the first sample data copy latent vector, and the second sample data copy latent vector; The obtaining of the first latent vector and the second latent vector according to the first sample data latent vector, the second sample data latent vector, the first sample data copy latent vector and the second sample data copy latent vector comprises: concatenating the first sample data latent vector with the first sample data copy latent vector to obtain the first latent vector; and concatenating the second sample data latent vector with the second sample data copy latent vector to obtain the second latent vector.

7. A text matching training device based on reinforcement learning, characterized in that: The reinforcement learning-based text matching training device includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by one or more of the processors, the one or more processors execute the reinforcement learning-based text matching training method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The storage medium can be read and written by a processor, and the storage medium stores computer instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the text matching training method based on reinforcement learning as described in any one of claims 1 to 5.

Images