Retrieval dialogue modeling and optimization method based on generative pre-training language model

Abstract

The application discloses a retrieval dialogue modeling and optimization method based on a generative pre-training language model, and comprises the following steps: establishing a multi-turn dialogue dataset; constructing a network model based on BART improvement and training the network model based on BART improvement; using the network model based on BART improvement after training to construct a retrieval dialogue system, the retrieval dialogue system accepts any input as a dialogue context, reads a stored candidate reply vector sequence, and uses the network model based on BART improvement to calculate the vector sequence of a certain candidate reply and the dialogue context together, calculates the probability that the candidate reply matches the dialogue context, and selects the highest probability as the reply output by the dialogue system.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and in particular to a retrieval-based dialogue modeling and optimization method based on a generative pre-trained language model. Background Technology

[0002] Dialogue systems are a crucial application of natural language processing (NLP) technology. They simulate human-computer interaction, enabling the input and output of natural language to complete tasks or provide services. With the development of the internet and the advent of the big data era, deep learning has achieved groundbreaking progress thanks to the easier access to massive amounts of data, driving advancements in NLP technology. Dialogue systems have also benefited, finding wide application in numerous fields such as customer service and intelligent assistants, finance and insurance, healthcare, education and training, entertainment, and social networking. Retrieval-based dialogue systems are an important category within dialogue systems. Their goal is to select the most suitable candidate response from a predefined pool of candidates, reflecting the current dialogue context. These systems are simple to implement, offer diverse and controllable responses, and have significant industry applications.

[0003] Pre-trained language models represent a significant achievement in natural language processing (NLP) in recent years. These models, with their massive parameter sets, have demonstrated remarkable performance improvements across various NLP tasks, promising to solve more challenging problems in practical applications and further improve human life and productivity. Similarly, these models hold immense potential for application in dialogue systems. However, their large parameter sets often result in longer computation times, and response time is frequently a crucial metric for dialogue systems serving online scenarios. Especially for retrieval-based dialogue systems, the diversity and accuracy of responses depend on the size of the candidate pool; a larger pool translates to increased computation and longer user wait times, posing a challenge to the application of pre-trained language models. Furthermore, existing retrieval-based dialogue modeling methods based on pre-trained language models fail to consider the characteristics of dialogue data structures, resulting in insufficient modeling of context and candidate responses, leaving room for improvement in system recall.

[0004] A typical existing method is the approach proposed by Whang et al., which uses BERT to model retrieval-based dialogue systems (Whang T, Lee D, Lee C, et al. An effective domain adaptive post-training method for BERT in response selection[J]. arXiv preprint arXiv:1908.04812,2019). This method concatenates the dialogue context and candidate responses into a long sequence as input to the model. Modeling in this way means the computation time is proportional to the number of candidate responses, which can be very long in online scenarios, potentially causing users to lose patience and leave the system. Furthermore, this method uses only a simple training approach and fails to fully utilize the structural characteristics of the dialogue data. Summary of the Invention

[0005] The primary objective of this invention is to propose a modeling method for retrieval-based dialogue systems based on generative and trained language models. This method effectively decouples the dialogue context and candidate responses, thereby reducing the computation time of the system in online scenarios.

[0006] The second objective of this invention is to propose a model optimization method based on denoising autoencoder pre-training. By using three source text destruction strategies, the model reconstructs the damaged text using a denoising autoencoder, making full use of the expressive power of generative models and effectively improving the recall rate of retrieval-based dialogue systems.

[0007] The present invention is achieved by at least one of the following technical solutions.

[0008] A retrieval-based dialogue modeling and optimization method based on generative pre-trained language models includes the following steps:

[0009] Establish a multi-turn dialogue dataset;

[0010] Construct a network model based on BART improvement and train the network model based on BART improvement on a multi-turn dialogue dataset;

[0011] A retrieval-based dialogue system is constructed using a BART-based improved network model after training. The retrieval-based dialogue system accepts any input as dialogue context, reads the stored candidate response vector sequence, calculates the probability that the candidate response matches the dialogue context using the BART-based improved network model, and selects the response with the highest probability as the output of the dialogue system.

[0012] Furthermore, the multi-turn dialogue dataset D includes N samples:

[0013]

[0014] Each sample is composed of c i r i y i The triple formed by c i r i y i Let c represent the dialogue context, response, and label of the i-th sample in the dataset, respectively, where the dialogue context is c. i ={u i,1 ,u i,2 ,…,u i,j ,…,u i,m}, u i,j This represents the alternating messages generated by the user and the system in the j-th round, with each message u being a unique identifier. i,j ={w i,j,1 ,…,w i,j,k Let} be a sentence consisting of several different words k, and m be the total number of dialogue turns in the context of the dialogue; reply r i Is related to context c i Pairs of single sentences, consisting of several words; tag y i The value can be 0 or 1, indicating a response r i Is it context c? i The correct response, y i =0 indicates a response r i It is context c i Incorrect response, y i =1 indicates a response r i It is context c i The correct response.

[0015] Furthermore, the BART-based improved network model includes a Transformer encoder module, an improved Transformer decoder module, and a softmax classifier;

[0016] The Transformer encoder module is composed of several layers of Transformer encoders stacked together, and the improved Transformer decoder module is composed of several layers of Transformer decoders stacked together.

[0017] Furthermore, the input to the Transformer encoder module: r i =([s],r i ,[ / s]) is a sentence of a candidate response. The input sequence is first converted into word embedding vectors. Then, the word embedding vectors and position embedding vectors are added to obtain a sequence of Q-dimensional vectors. This sequence will be passed through several layers of Transformer encoders to obtain the final state vector sequence of the candidate responses.

[0018] Furthermore, the input to the improved Transformer decoder module is c. o =([s],u i,1 [EOT],u i,2 ,[EOT],…,u i,m [s] is a sequence of all sentences in the dialogue context, where [s] and [ / s] are special symbols marking the beginning and end of sentences, and [EOT] is a special symbol inserted between each speech in the dialogue context, indicating the end of a turn. i,m This represents the alternating speech generated by the user and the system in the m-th round; the input sequence is first converted into word embedding vectors, and then the word embedding vectors and position embedding vectors are added to obtain a sequence of Q-dimensional vectors; the obtained sequence is passed through several layers of Transformer encoders, which also accept the final state vector sequence of candidate responses from the encoders, to obtain a sequence of final state vectors of the context.

[0019] Furthermore, after obtaining the final state of the decoder, a linear layer is added to the final state of the special symbol [s], and a softmax classifier is used for classification to determine whether the candidate response input from the encoder is a suitable response to the dialogue context input from the decoder. The final state of the special symbol [s] is denoted as E. [s] Then the candidate response r i and context c i The probability p of the classification result i Represented as:

[0020] p i =softmax(tanh(WE) [s] +b))

[0021] Where W and b are the parameters and biases of the linear neural network layer, respectively.

[0022] Furthermore, the BART-based improved network model is trained using a denoising autoencoder, the training comprising the following steps:

[0023] Noise is added to the training data using text destruction strategies;

[0024] The BART-based improved network model is pre-trained using noisy text. The noisy text is input to the encoder of the BART-based improved network model, and the denoised text is output to the decoder. The training objective function is to minimize the cross-entropy between the model's output text and the source text.

[0025]

[0026] Where H represents the total number of words masked during the noise addition process, and p i To predict the probability of the correct word using a network model based on BART.

[0027] Furthermore, the BART-based improved network model is fine-tuned by minimizing the loss function through binary classification cross-entropy:

[0028] l = -y i log(p i )-(1-y i log(1-p) i )

[0029] Where l is the loss function for a single triplet sample in the dataset, y i For candidate responses r i With context c i The tag has a value of 0 or 1; p i This represents the context and the probability that the candidate response is correct, as predicted by the BART-based improved network model.

[0030] Furthermore, text destruction strategies include:

[0031] If the input sentence for training comes from candidate responses or a dialogue context with 1 turn, a text-filling strategy is used to destroy the text: Iteratively, a word is randomly selected as the starting point. If the word is selected, the length n of the text to be destroyed is sampled from a Poisson distribution with λ=3. The selected word and the next n consecutive words are replaced with a single [MASK] special character. The next n consecutive words are called a span. If n=0, it is equivalent to inserting a [MASK] special character after the selected word. This step is repeated until 15% of the words in the dialogue context are replaced with [MASK] characters.

[0032] Furthermore, text destruction strategies include:

[0033] If the input sentence for training comes from a dialogue context with more than 1 rounds, a speech masking strategy is used to corrupt the text: a speech is randomly selected from the context and directly replaced with a single special character [MASK].

[0034] Compared with existing technologies, the beneficial effects of the present invention are as follows:

[0035] This invention fully leverages the characteristics of generative language models and retrieval-based dialogue data, innovatively proposing three pre-training text disruption strategies. Through more challenging generative training tasks, the model is forced to learn more fundamental language patterns and rules during the pre-training phase, thereby improving its performance on downstream tasks. Furthermore, by using an encoder-decoder structure to model retrieval-based dialogues, the computation of dialogue context and candidate responses can be effectively decoupled, transferring some computational tasks offline and reducing online computation time. Attached Figure Description

[0036] Figure 1 This is a schematic diagram illustrating the process of a retrieval-based dialogue modeling and optimization method based on a generative pre-trained language model, as an example.

[0037] Figure 2 A schematic diagram of the improved transformer decoder module for this embodiment;

[0038] Figure 3 This is a schematic diagram illustrating the training steps of a network model based on the BART improvement, as shown in the example.

[0039] Figure 4 This is a schematic diagram illustrating the application of a network model based on BART improvement in an example. Detailed Implementation

[0040] To enable those skilled in the art to better understand the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0041] like Figure 1 As shown, this embodiment describes a retrieval-based dialogue modeling and optimization method based on a generative pre-trained language model, which includes the following steps:

[0042] Building a multi-turn dialogue dataset: Considering multi-turn dialogue datasets between users and the system This dataset contains N triples, where c i r i y i Let c represent the dialogue context, response, and label of the i-th sample in the dataset, respectively. i ={u i,1 ,u i,2 ,…,u i,j ,…,u i,m The m rounds of past speeches generated alternately by the user and the system. i,j Composition, each speechi,j ={w i,j,1 ,…,w i,j,k Let} be a sentence composed of several different words k, and m be the total number of dialogue turns for each context, where m may be different for each context. Reply r i Is related to context c i A pair of single sentences is also composed of several words. (Tag y) i The value of r is either 0 or 1, indicating the response r. i Is it context c? i The correct response, y i =0 indicates a response r i It is context c i Incorrect response, y i =1 indicates a response r i It is context c i The correct response.

[0043] A BART-based improved network model is constructed and trained. The trained BART-based improved network model is then used to build a retrieval-based dialogue system.

[0044] A candidate response pool of arbitrary size is constructed from the dataset, and all candidate responses are encoded into a sequence of vectors using a BART encoder, which is then pre-stored in any manner. The dialogue system accepts any input as the dialogue context, reads the stored candidate response vector sequence, and uses a BART decoder and encoder to compute the probability that a candidate response matches the dialogue context along with its vector sequence. The matching probabilities of all candidate responses are calculated in this way, and the response with the highest probability is selected as the output of the dialogue system. This retrieval-based dialogue system can be used in scenarios such as intelligent telephone customer service and web navigation.

[0045] The BART-based improved network model includes a Transformer encoder module, an improved Transformer decoder module, and a softmax classifier.

[0046] like Figure 1 As shown, at the encoder end, the input sequence is a sentence of a candidate response: r i =([s],r i ,[ / s]). Here, [s] and [ / s] are special symbols used by BART to mark the beginning and end of a sentence. At the decoder end, the input is: c i =([s],u i,1 [EOT],u i,2 ,[EOT],…,u i,m,[ / s]), is a sequence composed of all sentences in the dialogue context. At the encoder, the input sequence is first converted into word embeddings, then added to the positional embeddings to obtain a sequence of 768-dimensional vectors. These vectors are passed through several layers of Transformer encoder modules to obtain a sequence of final word states. At the decoder, the input sequence is again converted into a vector sequence through word and positional embeddings, and then passed through a speaker embedding sequence. The speaker embedding is set as follows: for sentences from the same speaker as the candidate response, the speaker embedding is 1 at each position in its sequence; otherwise, it is 0. The vector sequence is then passed through several layers of Transformer decoders, where the self-attention mechanism is improved to a bidirectional attention mechanism to enhance model performance. After obtaining the final state of the decoder, a linear layer is added to the final state of the special symbol [s], and a softmax classifier is used for classification to determine whether the candidate response input from the encoder is a suitable response to the dialogue context input from the decoder. Let the final state of the special symbol [s] be E. [s] Then the candidate response r i and context c i Classification result p i The probability can be expressed as:

[0047] p i =softmax(tanh(WE) [s] +b))

[0048] Where W and b are the parameters and biases of the linear neural network layer, respectively. The entire model is trained by minimizing the loss function through binary classification cross-entropy.

[0049] l = -y i log(p i )-(1-y i log(1-p) i )

[0050] In a preferred embodiment, the Transformer encoder module is composed of 6 stacked Transformer encoder layers, and the improved Transformer decoder module is composed of 6 stacked Transformer decoder layers; the input of the Transformer encoder module is: r i =([s],r i ,[ / s]) is a sentence of a candidate response. The Transformer encoder module first converts the input sequence into word embedding vectors, and then adds the word embedding vectors and position embedding vectors to obtain a sequence of 1024-dimensional vectors. This sequence will be passed through 6 layers of Transformer encoders to obtain the final state vector sequence of the candidate responses.

[0051] Figure 2 The diagram shows the improved Transformer decoder module. The improvements made to the original Transformer decoder are as follows: the original Transformer decoder's self-attention layer was unidirectional, while this invention improves it to bidirectional self-attention. Compared to unidirectional attention, bidirectional attention allows semantic matching between the word at the current position and the words before and after it, effectively enhancing the model's understanding ability.

[0052] The input to the improved Transformer decoder module is c o =([s],u i,1 [EOT],u i,2 ,[EOT],…,u i,m [s] is a sequence of all sentences in the dialogue context, where [s] and [ / s] are special symbols marking the beginning and end of sentences, and [EOT] is a special symbol inserted between each speech in the dialogue context, indicating the end of a turn. i,m This represents the alternating speech generated by the user and the system in the m-th round; the input sequence is first converted into word embedding vectors, and then the word embedding vectors and position embedding vectors are added to obtain a sequence of 1024-dimensional vectors; the obtained sequence is passed through a 6-layer Transformer encoder, which also accepts the final state vector sequence of candidate responses from the encoder, to obtain a sequence of final state vectors of the context.

[0053] A BART-based improved network model is pre-trained using a denoising autoencoder. First, noise is added to the source text using three text destruction methods:

[0054] If the input text is a candidate response, and if the pre-trained input sentence comes from a candidate response or a dialogue context with one speaking turn, a text-filling strategy is used to destroy the text. Iteratively, a word is randomly selected as the starting point. If a word is selected, the length n of the text to be destroyed is sampled from a Poisson distribution with λ = 3. This word, along with the next n consecutive words, is replaced with a single [MASK] special character. These n consecutive words are called a span. If n = 0, it is equivalent to inserting a [MASK] special character after the selected word. This step is repeated until 15% of the words in the dialogue context have been replaced with [MASK] characters.

[0055] If the input sentences for pre-training come from dialogue contexts with more than one round, a speech masking strategy is used to disrupt the text. A speech is randomly selected from the context and directly replaced with a single special character [MASK]. This strategy can be seen as a special case of text filling strategies, which can teach the model to learn contextual relationships.

[0056] As one example, suppose we are given a dialogue context c = {[s], u1, [EOT], ..., [EOT], u m After replacement, c = {[s],[MASK],[EOT],…,[EOT],u m ,[ / s]}, where the special character [MASK] replaces all words in speech u1.

[0057] If the pre-training input sentences come from the dialogue context, after the speech masking strategy disrupts the source text, the context speech scrambling strategy is applied again. The original context text is segmented into multiple complete speeches using the special characters [s], [ / s], and [EOT] as separators, and then the speeches are scrambled. This strategy can teach the model to understand the logical relationships within the dialogue context.

[0058] As another embodiment, given a dialogue context c = {[s], u1, [EOT], ..., [EOT], u m ,[ / s]}, to include all dialogue messages u1,…,u m Shuffling the order to obtain a new dialogue context can be c ′ ={[s],u3,[EOT],u m ,[EOT],…,u1,[ / s]).

[0059] After corrupting the text using the above methods, it is used as input to the encoder for pre-training the model using a denoising autoencoder approach to obtain a dialogue retrieval model based on BART improvement. The denoising autoencoder approach involves inputting noisy text into the model and expecting the model to reconstruct the noise-free text. Specifically, pre-training is performed by optimizing the reconstruction loss, with the objective of minimizing the cross-entropy between the decoder output and the source text. The formula is:

[0060]

[0061] Where H is the total number of all masked words, p i The probability of the model predicting the correct word.

[0062] like Figure 4As shown, after obtaining the trained model through this invention, it can be used to build a retrieval-based dialogue system. Taking e-commerce customer service as a specific application scenario, the original dialogue data can come from the existing dialogue between human customer service and customers. By constructing a training dialogue dataset from the original data, and after pre-training and fine-tuning using the method proposed in this invention, the model is put into the actual dialogue system to interact with users as an intelligent customer service representative. E-commerce dialogue is usually initiated by the user, who first asks the customer service representative: "How do I initiate a refund?" Assume that there are the following 3 responses in the candidate pool at this time: (1) "What problem do you have that requires a refund?" (2) "We will ship your order as soon as possible" (3) "Orders placed today will be shipped tomorrow". The dialogue system using this invention has pre-calculated the vector representations for these 3 candidate responses. After the user asks the question, the model proposed in this invention is used to calculate the matching probability between the dialogue context "How do I initiate a refund?" and the 3 candidate responses, and then the response with the highest probability (1) is selected as the response.

[0063] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A retrieval-based dialogue modeling and optimization method based on a generative pre-training language model, characterized in that, The method comprises the following steps: establishing a multi-turn dialogue dataset; constructing a network model improved based on BART and training the network model improved based on BART on the multi-turn dialogue dataset; the network model improved based on BART comprises a Transformer encoder module, an improved Transformer decoder module and a softmax classifier; the Transformer encoder module is stacked by several layers of Transformer encoders, and the improved Transformer decoder module is stacked by several layers of Transformer decoders; The input of the Transformer encoder module: For a sentence of a candidate reply, the input sequence is first converted into a word embedding vector, and then the word embedding vector and the position embedding vector are added to obtain a sequence composed of Q-dimensional vectors. The sequence will pass through several layers of Transformer encoders to obtain a final state vector sequence of the candidate reply. The input of the improved Transformer decoder module is a sequence of all sentences concatenated in the dialog context, where [s] and [ / s] are special symbols marking the beginning and end of a sentence, and [EOT] is a special symbol inserted between each utterance in the dialog context, indicating the end of a turn (End Of Turn), denotes the turn of the user and the system alternately produced utterances; the input sequence is first converted into a word embedding vector, and then the word embedding vector and the position embedding vector are added to obtain a sequence composed of Q-dimensional vectors; the obtained sequence passes through several layers of Transformer encoder, which also accepts the final state vector sequence of the candidate reply from the encoder to obtain a sequence composed of the final state vectors of the context; After obtaining the final state of the decoder, a linear layer is concatenated on the final state of the special symbol [s], and a softmax classifier is used to classify whether the candidate reply input at the encoder end is a suitable reply to the conversation context input at the decoder end, and the final state of the special symbol [s] is denoted as Then the candidate reply and the context The probability of the classification result is is expressed as: wherein W and b are parameters and biases of linear neural network layers, respectively; using the network model improved based on BART after training to construct a retrieval dialogue system, the retrieval dialogue system accepts any input as a dialogue context, reads the stored candidate reply vector sequence, and calculates the probability of matching the candidate reply and the dialogue context by using the network model improved based on BART, and selects the one with the highest probability as the reply output by the dialogue system.

2. The retrieval-based dialog modeling and optimization method based on a generative pre-training language model according to claim 1, characterized in that, The multi-turn dialogue dataset , comprising samples: Each sample is composed of , , The triplet formed, , , They represent the first and second elements in the dataset, respectively. The dialogue context, responses, and tags for each sample, where the dialogue context is... , Indicates the first The messages generated alternately by users and the system, each message For several different words The sentence is composed of m, where m is the total number of dialogue turns in the context of the dialogue; the reply Is with context Pairs of single sentences, consisting of several words; tags A value of 0 or 1 indicates a response. Is it context? The correct response, This indicates a reply. Context Incorrect response, This indicates a reply. Context The correct response.

3. The retrieval-based dialog modeling and optimization method based on a generative pre-training language model according to any one of claims 1-2, characterized in that, The network model improved based on BART is trained by a denoising autoencoder, and the training comprises the following steps: using a text destruction strategy to add noise to the training data; using the noisy text to pre-train the network model improved based on BART, inputting the noisy text at the encoder end of the network model improved based on BART, and outputting the denoised text at the decoder end, and the training target function is to minimize the cross entropy between the model output text and the source text: wherein is the number of all covered words in the noise adding process, is the probability of predicting the correct word based on the improved network model of BART.

4. The retrieval-based dialog modeling and optimization method based on a generative pre-training language model according to claim 3, characterized in that, fine-tuning the network model improved based on BART by minimizing the loss function of binary classification cross entropy: wherein is the loss function for a single triple sample in the data set, is the candidate reply is the label of the context , taking values 0 or 1 ; is the probability that the context and the candidate reply are correct, predicted by the network model improved based on BART.

5. The retrieval-based dialog modeling and optimization method based on a generative pre-training language model according to claim 3, characterized in that: The text destruction strategy comprises: If the input sentences for training come from candidate responses or the context of a dialogue with one turn, a text-filling strategy is used to decompile the text: words are iteratively and randomly selected as starting points; if a word is selected, the text is filled in from the next point. The length of the text to be destroyed is sampled from the Poisson distribution. This word and its subsequent continuation Each word is replaced with a single [MASK] special character, consecutively. One word is called a span; if This is equivalent to inserting a [MASK] special character after the selected word. Repeat this step until 15% of the words in the dialogue context have been replaced with the [MASK] character.

6. The retrieval-based dialog modeling and optimization method based on a generative pre-training language model according to claim 3, characterized in that: The text destruction strategy comprises: If the input sentence of the training comes from a dialogue context with a turn number greater than 1, use the utterance masking strategy to destroy the text: randomly select a sentence in the context, and directly replace the sentence with a single special character [MASK].

Images