Method, apparatus and device for training generative language model

Abstract

The application provides a generative language model training method, device and equipment, the method inputs multiple sets of dialogue data into a generative language model; based on the generative language model, determines semantic representations corresponding to dialogue context and dialogue context in each set of dialogue data respectively, and a predicted response corresponding to each set of dialogue data; according to the semantic representations corresponding to the multiple sets of dialogue data, determines a mutual information matrix; each value in the mutual information matrix is used to represent the correlation degree between the corresponding two semantic representations; according to the mutual information matrix and the predicted response corresponding to the multiple sets of dialogue data, the generative language model is updated, the mutual information between the predicted response output by the model and the semantic representation of the dialogue context is increased by updating the model based on the mutual information matrix, the predicted response of the output universal reply is reduced, and thus the accuracy of the generated response is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a generative language model training method, apparatus, and equipment. Background Technology

[0002] With the rapid development of mobile internet and artificial intelligence technologies, dialogue systems are being used more and more widely. Dialogue systems can intelligently provide information services to users through human-computer interaction. Among them, dialogue generation is a key link in dialogue systems.

[0003] Open-domain dialogue systems primarily aim to facilitate communication with people on any topic using natural language, characterized by a wide range of topics and diverse expression methods. Currently, open-domain dialogue systems mainly fall into three categories: retrieval-based dialogue systems, generative dialogue systems, and coupled retrieval-generative dialogue systems. Retrieval-based dialogue systems select responses from a dialogue corpus corresponding to the current question; however, their performance is limited by the size and quality of the corpus, and accurate responses cannot be obtained when the scenario is not covered in the corpus. Generative dialogue systems are trained on datasets of human-to-human dialogue, enabling the model to learn features from people's dialogue history to generate corresponding responses; however, the responses generated by generative dialogue systems are often generic. Coupled retrieval-generative dialogue systems combine retrieval and generative methods sequentially or in parallel; however, this approach is often heavily influenced by the results of the retrieval method, weakening the role of the generative method and failing to effectively integrate the two approaches.

[0004] Therefore, existing dialogue generation methods suffer from low accuracy in generating responses. Summary of the Invention

[0005] This invention provides a generative language model training method, apparatus, and device to address the problem of low accuracy in responses generated by existing dialogue generation methods.

[0006] In a first aspect, the present invention provides a generative language model training method, the method comprising:

[0007] Multiple sets of dialogue data are input into the generative language model;

[0008] Based on the generative language model, the semantic representations corresponding to the dialogue context and dialogue background in each set of dialogue data are determined, as well as the predicted response corresponding to each set of dialogue data.

[0009] Based on the semantic representations corresponding to multiple sets of dialogue data, a mutual information matrix is ​​determined; each value in the mutual information matrix is ​​used to represent the degree of correlation between the corresponding two semantic representations.

[0010] The generative language model is updated based on the mutual information matrix and the predicted responses corresponding to multiple sets of dialogue data.

[0011] Optionally, the semantic representation set is composed of the semantic representations of the preceding and following dialogue texts corresponding to all groups of dialogue data; based on the semantic representations corresponding to multiple groups of dialogue data, a mutual information matrix is ​​determined, including:

[0012] For each semantic representation in the semantic representation set, calculate the inner product of that semantic representation with every other semantic representation in the semantic representation set except for that semantic representation;

[0013] Based on the calculated inner products, the mutual information matrix corresponding to the multiple sets of dialogue data is determined.

[0014] Optionally, the inner products corresponding to a semantic representation set form an inner product matrix; based on the calculated inner products, the mutual information matrix corresponding to the multiple sets of dialogue data is determined, including:

[0015] Normalize the inner product of each row or column in the inner product matrix respectively;

[0016] The mutual information matrix is ​​determined based on the results of the normalization operation corresponding to each row or column of data.

[0017] Optionally, the generative language model is updated based on the mutual information matrix and the predicted responses corresponding to multiple sets of dialogue data, including:

[0018] The first loss value is determined based on the mutual information matrix and the corresponding reference mutual information matrix;

[0019] The second loss value is determined based on the predicted response and the corresponding dialogue context;

[0020] The sum of the first loss value and the second loss value is calculated, and the generative language model is updated based on the sum.

[0021] Optionally, the inner product of the semantic representation of any dialogue context and the corresponding semantic representation of the dialogue context in the reference mutual information matrix is ​​greater than the inner product of the semantic representation of the dialogue context and other semantic representations; the other semantic representations are semantic representations other than the semantic representation of the dialogue context corresponding to the dialogue context.

[0022] Optionally, based on the generative language model, determine the semantic representations corresponding to the preceding and following dialogue texts in each set of dialogue data, including:

[0023] For each set of dialogue data, a first input data and a second input data are generated based on the preceding and following dialogue text; wherein, the first input data is the data obtained by concatenating the following dialogue text after the preceding dialogue text; and the second input data is the data obtained by concatenating the preceding dialogue text after the preceding dialogue text.

[0024] The first and second input data corresponding to each set of dialogue data are respectively input into the generative language model;

[0025] The output of the hidden layer of the generative language model corresponding to the first input data is determined as the semantic representation of the preceding context of the dialogue; the output of the hidden layer of the generative language model corresponding to the second input data is determined as the semantic representation of the following context of the dialogue.

[0026] Optionally, normalization operations are performed on the inner product of each row or each column in the inner product matrix, including:

[0027] Perform a masking operation on the inner product at the diagonal positions of the inner product matrix to obtain the inner product matrix after the masking operation;

[0028] Normalize the inner product of each row or column in the inner product matrix after the masking operation.

[0029] In a second aspect, the present invention provides a generative language model training device, the device comprising:

[0030] The input module is used to input multiple sets of dialogue data into the generative language model;

[0031] The first determining module is used to determine, based on the generative language model, the semantic representations corresponding to the dialogue context and dialogue background in each set of dialogue data, as well as the predicted response corresponding to each set of dialogue data.

[0032] The second determining module is used to determine the mutual information matrix based on the semantic representations corresponding to multiple sets of dialogue data; each value in the mutual information matrix is ​​used to represent the degree of correlation between the corresponding two semantic representations.

[0033] The update module is used to update the generative language model based on the mutual information matrix and the predicted responses corresponding to multiple sets of dialogue data.

[0034] Thirdly, the present invention provides an electronic device, comprising: at least one processor and a memory;

[0035] The memory stores computer-executed instructions;

[0036] The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method as described in any of the first aspects.

[0037] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in any one of the first aspects.

[0038] Fifthly, the present invention provides a program product including a computer program that, when executed by a processor, implements the method described in any of the first aspects.

[0039] This invention provides a generative language model training method, apparatus, and device. The method includes: inputting multiple sets of dialogue data into a generative language model; determining, based on the generative language model, the semantic representations corresponding to the preceding and following dialogue texts in each set of dialogue data, and the predicted response corresponding to each set of dialogue data; determining a mutual information matrix based on the semantic representations corresponding to the multiple sets of dialogue data; each value in the mutual information matrix represents the degree of correlation between the corresponding two semantic representations; updating the generative language model based on the mutual information matrix and the predicted responses corresponding to the multiple sets of dialogue data. By updating the model based on the mutual information matrix, the mutual information between the predicted response output by the model and the semantic representation of the preceding dialogue text is increased, reducing the predicted response of the output universal response, thereby improving the accuracy of the generated response. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0041] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present invention;

[0042] Figure 2 A flowchart illustrating a generative language model training method provided in an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram illustrating the use of a generative language model to obtain semantic representations and predict responses, as provided in an embodiment of the present invention.

[0044] Figure 4 A schematic diagram illustrating the adjustment of attention allocation coefficients according to an embodiment of the present invention;

[0045] Figure 5 This is a schematic diagram illustrating the generation of a mutual information matrix according to an embodiment of the present invention;

[0046] Figure 6 A method provided for embodiments of the present invention and Figure 5 A schematic diagram of the corresponding reference mutual information matrix;

[0047] Figure 7 This is a schematic diagram of the structure of a generative language model training device provided in an embodiment of the present invention;

[0048] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0049] The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0050] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0051] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0052] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present invention, such as... Figure 1 As shown, when a user has a need for casual conversation, they can chat with a web chatbot. After the user sends a message, the web page processor can obtain the message and transmit it to the backend server. The backend server has a dialogue system that can generate and output responses based on the messages sent by the user and display them on the page. This dialogue system is implemented based on a generative language model.

[0053] Existing dialogue systems are generally categorized into retrieval-based, generative, or coupled-retrieval-generated dialogue systems. Retrieval-based systems typically retrieve a small candidate set from a dialogue corpus (comprising context and response) based on context, and then select a suitable response from this set for that context. This method is usually limited by the size and quality of the dialogue corpus. Generative dialogue systems, on the other hand, generate responses that are largely generic due to the large proportion of generic responses in the dialogue corpus. Generative responses reduce the accuracy of the dialogue system and negatively impact the user experience. Coupled-retrieval-generated dialogue systems usually fuse the two methods sequentially or in parallel. This method is heavily influenced by the results of the retrieval method and cannot effectively integrate the two approaches. For example, in a parallel approach, responses from both methods are treated as different alternatives, and the two results do not affect each other. For instance, if the score of the retrieved response is higher than a threshold, the retrieval response is used as the final result; otherwise, the result from the generative method is returned.

[0054] In view of the above problems, the generative language model training method provided by this invention enhances the ability of the generative algorithm by means of the idea of ​​retrieval method. That is, by improving the training method of generative language model, the model is trained not only based on the predicted response, but also based on the mutual information matrix that represents the degree of correlation between the semantic representations corresponding to multiple sets of dialogue data. This increases the mutual information between the output response and the input dialogue context, which means that the trained model reduces the occurrence of generalized responses (e.g., I don't know, thank you, okay, etc.) and improves the accuracy of the generated response.

[0055] Figure 2 This is a flowchart illustrating a generative language model training method provided in an embodiment of the present invention; as shown below. Figure 2 As shown, the method includes:

[0056] Step S201: Input multiple sets of dialogue data into the generative language model.

[0057] Among them, it is possible to Figure 1 The backend server is configured with a generative language model, which is pre-trained. After training, the trained generative language model is used to output the response.

[0058] When training the model, multiple sets of dialogue data can be acquired first and then input into the generative language model. These multiple sets of dialogue data come from various open-ended casual conversation corpora.

[0059] During training, all the obtained conversation data can be divided into multiple batches, and the model can be trained based on multiple groups of conversation data in each batch. That is, steps S201 to S204 can be executed based on multiple groups of conversation data in each batch until the condition for stopping training is met, such as reaching the number of training times, then it is determined that the generative language model training is completed.

[0060] Among them, the conversation data includes the conversation context and the conversation continuation. According to the conversation context and the conversation continuation, input data can be constructed. The constructed input data not only includes the conversation context and the conversation continuation, but also includes the start identifier CLS and the segmentation identifier SEP. The start identifier indicates the start of the input data, and the segmentation identifier indicates the segmentation of every two sentences of the input.

[0061] The generative language model here refers to an improved bert (Bidirectional Encoder Representations from Transformer) model.

[0062] The traditional bert model is a self-encoding pre-training model or a self-regressive pre-training model. The self-encoding pre-training model means that when generating the semantic representation of the conversation history or outputting the predicted response, it can be determined according to the context. That is, when generating the semantic representation of the conversation history, it is determined according to the complete conversation context of the input. Exemplarily, it can be determined according to "CLS What do you drink? SEP". When outputting the predicted response, it is determined according to "CLS What do you drink? SEP Plain water", but when predicting the response, it is actually impossible to predict according to the actual output. The self-encoding pre-training model is more suitable for language understanding rather than language generation.

[0063] The self-regressive pre-training model means that when generating the semantic representation of the conversation history or predicting the response, it can only be determined according to the above data. Then, when generating the semantic representation of the conversation history, an accurate semantic representation cannot be generated, which is equivalent to predicting "What do you drink?" as well, and this part does not need to be predicted. Therefore, the self-regressive pre-training model is not suitable for application in this conversation system either.

[0064] Figure 3 It is a schematic diagram of obtaining semantic representation and predicted response by using a generative language model provided by an embodiment of the present invention. As Figure 3 shown, the improved bert model (generative language model) is a fusion of a self-encoding pre-training model and a self-regressive pre-training model. When generating the semantic representation of the conversation history (the first box output), it is implemented based on the self-encoding pre-training model; when generating the predicted response, it is implemented based on the self-regressive pre-training model.

[0065] When generating the semantic representation of the dialogue history (the first box in the output), it is determined according to the context of the input dialogue above, so as to accurately understand the dialogue history data. The first box in the output is the output vector corresponding to the CLS identifier, and this vector can fairly integrate the semantic information of each word in the dialogue above. When generating the predicted response, it is determined only according to the above data, that is, when generating "white", it is determined only according to "CLS What would you like to drink? SEP", and when generating "open", it is determined only according to "CLS What would you like to drink? SEP white", so as to predict the output data.

[0066] Among them, the improvement of the bert model can be achieved by adjusting the attention distribution coefficient. A variable M is added to the existing attention distribution coefficient, and by setting the value of M corresponding to each position to 0 or negative infinity, the improved bert model is obtained. Set M in the part of generating the semantic representation of the dialogue history to 0; Figure 4 It is a schematic diagram of adjusting the attention distribution coefficient provided by an embodiment of the present invention; as Figure 4 shown, the horizontal direction represents the input, and the vertical direction represents the output of the hidden layer and the predicted response of the model. Set the value of M in the attention distribution coefficient corresponding to the shaded part to 0, and set the value of M in the attention distribution coefficient corresponding to the rest to negative infinity. Among them, when set to 0, it means that when generating the semantic representation of the dialogue above of the hidden layer output, all the dialogue above can be seen. When outputting the predicted response, only the input before this character can be seen. For example, when predicting "open", only "CLS What would you like to drink? SEP white" can be seen, but not "boiled water".

[0067] By improving the generative language model, the hidden layer can output the accurate semantic representation of the dialogue above, and at the same time, the model has the language generation ability.

[0068] Step S202: Based on the generative language model, determine the semantic representations corresponding to the dialogue above and the dialogue below in each group of dialogue data, and the predicted response corresponding to each group of dialogue data.

[0069] After inputting multiple groups of dialogue data into the generative language model, the hidden layer of the generative language model can output the semantic representation corresponding to each group of dialogue data. The semantic representation corresponding to each group of dialogue data includes the semantic representation of the dialogue above and the semantic representation of the dialogue below.

[0070] Exemplarily, for single-round dialogue data, the dialogue above is "What would you like to drink?", and the dialogue below is "boiled water", then the semantic representations of "What would you like to drink?" and "boiled water" need to be obtained respectively. For multi-round dialogue data, if the dialogue round is N, the dialogue above is the dialogue data including N - 1 rounds and the dialogue above data of the Nth round of dialogue; the dialogue below is the dialogue below of the Nth round of dialogue.

[0071] Optionally, based on the generative language model, determine the semantic representations corresponding to the preceding and following dialogue texts in each set of dialogue data, including:

[0072] For each set of dialogue data, first input data and second input data are generated based on the preceding and following dialogue texts; wherein, the first input data is data obtained by concatenating the following dialogue texts after the preceding dialogue texts; the second input data is data obtained by concatenating the preceding dialogue texts after the preceding dialogue texts; the first input data and second input data corresponding to each set of dialogue data are respectively input into the generative language model; the output of the hidden layer of the generative language model corresponding to the first input data is determined as the semantic representation of the preceding dialogue texts; the output of the hidden layer of the generative language model corresponding to the second input data is determined as the semantic representation of the following dialogue texts.

[0073] Specifically, when generating semantic representations, only the semantic representation of the dialogue context input into the model can be generated. Therefore, in order to obtain the semantic representations corresponding to the dialogue context and the dialogue background in a set of dialogue data, the input data can be constructed.

[0074] Specifically, when generating the semantic representation of the preceding dialogue, the following dialogue from a set of dialogue data can be concatenated after the preceding dialogue to obtain the first input data. Similarly, when generating the semantic representation of the following dialogue, the preceding dialogue from a set of dialogue data can be concatenated after the following dialogue to obtain the second input data. For example, if the preceding dialogue is "What do you want to drink?" and the following dialogue is "Plain water", denoted as SENT_A and SENT_B respectively, then for a set of dialogue data, the corresponding first input data is [CLS]SENT_A[SEP]SENT_B[SEP]; the corresponding second input data is [CLS]SENT_B[SEP]SENT_A[SEP]. The first and second input data are then input into the generative language model.

[0075] After inputting the first and second input data into the model, the semantic representations of the preceding and following dialogues can be obtained. Specifically, when the first input data is input into the model, the semantic representation of the preceding dialogue is obtained; when the second input data is input into the model, the semantic representation of the following dialogue is obtained. For example, when the dialogue data is "What do you drink? Plain water," this step can obtain the semantic representations of "What do you drink?" and "Plain water."

[0076] By constructing the two types of input data as described above, we can accurately obtain the semantic representation of the preceding and following text of the dialogue, while also having the advantage of simple operation.

[0077] In addition, it is necessary to obtain the predicted response corresponding to each set of dialogue data, which can be obtained for each first input data. For example, when the input is "CLS what do you drink? SEP plain water? SEP", the predicted response can be obtained based on the semantic representation of "what do you drink?".

[0078] Step S203: Determine the mutual information matrix based on the semantic representations corresponding to multiple sets of dialogue data; each value in the mutual information matrix is ​​used to represent the degree of correlation between the corresponding two semantic representations.

[0079] After obtaining the semantic representations corresponding to multiple sets of dialogue data in a batch, a mutual information matrix can be calculated based on the semantic representations. Each value in the mutual information matrix represents the degree of correlation between two semantic representations. When a batch of dialogue data consists of N sets, the size of the corresponding mutual information matrix is ​​2N*2N.

[0080] The semantic representations of the preceding or following dialogue context here are multi-dimensional vectors. Mutual information between two vectors can be obtained through vector operations. The greater the mutual information, the stronger the correlation between the two semantic representations; the smaller the mutual information, the weaker the correlation between the two semantic representations.

[0081] Figure 5 This is a schematic diagram illustrating the generation of a mutual information matrix according to an embodiment of the present invention. Semantic representation 1 corresponds to the context of a set of dialogue data, and semantic representation 2 corresponds to the context of the same set of dialogue data. That is, semantic representation 1 and semantic representation 2 match, while semantic representation 1 does not match the other semantic representations. Similarly, semantic representation 3 and semantic representation 4 match, while semantic representation 3 does not match the other semantic representations. Matrix A, containing 16 data points, can be calculated based on the vector of each semantic representation.

[0082] This step embodies the idea of ​​retrieval, which is to select the semantic representation with the highest mutual information with a certain semantic representation from multiple semantic representations. Therefore, for responses generated by generative methods, since responses such as universal responses usually have low mutual information with the context of the dialogue, the generation of universal responses will be reduced.

[0083] Step S204: Update the generative language model based on the mutual information matrix and the predicted responses corresponding to multiple sets of dialogue data.

[0084] After obtaining the mutual information matrix and predicted responses corresponding to multiple sets of dialogue data in a batch, the generative language model can be updated based on the mutual information matrix and predicted responses. Specifically, the loss value can be calculated based on the mutual information matrix and predicted responses, and the generative language model can be updated based on the loss value.

[0085] Specifically, the process of updating the model can involve gradient descent, backpropagating the loss value to update the parameters of the entire generative language model. These parameters include those of the improved BERT model, such as the parameters between each hidden layer and the parameters between the hidden layer and the output layer (outputting the predicted response); they also include the parameters of the fully connected layer when generating the mutual information matrix. By updating the parameters, the trained model can reduce the predicted response of the generalized output response.

[0086] In practice, the model can first be trained on a large amount of training data. All training data can be divided into multiple batches. For each batch, the semantic representation of each group of dialogue data within that batch can be calculated, and then the mutual information matrix corresponding to that batch can be calculated. Based on the mutual information matrix and the predicted response, the loss value is determined, and the model is trained based on this loss value. After obtaining the trained model, the inference stage can begin. In this stage, the user input dialogue context can be received, and the trained model can generate and output the predicted response corresponding to the dialogue context.

[0087] The generative language model training method provided in this application involves inputting multiple sets of dialogue data into a generative language model. Based on the generative language model, it determines the semantic representations corresponding to the preceding and following dialogue texts in each set of dialogue data, as well as the predicted response for each set of dialogue data. A mutual information matrix is ​​determined based on the semantic representations corresponding to the multiple sets of dialogue data. Each value in the mutual information matrix represents the degree of correlation between the corresponding two semantic representations. The generative language model is updated based on the mutual information matrix and the predicted responses corresponding to the multiple sets of dialogue data. By updating the model based on the mutual information matrix, the mutual information between the predicted response output by the model and the semantic representation of the preceding dialogue text is increased, reducing the predicted response of a universal response and thus improving the accuracy of the generated response.

[0088] Optionally, the semantic representations of the preceding and following dialogue texts corresponding to all groups of dialogue data constitute a semantic representation set; based on the semantic representations corresponding to multiple groups of dialogue data, a mutual information matrix is ​​determined, including: for each semantic representation in the semantic representation set, calculating the inner product of the semantic representation with each other semantic representation in the semantic representation set except for the semantic representation; and determining the mutual information matrix corresponding to the multiple groups of dialogue data based on the calculated inner products.

[0089] In determining the mutual information matrix, a semantic representation set can be constructed first, placing the semantic representations of the preceding and following dialogue texts corresponding to all groups of dialogue data in a batch into a single set. When calculating the mutual information matrix, the inner product of one semantic representation with every other semantic representation in the set can be calculated first. For example... Figure 5As shown, for semantic representation 1, its inner product with semantic representation 2, semantic representation 3, and semantic representation 4 is calculated respectively. Here, a semantic representation is a multi-dimensional vector, and the inner product of two vectors reflects the correlation between them. After obtaining the inner product, the corresponding mutual information matrix needs to be calculated further.

[0090] Determining the mutual information matrix based on the inner product has the advantages of being computationally simple and accurate.

[0091] Optionally, the inner products corresponding to a semantic representation set form an inner product matrix; based on the calculated inner products, the mutual information matrix corresponding to the multiple sets of dialogue data is determined, including:

[0092] Normalize the inner product of each row or column in the inner product matrix; determine the mutual information matrix based on the result of the normalization operation corresponding to each row or column of data.

[0093] In this context, the inner products of multiple semantic representation sets corresponding to a batch can form an inner product matrix. When calculating the mutual information matrix, a normalization operation can be performed on each row or column of the inner product matrix. This normalization operation involves processing the data in each row or column using a softmax function, mapping each data point in each row or column to a value between 0 and 1, ensuring that the sum of all data points in each row or column is 1. The matrix resulting from this normalization operation can be considered the mutual information matrix.

[0094] By normalizing the values ​​in the inner product matrix, the convergence speed of the model can be improved, and the influence or contribution of the inner product of all semantic representations on the model parameters can be made relatively uniform, so as not to be affected by individual semantic representations, thereby improving the accuracy of the trained model.

[0095] Optionally, the generative language model is updated based on the mutual information matrix and the predicted responses corresponding to multiple sets of dialogue data, including:

[0096] A first loss value is determined based on the mutual information matrix and the corresponding reference mutual information matrix; a second loss value is determined based on the predicted response and the corresponding dialogue context; the sum of the first loss value and the second loss value is calculated, and the generative language model is updated based on the sum.

[0097] When determining the loss value based on the mutual information matrix and the reference mutual information matrix, two loss values ​​can be calculated. The first loss value is the difference between the actual determined mutual information matrix, the reference mutual information matrix, and the loss function. This process can reduce the likelihood of the model outputting a universal response.

[0098] Optionally, the inner product of the semantic representation of any conversation context and the corresponding conversation continuation in the reference mutual information matrix is greater than the inner product of the semantic representation of the conversation context and other semantic representations; the other semantic representations are semantic representations other than the semantic representation of the conversation continuation corresponding to the conversation context.

[0099] Figure 6 A provided by an embodiment of the present invention is related to Figure 5 Schematic diagram of the corresponding reference mutual information matrix, as Figure 6 shown, semantic representation 1 is the semantic representation of the conversation context, and semantic representation 2 is the semantic representation of the conversation continuation. Since semantic representation 1 and semantic representation 2 match, the corresponding inner product is larger; semantic representation 1 and the remaining semantic representations do not match, so the inner product of semantic representation 1 and each of the remaining semantic representations is smaller. That is, the similarity between two matching semantic representations is increased, and the similarity between two non-matching semantic representations is decreased.

[0100] Exemplarily, the mutual information between semantic representation 1 and semantic representation 2 in the reference mutual information matrix can be 1, and the mutual information between semantic representation 1 and semantic representation 3 or semantic representation 4 is 0.

[0101] The first loss function is determined by calculating the difference between the mutual information matrix and the corresponding reference mutual information matrix, and the generative language model is updated based on the first loss function, so that the first loss function becomes smaller and smaller during training, that is, the actually calculated mutual information matrix approaches the reference mutual information matrix, so that the model outputs a predicted response with a higher mutual information with the conversation context and reduces the predicted response that outputs a general reply.

[0102] In addition, when training the model, the model is also updated based on the second loss value determined by the predicted response and the conversation continuation. The second loss value is the difference determined according to the predicted response, the conversation continuation and the loss function. Exemplarily, when a set of input conversation data is "CLS What do you drink SEP Plain water", if the predicted response is "Beijing", then the loss value between "Beijing" and "Plain water" is calculated.

[0103] After calculating the first loss value and the second loss value, the generative language model is updated according to the sum of the two.

[0104] Optionally, the first loss value and the second loss value can be directly summed to obtain the sum result.

[0105] Optionally, the first weight corresponding to the first loss value and the second weight corresponding to the second loss value are determined, the first loss value is multiplied by the first weight to obtain the first multiplication result, the second loss value is multiplied by the second weight to obtain the second multiplication result, and the sum of the first multiplication result and the second multiplication result is determined as the sum result.

[0106] The first and second weights can be adjusted according to the training stage. For example, in the early stage of training, the second weight is set to be greater than the first weight so that the model can quickly output the predicted response corresponding to the dialogue context. In the later stage of training, the first weight is set to be greater than the second weight so that the model parameters can be further adjusted to reduce the possibility of the model outputting a general response.

[0107] The above method enables the model to output a response that corresponds to the context of the dialogue. Furthermore, it can reduce the likelihood of outputting a generic response and increase the similarity between the predicted response and the context of the dialogue.

[0108] Optionally, normalization operations are performed on the inner product of each row or each column in the inner product matrix, including:

[0109] Perform a masking operation on the inner product at the diagonal positions of the inner product matrix to obtain the inner product matrix after the masking operation; then perform a normalization operation on the inner product of each row or column of the inner product matrix after the masking operation.

[0110] To facilitate the calculation of the inner product, the semantic representations of multiple sets of dialogue data corresponding to a batch can be generated into two matrices. The first matrix is ​​of size M*N, where M represents the number of semantic representations and N represents the dimension of each semantic representation. The second matrix is ​​of size N*M. Multiplying the first and second matrices directly yields an M*M inner product matrix, which can improve the speed of calculating the inner product matrix.

[0111] However, the inner product matrix calculated using the above method contains an inner product of a semantic representation and itself. This inner product does not participate in the calculation of the mutual information matrix and cannot be approximated to 1 or 0; it needs to be masked. Specifically, the values ​​at the diagonal positions of the inner product matrix can be hidden by a masking operation, allowing for normalization based on the masked inner product matrix.

[0112] By using the above method, the accuracy of the determined mutual information matrix can be guaranteed while improving the speed of calculating the inner product matrix, thereby improving the accuracy of training the generative language model and the accuracy of the model's predicted response.

[0113] In practice, chatbot services targeting open domains or topics can be provided to users through web-based or voice-based dialogue systems. For web-based systems, users simply need to open a human-computer interaction page to chat with the chatbot. After receiving user input, the chatbot uses a generative language model deployed on a backend server to calculate the corresponding response. In multi-turn dialogue scenarios, for a particular turn of user input, previous turns of conversation can be incorporated as part of the preceding dialogue to accurately predict the response for that turn.

[0114] Furthermore, to verify the merits of the training method in this application, two sets of comparative models for the generation methods were set up: Seq2Seq-Att and Transfor. Transfor refers to the original Transformer architecture (the original BERT model), which includes 6 encoders and 6 decoders; Seq2Seq-Att adds an attention mechanism to the basic Seq2Seq framework, using a GRU (Gate Recurrent Unit) recurrent neural network as the base network for both encoders and decoders; the model in this application uses an improved BERT model. All three models have the same word embedding dimension, and all models have a training dimension of 10 epochs, with the same training size and learning rate. Using the above settings, each model was evaluated from three dimensions: quality, diversity, and fluency of the generated text. Quality and diversity were evaluated based on word overlap rate, while fluency was determined using language perplexity. The evaluation results are shown in the table below.

[0115] In the table, BLEU-2 and BLEU-4 represent the cumulative BLEU (Bilingual Evaluation Understudy) with two different weights; Dist-1 and Dist-2 represent the diversity determined based on the number of single-character and two-character categories in the generated responses, respectively.

[0116]

[0117]

[0118] As shown in the table above, the evaluation results of this application are improved in both response quality and diversity compared to the Transfor model, and also improved in both response quality and diversity compared to the Seq2Seq-Att model.

[0119] Figure 7 This is a schematic diagram of the structure of a generative language model training device provided in an embodiment of the present invention, as shown below. Figure 7 As shown, the device 70 includes:

[0120] Input module 701 is used to input multiple sets of dialogue data into the generative language model;

[0121] The first determining module 702 is used to determine, based on the generative language model, the semantic representations corresponding to the dialogue context and dialogue background in each set of dialogue data, as well as the predicted response corresponding to each set of dialogue data.

[0122] The second determining module 703 is used to determine a mutual information matrix based on the semantic representations corresponding to multiple sets of dialogue data; each value in the mutual information matrix is ​​used to represent the degree of correlation between the corresponding two semantic representations.

[0123] The update module 704 is used to update the generative language model based on the mutual information matrix and the predicted responses corresponding to multiple sets of dialogue data.

[0124] Optionally, the semantic representations of the preceding and following dialogue texts corresponding to all groups of dialogue data constitute a semantic representation set; when determining the mutual information matrix based on the semantic representations corresponding to multiple groups of dialogue data, the second determining module 703 is specifically used for:

[0125] For each semantic representation in the semantic representation set, calculate the inner product of that semantic representation with every other semantic representation in the semantic representation set except for that semantic representation;

[0126] Based on the calculated inner products, the mutual information matrix corresponding to the multiple sets of dialogue data is determined.

[0127] Optionally, the inner product corresponding to a semantic representation set forms an inner product matrix; when the second determining module 703 determines the mutual information matrix corresponding to the multiple sets of dialogue data based on the calculated inner products, it is specifically used for:

[0128] Normalize the inner product of each row or column in the inner product matrix respectively;

[0129] The mutual information matrix is ​​determined based on the results of the normalization operation corresponding to each row or column of data.

[0130] Optionally, when updating the generative language model based on the mutual information matrix and the predicted responses corresponding to multiple sets of dialogue data, the update module 704 is specifically used for:

[0131] The first loss value is determined based on the mutual information matrix and the corresponding reference mutual information matrix;

[0132] The second loss value is determined based on the predicted response and the corresponding dialogue context;

[0133] The sum of the first loss value and the second loss value is calculated, and the generative language model is updated based on the sum.

[0134] Optionally, the inner product of the semantic representation of any dialogue context and the corresponding semantic representation of the dialogue context in the reference mutual information matrix is ​​greater than the inner product of the semantic representation of the dialogue context and other semantic representations; the other semantic representations are semantic representations other than the semantic representation of the dialogue context corresponding to the dialogue context.

[0135] Optionally, based on the generative language model, the first determining module 702, when determining the semantic representations corresponding to the dialogue context and dialogue background in each set of dialogue data, is specifically used for:

[0136] For each set of dialogue data, a first input data and a second input data are generated based on the preceding and following dialogue text; wherein, the first input data is the data obtained by concatenating the following dialogue text after the preceding dialogue text; and the second input data is the data obtained by concatenating the preceding dialogue text after the preceding dialogue text.

[0137] The first and second input data corresponding to each set of dialogue data are respectively input into the generative language model;

[0138] The output of the hidden layer of the generative language model corresponding to the first input data is determined as the semantic representation of the preceding context of the dialogue; the output of the hidden layer of the generative language model corresponding to the second input data is determined as the semantic representation of the following context of the dialogue.

[0139] Optionally, when the second determining module 703 performs a normalization operation on the inner product of each row or each column in the inner product matrix, it is specifically used for:

[0140] Perform a masking operation on the inner product at the diagonal positions of the inner product matrix to obtain the inner product matrix after the masking operation;

[0141] Normalize the inner product of each row or column in the inner product matrix after the masking operation.

[0142] The generative language model training apparatus provided in this application embodiment can be used to perform the above-mentioned... Figures 1 to 6 The technical solutions of the embodiments shown are similar in principle and in effect, and will not be described again here.

[0143] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device in this embodiment may include:

[0144] At least one processor 801; and

[0145] Memory 802 communicatively connected to the at least one processor;

[0146] The memory 802 stores instructions that can be executed by the at least one processor 801, which, when executed by the at least one processor 801, cause the electronic device to perform the method as described in any of the above embodiments.

[0147] Alternatively, the memory 802 can be either standalone or integrated with the processor 801.

[0148] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the foregoing embodiments, and will not be repeated here.

[0149] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in any of the foregoing embodiments.

[0150] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the methods described in any of the foregoing embodiments.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0152] The integrated modules implemented as software functional modules described above can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application.

[0153] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor. The memory may include high-speed RAM, and may also include non-volatile memory (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk, or optical disc, etc.

[0154] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0155] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. Both the processor and the storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic device or host device.

[0156] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0157] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0158] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0159] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A generative language model training method, characterized in that, The method includes: Multiple sets of dialogue data are input into the generative language model; Based on the generative language model, the semantic representations corresponding to the dialogue context and dialogue background in each set of dialogue data are determined, as well as the predicted response corresponding to each set of dialogue data. Based on the semantic representations corresponding to multiple sets of dialogue data, a mutual information matrix is ​​determined; each value in the mutual information matrix is ​​used to represent the degree of correlation between the corresponding two semantic representations. The generative language model is updated based on the mutual information matrix and the predicted responses corresponding to multiple sets of dialogue data.

2. The method according to claim 1, characterized in that, The semantic representation set consists of the semantic representations of the preceding and following dialogues corresponding to all groups of dialogue data; Based on the semantic representations corresponding to multiple sets of dialogue data, a mutual information matrix is ​​determined, including: For each semantic representation in the semantic representation set, calculate the inner product of that semantic representation with every other semantic representation in the semantic representation set except for that semantic representation; Based on the calculated inner products, the mutual information matrix corresponding to the multiple sets of dialogue data is determined.

3. The method according to claim 2, characterized in that, The inner products corresponding to a semantic representation set form an inner product matrix; based on the calculated inner products, the mutual information matrix corresponding to the multiple sets of dialogue data is determined, including: Normalize the inner product of each row or column in the inner product matrix respectively; The mutual information matrix is ​​determined based on the results of the normalization operation corresponding to each row or column of data.

4. The method according to claim 1, characterized in that, The generative language model is updated based on the mutual information matrix and the predicted responses corresponding to multiple sets of dialogue data, including: The first loss value is determined based on the mutual information matrix and the corresponding reference mutual information matrix; The second loss value is determined based on the predicted response and the corresponding dialogue context; The sum of the first loss value and the second loss value is calculated, and the generative language model is updated based on the sum.

5. The method according to claim 4, characterized in that, The inner product of the semantic representation of any dialogue context and the corresponding semantic representation of the dialogue context in the reference mutual information matrix is ​​greater than the inner product of the semantic representation of the dialogue context and other semantic representations; the other semantic representations are semantic representations other than the semantic representation of the dialogue context corresponding to the dialogue context.

6. The method according to any one of claims 1-5, characterized in that, Based on the generative language model, the semantic representations corresponding to the preceding and following contexts of each dialogue data set are determined, including: For each set of dialogue data, a first input data and a second input data are generated based on the preceding and following dialogue text; wherein, the first input data is the data obtained by concatenating the following dialogue text after the preceding dialogue text; and the second input data is the data obtained by concatenating the preceding dialogue text after the preceding dialogue text. The first and second input data corresponding to each set of dialogue data are respectively input into the generative language model; The output of the hidden layer of the generative language model corresponding to the first input data is determined as the semantic representation of the preceding context of the dialogue; the output of the hidden layer of the generative language model corresponding to the second input data is determined as the semantic representation of the following context of the dialogue.

7. The method according to claim 3, characterized in that, Normalize the inner product of each row or column in the inner product matrix, including: Perform a masking operation on the inner product at the diagonal positions of the inner product matrix to obtain the inner product matrix after the masking operation; Normalize the inner product of each row or column in the inner product matrix after the masking operation.

8. A generative language model training device, characterized in that, The device includes: The input module is used to input multiple sets of dialogue data into the generative language model; The first determining module is used to determine, based on the generative language model, the semantic representations corresponding to the dialogue context and dialogue background in each set of dialogue data, as well as the predicted response corresponding to each set of dialogue data. The second determining module is used to determine the mutual information matrix based on the semantic representations corresponding to multiple sets of dialogue data; each value in the mutual information matrix is ​​used to represent the degree of correlation between the corresponding two semantic representations. The update module is used to update the generative language model based on the mutual information matrix and the predicted responses corresponding to multiple sets of dialogue data.

9. An electronic device, characterized in that, include: At least one processor and memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images