Methods, apparatus, devices, storage media, and products for generating dialogue text

Abstract

This application relates to the field of artificial intelligence technology, providing a method, apparatus, device, storage medium, and product for generating dialogue text. The method includes: acquiring input text during a dialogue; inputting the input text into a text generation model to obtain the dialogue text output by the text generation model; the text generation model generating the dialogue text based on a sentiment type weight matrix and a sentiment intensity weight matrix obtained through feature learning of the sentiment type and sentiment intensity labels corresponding to the sample input text and the sample dialogue text. Through this method, the text generation model can generate dialogue text with emotional content based on the sentiment type and sentiment intensity weight matrices, thereby accurately simulating real service scenarios, making the dialogue process in customer service training closer to reality, and improving the quality of customer service training.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a method, apparatus, device, storage medium, and product for generating dialogue text. Background Technology

[0002] Customer service training has always been a crucial part of corporate training because customer service personnel interact directly with customers, directly impacting customer satisfaction and corporate reputation. Currently, human-computer dialogue models have become an effective tool for training customer service personnel.

[0003] Existing human-computer dialogue models can automatically generate dialogue text based on the input text of customer service personnel, thereby simulating the dialogue process between customers and customer service personnel and realizing simulated training for customer service personnel.

[0004] However, in real-world service scenarios, customer conversations are often imbued with subjective emotions, and existing models are unable to generate dialogue texts that convey human emotions, making it difficult to accurately simulate real service scenarios and limiting the quality of customer service training. Summary of the Invention

[0005] This application provides a method, apparatus, device, storage medium, and product for generating dialogue text, in order to solve the technical problem that existing models cannot generate dialogue text with human emotions, making it difficult to accurately simulate real service scenarios and limiting the quality of customer service training.

[0006] In a first aspect, embodiments of this application provide a method for generating dialogue text, comprising: acquiring input text during a dialogue; inputting the input text into a text generation model to obtain dialogue text output by the text generation model; wherein the text generation model is trained based on sample input text, sample dialogue text corresponding to the sample input text, sentiment type labels and sentiment intensity labels corresponding to the sample input text, and sentiment type labels and sentiment intensity labels corresponding to the sample dialogue text; the text generation model generates dialogue text based on sentiment type weight matrices and sentiment intensity weight matrices obtained by feature learning of sentiment type labels and sentiment intensity labels corresponding to the sample input text and sentiment type labels and sentiment intensity labels corresponding to the sample dialogue text.

[0007] In one embodiment, the text generation model is a neural network model based on the Transformer network architecture, including a feature extraction network, a self-attention network, and a multi-task learning network. The feature extraction network extracts features from the input text, generating text vectors, sentiment type vectors, and sentiment intensity vectors corresponding to the input text. The self-attention network calculates attention scores on the text vectors, sentiment type vectors, and sentiment intensity vectors based on the sentiment type weight matrix and the sentiment intensity weight matrix, generating a sentiment score for the input text and generating multiple word vectors based on the sentiment score. The multi-task learning network performs multi-task learning representations on the multiple word vectors to generate dialogue text.

[0008] In one embodiment, inputting input text into a text generation model to obtain dialogue text output by the text generation model includes: inputting the input text into a feature extraction network of the text generation model to obtain text vectors, sentiment type vectors, and sentiment intensity vectors corresponding to the input text output by the feature extraction network; inputting the text vectors, sentiment type vectors, and sentiment intensity vectors into a self-attention network of the text generation model to obtain multiple word vectors output by the self-attention network; and inputting the multiple word vectors into a multi-task learning network of the text generation model to obtain dialogue text output by the multi-task learning network.

[0009] In one embodiment, before inputting the input text into the text generation model and obtaining the dialogue text output by the text generation model, the method further includes: acquiring sample input text, sample dialogue text corresponding to the sample input text, sentiment type label and sentiment intensity label corresponding to the sample input text, and sentiment type label and sentiment intensity label corresponding to the sample dialogue text; training the initial text generation model based on the sample input text, the sample dialogue text corresponding to the sample input text, the sentiment type label and sentiment intensity label corresponding to the sample input text, and the sentiment type label and sentiment intensity label corresponding to the sample dialogue text to obtain the text generation model; wherein, the sentiment type label includes positive label, neutral label, and negative label; the sentiment intensity label includes multiple sentiment intensity level labels.

[0010] In one embodiment, the text generation model is trained using a preset loss function. The preset loss function is calculated by weighting and aggregating the text generation loss and its corresponding first weight, the sentiment type loss and its corresponding second weight, and the sentiment intensity loss and its corresponding third weight.

[0011] In one embodiment, the text generation loss is determined based on the text vector similarity between the sample input text and the corresponding sample dialogue text, and the text vector similarity is determined based on the cross-entropy loss function; the sentiment type loss is determined based on the sentiment type vector similarity between the sample input text and the corresponding sample dialogue text, and the sentiment type vector similarity is determined based on the mean squared error loss function; the sentiment intensity loss is determined based on the sentiment intensity vector similarity between the sample input text and the corresponding sample dialogue text, and the sentiment intensity vector similarity is determined based on the mean squared error loss function.

[0012] Secondly, embodiments of this application provide a dialogue text generation apparatus, comprising: an acquisition module for acquiring input text during a dialogue process; and a generation module for inputting the input text into a text generation model to obtain dialogue text output by the text generation model; wherein the text generation model is trained based on sample input text, sample dialogue text corresponding to the sample input text, sentiment type labels and sentiment intensity labels corresponding to the sample input text, and sentiment type labels and sentiment intensity labels corresponding to the sample dialogue text; the text generation model generates dialogue text based on sentiment type weight matrices and sentiment intensity weight matrices obtained by feature learning of the sentiment type labels and sentiment intensity labels corresponding to the sample input text and the sentiment type labels and sentiment intensity labels corresponding to the sample dialogue text.

[0013] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the dialogue text generation method described above.

[0014] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the dialog text generation method as described above.

[0015] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the dialog text generation method as described above.

[0016] The methods, apparatus, devices, storage media, and products for generating dialogue text provided in this application embodiment acquire input text during a dialogue process; input the input text into a text generation model to obtain the dialogue text output by the text generation model; wherein, the text generation model is trained based on sample input text, sample dialogue text corresponding to the sample input text, sentiment type labels and sentiment intensity labels corresponding to the sample input text, and sentiment type labels and sentiment intensity labels corresponding to the sample dialogue text; the text generation model generates dialogue text based on sentiment type weight matrices and sentiment intensity weight matrices obtained by feature learning of the sentiment type labels and sentiment intensity labels corresponding to the sample input text and the sentiment type labels and sentiment intensity labels corresponding to the sample dialogue text. Through the above method, feature learning of sentiment type labels and sentiment intensity labels of text is introduced during the training process of the text generation model to obtain sentiment type weight matrices and sentiment intensity weight matrices, enabling the text generation model to generate dialogue text with emotions based on the sentiment type weight matrices and sentiment intensity weight matrices, thereby accurately simulating real service scenarios, making the dialogue process of customer service training closer to reality, and improving the quality of customer service training. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the method for generating dialogue text provided in an embodiment of this application;

[0019] Figure 2 This is a schematic diagram of the network architecture of the text generation model provided in the embodiments of this application;

[0020] Figure 3 This is a schematic diagram of the training process of the text generation model provided in the embodiments of this application;

[0021] Figure 4 This is a schematic diagram illustrating the composition of the combined loss function provided in the embodiments of this application;

[0022] Figure 5 This is a schematic diagram of the structure of the dialog text generation device provided in the embodiments of this application;

[0023] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] Please see Figure 1 , Figure 1 This is a flowchart illustrating the method for generating dialogue text provided in this application embodiment. In this embodiment, the method for generating dialogue text specifically includes steps S110 to S120, each step as follows:

[0026] S110: Obtain the input text during the dialogue process.

[0027] S120: Input the input text into the text generation model and obtain the dialogue text output by the text generation model.

[0028] The text generation model is trained based on the sample input text, the sample dialogue text corresponding to the sample input text, the sentiment type label and sentiment intensity label corresponding to the sample input text, and the sentiment type label and sentiment intensity label corresponding to the sample dialogue text.

[0029] The text generation model generates dialogue text based on the emotion type and emotion intensity labels corresponding to the sample input text and the emotion type and emotion intensity labels corresponding to the sample dialogue text, which are obtained through feature learning.

[0030] The dialogue text generation method provided in this application involves acquiring input text during a dialogue process; inputting the input text into a text generation model to obtain the dialogue text output by the text generation model; wherein, the text generation model is trained based on sample input text, sample dialogue text corresponding to the sample input text, sentiment type labels and sentiment intensity labels corresponding to the sample input text, and sentiment type labels and sentiment intensity labels corresponding to the sample dialogue text; the text generation model generates dialogue text based on sentiment type weight matrices and sentiment intensity weight matrices obtained by feature learning on the sentiment type labels and sentiment intensity labels corresponding to the sample input text and the sentiment type labels and sentiment intensity labels corresponding to the sample dialogue text. Through the above method, feature learning of sentiment type labels and sentiment intensity labels of text is introduced during the training process of the text generation model to obtain sentiment type weight matrices and sentiment intensity weight matrices, enabling the text generation model to generate dialogue text with emotions based on the sentiment type weight matrices and sentiment intensity weight matrices, thereby accurately simulating real service scenarios, making the dialogue process of customer service training closer to reality, and improving the quality of customer service training.

[0031] In some embodiments, the text generation model is a neural network model based on the Transformer network architecture, including a feature extraction network, a self-attention network, and a multi-task learning network. The feature extraction network extracts features from the input text to generate text vectors, sentiment type vectors, and sentiment intensity vectors corresponding to the input text. The self-attention network calculates attention scores on the text vectors, sentiment type vectors, and sentiment intensity vectors based on the sentiment type weight matrix and the sentiment intensity weight matrix, generating a sentiment score corresponding to the input text, and generating multiple word vectors based on the sentiment score. The multi-task learning network performs multi-task learning representations on the multiple word vectors to generate dialogue text.

[0032] Please see Figure 2 , Figure 2 This is a schematic diagram of the network architecture of the text generation model provided in the embodiments of this application.

[0033] In this embodiment, the text generation model is a neural network model based on the Transformer network architecture.

[0034] The core idea of ​​the Transformer network architecture is to overcome the gradient vanishing and gradient explosion problems of traditional sequence models when dealing with long sequences by introducing a self-attention mechanism.

[0035] Thanks to the use of self-attention mechanisms, the Transformer network architecture is highly parallelizable. It can capture the correlation patterns between different features by calculating attention scores in parallel on different features, and uses fully connected layers to perform linear transformations and provide the final output. The Transformer network architecture performs exceptionally well in the field of natural language processing.

[0036] like Figure 2 As shown, the text generation model includes a feature extraction network, a self-attention network, and a multi-task learning network.

[0037] The feature extraction network can be understood as the feature extraction module of the text generation model, which is used for feature learning and feature extraction.

[0038] Specifically, the feature extraction network is used to extract features from the input text, generating text vectors, sentiment type vectors, and sentiment intensity vectors corresponding to the input text. This transforms the text sequence corresponding to the input text, the sentiment type corresponding to the text sequence, and the sentiment intensity corresponding to the sentiment type into vectorized representation codes, which facilitates subsequent attention score calculation for different feature vectors through a self-attention mechanism.

[0039] Self-attention networks and multi-task learning networks can be understood as generative multi-task learning modules in text generation models. They are used to calculate attention scores on different features in parallel, capture the correlation patterns between different features, and achieve multi-feature learning representations to generate the final dialogue text.

[0040] Specifically, the self-attention network is used to calculate attention scores on text vectors, sentiment type vectors, and sentiment intensity vectors based on sentiment type weight matrices and sentiment intensity weight matrices, generate sentiment scores corresponding to the input text, and generate multiple word vectors based on the sentiment scores.

[0041] Self-attention networks consist of multiple Transformer layers, each of which generates a word vector.

[0042] Specifically, in each Transformer layer, attention scores can be calculated on the text vector, sentiment type vector, and sentiment intensity vector based on the sentiment type weight matrix and the sentiment intensity weight matrix to generate a sentiment score corresponding to the input text and generate an initial word embedding vector. The sentiment score is added to the initial word embedding vector to generate a word vector with sentiment connotation. The word vector with sentiment connotation is input into the next Transformer layer, and in the next Transformer layer, the sentiment score is added to the word vector with sentiment connotation again to generate the next word vector, realizing the prediction of the next generated word. After multiple Transformer layers, multiple word vectors can be obtained.

[0043] Since self-attention networks can incorporate the sentiment type and sentiment intensity information of text into the calculation of attention scores, and generate multiple word vectors based on the sentiment type weight matrix and sentiment intensity weight matrix, thus realizing the adjustment of the sentiment color of word vectors, self-attention networks can be understood as the sentiment attention module of text generation models.

[0044] Multi-task learning networks are used to learn representations from multiple word vectors in a multi-task manner to generate dialogue text.

[0045] like Figure 2 As shown, to enhance the expressive power of the text generation model, the multi-task learning network adopts a multi-connection layer design.

[0046] Specifically, the multi-task learning network (i.e., multi-task classifier) ​​includes a text generation fully connected layer, a sentiment type fully connected layer, and a sentiment intensity fully connected layer, which are used to perform multi-task learning representations, fit the information features between different word vectors, and transform different word vectors into readable dialogue text.

[0047] The dialogue text generation method provided in this application uses a Transformer network architecture to design a text generation model, which avoids the error propagation problem caused by multi-level model architecture. By improving the existing Transformer network architecture, sentiment type weight matrix and sentiment intensity weight matrix are introduced into the self-attention network as control factors for the sentiment type and sentiment intensity of the generated word vectors. This achieves end-to-end control of the sentiment of the model's output text, enabling the model to generate text with sentiment bias. This more accurately simulates the dialogue process between real users and customer service personnel, making the customer service training process closer to real-world scenarios.

[0048] In some embodiments, inputting input text into a text generation model to obtain dialogue text output by the text generation model includes: inputting the input text into a feature extraction network of the text generation model to obtain text vectors, sentiment type vectors, and sentiment intensity vectors corresponding to the input text output by the feature extraction network; inputting the text vectors, sentiment type vectors, and sentiment intensity vectors into a self-attention network of the text generation model to obtain multiple word vectors output by the self-attention network; and inputting the multiple word vectors into a multi-task learning network of the text generation model to obtain dialogue text output by the multi-task learning network.

[0049] Specifically, the input text is fed into the feature extraction network of the text generation model to obtain the text vector, sentiment type vector, and sentiment intensity vector corresponding to the input text output by the feature extraction network.

[0050] Suppose the input text is denoted as text sequence X, and the sentiment type corresponding to text sequence X is denoted as X. EmotionType The intensity of the emotion corresponding to this emotion type is denoted as X. EmotionIntensity Then, after inputting the input text into the feature extraction network of the text generation model, the text vector, sentiment type vector, and sentiment intensity vector corresponding to the input text output by the feature extraction network can be obtained.

[0051] The generation process of the text vector InputTextEmbed can be represented as follows:

[0052] InputTextEmbed=Embedding(X);

[0053] The generation process of the emotion type vector EmotionTypeEmbed can be represented as follows:

[0054] EmotionTypeEmbed = Embeding(X EmotionType );

[0055] The generation process of the emotion intensity vector EmotionIntensityEmbed can be represented as follows:

[0056] EmotionIntensityEmbed=Embeding(X EmotionIntensity );

[0057] Here, Embedding represents the embedding vector computation function.

[0058] like Figure 2 As shown, since Embedding is an embedding vector calculation function, the text vector can also be denoted as the input text embedding vector, the sentiment type vector can also be denoted as the sentiment type embedding vector, and the sentiment intensity vector can also be denoted as the sentiment intensity embedding vector.

[0059] Furthermore, the text vector, sentiment type vector, and sentiment intensity vector are input into the self-attention network of the text generation model to obtain multiple word vectors output by the self-attention network.

[0060] Based on the Transformer layer of the self-attention network, emotion type and emotion intensity factors are introduced and incorporated into the attention score calculation. Emotion type weight matrix and emotion intensity weight matrix are then used.

[0061] Among them, the sentiment type weight matrix E can be used as an additive factor, added to the original attention key matrix of the Transformer layer, to increase the original attention key matrix's capture of sentiment type information; the sentiment intensity weight matrix I can be used as a multiplicative factor, multiplied with the original attention key matrix of the Transformer layer, to control the scaling of the generated text with respect to sentiment intensity.

[0062] Specifically, in each Transformer layer, attention scores can be calculated on the text vector, sentiment type vector, and sentiment intensity vector based on the sentiment type weight matrix and the sentiment intensity weight matrix to generate a sentiment score corresponding to the input text and generate an initial word embedding vector. The sentiment score is added to the initial word embedding vector to generate a word vector with sentiment connotation. The word vector with sentiment connotation is input into the next Transformer layer, and in the next Transformer layer, the sentiment score is added to the word vector with sentiment connotation again to generate the next word vector, realizing the prediction of the next generated word. After multiple Transformer layers, multiple word vectors can be obtained.

[0063] Therefore, the process of generating word vectors for the emotion attention module (i.e., the self-attention network) can be represented as:

[0064] Attention(Q,K,V,E,I)=softmax(score(Q,K,E,I))·V;

[0065]

[0066] Where score(Q,K,E,I) represents the sentiment score calculation function; softmax is the activation function; E is the sentiment type weight matrix; I is the sentiment intensity weight matrix; Q, K, and V are the query matrix, key matrix, and value matrix of the existing Transformer network architecture, respectively.

[0067] The formulas for calculating Q, K, V, E, and I are as follows:

[0068] Q = InputTextEmbed·W Q +b Q ;

[0069] K = InputTextEmbed·W K +b K ;

[0070] V = InputTextEmbed·W V +b V ;

[0071] E = EmotionTypeEmbed·W E +b E ;

[0072] I = EmotionIntensityEmbed·W I +b I ;

[0073] Among them, W Q W K W V W E W I The text generation model learns multiple weight matrices acquired during model training; b Q b K b V b E b I The corresponding weight matrix W Q W K W V W E W I The bias term.

[0074] Furthermore, multiple word vectors are input into the multi-task learning network of the text generation model to obtain the dialogue text output by the multi-task learning network.

[0075] The dialogue text generation method provided in this application allows input text to be fed into a text generation model, which in turn generates dialogue text with emotional bias. This more accurately simulates the dialogue process between real users and customer service personnel, making the customer service training process closer to real-world scenarios.

[0076] In some embodiments, before inputting the input text into the text generation model and obtaining the dialogue text output by the text generation model, the method further includes: acquiring sample input text, sample dialogue text corresponding to the sample input text, sentiment type label and sentiment intensity label corresponding to the sample input text, and sentiment type label and sentiment intensity label corresponding to the sample dialogue text; training the initial text generation model based on the sample input text, the sample dialogue text corresponding to the sample input text, the sentiment type label and sentiment intensity label corresponding to the sample input text, and the sentiment type label and sentiment intensity label corresponding to the sample dialogue text to obtain the text generation model; wherein, the sentiment type label includes positive label, neutral label, and negative label; the sentiment intensity label includes multiple sentiment intensity level labels.

[0077] Please see Figure 3 , Figure 3 This is a schematic diagram of the training process of the text generation model provided in the embodiments of this application.

[0078] like Figure 3 As shown, before training the text generation model, it is necessary to obtain the sample input text, the sample dialogue text corresponding to the sample input text, the sentiment type label and sentiment intensity label corresponding to the sample input text, and the sentiment type label and sentiment intensity label corresponding to the sample dialogue text.

[0079] Since there are no refined evaluation standards or quantitative standards for sentiment expression in text in the current technology, it is necessary to improve the existing training data by introducing sentiment type labels and sentiment intensity labels into the existing training data, and to classify and quantify the sentiment type and sentiment intensity corresponding to the input text of the sample, so as to improve the model's ability to understand and express complex emotional states.

[0080] Specifically, each sample input text is first labeled with its corresponding sentiment type and sentiment intensity labels.

[0081] The sentiment type labels include positive labels (E_POS), neutral labels (E_NEU), and negative labels (E_NEG).

[0082] Optionally, each emotion intensity label may include multiple emotion intensity level labels.

[0083] For example, each emotion intensity label can include 5 emotion intensity level labels, which are labeled as level 1, level 2, level 3, level 4, and level 5 respectively. The higher the emotion intensity level, the more intense the emotion. The emotion type corresponding to the sample input text labeled as positive is positive, and the intensity level of positive emotion can be divided into 5 levels.

[0084] Preferably, a positive label corresponds to multiple sentiment intensity level labels; a negative label corresponds to multiple sentiment intensity level labels; a neutral label corresponds to only one sentiment intensity level label, and the sentiment intensity level label is always 0, that is, the sample input text labeled as neutral does not contain obvious sentiment tendency.

[0085] Since sentiment type and sentiment intensity are relatively subjective concepts, different annotators may have different understandings of the sentiment in the same text. Therefore, the sentiment type and sentiment intensity labels corresponding to the sample input text can be labeled according to the following labeling rules to reduce the subjectivity of the labeling:

[0086] Annotation rule 1: For each sample input text, multiple annotators will annotate it separately, and the majority of the annotation results will be used to reduce the subjectivity of the annotation.

[0087] Preferably, the number of people labeled is an odd number of no less than 3.

[0088] Labeling rule 2: For each sample input text, select the sentiment type that is most frequently labeled by the labelers as the sentiment type label corresponding to that sample input text.

[0089] Labeling rule 3: For each sample input text, after determining the sentiment type label corresponding to the sample input text, determine the labeling results of multiple sentiment intensity levels corresponding to the sentiment type label, calculate the average value of all sentiment intensity level labeling results and round down to obtain the sentiment intensity label corresponding to the sample input text.

[0090] After labeling each input text with its corresponding sentiment type and sentiment intensity labels, the output text corresponding to each input text is rewritten, expanded, and labeled according to the expected sentiment type and sentiment intensity labels to generate sample dialogue texts with sentiment type and sentiment intensity labels, thereby constructing a training dataset labeled with sentiment type and sentiment intensity labels.

[0091] The training dataset includes sample input text, sample dialogue text corresponding to the sample input text, sentiment type labels and sentiment intensity labels corresponding to the sample input text, and sentiment type labels and sentiment intensity labels corresponding to the sample dialogue text.

[0092] Furthermore, based on the training dataset, the initial text generation model is iteratively trained until it converges to the expected effect, thus obtaining the text generation model.

[0093] The dialogue text generation method provided in this application proposes a refined emotional expression and quantification standard, which realizes the accurate expression of emotions and enhances the model's understanding and control of text emotions when processing and generating text. This enables the model to understand and express complex emotional states, solving the problem of insufficient emotional expression in generated text. In addition, the method classifies emotional intensity and provides a definition of the judgment standard, making the judgment of emotional intensity easy to quantify and calculate. It can control the emotions of generated content at the model parameter level, realizing personalized, dynamic, and parameter-level emotional tendency intervention.

[0094] In some embodiments, the text generation model is trained using a preset loss function; the preset loss function is obtained by weighted aggregation of the text generation loss and its corresponding first weight, the sentiment type loss and its corresponding second weight, and the sentiment intensity loss and its corresponding third weight.

[0095] In this embodiment, the text generation model is trained using a preset loss function L.

[0096] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating the composition of the combined loss function provided in this embodiment. In this embodiment, the preset loss function L is a combined loss function, which includes text generation loss, sentiment type loss, and sentiment intensity loss.

[0097] It should be noted that, assuming the preset loss function is denoted as L, after introducing emotion type and emotion intensity factors into the emotion attention module (i.e., self-attention network), the gradient output by the emotion attention module can be expressed as:

[0098]

[0099] in, This represents the gradient of the predefined loss function L with respect to the output of the attention mechanism; This represents the gradient of the preset loss function L with respect to the value matrix V.

[0100] Similarly, the gradient formula for the query matrix Q can be expressed as:

[0101]

[0102] in, It needs to be calculated according to the score formula.

[0103] Substitute into the score formula (i.e.) If the gradient formula for the query matrix Q is given by:

[0104]

[0105] Similarly, the gradient formula for the bond matrix K can be expressed as:

[0106]

[0107] Similarly, It needs to be calculated according to the score formula.

[0108] Substituting into the score formula, the gradient formula for the key matrix K can be expressed as:

[0109]

[0110] Similarly, the gradient formula for the value matrix V can be expressed as:

[0111]

[0112] Similarly, It needs to be calculated according to the score formula.

[0113] Similarly, the gradient formula for the sentiment type vector can be expressed as:

[0114]

[0115] Similarly, It needs to be calculated according to the score formula.

[0116] Substituting into the score formula, the gradient formula for the sentiment type vector can be expressed as:

[0117]

[0118] Similarly, the gradient formula for the emotion intensity vector can be expressed as:

[0119]

[0120] Similarly, It needs to be calculated according to the score formula.

[0121] Substituting into the score formula, the gradient formula for the sentiment intensity vector can be expressed as:

[0122]

[0123] It should be noted that for each model using the Transformer network architecture, there are three important weight matrices: the query matrix Q, the key matrix K, and the value matrix V. To further improve the model's performance, a multi-head attention mechanism can be introduced into the Transformer network architecture.

[0124] Multi-head attention mechanisms learn different contextual representations by applying self-attention mechanisms to multiple different sets of query matrices Q, key matrices K, and value matrices V. Specifically, the input sequence is transformed into multiple different sets of query matrices Q, key matrices K, and value matrices V through different linear transformations, and then these are fed into multiple parallel self-attention mechanisms for processing.

[0125] Therefore, when using a multi-head (h) attention mechanism, the function is expressed as follows:

[0126] MultiHead(Q,K,V,E,I)=Concat(head1,...,head h W O ;

[0127] head=Attention(Q,K,V,E,I);

[0128] Among them, W O represents the model parameters; Concat is the concatenation function for multi-head attention mechanisms, used to concatenate different self-attention mechanisms to obtain the output value of multi-head self-attention.

[0129] Preferably, the text generation fully connected layer includes a feedforward layer and a fully connected layer, the sentiment type fully connected layer includes a feedforward layer and a fully connected layer, and the sentiment intensity fully connected layer includes a feedforward layer and a fully connected layer; wherein, the ReLU nonlinear transformation activation function is used as the feedforward layer and the fully connected layer.

[0130] The feedforward layer can be represented by the following formula:

[0131] FFN(x)=max(0,xW1+b1)W2+b2;

[0132] Where x represents the input of the feedforward layer; max is a function; and W1, W2, b1, and b2 are model parameters.

[0133] Alternatively, assuming the fully connected layer uses a conventional softmax function to implement multi-task classification, and the notation of the fully connected layer is denoted by LastLayerOutput, then the output of the fully connected layer in the model can be expressed by the following formula:

[0134] Output=LastLayerOutput·W0+b0;

[0135] Where W0 and b0 are model parameters.

[0136] Preferably, to alleviate the problems of internal covariate shift, gradient vanishing, and training convergence difficulties in deep neural network training, layer normalization and residual connections can be introduced into the multi-task learning network of the text generation model. After introducing layer normalization and residual connections, the output of the multi-task learning network can be expressed by the following formula:

[0137] Output=LayerNorm(x+Sublayer(x));

[0138] Where Sublayer(x) represents the output of any sublayer in the multi-task learning network, and x represents the input of that layer.

[0139] Specifically, the preset loss function is obtained by weighted aggregation of text generation loss and its corresponding first weight, sentiment type loss and its corresponding second weight, and sentiment intensity loss and its corresponding third weight.

[0140] like Figure 2 As shown, during model training, the fully connected layer for text generation, the fully connected layer for sentiment type, and the fully connected layer for sentiment intensity are used to output the text generation loss L, respectively. text Emotional type loss L type Emotional intensity loss L intensity These are combined to form a preset loss function L, and the formula for calculating the preset loss function L is as follows:

[0141] L=αL text +βL type +γL intensity ;

[0142] Where α is the first weight corresponding to the text generation loss; β is the second weight corresponding to the sentiment type loss; and γ is the third weight corresponding to the sentiment intensity loss.

[0143] The dialogue text generation method provided in this application has designed a combined loss function for the text generation model, which comprehensively considers text generation loss, sentiment type loss and sentiment intensity loss, and balances the influence of different losses through different weight coefficients. This allows the preset loss function to more comprehensively measure and guide the learning and training process of the text generation model, and improve the model's performance and efficiency in generating text that meets specific sentiment requirements.

[0144] In some embodiments, the text generation loss is determined based on the text vector similarity between the sample input text and the corresponding sample dialogue text, and the text vector similarity is determined based on the cross-entropy loss function.

[0145] Specifically, a portion of the training data in the training dataset can be used as a validation set.

[0146] The model inputs the sample input text, the corresponding sentiment type label, and the sentiment intensity label from the validation set. The text generation loss is determined by comparing the text vector similarity between the model-generated text and the sample dialogue text corresponding to the sample input text in the validation set.

[0147] Text vector similarity is determined based on the cross-entropy loss function, and the text generation loss L... text The calculation formula is as follows:

[0148] L text =-∑ i target i log(output i );

[0149] Among them, target i Let i be the true probability distribution of the i-th tag in the sample dialogue text, and output i = true probability distribution of the sample dialogue text. i It is the predicted probability distribution of the i-th tag in the text generated by the model.

[0150] The sentiment type loss is determined based on the similarity of sentiment type vectors between the sample input text and the corresponding sample dialogue text. The sentiment type vector similarity is determined based on the mean squared error loss function.

[0151] The model inputs the sample input text, the corresponding sentiment type label, and the sentiment intensity label from the validation set. The sentiment type loss is determined by comparing the sentiment type vector similarity between the model-generated text and the sample dialogue text corresponding to the sample input text in the validation set.

[0152] Sentiment type vector similarity is determined based on the Mean Squared Error Loss function, where the sentiment type loss L... type The calculation formula is as follows:

[0153]

[0154] Among them, target_type i The output_type is the i-th element of the sentiment type vector of the sample dialogue text. i It is the i-th element of the sentiment type vector generated by the model, and N is the dimension of the sentiment type vector.

[0155] The sentiment intensity loss is determined based on the similarity of the sentiment intensity vectors between the sample input text and the corresponding sample dialogue text. The sentiment intensity vector similarity is determined based on the mean squared error loss function.

[0156] The model inputs sample text, the corresponding sentiment type label, and the sentiment intensity label from the validation set are used as inputs. The sentiment type loss is determined by comparing the sentiment intensity vector similarity between the text generated by the model and the sample dialogue text corresponding to the sample input text in the validation set.

[0157] The similarity of sentiment intensity vectors is determined based on the Mean Squared Error Loss function, where the sentiment intensity loss L... intensity The calculation formula is as follows:

[0158]

[0159] Among them, target_intensity j The output_intensity is the j-th element of the sentiment intensity vector of the sample dialogue text. j It is the j-th element of the emotion intensity vector generated by the model, and M is the dimension of the emotion intensity vector.

[0160] The dialogue text generation method provided in this application calculates text generation loss, sentiment type loss, and sentiment intensity loss using different functions to form a combined loss function. This combined loss function comprehensively considers the impact of text generation loss, sentiment type loss, and sentiment intensity loss, thereby enabling the combined loss function to more comprehensively measure and guide the learning and training process of the text generation model, and improve the model's performance and efficiency in generating text that meets specific sentiment requirements.

[0161] This application also provides an apparatus for generating dialogue text. Please refer to... Figure 5 , Figure 5 This is a schematic diagram of the structure of the dialog text generation device provided in this application embodiment. In this embodiment, the dialog text generation device includes an acquisition module 510 and a generation module 520.

[0162] The acquisition module 510 is used to acquire the input text during the dialogue process.

[0163] The generation module 520 is used to input the input text into the text generation model and obtain the dialogue text output by the text generation model.

[0164] The text generation model is trained based on the sample input text, the sample dialogue text corresponding to the sample input text, the sentiment type label and sentiment intensity label corresponding to the sample input text, and the sentiment type label and sentiment intensity label corresponding to the sample dialogue text.

[0165] The text generation model generates dialogue text based on the emotion type and emotion intensity labels corresponding to the sample input text and the emotion type and emotion intensity labels corresponding to the sample dialogue text, which are obtained through feature learning.

[0166] In some embodiments, the text generation model is a neural network model based on the Transformer network architecture, including a feature extraction network, a self-attention network, and a multi-task learning network. The feature extraction network extracts features from the input text to generate text vectors, sentiment type vectors, and sentiment intensity vectors corresponding to the input text. The self-attention network calculates attention scores on the text vectors, sentiment type vectors, and sentiment intensity vectors based on the sentiment type weight matrix and the sentiment intensity weight matrix, generating a sentiment score for the text and generating multiple word vectors based on the sentiment score. The multi-task learning network performs multi-task learning representations on the multiple word vectors to generate dialogue text.

[0167] In some embodiments, the generation module 520 is configured to input the input text into the feature extraction network of the text generation model to obtain the text vector, sentiment type vector, and sentiment intensity vector corresponding to the input text output by the feature extraction network; input the text vector, sentiment type vector, and sentiment intensity vector into the self-attention network of the text generation model to obtain multiple word vectors output by the self-attention network; and input the multiple word vectors into the multi-task learning network of the text generation model to obtain the dialogue text output by the multi-task learning network.

[0168] In some embodiments, the generation module 520 is used to acquire sample input text, sample dialogue text corresponding to the sample input text, sentiment type label and sentiment intensity label corresponding to the sample input text, and sentiment type label and sentiment intensity label corresponding to the sample dialogue text; and to train an initial text generation model based on the sample input text, the sample dialogue text corresponding to the sample input text, the sentiment type label and sentiment intensity label corresponding to the sample input text, and the sentiment type label and sentiment intensity label corresponding to the sample dialogue text to obtain a text generation model; wherein, the sentiment type label includes positive label, neutral label, and negative label; and the sentiment intensity label includes multiple sentiment intensity level labels.

[0169] In some embodiments, the text generation model is trained using a preset loss function; the preset loss function is obtained by weighted aggregation of the text generation loss and its corresponding first weight, the sentiment type loss and its corresponding second weight, and the sentiment intensity loss and its corresponding third weight.

[0170] In some embodiments, the text generation loss is determined based on the text vector similarity between the sample input text and the corresponding sample dialogue text, and the text vector similarity is determined based on the cross-entropy loss function; the sentiment type loss is determined based on the sentiment type vector similarity between the sample input text and the corresponding sample dialogue text, and the sentiment type vector similarity is determined based on the mean squared error loss function; the sentiment intensity loss is determined based on the sentiment intensity vector similarity between the sample input text and the corresponding sample dialogue text, and the sentiment intensity vector similarity is determined based on the mean squared error loss function.

[0171] This application also provides an electronic device. Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 6 As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a method for generating dialogue text.

[0172] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0173] This application also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the dialog text generation method provided by the above methods.

[0174] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the dialog text generation method described above.

[0175] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0176] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for generating a dialogue text, characterized by, The method comprises the following steps: obtaining input text in a dialogue process; inputting the input text into a text generation model to obtain dialogue text output by the text generation model; wherein the text generation model is trained based on sample input text, sample dialogue text corresponding to the sample input text, emotion type labels and emotion intensity labels corresponding to the sample input text, emotion type labels and emotion intensity labels corresponding to the sample dialogue text; the text generation model generates the dialogue text based on emotion type weight matrices and emotion intensity weight matrices obtained by feature learning on the emotion type labels and emotion intensity labels corresponding to the sample input text and the emotion type labels and emotion intensity labels corresponding to the sample dialogue text; the text generation model is a neural network model based on a Transformer network architecture, comprising a feature extraction network, a self-attention network and a multi-task learning network; wherein the feature extraction network is used for feature extraction of the input text to generate a text vector, an emotion type vector and an emotion intensity vector corresponding to the input text; the self-attention network is used for attention score calculation of the text vector, the emotion type vector and the emotion intensity vector based on the emotion type weight matrices and the emotion intensity weight matrices to generate an emotion score corresponding to the input text, and generate a plurality of word vectors based on the emotion score; the multi-task learning network is used for multi-task learning representation of the plurality of word vectors to generate the dialogue text; the inputting of the input text into the text generation model to obtain the dialogue text output by the text generation model comprises: inputting the input text into the feature extraction network of the text generation model to obtain the text vector, the emotion type vector and the emotion intensity vector corresponding to the input text output by the feature extraction network; inputting the text vector, the emotion type vector and the emotion intensity vector into the self-attention network of the text generation model to obtain a plurality of word vectors output by the self-attention network; inputting the plurality of word vectors into the multi-task learning network of the text generation model to obtain the dialogue text output by the multi-task learning network.

2. The method of claim 1, wherein, Before the inputting of the input text into the text generation model to obtain the dialogue text output by the text generation model, the method further comprises the following steps: obtaining sample input text, sample dialogue text corresponding to the sample input text, emotion type labels and emotion intensity labels corresponding to the sample input text, and emotion type labels and emotion intensity labels corresponding to the sample dialogue text; training an initial text generation model based on the sample input text, the sample dialogue text corresponding to the sample input text, the emotion type labels and the emotion intensity labels corresponding to the sample input text, and the emotion type labels and the emotion intensity labels corresponding to the sample dialogue text to obtain the text generation model; wherein the emotion type labels comprise positive labels, neutral labels and negative labels; the emotion intensity labels comprise a plurality of emotion intensity level labels.

3. The method of claim 1, wherein, The text generation model is trained by using a preset loss function. The preset loss function is obtained by weighted aggregation calculation of a text generation loss and a first weight corresponding to the text generation loss, an emotion type loss and a second weight corresponding to the emotion type loss, and an emotion intensity loss and a third weight corresponding to the emotion intensity loss.

4. The method of claim 3, wherein The text generation loss is determined based on a text vector similarity between the sample input text and the sample dialogue text corresponding to the sample input text, and the text vector similarity is determined based on a cross-entropy loss function. The emotion type loss is determined based on an emotion type vector similarity between the sample input text and the sample dialogue text corresponding to the sample input text, and the emotion type vector similarity is determined based on a mean square error loss function. The emotion intensity loss is determined based on an emotion intensity vector similarity between the sample input text and the sample dialogue text corresponding to the sample input text, and the emotion intensity vector similarity is determined based on a mean square error loss function.

5. An apparatus for generating a dialogue text, characterized by The method comprises: an acquisition module configured to acquire an input text in a dialogue process; a generation module configured to input the input text into a text generation model to obtain a dialogue text output by the text generation model; The text generation model is trained based on sample input texts, sample dialogue texts corresponding to the sample input texts, emotion type labels and emotion intensity labels corresponding to the sample input texts, and emotion type labels and emotion intensity labels corresponding to the sample dialogue texts; The text generation model generates the dialogue text based on an emotion type weight matrix and an emotion intensity weight matrix obtained by feature learning on the emotion type labels and emotion intensity labels corresponding to the sample input texts and the emotion type labels and emotion intensity labels corresponding to the sample dialogue texts; The text generation model is a neural network model based on a Transformer network architecture, comprising a feature extraction network, a self-attention network, and a multi-task learning network. The feature extraction network is configured to perform feature extraction on the input text to generate a text vector, an emotion type vector, and an emotion intensity vector corresponding to the input text. The self-attention network is configured to perform attention score calculation on the text vector, the emotion type vector, and the emotion intensity vector based on the emotion type weight matrix and the emotion intensity weight matrix to generate an emotion score corresponding to the input text, and generate a plurality of word vectors based on the emotion score. The multi-task learning network is configured to perform multi-task learning representation on the plurality of word vectors to generate the dialogue text. The generation module is configured to input the input text into a feature extraction network of the text generation model, obtain a text vector, a sentiment type vector and a sentiment intensity vector corresponding to the input text output by the feature extraction network, input the text vector, the sentiment type vector and the sentiment intensity vector into a self-attention network of the text generation model, obtain a plurality of word vectors output by the self-attention network, and input the plurality of word vectors into a multi-task learning network of the text generation model, and obtain a dialogue text output by the multi-task learning network.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the dialogue text generation method according to any one of claims 1 to 4 when executing the program.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the dialogue text generation method according to any one of claims 1 to 4.

8. A computer program product comprising a computer program, characterized in that, The computer program, when executed by the processor, implements the dialogue text generation method according to any one of claims 1 to 4. The computer program, when executed by the processor, implements the dialogue text generation method according to any one of claims 1 to 4.

Images