A method for generating conversation content in combination with a hybrid model

Abstract

The application discloses a conversation content generation method combined with a hybrid model, and belongs to the technical field of open-domain multi-turn dialogue. The method first acquires a conversation corpus, which is used as a candidate corpus; then acquires a training data set and performs preprocessing; fine-grained context construction is performed on the training data, concept entities in the training data are searched, and neighbor entities of the concept entity set are used as auxiliary embedding; the structure of a conversation search model is improved, and the conversation context and the reply in the training data set are encoded respectively to obtain a search matching score; a Chinese pre-training language model is used to encode the original conversation context and the reply of the training data set and neighbor concept entity information respectively to obtain a text context vector; and the text context vector is decoded to generate a corresponding conversation reply. The application improves the relevance between the generated conversation reply and the conversation context, and improves the semantic richness and quality of the generated reply.

Description

Technical Field

[0001] This invention relates to the field of open-domain multi-turn dialogue, and more particularly to a method for generating conversation content that combines a hybrid model. Background Technology

[0002] In recent years, with the development of computer technology and artificial intelligence, a large amount of available conversational data and conversational AI neural methods have flourished, making the development of intelligent conversational systems a major trend and field. Conversational systems primarily focus on conversational interaction between humans and intelligence. As early as the 1950s, the Turing Test used experiments similar to human-computer dialogue to determine the level of machine intelligence, opening a new door for researchers to explore human-intelligent conversations. Due to the rapid development of artificial intelligence and deep learning model technologies, combined with the powerful computing capabilities of hardware and the massive amounts of data brought by the internet age, a number of practical domain-specific products have emerged in the field of intelligent conversational systems. With the rapid development of internet technology and humanity's fervent pursuit of high-tech products, intelligent conversational systems have become an indispensable tool in people's lives.

[0003] Currently, most conversation content generation models are either retrieval-based or generation-based. However, retrieval-based methods are limited by the size of the retrieval corpus and cannot generate new response statements; generation-based methods are limited by common problems in the field of text generation, making it difficult to generate high-quality response statements. Because conversation systems based on retrieval and generation methods each have their own drawbacks and limitations, some studies have attempted to integrate retrieval and generation methods to build hybrid conversation systems. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a session content generation method that combines a hybrid model, which helps to solve the problems of relevance errors and poor quality of generated response statements in current session content generation.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A method for generating session content using a hybrid model, comprising:

[0007] S1: Obtain the training corpus and use natural language processing tools to parse and process the training dataset to obtain contextual information of the sentences and responses. Simultaneously, construct fine-grained contextual information and use a commonsense knowledge graph to retrieve neighboring conceptual entities in the sentences for external knowledge enhancement. Specifically, this includes:

[0008] S11: Segment the conversation context sentences and response sentences in the training dataset to obtain the sentence segmentation results;

[0009] S12: Query the context dictionary through the sentence segmentation results to obtain the triple score, which serves as the contextual state information contained in the segmentation.

[0010] S13: Simultaneously segment the conversation context and response statement into context-response pairs with fewer sentences, and simultaneously filter out concept entities from the sentence segmentation results. By querying the common sense knowledge graph, find the neighboring entities of the concept entities, and embed the neighboring entity information as external information.

[0011] S2: Using a pre-trained Chinese language generation model, the context text and response text of the training dataset, as well as neighbor entity information, are input into the pre-trained language model for encoding to obtain a text context vector. The vector is then decoded to obtain the generated response, specifically including:

[0012] S21: Group longer corpora in the discourse context according to the actual situation to construct multiple sub-contexts for subsequent training;

[0013] S22: Combine common sense knowledge graphs and embed corresponding neighbor entity semantic information to improve the semantic richness of generated response statements.

[0014] S23: Concatenate the set of neighboring entities and the context response statements, and embed them from three dimensions to obtain the input vector.

[0015] S24: Input the input vector into the pre-trained language generation model, decode it, and generate a response statement.

[0016] S3: Using a conversational retrieval model, the contextual text and response text of the training dataset, along with their contextual information, are input into the retrieval model to encode an aggregated scoring vector. Then, the matching score between the context and the response is obtained, specifically including:

[0017] S31: Represent the words in the training set context text and the response text using pre-trained word vectors and character embeddings of the words themselves;

[0018] S32: Encode the contextual state information corresponding to the conversation text, and use multiple bidirectional long short-term memory networks for encoding, and recursively encode each input sentence;

[0019] S33: Input the contextual state information matrix and the sentence encoding information matrix into the cross-attention matching layer in the retrieval model, use the information to perform attention calculation on the interaction between context and response, and reconstruct the representation of the other after incorporating their respective contextual state information, so as to obtain a new matrix after reconstruction calculation as a new representation, thereby cross-fusing the matching information between them.

[0020] S34: The reconstructed context representation and response representation are further aggregated, and a bidirectional long short-term memory network is used to further capture the temporal relationship between the two at the word granularity and sentence granularity to obtain the final matching feature vector.

[0021] S35: The obtained feature vectors are transformed into the final matching scores of the conversation context statements and response statements through a multilayer perceptron model.

[0022] S4: Using a hybrid approach, the aggregated score vector obtained from the retrieval model is used to assist in the generation of the final response statement, which specifically includes:

[0023] S41: Combine the context statement aggregation information and the response statement aggregation information in the retrieval model;

[0024] S42: Input the concatenated aggregated information, along with the original context text and the response text, into the conversation generation model;

[0025] S43: Use the last continuous output of the stacked layer in the pre-trained language generation model as the output response.

[0026] This invention uses a hybrid model composed of a retrieval model and a pre-trained language generation model to improve the quality of generated responses. At the same time, it makes full use of external common sense knowledge to solve the problem of low semantic richness of generated responses. Attached Figure Description

[0027] Figure 1 This is a flowchart of the session content generation method in this paper;

[0028] Figure 2 This is a schematic diagram illustrating the technical principle of the retrieval model used in this method.

[0029] Figure 3 This is a schematic diagram illustrating the generative model technology principle of this method.

[0030] Figure 4 This is a schematic diagram illustrating the embedding of external knowledge in this method. Detailed Implementation

[0031] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described with reference to the accompanying drawings.

[0032] Reference Figure 1 As shown, Figure 1 This paper presents a method for generating session content using a hybrid model, which includes:

[0033] S1: Obtain the training dataset and use natural language processing tools to parse and process it, obtaining contextual information of the sentences and responses. Simultaneously, construct fine-grained contextual information and use a commonsense knowledge graph to retrieve neighboring conceptual entities within the sentences for external knowledge enhancement. Specifically, this includes:

[0034] S11: Obtain the training dataset, which is the dataset required for training the conversation content generation task, including context text and standard response text; use the Jieba NLP segmentation tool to segment the conversation context sentences and response sentences in the training dataset. The Jieba segmentation tool is an open-source third-party library that combines rule-based and statistical methods. It performs word graph scanning based on a prefix dictionary to generate a directed acyclic graph consisting of all possible word combinations in the input sentence. During use, the cut method is used to input the sentences to be segmented in the training set. After setting the dictionary and stop words, the segmented results are saved in JSON file format to obtain the sentence segmentation results.

[0035] S12: Query the context dictionary through the sentence segmentation results. The context dictionary is an NRC dictionary. Each word can be searched in the dictionary and the corresponding context score can be retrieved. The score is represented from three dimensions. By searching the context dictionary, a triplet represented by three discrete values ​​between 0 and 1 is finally obtained, which serves as the context state information contained in the segmentation.

[0036] S13: Segment the context text and standard response text into context response pairs with fewer sentences, and take each word in the sentence segmentation result set as input. Then, search in the common sense knowledge graph, filter out the concept entities contained in the common sense knowledge graph, set the neighbor entities of the found concept entities, and use the information contained in the retrieved neighbor concept entities as auxiliary embedding information.

[0037] In specific practice, such as referring to Figure 3 As shown, the original context statements in the training data are U1, U2, U3, ..., U n The number of statements is n, and the granularity K is set to 2. Granularity K represents the density of response statements in the context-response pair. A larger K indicates more response statements in the context-response pair. K is the number of response statements, and its value can generally be set according to the specific dataset. The original context statements U1, U2, U3, ..., U... n Divide into l K Group context response pair, l K The calculation process is as follows:

[0038]

[0039] The session context statements U1, U2, U3, ..., U n Extract one of the statements U1, where statement U1 consists of m words, including Some of the words can be retrieved as concept entities in a commonsense knowledge graph. This graph is available as a file on an open-source website and consists of edges and vertices. Each vertex represents entity information, and each edge represents the relationship between entities. The set of retrieved concept entities in statement U1 is extracted, containing t retrieved concept entities e1, e2, ..., e... t The system filters based on a threshold η, which is the hop count for retrieving neighboring entities; the default value is 1. In the graph, two points with a directly connected edge are considered neighboring entities with a hop count of 1, meaning they are adjacent to the current entity. This results in the final set of available neighboring entities: ne2, ne5, ne6, ne7. (See reference...) Figure 4 As shown, the set of available neighbor entities is finally concatenated with entity e1 as auxiliary embedding information.

[0040] S2: Using a pre-trained Chinese language model, the context text, standard response text, and neighbor entity information from the training dataset are input into the pre-trained Chinese language model for encoding to obtain a text context vector. The vector is then decoded to obtain the generated response, specifically including:

[0041] S21: Group the corpus in the discourse context according to the actual situation to construct multiple sub-contextual contexts for subsequent training;

[0042] S22: Combine common sense knowledge graphs and embed corresponding neighbor entity semantic information to improve the semantic richness of generated response statements.

[0043] S23: Concatenate the neighbor entity set and the context response pair to obtain the input statement.

[0044] S24: Input the input sentence into the Chinese pre-trained language generation model CDial-GPT for training. Encode and decode it through a stacked Transformer network to obtain the text context vector. Then, input the text context vector into the trained generation model CDial-GPT. Through model calculation, the output vector is finally obtained. Then, the output vector is used to retrieve the corresponding dictionary to obtain the generated sentence.

[0045] S3: Using a conversational retrieval model, the context text, response text, and their situational information from the training dataset are input into the retrieval model, encoded to obtain an aggregated scoring vector, and the matching score between the context text and the response text is obtained. Specifically, this includes:

[0046] S31: Represent the words in the training set context text and the response text using pre-trained word vectors and character embeddings of the words themselves;

[0047] Specifically, for better representation, pre-trained word vectors and word embedding vectors constructed from existing local dictionaries in the training set were used together for representation. (See reference...) Figure 2 As shown, the input context statement is The number of statements is n, and the matching responses used for training are R. 0 Taking the k-th word in the context as an example, the statement The vocabulary u contained in the set is shown below, where i represents the i-th statement in the set of reply statements:

[0048]

[0049] Candidate Response R 0 The vocabulary r contained in the candidate response is shown below, where j represents the j-th vocabulary in the candidate response:

[0050]

[0051] in For statement The number of words in l R For candidate response R 0 The number of words in the dictionary is set to a fixed dimension of dim = 200 for the word embedding vectors constructed from the local dictionary.

[0052] S32: Encode the contextual state information corresponding to the conversation text, and recursively encode each input sentence using multiple bidirectional long short-term memory networks;

[0053] Specifically, will and R 0 The input is given to the context state information encoding layer, and U k-emb and R emb The input is then fed into the next sentence encoding layer for encoding, where U... k-emb and R emb These are the embedding vector matrices for the context statement and the response statement, respectively. The encoding process is as follows:

[0054]

[0055] R emb =embedding_lookup(Emb,R) 0 )

[0056] The sentence encoding layer in the retrieval model uses multiple Bidirectional Long Short-Term Memory (BiLSTM) networks. BiLSTM includes forward LSTM and backward LSTM, which acquire sentence encoding information by stacking their hidden states. The output of each layer serves as the input to the next layer. Each sentence undergoes M BiLSTM network operations. Utilizing multiple deep networks to encode the input sentence helps the model capture deeper and more useful information. The input includes U from the output of the lexical representation layer. emb , including U 1-emb U 2-emb ,...,U n-emb W b and W f These are the required training parameters for the inverse LSTM and the forward LSTM, respectively. and These are the training parameters for the inverse LSTM. and These are the training parameters for the forward LSTM. Training is performed by setting these parameters in the code, where the values ​​of i and j are between 0 and the number of input context statements. The output is the single-layer encoding matrix information. The single-layer encoding information of the response statement also serves as the input to the next bidirectional LSTM layer. Finally, by stacking multiple layers, the output of each layer adaptively sets the training parameters during the training and fitting process by minimizing the loss function, causing the loss function to converge. The output values ​​of each layer are then summed, and the sum is used as the final result of the sentence encoding.

[0057] The process of encoding contextual state information begins with encoding the input context statements. Each word in each sentence is retrieved from the VAD dictionary to obtain the corresponding three-dimensional vector. Here, we take... The word u in 1,i For example, by searching the VAD dictionary, triples can be obtained. The triplet consists of discrete values ​​between 0 and 1. The length is Similarly, the response statement R 0 The word r in j The triplet was obtained by searching the VAD dictionary. The triplet also contains discrete values ​​between 0 and 1, where the response statement R... 0 The length is Then, the three-dimensional vector is encoded using a bidirectional long short-term memory network to obtain the encoded contextual state information matrix. and response context state information matrix R vadAfter obtaining the context and response state encoding information, the matrix is ​​used in the subsequent cross-attention matching layer for computation.

[0058] S33: Input the contextual state information matrix and the sentence encoding information matrix into the cross-attention matching layer in the retrieval model, use the information to perform attention calculation on the interaction between context and response, and reconstruct the representation of the other after incorporating their respective contextual state information, so as to obtain a new matrix after reconstruction calculation as a new representation, thereby cross-fusing the matching information between them.

[0059] S34: The reconstructed context representation and response representation are further aggregated, and the temporal relationship between the two is further captured at the word granularity and sentence granularity using a bidirectional long short-term memory network. That is, matrix calculations are performed using sentence vector matrix and word vector matrix respectively to obtain the final matching feature vector.

[0060] S35: The obtained feature vectors are transformed into the final matching scores of the conversation context statements and response statements through a multilayer perceptron model.

[0061] S4: Using a hybrid approach, the aggregated score vector obtained from the retrieval model is used to assist in the generation of the final response statement, which specifically includes:

[0062] S41: Combine the context statement aggregation information and the response statement aggregation information in the retrieval model;

[0063] Specifically, the input statement first passes through a lexical representation layer, then a contextual state information encoding layer, and finally a sentence encoding layer to obtain an encoding vector. The encoding vectors of the context statement and the response statement are then input into a cross-attention matching layer, allowing the context statement and the response statement to reconstruct their representations. This input is then fed into an aggregation scoring layer, where a bidirectional long short-term memory network is used for fine-grained encoding. Finally, the data is aggregated to obtain the context statement aggregation matrix C. agr Response statement aggregation matrix R agr Then, the two are concatenated to obtain the concatenated aggregated information.

[0064] S42: Input the concatenated aggregated information, along with the original context text and the response text, into the conversation generation model;

[0065] Specifically, the concatenated aggregated information is input into the conversation generation model, along with the original context statements and responses. After the fine-grained encoding and external knowledge embedding mentioned earlier, the information is fed into the CDial-GPT conversation generation model, resulting in the continuous output of the last layer of the stacked Transformer, thus obtaining the output matrix. For negative samples containing incorrect responses, the final score is reduced through parameter training; the model's objective function L... h Specifically as follows:

[0066]

[0067] Where f(U) i ,R i The expression represents the output vector obtained by the session generation model for the i-th session. Similarity is calculated on the vector to obtain the corresponding score. I is the number of session groups in the corpus, and ε is the network parameter in the generation model. λ is the regularization term, and λ is the parameter corresponding to the regularization term.

[0068] S43: Use the last continuous output of the stacked layer in the pre-trained language generation model as the output response.

[0069] As shown in Tables 1, 2, and 3, this embodiment selects the Douban conversation dataset and the e-commerce conversation dataset as training and testing data. Examples of the two datasets are as follows. Both open-source datasets are real conversation information collected online:

[0070] Table 1: Sample Information from the Douban Dataset

[0071]

[0072] Table 2: Sample Information of E-commerce Datasets

[0073]

[0074] In this embodiment, multiple methods of the prior art are compared simultaneously, and the specific experimental results are shown in Tables 4 and 5:

[0075] Table 4: Experimental results on the Douban dataset

[0076]

[0077] Table 5: Experimental results on the e-commerce dataset

[0078]

[0079]

[0080] In Tables 4 and 5, SEM represents the method of this invention, while the other models are comparative models. 10 @1 represents the proportion of the 10 retrieved responses with the highest probability of being genuine replies; the higher the proportion, the better the model's response generation performance. 10@5 represents the proportion of the top 5 most likely true responses out of 10 retrieved responses; a higher proportion indicates better response generation performance from the model. AUC and MAP values ​​represent retrieval accuracy; higher values ​​indicate more accurate retrieval. Experimental results show that the proportion and accuracy of true responses obtained using this method are superior to most existing conversation generation models.

[0081] Table 6: Experimental results of the HDM model with different K values ​​on the Douban dataset.

[0082]

[0083] Where K is the granularity value mentioned above, that is, the number of reply statements in the context reply pair.

[0084] In Table 6, ROUGE-2, BLEU, and METEOR values ​​represent the similarity between the generated and real responses in terms of sentence structure. The higher the similarity to the real response, the better the model's generation performance. After assembling the hybrid model, the generated responses with higher BLEU and ROUGE values ​​outperform other models. Furthermore, after incorporating sentence context information, the accuracy of retrieving correct responses is also superior to other models.

[0085] The embodiments described above are merely preferred embodiments of the present invention, and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A method for generating session content using a hybrid model, characterized in that, Includes the following steps: S1: Obtain the training corpus and use natural language processing tools to parse and process the training dataset to obtain contextual information of the sentences and responses. Simultaneously, construct fine-grained contextual information and use a commonsense knowledge graph to retrieve neighboring conceptual entities in the sentences for external knowledge enhancement. Specifically, this includes: S11: Perform word segmentation on the conversation context statements and response statements in the training dataset to obtain the sentence word segmentation results; S12: Query the context dictionary through the sentence segmentation results to obtain the triple score, which serves as the contextual state information contained in the segmentation. S13: Simultaneously segment the conversation context statement and the response statement into context response pairs with fewer sentences, and simultaneously filter out concept entities from the sentence segmentation results. By querying the common sense knowledge graph, find the neighbor entities of the concept entities, and embed the neighbor entity information as external information. S2: Use a pre-trained Chinese language generation model, and input the context text and response text of the training dataset, as well as the neighbor entity information, into the pre-trained language model for encoding to obtain a text context vector. Decode the vector to obtain the generated response, specifically including: S21: Group the corpus in the discourse context according to the actual situation to construct multiple sub-contextual contexts for subsequent training; S22: Combine common sense knowledge graphs and embed corresponding neighbor entity semantic information to improve the semantic richness of generated response statements; S23: Concatenate the set of neighboring entities and the context response statements, and embed them from three dimensions to obtain the input vector; S24: Input the input vector into the pre-trained language generation model, decode it, and generate a response statement; S3: Using a conversation retrieval model, the contextual statements and response statements from the training dataset, along with their contextual information, are input into the retrieval model to encode an aggregated scoring vector. Then, the matching score between the context and the response is obtained, specifically including: S31: Represent the words in the context sentences and response sentences of the training dataset using pre-trained word vectors and character embeddings of the words themselves; S32: Encode the contextual state information corresponding to the conversation text, and use multiple bidirectional long short-term memory networks for encoding. Recursively encode each input sentence to obtain the contextual state information matrix and the sentence encoding information matrix. S33: Input the context state information matrix and the sentence encoding information matrix into the cross-attention matching layer in the retrieval model, use the context state information matrix and the sentence encoding information matrix to perform attention calculation on the interaction between context and response, and reconstruct the representation of each other after incorporating their respective context state information, and obtain a new matrix after reconstruction calculation as a new representation, thereby cross-fusing the matching information between them. S34: The reconstructed context representation and response representation are further aggregated, and the temporal relationship between the two is further captured at the word granularity and sentence granularity using a bidirectional long short-term memory network. That is, matrix calculations are performed using sentence vector matrix and word vector matrix respectively to obtain the final matching feature vector. S35: The obtained feature vectors are transformed into the final matching scores of the conversation context statements and response statements through a multilayer perceptron model; S4: Using a hybrid approach, the aggregated score vector obtained from the retrieval model is used to assist in the generation of the final response statement, which specifically includes: S41: Combine the context statement aggregation information and the response statement aggregation information in the retrieval model; S42: Input the concatenated aggregated information, along with the original context statement and response statement, into the conversation generation model; S43: Use the last continuous output of the stacked layer in the pre-trained language generation model as the output response.

2. The session content generation method according to claim 1, characterized in that, In S11, the word segmentation tool is Jieba NLP word segmentation tool. During the process, the cut method is used to input the sentence to be segmented in the training set. After setting the dictionary and stop words, the segmented result is saved in JSON file format to obtain the sentence word segmentation result. The context dictionary is an NRC dictionary. Each word can be searched in the dictionary and the corresponding context score can be retrieved. The score is represented from three dimensions. By searching the context dictionary, a triplet represented by three discrete values ​​between 0 and 1 is finally obtained as the context state information contained in the word segmentation.

3. The session content generation method according to claim 2, characterized in that, Step S13 is as follows: The original context statements in the training data are U1, U2, U3, ..., U n The number of statements is n, and the granularity K represents the density of response statements in the context-response pair, i.e., the number of response statements. The value of K is set according to the specific dataset. The original context statements U1, U2, U3, ..., U n Divide into l K Group context response pair, l K The calculation process is as follows: The session context statements U1, U2, U3, ..., U n Extract one of the statements U1, where statement U1 consists of m words, including Some of the words can be retrieved as concept entities in a commonsense knowledge graph. This graph is available as a file on an open-source website and consists of edges and vertices. Each vertex represents entity information, and each edge represents the relationship between entities. The set of retrieved concept entities in statement U1 is extracted, containing t retrieved concept entities e1, e2, ..., e... t The system filters based on a threshold η, which is the number of hops to retrieve neighboring entities. The default value is 1. If two points in the graph are directly connected by an edge, then these two entities are neighboring entities with a hop count of 1, that is, concept entities adjacent to the current entity. This yields the final set of available neighboring entities. Finally, the set of available neighboring entities is concatenated with entity e1 as auxiliary embedded information.

4. The session content generation method according to claim 3, characterized in that, The representation and encoding operations in step S3 are as follows: To achieve better representation, pre-trained word vectors are used together with word embedding vectors constructed from existing local dictionaries in the training set; the input context is... The number of statements is n, and the number of candidate responses used for training is R. 0 Taking the k-th word in the context as an example, the statement The vocabulary u contained in the set is shown below, where i represents the i-th statement in the set of reply statements: Candidate Response R 0 The vocabulary r contained in the candidate response is shown below, where j represents the j-th vocabulary in the candidate response: in For statement The number of words in l R For candidate response R 0 The number of words in the dictionary is set to a fixed dimension of dim = 200 for the word embedding vectors constructed from the local dictionary. The contextual state information and sentences corresponding to the conversation text are encoded; specifically, the contextual state information and sentences corresponding to the conversation text are encoded. and R 0 The input is given to the context state information encoding layer, and U k-emb and R emb The input is then fed into the next sentence encoding layer for encoding, where U... k-emb and R emb These are the embedding vector matrices for the context statement and the response statement, respectively. The encoding process is as follows: R emb =embedding_lookup(Emb,R 0 ) The sentence encoding layer in the retrieval model utilizes multiple bidirectional long short-term memory (BSSM) networks, including forward and backward BSSM networks. Sentence encoding information is obtained by stacking their hidden states. The output of each layer serves as the input to the next layer. Each sentence undergoes M BSSM network operations. Encoding the input sentence using multiple deep networks helps the model capture deeper and more useful information. The input includes U from the lexical representation layer. emb , including U 1-emb U 2-emb ,...,U n-emb W b and W f These are the required training parameters for the inverse LSTM and the forward LSTM, respectively. and These are the training parameters for the inverse LSTM. and These are the training parameters for the forward LSTM. Training is performed by setting these parameters in the code. The values ​​of i and j are between 0 and the number of input context statements; the output is a single-layer encoding matrix. The single-layer encoding information of the response statement is also used as the input to the next layer of the bidirectional LSTM. Finally, after stacking multiple layers, the output of each layer is obtained by adaptively setting the above training parameters through the calculation method of minimizing the loss function during the training and fitting process, so that the loss function converges and the output value of each layer is obtained. The values ​​are summed and the sum is used as the final result of sentence encoding. The process of encoding contextual state information begins with encoding the input context statements. Each word in each sentence is retrieved from the VAD dictionary to obtain the corresponding three-dimensional vector. Here, we take... The word u in 1,i For example, by searching the VAD dictionary, triples can be obtained. The triplet consists of discrete values ​​between 0 and 1. The length is Similarly, the response statement R 0 The word r in j The triplet was obtained by searching the VAD dictionary. The triplet also contains discrete values ​​between 0 and 1, where the response statement R... 0 The length is Then, the three-dimensional vector is encoded using a bidirectional long short-term memory network to obtain the encoded contextual state information matrix. and response context state information matrix R vad ; After obtaining the context and response state encoding information, the matrix is ​​used in the subsequent cross-attention matching layer for computation.

5. The session content generation method according to claim 4, characterized in that, The specific operation of step S41 is as follows: The input statement first passes through a lexical representation layer, then a contextual state information encoding layer, and finally a sentence encoding layer to obtain an encoding vector. The encoding vectors of the context statement and the response statement are then input into a cross-attention matching layer, allowing the context statement and the response statement to reconstruct their representations. This input is then fed into an aggregation scoring layer, where a bidirectional long short-term memory network is used for fine-grained encoding. Finally, the input is aggregated to obtain the context statement aggregation matrix C. agr Response statement aggregation matrix R agr Then, the two are concatenated to obtain the concatenated aggregated information.

6. The session content generation method according to claim 5, characterized in that, The specific operation of step S42 is as follows: The concatenated aggregated information is input into the conversation generation model, along with the original context statements and responses. After the fine-grained encoding and external knowledge embedding mentioned earlier, the input is fed into the CDial-GPT conversation generation model, resulting in the continuous output of the last layer of the stacked Transformer, thus obtaining the output matrix. For negative samples containing incorrect responses, the final score is reduced through parameter training; the model's objective function L... h Specifically as follows: Where f(U) i ,R i The expression represents the output vector obtained by the session generation model for the i-th session. Similarity is calculated on the vector to obtain the corresponding score. I is the number of session groups in the corpus, and ε is the network parameter in the generation model. λ is the regularization term, and λ is the parameter corresponding to the regularization term.

Images