Large language model for generating multi-session dialogue

Abstract

The invention provides a large language model for generating a multi-session conversation. The large language model comprises an original large language model, a VAE unit, a LoRA unit and a hidden state generation unit, the VAE unit receives the current query or extracts the query in each round of dialogue from the dialogue history, then generates implicit vector representation corresponding to the current query and the dialogue history, and transmits the implicit vector representation to the hidden state generation unit; the LoRA unit is connected to the output of each layer of the original large language model, then superposes the output of each layer with a weight matrix, and then transmits a result to the hidden state generation unit; and the hidden state generation unit receives the outputs of the LoRA unit and the VAE unit and the parameters of each layer of the original large language model, and adds the outputs of the LoRA unit and the VAE unit to obtain a new hidden state of each layer of the original large language model. The reply consistent with the dialogue history can be generated, the dialogue ability of the large language model is improved, the dialogue experience of the user can be enriched, and the dialogue interest of the user can be attracted.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing and deep learning technology, and in particular to a large language model for generating multi-conversation dialogues. Background Art

[0002] Multi-session dialogue generation has become a key issue and important challenge in open-domain dialogue systems. Its goal is to generate responses that are consistent with the dialogue history. Multi-session dialogues contain dialogue histories from multiple sessions, significantly surpassing the limitation of traditional personalized dialogues that are limited to 16 turns [1].

[0003] Most existing multi-session dialogue generation models currently focus only on the conversation history of the current session. They are still unable to establish long-term interaction with humans in multi-session dialogues, especially in multi-session personalized dialogue generation. In real conversation scenarios, dialogue models usually need to have the ability to accompany and familiarize with each other for a long time [2].

[0004] Existing multi-session dialogue generation methods mainly focus on historical modeling, which can be further divided into summary-based methods, extended context window methods, and retrieval-based methods. Summary-based methods usually use prompts or instructions to enable the generative model to summarize the dialogue history. For example, reference [3] proposes to recursively use previous memories and subsequent contexts to generate new memories, thereby helping chatbots generate highly consistent responses in long-term dialogues. The extended context window method increases the input capacity by changing the architecture of the generative model. For example, reference [4] designs different attention mask matrices based on long-term memory and short-term memory to expand the input scale. While ensuring performance, the efficiency is significantly improved. However, this method may face the risk of losing important historical information or introducing noise. In order to guide the model to pay attention to important historical information, many studies have explored the use of retrieval-based methods. They use retrieval models to obtain relevant dialogue information. For example, reference [5] proposes to use a memory pool and store the dialogue history in the memory pool. The memory pool accumulates information as the dialogue proceeds. Before the dialogue is generated, a retriever such as DPR is used to retrieve personality memory or historical event memory to maintain the consistency of the dialogue model. Although these methods can access relevant historical information, they may destroy the dialogue structure and hinder the model's understanding of the dialogue history.

[0005] References:

[0006] [1] Huang, M., Zhu, X., & Gao, J. (2019). Challenges in BuildingIntelligent Open-domain Dialog Systems. ACM Transactions on InformationSystems (TOIS), 38, 1 - 32.

[0007] [2] Li, Hao, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang and Tat-Seng Chua. “Hello Again! LLM-powered Personalized Agent for Long-termDialogue.” ArXiv abs / 2406.05925 (2024): n. when.

[0008] [3] Wang, Qingyue, Liang Ding, Yanan Cao, Zhiliang Tian, ​​Shi Wang,Dacheng Tao and Li Guo. “Recursively Summarizing Enables Long-Term DialogueMemory in Large Language Models.” ArXiv abs / 2308.15022 (2023): n. when.

[0009] [4] Kim, Seo Hyun, Keummin Ka, Yohan Jo, Seung-won Hwang, Dongha Leeand Jinyoung Yeo. “Ever-Evolving Memory by Blending and Refining the Past.” ArXiv abs / 2403.04787 (2024): n. when.

[0010] [5] Lee, Gibbeum, Volker Hartmann, Jongho Park, DimitrisPapailiopoulos and Kangwook Lee. “Prompted LLMs as Chatbot Modules for LongOpen-domain Conversation.” Annual Meeting of the Association for Computational Linguistics (2023). Summary of the invention

[0011] In order to solve the above problems and enhance the generation capability of the large language model in the multi-session dialogue generation task, the present invention improves the existing open source large language model, adds VAE units, LoRA units and hidden state generation units in each layer, provides the large language model with useful prior knowledge in the multi-session dialogue generation context, and enhances the multi-session dialogue generation capability of the large language model.

[0012] The large language model for generating multi-session dialogues proposed by the present invention includes an original large language model and a hidden state improvement module; the hidden state improvement module is used to generate a new hidden state for each layer of the original large language model, including a VAE unit, a LoRA unit and a hidden state generation unit, wherein:

[0013] The VAE unit is used to receive the current query or extract the query in each round of dialogue from the dialogue history, then generate an implicit vector representation corresponding to the current query and the dialogue history, and transmit the implicit vector representation to the hidden state generation unit;

[0014] The LoRA unit has a LoRA weight matrix (i.e., a low-rank matrix) connected after the output of each layer of the original large language model, and is used to superimpose the weight matrix on the output of each layer, and then transmit the result to the hidden state generation unit;

[0015] The hidden state generation unit is used to receive the output of the LoRA unit and the VAE unit and the parameters of each layer of the original large model, and add the three together to obtain the new hidden state of each layer of the original large language model.

[0016] Furthermore, it also includes a historical conversation acquisition module, which is used to process the conversation history, calculate the similarity score between each conversation history and the current query, select the k conversation histories with the highest relevance scores as the conversation history of the current query, and then input the conversation history into the original large language model.

[0017] Furthermore, in the historical dialogue acquisition module, the method for acquiring the similarity score includes:

[0018] Calculate the time interval between all conversation histories in the conversation and the current query, and assign a weight to each conversation history according to the time decay formula:

[0019]

[0020] Where t is the time interval, The temperature coefficient is a hyperparameter that can be adjusted by yourself;

[0021] The similarity between all conversation histories in the conversation and the current query is calculated and multiplied by the time decay factor to obtain the final similarity score.

[0022] Furthermore, in the VAE unit, a variational autoencoder encoder is used to encode and reconstruct the input data, encode the queries in the conversation to better extract the implicit information in the current query, and finally generate a vector containing implicit information.

[0023] Furthermore, the VAE unit also includes an adapter for bridging the gap between the implicit vectors generated by the VAE module and the vector space of the original large language model.

[0024] Furthermore, in the VAE unit, during training, the current query and the standard answer are input to the encoder to obtain a representation vector; then the two representation vectors are sampled using a re-parameterization technique to ensure that the gradients can be calculated at all steps in the training.

[0025] Furthermore, the KL divergence is used to approximate the two sampled vectors, and the loss function is as follows:

[0026]

[0027] in, and are the approximate posterior distribution and the approximate prior distribution respectively. and is a hyperparameter, is the representation of the query, is the representation of the standard answer. is the sampling distribution of VAE.

[0028] Furthermore, it also includes a data cleaning module for data cleaning and segmentation to obtain a number of conversations and corresponding labels. Each piece of data in the processed text contains the user's personality information, the chatbot's personality information, the user's current query, the chatbot's standard reply, and previous conversation history.

[0029] Furthermore, a training module is included for training parameters of the VAE unit and the LoRA unit. The training method includes:

[0030] S1. Clean and segment the data set to obtain several conversations and corresponding labels.

[0031] S2, retrieve the conversation history related to the current query;

[0032] S3, extract the query in each round of dialogue and input it into the VAE unit to obtain implicit representation;

[0033] S4. Add the implicit representation, the parameters of the original large language model, and the LoRA unit output, set the training parameters, and update the training parameters using the back propagation algorithm.

[0034] Furthermore, in the training module, the loss function is:

[0035]

[0036]

[0037] in, is the generation loss of the original large language model, It is LoRA With hidden state The loss between them, MIM is a method to obtain the maximum mutual information, is a hyperparameter.

[0038] The beneficial effects of the present invention are as follows:

[0039] (1) Modeling the memory mechanism based on the time interval between different conversations can effectively filter the conversation history related to the current query and provide useful historical information for conversation generation.

[0040] (2) The creative combination of VAE and LoRA is a new and efficient fine-tuning method that can effectively provide rich prior knowledge for large language models and improve the generation ability of large language models.

[0041] (3) This method has achieved the best results on the MSC dataset, indicating that this method can achieve good performance in multi-session dialogue generation tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 A schematic diagram of the structure of a large language model according to an embodiment of the present invention;

[0044] Figure 2 A schematic diagram of the improved structure of each layer of a large language model according to an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of a flow chart of a historical conversation acquisition module according to an embodiment of the present invention;

[0046] Figure 4 A schematic diagram of a process of a VAE unit according to an embodiment of the present invention;

[0047] Figure 5 The figure is a flowchart of large language model training according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be checked and described in full in combination with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The present invention provides a large language model for generating multi-session dialogues, such as Figure 1 , 2 As shown, it includes the original large language model and the hidden state improvement module, where:

[0050] Original large language models, which are existing open source large language models, such as ChatGLM and Llama2;

[0051] Hidden state improvement module, used to generate new hidden states for each layer of the original large language model, including VAE units, LoRA units and hidden state generation units, where:

[0052] The VAE unit is used to receive the current query, or extract the query in each round of dialogue from the dialogue history, and then generate an implicit vector representation corresponding to the current query and the dialogue history, and transmit the implicit vector representation to the hidden state generation unit.

[0053] The LoRA unit, which has a LoRA weight matrix (i.e., a low-rank matrix), is connected after the output of each layer of the original large language model to superimpose the parameter matrix on the output of each layer and then transmit the result to the hidden state generation unit.

[0054] The hidden state generation unit is used to receive the output of the LoRA unit and the VAE unit and the parameters of each layer of the original large model, and add the three together to obtain the new hidden state of each layer of the original large language model.

[0055] The hidden state of the original large language model refers to the output of each layer, which is a series of vectors. The hidden state of the last layer plus an additional network similar to MLP is mapped to the vocabulary of the large language model, which is the final generation result. This part is part of the structure of the original large language model itself and has not changed, so I will not go into details.

[0056] In one embodiment, each layer of the original large language model corresponds to a LoRA unit and a VAE unit.

[0057] In one embodiment, because the conversation history is very long and cannot be directly input into the large language model, the present invention designs a historical conversation acquisition module for processing the conversation history. Specifically, it includes: using the DPR model and the time decay formula to calculate the relevance score of each conversation history with the current query, selecting the k conversation histories with the highest relevance scores as the conversation history of the current query, and then transmitting the conversation history to the original large language model to generate the hidden state of each layer.

[0058] In the historical dialogue acquisition module, a memory mechanism is established for the large language model to retrieve the dialogue history related to the current query from previous dialogues, such as Figure 3 As shown, first calculate the time interval between all conversation histories in the conversation and the current query, and assign a weight to each conversation history according to the time decay formula:

[0059]

[0060] Where t is the time interval, The temperature coefficient is a hyperparameter that can be adjusted by yourself. Then, the similarity between all the conversation histories in the conversation and the current query is calculated (for example, using the DPR model), and the final similarity score is obtained by multiplying the time decay factor. Based on the similarity score, the k conversations with the highest similarity are selected as the relevant conversation histories and input into the VAE unit.

[0061] The DPR model (Dense Passage Retrieval) was proposed by Facebook AI Research. It aims to improve the efficiency and accuracy of retrieval through dense vector representation and select the most relevant paragraphs as retrieval results.

[0062] In one embodiment, the historical conversation acquisition module may use Hamming distance, cosine similarity, etc. to calculate the similarity between all conversation history calculations in the conversation and the current query.

[0063] In the VAE unit, a variational autoencoder is used to generate new data and compress and reconstruct the input data. The variational autoencoder combines the encoder and decoder parts, and can perform effective random sampling in the latent space to generate new samples similar to the training data. In the present invention, Figure 4 As shown in the figure, only the encoder of the variational autoencoder is used and its ability to encode and reconstruct the input data is utilized to encode the query in the conversation to better extract implicit information such as the topic, emotion, personality, etc. in the current query, and finally generate the corresponding vector containing implicit information. When training the large language model, the current query and the standard answer are input to the encoder at the same time to obtain the representation vector. The introduction of the standard answer is to provide effective prior knowledge to the original large language model and improve its ability to generate meaningful answers. For the encoder, any model with encoding function can be used, such as BERT. Then the implicit space of the two representation vectors is sampled using the re-parameterization technique to ensure that the gradient can be calculated in all steps of the training:

[0064]

[0065] Finally, in order to ensure the consistency of prior knowledge, KL divergence is used to approximate the two sampled vectors, and the loss function is as follows:

[0066]

[0067] in, and are the approximate posterior distribution and the approximate prior distribution respectively. and is a hyperparameter, is the representation of the query, is the representation of the standard answer. is the sampling distribution of VAE. During inference, we only need to perform the same encoding and resampling on the current query to obtain the vector.

[0068] In one embodiment, the VAE unit further includes an adapter for connecting after the VAE unit to bridge the gap between the latent representation vector extracted by the VAE module and the vector space of the original large language model.

[0069] In the LoRA unit, LoRA (Low-Rank Adaptation) is an efficient model fine-tuning technology designed to optimize the adaptability of large-scale pre-trained models. It has a low-rank matrix (i.e., weight matrix) that enables fast and efficient model adjustment, thereby significantly reducing the required computing resources and storage space while maintaining the performance of the original model. This allows high-quality fine-tuning even with limited resources.

[0070] In the hidden state generation unit, the output result of the VAE unit, the output result of the LoRA unit, and the hidden state of the original large language model are received, and then the three are added as the new hidden state of the original large language model. The formula is as follows:

[0071]

[0072] in, is the new hidden state of the original large language model, is the initial hidden state of the original large language model, are the parameters of the original large language model, and is the weight matrix of the LoRA unit, and is the parameter matrix of the adapter.

[0073] The present invention creatively integrates the concepts of variational autoencoders (VAE) and low rank adaptation (LoRA). Although LoRA can effectively improve the performance of large language models in many tasks, its improvement is always limited in specific tasks. In order to further enhance the performance of LoRA, the parameters (weights, essentially matrices) of the LoRA unit, the sampling results of VAE or the results of multiplication with the adapter (also matrices), and the output of each layer of the large language model (also matrices) are added to obtain the hidden state of the new large language model, thereby providing the large language model with more effective prior knowledge to improve the generation ability of the language model while ensuring its performance in downstream tasks.

[0074] In one embodiment, a data cleaning module is also included for data cleaning and segmentation to obtain a number of conversations and corresponding labels. The text input to the large model may include conversation data, which includes all conversation histories and the time intervals between different conversations. All conversation data are described in natural languages ​​such as English and Chinese. In order to facilitate subsequent model training, the input text is processed, the conversation histories of different conversations are segmented according to the time intervals in the text, and sensitive information appearing in open domain conversations is filtered out. Each piece of data in the processed text contains the user's personality information, the chatbot's personality information, the user's current query, the chatbot's standard response, and the past conversation history.

[0075] In one embodiment, a training module is also included to set the hyperparameters of the VAE unit and the LoRA unit, including the learning rate, batch size, and number of training iterations (the parameters of the original large language model are frozen at this time), and then the parameters of the VAE unit and the LoRA unit are updated using a back propagation algorithm (such as an AdamW optimizer) according to the loss function. The loss function is as follows:

[0076]

[0077]

[0078] in, is the generation loss of the original large language model, yes With the new hidden state The loss between them, MIM is a method to obtain the maximum mutual information. In fact, mean square error and KL divergence can also be applied. is a hyperparameter.

[0079] The specific training process is as follows: Figure 5 As shown, including:

[0080] S1. Clean and segment the data set to obtain several conversations and corresponding labels.

[0081] S2, retrieve the conversation history related to the current query;

[0082] S3, extract the query in each round of dialogue and input it into the VAE unit to obtain implicit representation;

[0083] S4. Add the implicit representation, the parameters of the original large language model, and the output of the LoRA unit to obtain a new hidden state, set the training parameters, generate a multi-session dialogue, and then use the back-propagation algorithm to update the training parameters.

[0084] S1 can be completed through the data cleaning module, S2 can be completed through the historical dialogue acquisition module, S3 can be completed through the VAE unit, and S4 can be completed through the LoRA unit and the hidden state generation unit, which will not be repeated here.

[0085] The present invention improves the architecture of the original large language model. On the basis of establishing a memory mechanism, the VAE-LoRA unit is trained, which can not only maintain the dialogue structure so that the large language model can fully understand the dialogue context of long-term dialogues, but also enable the large language model to deeply understand and mine user portraits, and generate responses that are consistent with the dialogue history and personalized information.

[0086] Experimental verification:

[0087] The input text is the corpus MSC dataset. The 2022 ACL conference paper Beyond Goldfish Memory: Long-Term Open-Domain Conversation first proposed open-domain conversations with multi-session conversations and proposed this dataset.

[0088] The present invention effectively enhances the performance of large language models on the MSC dataset and achieves SOTA. As shown in Table 1, experiments were conducted on the MSC dataset using ChatGLM-6B and Llama2-7B and the method of the present invention, and BlenderBot, HAHT, ChatGLM-6B (Base), Llama2-7B (Base), ChatGPT, CPD, StreamingDialogue, and LD-Agent were used as baseline models for comparison on BLEU-1, BLEU-2, BLEU-3, and ROUGE-L indicators.

[0089] BlenderBot is an advanced open-domain dialogue model developed by large-scale pre-training on large-scale datasets, combined with a retrieval-refinement mechanism and an optimized decoding strategy. HAHT is a model that adopts hierarchical encoding and attention mechanisms to maintain and utilize long-term historical dialogue memory to generate context-relevant replies through a history-aware response generator. ChatGPT is a closed-source large language model based on the GPT architecture, and we use the API service of OpenAI's gpt-3.5-turbo model. CPD is a causal-aware multi-turn dialogue framework that uses a perturbation-based causal variable discovery method to extract statements with high causal relevance from historical dialogues, thereby enhancing the causal awareness of large language models. StreamingDialogue is a model that effectively handles long-context dialogues by compressing the dialogue history into a "dialogue attention sink" and adopting short-term memory reconstruction (SMR) and long-term memory activation (LMR) learning strategies. LD-Agent is a dialogue agent framework that supports coherent dialogues by integrating event memory and personalized role modeling. It is the previous state-of-the-art method on the MSC dataset. As shown in Table 1, the best result is in bold and the second best result is underlined.

[0090] Table 1

[0091] Model BLEU-1 BLEU-2 BLEU-3 ROUGE-L BlenderBot - 4.91 1.53 16.06 HAHT - 5.10 1.59 16.58 ChatGLM(Base) 19.20 5.54 1.50 16.49 Llama2(Base) 17.34 4.37 1.21 10.29 ChatGPT 18.96 5.77 1.51 16.84 CPD 12.45 4.41 - 12.14 StreamingDialogue 19.33 - - 15.86 LD-Agent 19.54 7.31 2.51 18.44 Ours(ChatGLM) <![CDATA[ 19.87 ]]> <![CDATA[ 7.71 ]]> <![CDATA[ 2.55 ]]> 19.87 Ours(Llama2) 20.51 8.16 <![CDATA[ 2.59 ]]> <![CDATA[ 18.77 ]]>

[0092] Those skilled in the art will appreciate that the foregoing embodiments are merely intended to illustrate the technical solutions of the present invention rather than to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will appreciate that the technical solutions described in the foregoing embodiments may be modified or some or all of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope defined by the claims of the present invention.

Claims

1. A large language model for generating multi-conversation dialogues, characterized in that It includes an original large language model and a hidden state improvement module; the hidden state improvement module is used to generate a new hidden state for each layer of the original large language model, including a VAE unit, a LoRA unit and a hidden state generation unit, wherein: The VAE unit is used to receive the current query or extract the query in each round of dialogue from the dialogue history, then generate an implicit vector representation corresponding to the current query and the dialogue history, and transmit the implicit vector representation to the hidden state generation unit; The LoRA unit has a LoRA weight matrix (i.e., a low-rank matrix) connected after the output of each layer of the original large language model, and is used to superimpose the weight matrix on the output of each layer, and then transmit the result to the hidden state generation unit; The hidden state generation unit is used to receive the output of the LoRA unit and the VAE unit and the parameters of each layer of the original large model, and add the three together to obtain the new hidden state of each layer of the original large language model.

2. The large language model according to claim 1, characterized in that: It also includes a historical conversation acquisition module for processing conversation histories, calculating the similarity score between each conversation history and the current query, selecting the k conversation histories with the highest relevance scores as the conversation histories of the current query, and then inputting the conversation histories into the original large language model.

3. The large language model according to claim 2, characterized in that: In the historical dialogue acquisition module, the similarity score acquisition method includes: Calculate the time interval between all conversation histories in the conversation and the current query, and assign a weight to each conversation history according to the time decay formula: , where t is the time interval, The temperature coefficient is a hyperparameter that can be adjusted by yourself; The similarity between all conversation histories in the conversation and the current query is calculated and multiplied by the time decay factor to obtain the final similarity score.

4. The large language model according to claim 1, characterized in that In the VAE unit, a variational autoencoder encoder is used to encode and reconstruct the input data, encode the queries in the conversation to better extract the implicit information in the current query, and finally generate a vector containing implicit information.

5. The large language model according to claim 4, characterized in that: The VAE unit also includes adapters to bridge the gap between the implicit vectors generated by the VAE module and the vector space of the original large language model.

6. The large language model according to claim 4, characterized in that: In the VAE unit, during training, the current query and the standard answer are input into the encoder to obtain a representation vector; then the two representation vectors are sampled using the re-parameterization technique to ensure that the gradient can be calculated at all steps in the training.

7. The large language model according to claim 4, characterized in that: Using KL divergence to approximate two sampled vectors, the loss function is as follows: in, and are the approximate posterior distribution and the approximate prior distribution respectively. and is a hyperparameter, is the representation of the query, is the representation of the standard answer. is the sampling distribution of VAE.

8. The large language model according to claim 1, characterized in that: It also includes a data cleaning module for data cleaning and segmentation to obtain several conversations and corresponding labels. Each piece of data in the processed text contains the user's personality information, the chatbot's personality information, the user's current query, the chatbot's standard response, and previous conversation history.

9. The large language model according to claim 1, characterized in that: It also includes a training module for training the parameters of the VAE unit and the LoRA unit. The training methods include: S1. Clean and segment the data set to obtain several conversations and corresponding labels. S2, retrieve the conversation history related to the current query; S3, extract the query in each round of dialogue and input it into the VAE unit to obtain implicit representation; S4. Add the implicit representation, the parameters of the original large language model, and the LoRA unit output, set the training parameters, and update the training parameters using the back propagation algorithm.

10. The large language model according to claim 9, characterized in that: In the training module, the loss function is: , ,in, is the generation loss of the original large language model, It is LoRA With hidden state The loss between them, MIM is a method to obtain the maximum mutual information, is a hyperparameter.

Images