A method for enhancing role awareness of a large language model

Abstract

The application provides a method for enhancing role perception of a large language model, comprising: S1, processing a dialogue data set to obtain a plurality of conversations and corresponding labels; S2, establishing a memory mechanism to obtain long-term historical dialogues and short-term historical dialogues related to a current query; S3, combining personality information of a user and a knowledge graph to construct a user graph for the user; and S4, training by using prompt learning through a graph neural network.The application can generate a reply consistent with a dialogue history and personalized information, improve dialogue capability of the large language model, enrich dialogue experience of the user, and attract dialogue interest of the user.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing and deep learning technology, and in particular to a method for enhancing role perception of a large language model. Background Art

[0002] Personalized dialogue generation has become a key issue and a significant challenge in open-domain dialogue systems. The goal is to generate responses that are consistent with the user's personality and conversation history. Personality information can make generated responses more specific and comprehensive, thereby enhancing user engagement. Existing approaches to multi-session personalized dialogue generation primarily focus on history modeling and personality modeling.

[0003] Historical modeling can be mainly divided into establishing a memory mechanism and expanding the context window. Reference [1] integrates the forgetting curve into the memory mechanism of a large language model and uses a retrieval model to retrieve relevant conversation information. Although this method allows the large language model to obtain relevant historical information, it often destroys the conversation structure and hinders the conversation model's understanding of the conversation history. Reference [2] expands the input scale by modifying the internal architecture of the large language model. This method is prone to losing key historical information or introducing noise.

[0004] Personality information modeling is mainly divided into explicit personality modeling and implicit personality modeling. Explicit personality methods usually input data containing personality information into a large language model. For example, Reference [3] generates customized dialogue responses by combining the user's personal background and knowledge information on Wikipedia. However, faced with a long dialogue history and brief character information, the large language model is likely to fail to pay attention to the character information and output responses that are inconsistent with the character information. The implicit personality-based method represents personality information with a vector to cover more character attributes. For example, Reference [4] automatically learns the user's implicit profile by analyzing the user's historical dialogue records and designs a personalized decoder that combines generation and replication strategies to generate responses that are more in line with the user's personality. However, the character information in the dialogue is often sparse, and the large language model lacks effective prior knowledge, resulting in the generated results being not rich enough and not attractive enough to users [5].

[0005] While previous approaches focusing on history modeling and role modeling have advanced the field, these approaches have overlooked the importance of dialogue structure in helping large language models understand conversational context. Furthermore, these approaches fail to effectively scale and utilize personality information, thus reducing response consistency.

[0006] References:

[0007] [1] Zhong, W., Guo, L., Gao, Q., Ye, H. and Wang, Y. 2024.MemoryBank: Enhancing Large Language Models with Long-Term Memory.Proceedings of the AAAI Conference on Artificial Intelligence. 38, 17 (Mar.2024), 19724-19731. DOI:https: / / doi.org / 10.1609 / aaai.v38i17.29946.

[0008] [2] Jia-Nan Li, Quan Tu, Cunli Mao, Zhengtao Yu, Ji-Rong Wen, and RuiYan. 2024.

[0009] StreamingDialogue: Prolonged Dialogue Learning via Long ContextCompression with Minimal Losses. arXiv:2403.08312 [cs.CL] https: / / arxiv.org / abs / 2403.08312

[0010] [3] Jang, Y., Lim, J., Hur, Y., Oh, D., Son, S., Lee, Y., Shin, D.,Kim, S. and Lim, H. 2022. Call for Customized Conversation: CustomizedConversation Grounding Persona and Knowledge. Proceedings of the AAAIConference on Artificial Intelligence. 36, 10 (Jun. 2022), 10803-10812. DOI:https: / / doi.org / 10.1609 / aaai.v36i10.21326.

[0011] [4] Zhengyi Ma, Zhicheng Dou, Yutao Zhu, Hanxun Zhong, and Ji-RongWen. 2021. One Chatbot Per Person: Creating Personalized Chatbots based onImplicit User Profiles. In Proceedings of the 44th International ACM SIGIRConference on Research and Development in Information Retrieval (SIGIR '21). Association for Computing Machinery, New York, NY, USA, 555–564. https: / / doi.org / 10.1145 / 3404835.3462828

[0012] [5] Shixuan Fan, Wei Wei, Wendi Li, Xian-Ling Mao, Wenfeng Xie, andDangyang Chen. 2024. Position Debiasing Fine-Tuning for Causal Perception inLongTerm Dialogue. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, Kate Larson (Ed.). International Joint Conferences on Artificial Intelligence Organization,6261–6269. https: / / doi.org / 10.24963 / ijcai.2024 / 692 Main Track. Summary of the Invention

[0013] To address the above issues and enable the large language model to understand the conversation context and mine profound user portraits in multi-session personalized conversations, the present invention designs a method for enhancing role perception for the large language model, which can not only effectively mine deeper user portraits but also maintain the conversation structure and understand the conversation information of multi-session conversations.

[0014] The present invention provides a method for enhancing character perception of a large language model, comprising:

[0015] S1. Process the conversation dataset to obtain several conversations and corresponding labels;

[0016] S2. Establish a memory mechanism to obtain long-term and short-term historical conversations related to the current query;

[0017] S3. Combine the user's personality information and knowledge graph to build a user graph for the user;

[0018] S4. Use prompt learning for training through graph neural networks.

[0019] Furthermore, in step S1, the conversation data includes the user's personality information, the chatbot's personality information, all conversation histories, and the time intervals between different conversations. The processed data includes the user's personality information, the chatbot's personality information, the user's current query, the chatbot's standard response, and previous conversation histories.

[0020] Furthermore, in step S2, for the long-term conversation history, the top k conversation histories related to the current query are ranked and retrieved according to similarity, and the order of the conversation history is adjusted.

[0021] Furthermore, a method for adjusting the order of the dialogue histories includes: using a ConvED method to calculate the dialogue edit distance of the utterances in the k dialogue histories, obtaining a score, and adjusting the order according to the score.

[0022] Furthermore, in step S2, for the short-term conversation history, the topic relevance between the short-term conversation history and the current query is evaluated through a topic transition detector; if the utterance in the short-term conversation is irrelevant to the topic of the current query, it is discarded.

[0023] Furthermore, in step S3, the method for constructing the user graph includes:

[0024] Process the knowledge graph and filter out irrelevant personality information;

[0025] Using the personality triplet extractor, traverse the conversation dataset in step S1 and extract the personality triples that appear from the predefined personality information and conversation history;

[0026] Calculate the similarity between all extracted personality triplets and triplets in the knowledge graph, set a certain threshold, and link the knowledge graph triplets and personality triplets whose similarity is greater than the threshold;

[0027] Construct a user graph, where the nodes of the user graph include personality information or personality attributes, and the edges of the user graph consist of the relationships between personality information or personality attributes.

[0028] Furthermore, the knowledge graph contains common sense about characters and connections between characters.

[0029] Furthermore, in step S4, it includes:

[0030] (1) Input the representation of nodes and edges in the graph into the embedding layer of the large language model to obtain the corresponding encoding vector;

[0031] (2) Input the query into the embedding layer of the large language model to obtain the encoding vector of the query, and calculate the weight of the encoding vector of the query and the encoding vector of the node in the graph through the attention mechanism;

[0032] (3) Aggregate the representations of all neighboring nodes of a node according to different relationships in the user graph to update the representation of the current node. Through pooling operations, the user portrait representation vector is obtained;

[0033] (4) Replace the predefined special characters in the prompt word with the user portrait representation vector: set the hyperparameters, calculate the loss between the content generated by the large language model and the standard answer, and use backpropagation for training.

[0034] Furthermore, during the training process, the parameters of the large language model are frozen, and the parameters of the graph neural network and MLP are updated. The beneficial effects of the present invention are as follows:

[0035] (1) This paper proposes a new memory mechanism that retrieves relevant conversation history for the large language model while ensuring the conversation structure information, thereby improving the large language model's ability to understand the conversation history context.

[0036] (2) The present invention constructs a unique user graph for users to expand and mine their personality information, and designs a graph neural network based on the graph to enhance the role perception capability of the large language model.

[0037] (3) The present invention achieves good results on the MSC dataset, indicating that the present method can achieve good performance in multi-session dialogue generation tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1A flowchart of a method for enhancing role perception of a large language model according to an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of a flow chart of establishing a memory mechanism according to an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of a process for constructing a user graph according to an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of a prompt learning training process according to an embodiment of the present invention;

[0043] Figure 5 2 is a schematic diagram of the workflow of a trained large language model according to one embodiment of the present invention. DETAILED DESCRIPTION

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be reviewed and fully described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0045] The present invention provides a method for enhancing role perception of a large language model, such as Figure 1 As shown, the method includes the following steps:

[0046] S1. Clean and split the conversation dataset to obtain several conversations and corresponding labels.

[0047] S2. Establish a memory mechanism to obtain long-term and short-term historical conversations related to the current query;

[0048] S3. Combine the user's personality information and knowledge graph to build a user graph for the user;

[0049] S4. Use prompt learning and training through graph neural network.

[0050] In step S1, the input text contains a conversation dataset. This data includes the user's personality information, the chatbot's personality information, all conversation histories, and the time intervals between different conversations. All conversation data is described in natural languages ​​such as English and Chinese. To facilitate subsequent model training, the input text is processed, segmenting the conversation histories of different conversations according to the time intervals in the text, and filtering out sensitive information that appears in open-domain conversations. Each piece of data in the processed text contains the user's personality information, the chatbot's personality information, the user's current query, the chatbot's standard response, and previous conversation history.

[0051] In multi-session dialogue generation tasks, there will be multiple dialogue interactions between users, which can be divided into multiple sessions based on time intervals. Among them, long-term historical dialogue refers to the dialogue content in previous sessions, while short-term historical dialogue refers to the dialogue content in the current session.

[0052] In step S2, as Figure 2 As shown, in order to retrieve memories related to semantics and query topic information from historical conversations while retaining the original conversation structure as much as possible, the present invention establishes a memory mechanism for the large language model and designs topic-aware retrieval strategies for long-term historical conversations and short-term historical conversations respectively, thereby obtaining long-term historical conversations and short-term historical conversations related to the query.

[0053] (1) For long-term conversation histories, the top k conversation histories related to the current query are ranked and retrieved based on similarity, including queries and responses. Relevance can be calculated using the DPR model, Hamming distance, and other methods. The DPR model (Dense Passage Retrieval), proposed by Facebook AI Research, is a model for information retrieval that aims to improve retrieval efficiency and accuracy through dense vector representation. It converts user-entered questions and documents into vectors, then evaluates similarity by calculating the dot product between the query vector and the paragraph vector, thereby selecting the most relevant paragraphs as retrieval results.

[0054] Then, for the first k conversation histories, adjust the order, such as adjusting the order by time.

[0055] In one embodiment, the ConvED method is used to calculate the dialogue edit distance of the utterances in the k dialogue histories to obtain a score. The order is adjusted according to the score to preserve the structural information in the dialogue as much as possible. The formula is as follows:

[0056]

[0057] in, and They represent the i-th token of the first sentence and the j-th token of the second sentence respectively. and are their corresponding representations, and are their distance scores,

[0058] Conversation Edit Distance (ConvED) is a conversation similarity measurement method that combines edit distance and distributional semantics, taking into account factors such as the semantics of sentences, conversation flow, and participants in the conversation. This method reorders retrieved conversations based on their distance, partially restoring the conversational structure of conversations sorted by relevance.

[0059] (2) For short-term conversation histories, the topic transition detector is used to evaluate the topic relevance between the short-term conversation history and the current query. If the topic transition detector determines that an utterance in the short-term conversation is irrelevant to the topic of the current query, it is discarded. Ultimately, all conversation histories that are relevant to the current query topic and maintain the conversation structure are obtained.

[0060] Topic transition detectors can be trained using topic transition-aware conversation modeling datasets such as TIAGE. TIAGE is a human-annotated conversation dataset that focuses on topic transitions in open-domain conversations. It provides three tasks: topic transition detection, topic transition-triggered response generation, and topic-aware conversation generation, to study topic transition modeling in conversational settings.

[0061] In step 3, if Figure 3As shown, in order to overcome the problem of sparse persona in multi-session personalized dialogue, the present application introduces a knowledge graph containing character common sense and the connection between characters, such as PeaCoK, to expand the persona. First, the knowledge graph is processed to filter irrelevant personas; then, a persona triple extractor is trained using a natural language inference (NLI) task (such as the DNLI dataset) to traverse the dialogue dataset in step S1 and extract the present persona triple from the predefined persona and dialogue history. Then, in order to better link the knowledge graph, the triple is converted into a sentence form, such as triple <I am a bassist, feature, like music> is converted to "I am a bassist, here is my feature, like music", and the knowledge graph is linked by calculating the semantic similarity (for example, using the DPR model), that is, for all extracted persona triples and triples in the knowledge graph, similarity calculation is performed, and a certain threshold is set, the knowledge graph triples and persona triples with a similarity greater than the threshold are linked together, and then according to all extracted triples and linked triples, a user graph is constructed for each user. The nodes of the user graph include persona information or persona attributes, and the edges of the user graph consist of the relationships between the persona information or persona attributes, which include: having a family, having a job, favorite hobbies, etc.

[0062] In step S4, in order to better mine user portraits and filter irrelevant information, the present application designs a query-driven graph neural network, which adds an attention mechanism to calculate the weight of the query encoding vector and the encoding vector of the node in the graph compared to the existing graph neural network, as shown in Figure 4 As shown, the representation of each node in the graph is updated by introducing the weight of the query encoding vector. The specific operation is as follows:

[0063] (1) The representation of the nodes and edges in the graph is input into the embedding layer of the large language model to obtain the corresponding encoding vector;

[0064] (2) The query is input into the embedding layer of the large language model to obtain the encoding vector of the query, and the encoding vector of the query and the encoding vector of the node in the graph are calculated by the attention mechanism to assign weights. Assigning weights is a filtering method, because the weight of irrelevant nodes is relatively low, and after summing according to the weight, the irrelevant nodes contribute little to the final representation, which is equivalent to filtering irrelevant nodes in the graph. The existing graph neural network usually only focuses on the structure and node information on the graph, and ignores whether the nodes in the graph are related to the current dialogue or the current query. Therefore, the query-driven graph neural network of the present application can alleviate the influence of irrelevant noise on dialogue generation and mine deeper user portrait representation by filtering nodes irrelevant to the current query and guiding node aggregation and updating representation.

[0065] (3) The representations of all neighbor nodes of the current node according to different relationships in the user graph are aggregated to update the representation of the current node, and a user portrait representation vector is obtained through a pooling operation, that is, the representations of all nodes in the graph are averaged, maximized, etc., and the formula is as follows:

[0066]

[0067] wherein, wherein represents the node The neighbor index set under the relationship , is the node and the neighbor The attention score under the relationship , is the query representation in the current utterance, which is initialized in the same way as the node in the graph. , , , is a learnable parameter in the graph neural network. is a weight, and the user portrait is a pooling operation on all node representations, that is, all h are pooled to obtain the final user portrait.

[0068] DNLI (Dialogue Natural Language Inference) is a method of converting consistency problems in dialogue into a natural language inference (NLI) task. It creates a new NLI dataset containing sentence pairs labeled as entailment, neutral or contradiction to train the model to rearrange utterances in the dialogue model, thereby improving the consistency of the dialogue agent. In this invention, we introduce the dataset to train the personality triple extractor.

[0069] There are two main advantages of converting user representation into vectors. First, in multi-session dialogue generation, each person shows more personality information. If described in language, the content input to the large language model will be too long, which poses a challenge to the input size. However, the vector constructed by the present application only occupies one character (i.e. in the subsequent input, replace the [CHATBOTPERSONA] of only one character with the user portrait representation) to summarize the user's personality information. Existing methods mostly input personality information in the form of text, which generally has hundreds of characters to thousands, often exceeding the input length limit of the large language model. Moreover, not all personality information is directly related to the current conversation. Therefore, the present application uses a character to summarize a person's personality information, which is much smaller than the length of the input personality information text. Second, it is difficult to completely summarize a person's personality characteristics in language, so we describe the personality information in an abstract vector form (i.e. user portrait vector).

[0070] (4) Replace the predefined special character in the prompt word with the user portrait representation vector: set the hyperparameter, calculate the loss between the content generated by the large language model and the standard answer, and train using backpropagation.

[0071] The large language model inputs the following content: "As a communication expert with excellent communication habits, you embody the role of CHATBOT in the following conversation. Here are some of your unique personal characteristics: [CHATBOTPERSONA].\n<Conversation History>\nYour recent conversation with the user:\n[CONTEXT]\n\nNow, please continue the conversation between CHATBOT and the user as CHATBOT.\nThe user just said: [QUERY]\nPlease respond to the user using the following format (up to 50 words, must be in Chinese):\nResponse:\n", the input instruction prompt word contains the personality information, conversation history, current query and character description required in the task, which can be constructed by yourself. Among them, the predefined special character [CHATBOTPERSONA] in the prompt word is replaced by the user portrait representation vector according to the following formula:

[0072]

[0073] Where E is the embedded vector form of the user input (including queries, prompts, etc.), where , represent token and user portrait representation vector, is the dimension of the large language model, is the graph neural network, The representative graph is represented by q, the query, the MLP is a multi-layer perception to represent the semantic space of the large language model and the user portrait representation space, the learning rate, the number of iterations and other hyperparameters are set, the cross entropy is used as the loss function, the loss between the content generated by the large language model and the standard answer is calculated, the training is carried out by using the back propagation, the parameters of the original large language model are frozen during the training process, and the parameters of the graph neural network and the MLP are updated, and finally the trained model is obtained.

[0074] When the trained model is used, as shown in the formula I: Figure 5 The query, the dialogue history and the personality information are input, the user first processes the dialogue history, retrieves the relevant dialogue history under the premise of ensuring the dialogue structure, then extracts triples from the dialogue history and the personality information, and introduces the knowledge graph to construct the user graph by triple linkage, then the graph and the query are calculated by the graph neural network to obtain the user portrait representation, finally the query, the retrieved dialogue history and the user portrait representation are input into the large language model, and finally the personalized dialogue output is obtained, wherein the user portrait representation is inserted after the text is converted into a vector by the large language model.

[0075] The user input includes the prompt word, the current query, the dialogue history and the personality information.

[0076] The present application designs a user graph and a prompt learning based on a graph neural network on the basis of establishing a memory mechanism, which not only maintains the dialogue structure to enable the large language model to fully understand the dialogue background of long-term dialogue, but also enables the large language model to deeply understand and mine the user portrait, generate replies consistent with the dialogue history and personalized information, and attract the user's dialogue interest.

[0077] Experimental verification:

[0078] In one embodiment, the input text is the corpus MSC data set, and the paper Beyond Goldfish Memory: Long-Term Open-Domain Conversation of the 2022 ACL conference first proposes the open domain dialogue of multi-session dialogue, and proposes the data set.

[0079] The present application effectively enhances the performance of the large language model on the MSC data set. As shown in Table 1, ChatGLM-6B and Llama2-7B are used, and the method of the present application is used to do experiments on the MSC data set, and BlenderBot, HAHT, ChatGLM-6B (Base), Llama2-7B (Base), ChatGPT, CPD, StreamingDialogue, LD-Agent are used as baseline models to compare on the BLEU-1, BLEU-2, BLEU-3 and ROUGE-L indicators.

[0080] BlenderBot is an advanced open-domain dialogue model developed through large-scale pre-training on large-scale datasets, combined with a retrieval-refinement mechanism and optimized decoding strategies. HAHT is a model that uses hierarchical encoding and attention mechanisms to maintain and utilize long-term historical dialogue memory, generating contextually relevant responses through a history-aware response generator. ChatGPT is a closed-source large language model based on the GPT architecture, and we use the API service of the gpt-3.5-turbo model of OpenAI. CPD is a causal-aware multi-turn dialogue framework that uses a perturbation-based causal variable discovery method to extract statements with high causal relevance from historical dialogue, thereby enhancing the causal awareness of large language models. StreamingDialogue is a model that effectively handles long-context dialogues by compressing dialogue history into a "dialogue attention pool" and using short-term memory reconstruction (SMR) and long-term memory activation (LMR) learning strategies. LD-Agent is a dialogue agent framework that supports coherent dialogue by integrating event memory and personalized role modeling. It is the most advanced method on the MSC dataset. As shown in Table 1, the best results are in bold and the second best results are underlined.

[0081] Table 1

[0082] Model BLEU-1 BLEU-2 BLEU-3 ROUGE-L BlenderBot - 4.91 1.53 16.06 HAHT - 5.10 1.59 16.58 ChatGLM(Base) 19.20 5.54 1.50 16.49 Llama2(Base) 17.34 4.37 1.21 10.29 ChatGPT 18.96 5.77 1.51 16.84 CPD 12.45 4.41 - 12.14 StreamingDialogue 19.33 - - 15.86 LD-Agent 19.54 ]]> ​ 7.31 2.51 18.44 Ours(ChatGLM) 19.44 <![CDATA[ 7.51 ]]> 2.52 ]]> ​ 19.67 Ours(Llama2) 20.56 8.05 <![CDATA[ 2.53 ]]> ]]>

[0083] ​

[0083] Those skilled in the art can understand that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the above embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope defined by the claims of the present application.

Claims

1. A method for enhancing role perception of a large language model, characterized in that: The method comprises: S1. Process the conversation dataset to obtain several conversations and corresponding labels; S2. Establish a memory mechanism to obtain long-term and short-term historical conversations related to the current query; S3. Combine the user's personality information and knowledge graph to build a user graph for the user; S4, training using prompt learning through graph neural networks; In step S1, the conversation data includes the user's personality information, the chatbot's personality information, all conversation histories, and the time intervals between different conversations. The processed data includes the user's personality information, the chatbot's personality information, the user's current query, the chatbot's standard response, and previous conversation histories. In step S2, the long-term historical conversation refers to the conversation content in the previous session, and the short-term historical conversation refers to the conversation content in the current session; In step S3, the knowledge graph contains common sense about characters and connections between characters; the nodes of the user graph include personality information or personality attributes, and the edges of the user graph consist of relationships between personality information or personality attributes; In step S4, it includes: (1) Input the representation of nodes and edges in the graph into the embedding layer of the large language model to obtain the corresponding encoding vectors; (2) Input the query into the embedding layer of the large language model to obtain the encoding vector of the query, and calculate the weight of the encoding vector of the query and the encoding vector of the nodes in the graph through the attention mechanism; (3) Aggregate the representations of all neighboring nodes of a node according to different relationships in the user graph to update the representation of the current node. Through the pooling operation, the user portrait representation vector is obtained. The formula is as follows: Among them, Indicates that node i is in the relationship The neighbor index set under is the attention score between node i and neighbor j under relation r, H q is the query representation in the current utterance, which is initialized in the same way as the nodes in the graph; is a learnable parameter in the graph neural network; is the weight, and the user portrait is pooled by all node representations, that is, all h are pooled to obtain the final user portrait; (4) Replace the predefined special characters in the prompt word with the user portrait representation vector: set hyperparameters, calculate the loss between the large language model generated content and the standard answer, and use backpropagation for training; Replace the predefined special characters in the prompt word with the user profile representation vector according to the following formula: Among them, E is the embedding vector form of user input, where Represents token t i and user portrait representation vector, d is the dimension of the large language model, f φ is a graph neural network, Represents graph, q represents query, and MLP is a multi-layer perceptron that represents the semantic space and user portrait space of its large language model.

2. The method according to claim 1, characterized in that In step S2, for the long-term conversation history, the top k conversation histories related to the current query are ranked and retrieved according to similarity, and the order of the conversation histories is adjusted.

3. The method according to claim 2, characterized in that Methods for adjusting the order of conversation histories include: using a ConvED method, calculating conversation edit distances of k utterances in conversation histories, obtaining scores, and adjusting the order based on the scores.

4. The method according to claim 1, wherein In step S2, for the short-term dialogue history, the topic relevance between the short-term dialogue history and the current query is evaluated by a topic transition detector; if the utterance in the short-term dialogue is irrelevant to the topic of the current query, it is discarded.

5. The method according to claim 1, wherein In step S3, the method for constructing the user graph includes: Process the knowledge graph and filter out irrelevant personality information; Using the personality triplet extractor, traverse the conversation dataset in step S1 and extract the personality triples that appear from the predefined personality information and conversation history; Calculate the similarity between all extracted personality triplets and triplets in the knowledge graph, set a certain threshold, and link the knowledge graph triplets and personality triplets whose similarity is greater than the threshold; Construct a user graph, where the nodes of the user graph include personality information or personality attributes, and the edges of the user graph consist of the relationships between personality information or personality attributes.

6. The method according to claim 1, characterized in that During training, the parameters of the large language model are frozen, and the parameters of the graph neural network and MLP are updated.

Images