A long-term memory-based question and answer generation method, device and equipment
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA CONSTRUCTION BANK
- Filing Date
- 2026-03-23
- Publication Date
- 2026-06-19
Smart Images
Figure CN122242758A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of artificial intelligence technology and natural language processing, and in particular to a question-and-answer generation method, apparatus and device based on long-term memory. Background Technology
[0002] Currently, intelligent dialogue systems based on large language models have demonstrated powerful general-purpose capabilities. However, existing models have fundamental limitations in terms of memory and continuous learning mechanisms. The model's "memory" is strictly limited by its fixed context window length. Once the number of dialogue turns or the total amount of text exceeds this limit, or the conversation connection is interrupted, the model will completely forget the previous interaction content and will be unable to maintain long-term, coherent, personalized dialogue.
[0003] To address the aforementioned needs for long-term memory and specific knowledge learning, the current mainstream approach is supervised fine-tuning of pre-trained models. This method updates model parameters by preparing high-quality datasets specific to a particular domain, enabling the model to "memorize" new knowledge. However, this approach has inherent drawbacks that are difficult to overcome. The existing supervised fine-tuning of pre-trained models requires extremely high data and computational costs, primarily relying on large-scale, high-precision, and balanced labeled datasets. The preparation process is professional and expensive, and the training itself consumes significant GPU computing resources, is time-consuming, and the final performance improvement is uncertain. Summary of the Invention
[0004] This application provides a method, apparatus, device, computer-readable storage medium, and computer product for question-and-answer generation based on long-term memory. The method uses a first knowledge base and a second knowledge base to perform parallel retrieval to obtain the corresponding basic knowledge of the target and the target's historical dialogue information before the question-and-answer model is used. This constructs an enhanced input that integrates authoritative facts and historical experience, ensuring that the knowledge acquired by the question-and-answer model is both accurate and context-relevant, thereby improving the accuracy and timeliness of the generated answers.
[0005] In a first aspect, embodiments of this application propose a question-and-answer generation method based on long-term memory, comprising: The input text is subjected to intent recognition, and when the text is determined to be a first text carrying a question based on the recognized text intent, semantic extraction is performed on the first text. Based on the extracted semantic features, the associated target basic knowledge is retrieved from the first knowledge base, and the associated target historical dialogue information is retrieved from the second knowledge base. The first knowledge base includes at least one pre-stored basic knowledge, and the second knowledge base includes at least one set of dynamically updated historical dialogue information, which includes historical first text and corresponding historical text answers. The first text, the target basic knowledge, and the target historical dialogue information are simultaneously input into the question answering model. The question answering model is used to generate and output the target text answer corresponding to the first text. The question answering model is pre-trained to generate the text answer based on predefined prompt words, prioritizing historical dialogue information and combining it with basic knowledge. The first text and the target text answer are stored as a set of historical dialogue information in the second knowledge base to dynamically update the second knowledge base.
[0006] Using the above method, when the current input text is the first text carrying the question, the system simultaneously retrieves the most relevant basic knowledge and historical dialogue information from two knowledge bases, integrating them into enhanced prompts to guide the question-answering model to answer based on a complete knowledge background and past interaction experience. Furthermore, it supports real-time feedback and calibration of the model's output, and the calibrated high-quality dialogues are synchronously added to the historical dialogue database. Through this mechanism, the question-answering model achieves an organic combination of static knowledge retrieval and dynamic experience learning without retraining, gradually evolving into a more accurate and personalized domain expert through continuous interaction.
[0007] In some possible embodiments, retrieving associated target basic knowledge from the first knowledge base based on the extracted semantic features includes: Calculate the first correlation coefficient between the semantic features and each basic knowledge in the first knowledge base; The highest correlation coefficient is determined from the first correlation coefficients and taken as the target first correlation coefficient. The basic knowledge corresponding to the target first correlation coefficient is determined as the target basic knowledge corresponding to the first text.
[0008] In some possible embodiments, retrieving associated target historical dialogue information from a second knowledge base based on extracted semantic features includes: Calculate the second correlation coefficient between the semantic features and each set of historical dialogue information in the second knowledge base; The highest correlation coefficient is determined from the second correlation coefficients and taken as the target second correlation coefficient. The historical dialogue information corresponding to the target second correlation coefficient is determined as the target historical dialogue information corresponding to the first text.
[0009] In some possible embodiments, calculating the second association coefficient between the semantic features and each set of historical dialogue information in the second knowledge base includes: Obtain the attribute information of each set of historical dialogue information, and determine the confidence level of each set of historical dialogue information based on the attribute information; For each set of historical dialogue information, the second correlation coefficient is determined based on the confidence level and the degree of matching between the content of the historical dialogue information and the semantic features.
[0010] In some possible embodiments, the attribute information includes at least the dialogue time point and the job level of the dialogue initiator. Determining the confidence level of each set of historical dialogue information based on the attribute information includes: Based on the chronological order of the dialogue timestamps, a first confidence level is generated for each group of historical dialogue information. Based on the job level of the dialogue initiator, a second confidence level is generated for each set of historical dialogue information; The confidence level corresponding to each group of historical dialogue information is determined based on the first confidence level, the second confidence level, and the weight coefficients pre-assigned to the first confidence level and the second confidence level.
[0011] In some possible embodiments, the question-answering model is adjusted through the following steps: The training set is input into the question-answering model to be trained. The training set includes multiple sets of interrelated sample first text, sample basic knowledge, sample simulated historical dialogue information, and standard text answers. Based on predefined prompts, the question-answering model is guided to prioritize the historical dialogue knowledge of the samples and combine it with the basic knowledge of the samples to generate predicted text answers. The parameters of the question-answering model are adjusted with the goal of minimizing the difference between the predicted text answer and the standard text answer.
[0012] In some possible embodiments, after generating and outputting the target text answer corresponding to the first text, the method further includes: The input text is subjected to intent recognition, and when the text is determined to be a second text carrying a correction text answer based on the recognized text intent, the correction text answer is obtained; The first text answer and the corrected text answer are stored as a set of historical dialogue information in the second knowledge base to update the second knowledge base.
[0013] Secondly, embodiments of this application also propose a question-and-answer generation device based on long-term memory, comprising: The intent recognition module is used to recognize the intent of the input text, and when the text is determined to be a first text carrying a question based on the recognized text intent, semantic extraction is performed on the first text. The retrieval module is used to retrieve associated target basic knowledge from a first knowledge base and associated target historical dialogue information from a second knowledge base based on the extracted semantic features. The first knowledge base includes at least one pre-stored basic knowledge, and the second knowledge base includes at least one dynamically updated set of historical dialogue information, which includes a historical first text and the corresponding historical text answer. The answer generation module is used to synchronously input the first text, the target basic knowledge, and the target historical dialogue information into the question answering model, and use the question answering model to generate and output the target text answer corresponding to the first text. The question answering model is pre-trained to generate text answers based on predefined prompt words, prioritizing historical dialogue information and combining basic knowledge. The dynamic storage module is used to store the first text and the target text answer as a set of historical dialogue information in the second knowledge base, so as to dynamically update the second knowledge base.
[0014] The aforementioned device retrieves the most relevant basic knowledge and historical dialogue information from two knowledge bases and integrates them into enhanced prompts. This guides the question-answering model to answer based on a complete knowledge background and past interaction experience, achieving an organic combination of static knowledge retrieval and dynamic experience learning. Through continuous model interaction, the model gradually evolves into a more accurate and personalized domain expert.
[0015] In some possible embodiments, the retrieval module is specifically used for: Calculate the first correlation coefficient between the semantic features and each basic knowledge in the first knowledge base; The highest correlation coefficient is determined from the first correlation coefficients and taken as the target first correlation coefficient. The basic knowledge corresponding to the target first correlation coefficient is determined as the target basic knowledge corresponding to the first text.
[0016] In some possible embodiments, the retrieval module is specifically used for: Calculate the second correlation coefficient between the semantic features and each set of historical dialogue information in the second knowledge base; The highest correlation coefficient is determined from the second correlation coefficients and taken as the target second correlation coefficient. The historical dialogue information corresponding to the target second correlation coefficient is determined as the target historical dialogue information corresponding to the first text.
[0017] In some possible embodiments, the retrieval module is further configured to: Obtain the attribute information of each set of historical dialogue information, and determine the confidence level of each set of historical dialogue information based on the attribute information; For each set of historical dialogue information, the second correlation coefficient is determined based on the confidence level and the degree of matching between the content of the historical dialogue information and the semantic features.
[0018] In some possible embodiments, the attribute information includes at least the dialogue time point and the job level of the dialogue initiator, and the retrieval module is further used for: Based on the chronological order of the dialogue timestamps, a first confidence level is generated for each group of historical dialogue information. Based on the job level of the dialogue initiator, a second confidence level is generated for each set of historical dialogue information; The confidence level corresponding to each group of historical dialogue information is determined based on the first confidence level, the second confidence level, and the weight coefficients pre-assigned to the first confidence level and the second confidence level.
[0019] In some possible embodiments, the question-answering model is adjusted through the following steps: The training set is input into the question-answering model to be trained. The training set includes multiple sets of interrelated sample first text, sample basic knowledge, sample simulated historical dialogue information, and standard text answers. Based on predefined prompts, the question-answering model is guided to prioritize the historical dialogue knowledge of the samples and combine it with the basic knowledge of the samples to generate predicted text answers. The parameters of the question-answering model are adjusted with the goal of minimizing the difference between the predicted text answer and the standard text answer.
[0020] In some possible embodiments, after generating and outputting the target text answer corresponding to the first text, the intent recognition module is further configured to: The input text is subjected to intent recognition, and when the text is determined to be a second text carrying a correction text answer based on the recognized text intent, the correction text answer is obtained; The first text answer and the corrected text answer are stored as a set of historical dialogue information in the second knowledge base to update the second knowledge base.
[0021] Thirdly, embodiments of this application also propose an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform steps in a question-and-answer generation method based on long-term memory as described in any of the first aspects of the above embodiments.
[0022] Fourthly, embodiments of this application also propose a computer-readable storage medium storing computer-executable instructions for performing steps in a question-answer generation method based on long-term memory as described in any of the first aspects of the above embodiments.
[0023] Fifthly, embodiments of this application also propose a computer program product comprising: computer program code, which, when executed on a computer, causes the computer to perform the steps of a question-and-answer generation method based on long-term memory as described in any of the first aspects of the above embodiments.
[0024] The question-answering generation method, apparatus, device, computer-readable storage medium, and computer program product based on long-term memory described in the above embodiments of this application, by parallelly searching a first knowledge base and a dynamically updated second knowledge base before using the question-answering model, constructs an enhanced input that integrates authoritative facts and historical experience (or real-time correction), ensuring that the knowledge acquired by the question-answering model is both accurate and context-relevant. This enables the question-answering model to absorb real-time corrections from experts, deposit high-quality feedback into long-term memory, continuously improve the accuracy and reliability of subsequent question-answering, and promote the model's autonomous evolution towards becoming a domain expert.
[0025] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description
[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 This is a flowchart illustrating a question-and-answer generation method based on long-term memory, as described in an embodiment of this application. Figure 2 This is a schematic diagram of a process for determining basic knowledge of a target in an embodiment of this application; Figure 3 This is a schematic diagram of a process for determining target historical dialogue information in an embodiment of this application; Figure 4 This is a schematic diagram of a process for performing domain-adaptive fine-tuning of a base model in an embodiment of this application; Figure 5 This is a flowchart illustrating one of the correction and feedback learning stages in an embodiment of this application. Figure 6 This is a schematic diagram illustrating a process for completing one question and answer session according to an embodiment of this application; Figure 7 This is a schematic diagram of a question-and-answer generation device based on long-term memory in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation
[0028] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.
[0029] The terms "first" and "second" in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. The term "multiple" in this application can mean at least two, for example, two, three, or more, and the embodiments of this application do not impose limitations.
[0030] The following explanations of some terms used in the embodiments of the present invention are provided to facilitate understanding by those skilled in the art.
[0031] (1) Question answering model refers to an AI model specifically designed to understand questions raised by users and to find or generate the most relevant answers from a given information source. Its goal is to achieve precise "question and answer" interaction. (2) Fine-tuning training refers to the technique of using datasets of specific domains and tasks for additional training on the basis of a general large language model (which has been pre-trained on massive Internet texts). Through fine-tuning, the model can be taught to master professional terms, specific formats or follow specific instructions, making it more suitable for a certain vertical domain.
[0032] (3) Knowledge base, specifically referring to quantitative knowledge base, is an efficient database that converts unstructured text (such as historical chat records) into vectors (i.e. a set of numbers that can represent semantics) using an embedding model and stores them. (4) Long-term memory refers to the ability of AI systems to break through the context length limit of a single dialogue, to persistently remember and utilize earlier historical information in future dialogues; (5) Enhanced input, also known as “context enhancement” or “RAG (Retrieval-Augmented Generation)”, refers to finding relevant background information through retrieval (such as from knowledge bases or documents) before submitting the user’s original question to the question answering model. Then, the original question and the retrieved relevant background information are used as new, more informative input to the model.
[0033] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These embodiments should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that in the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solutions of this application, and do not imply that the applicant has already used or necessarily used such solutions.
[0034] The acquisition, transmission, storage, and use of data in this application all comply with the requirements of relevant national laws and regulations.
[0035] Before introducing the question-and-answer generation method, apparatus, device, computer-readable storage medium, and computer program product based on long-term memory provided in the embodiments of this application, the technical background of the embodiments of this application will be described in detail below for ease of understanding.
[0036] Currently, intelligent dialogue systems based on large language models have demonstrated powerful general-purpose capabilities. However, existing models have fundamental limitations in terms of memory and continuous learning mechanisms. Current AI large language models have limited memory capacity; their "memory" is strictly constrained by their fixed context window length. For a chat topic, once the number of dialogue rounds or the total amount of text exceeds this limit, or if the conversation connection is interrupted, the model will completely forget previous interactions, making it unable to maintain long-term, coherent, personalized dialogue.
[0037] To address the aforementioned needs for long-term memory and specific knowledge learning, the current mainstream technical approach is supervised fine-tuning of pre-trained models. This method updates model parameters by preparing high-quality datasets specific to a particular domain, enabling the model to "memorize" new knowledge. However, this approach has inherent and insurmountable drawbacks. The existing supervised fine-tuning of pre-trained models requires extremely high data and computational costs, primarily relying on large-scale, high-precision, and balanced labeled datasets. Furthermore, it necessitates creating three datasets (training, validation, and test sets), each rich in data and encompassing various scenarios, with a balanced proportion among the datasets. Poor dataset quality will severely impact the accuracy of the trained model.
[0038] Secondly, the preparation of the aforementioned datasets is a professional and expensive process, and the training itself consumes a huge amount of GPU computing resources. Currently, GPU resources are relatively scarce and it is difficult to meet the needs of model training. Moreover, the training effect cannot be guaranteed, and the final performance improvement is uncertain.
[0039] Therefore, in view of the technical problems existing in the above-mentioned prior art, such as model training relying on expensive fine-tuning and difficulty in achieving sustainable long-term memory and dynamic knowledge accumulation, this application proposes a question-answering generation method based on long-term memory, such as... Figure 1 As shown, the method includes: Step S101: Perform intent recognition on the input text, and if the text is determined to be a first text carrying a question based on the recognized text intent, perform semantic extraction on the first text. Step S102: Based on the extracted semantic features, retrieve the associated basic knowledge of the target from the first knowledge base and the associated historical dialogue information of the target from the second knowledge base. The first knowledge base includes at least one pre-stored basic knowledge, and the second knowledge base includes at least one dynamically updated set of historical dialogue information, wherein the historical dialogue information includes a historical first text and the corresponding historical text answer. Step S103: The first text, the target basic knowledge, and the target historical dialogue information are synchronously input into the question-answering model, and the question-answering model is used to generate and output the target text answer corresponding to the first text; The question-answering model is pre-trained to generate text answers based on predefined prompts, prioritizing historical dialogue information and combining it with basic knowledge. Step S104: Store the first text and the target text answer as a set of historical dialogue information in the second knowledge base to dynamically update the second knowledge base.
[0040] In this embodiment, when the currently input text is identified as the first text carrying a question, the system simultaneously retrieves the most relevant basic knowledge and historical dialogue information from two knowledge bases and integrates them into enhanced prompts. This guides the question-answering model to answer based on a complete knowledge background and past interaction experience. Furthermore, it supports real-time feedback and calibration of the model's output, and the calibrated high-quality dialogues are synchronously added to the historical dialogue database. Through this mechanism, the question-answering model achieves an organic combination of static knowledge retrieval and dynamic experience learning without retraining, gradually evolving into a more accurate and personalized domain expert through continuous interaction.
[0041] In step S101 above, any existing intent recognition model can be used, such as a fine-tuned classifier based on pre-trained models like BERT (Bidirectional Encoder Representations from Transformers) or RoBERTa (A Robustly Optimized BERT Approach), or a hybrid recognition system based on rules and machine learning. The specific implementation method can be flexibly selected according to the complexity of the actual scenario, the data scale, and the real-time requirements. This application embodiment does not impose specific limitations here.
[0042] Furthermore, semantic extraction (i.e. vectorization) of the first text can be performed using any existing text embedding model to achieve the conversion from text to high-dimensional semantic vectors. The specific implementation technology is not the core of this solution, and the embodiments of this application are not specifically limited here.
[0043] In some possible embodiments, each basic knowledge in the first knowledge base is stored in vector form, specifically: First, by parsing the source code files of basic knowledge using an abstract syntax tree, the definitions of entities such as modules, classes, interfaces, and functions, as well as their calls and dependencies, are automatically extracted, thus constructing a structured basic knowledge graph. Then, based on the clear hierarchical logic inherent in this graph, text content describing the same core functional entity is aggregated and identified and encapsulated using predefined delimiters, ensuring the integrity of the logical unit in subsequent processing. Finally, the structured text with delimiters is uniformly converted into a markup language format. When uploaded to the first vector knowledge base, the delimiter is explicitly specified as the basis for text segmentation, enabling the knowledge base to store each encapsulated logical unit completely in a single vector fragment when creating the basic knowledge vector index. This maintains the semantic integrity of basic knowledge during vectorized storage, laying a reliable data foundation for subsequent high-precision semantic retrieval and analysis.
[0044] In step S102 of this application embodiment, the relevant target basic knowledge is retrieved from the first knowledge base based on the extracted semantic features, such as... Figure 2 As shown, it includes: Step S201: Calculate the first correlation coefficient between the semantic features and each basic knowledge in the first knowledge base; Specifically, the semantic features are typically in vector form. After extracting the semantic feature vector of the first text, each piece of basic knowledge stored in the first knowledge base (i.e., the basic knowledge base) is traversed and queried. For each piece of basic knowledge, a first correlation coefficient is calculated between its vectorized representation and the semantic feature vector of the first text. The first correlation coefficient is typically measured using methods such as cosine similarity, inverse Euclidean distance, or dot product to quantify the degree of correlation between the two in the semantic space. Step S202: Determine the highest correlation coefficient from the first correlation coefficients as the target first correlation coefficient, and determine the basic knowledge corresponding to the target first correlation coefficient as the target basic knowledge corresponding to the first text; Specifically, the highest relevance can be determined by sorting or maximum value selection algorithms, and the basic knowledge corresponding to the first relevance coefficient can be determined as the target basic knowledge that matches the current first text query.
[0045] The method for determining the first correlation coefficient described in this application embodiment aims to identify the target basic knowledge that is semantically closest to and most relevant to the current query, and then use this target basic knowledge as the core reference for generating subsequent answers.
[0046] In step S102 of this embodiment, the retrieval of the aforementioned historical dialogue information not only considers its semantic matching degree but also introduces a multi-attribute weighting mechanism to achieve intelligent evaluation of the confidence level of the historical dialogue information, thereby improving the timeliness and authority of the retrieval results. The specific steps are as follows: Figure 3 As shown, it includes: Step S301: Calculate the second correlation coefficient between the semantic features and each group of historical dialogue information in the second knowledge base; This step is achieved through the following steps: Step 1: Obtain the attribute information of each group of historical dialogue information, and determine the confidence level of each group of historical dialogue information based on the attribute information. The attribute information includes at least the dialogue time point and the job level of the dialogue initiator. The step of determining the confidence level of each group of historical dialogue information based on the attribute information includes the following steps: Based on the chronological order of the dialogue timestamps, a first confidence level is generated for each group of historical dialogue information. Based on the job level of the dialogue initiator, a second confidence level is generated for each set of historical dialogue information; The confidence level corresponding to each group of historical dialogue information is determined based on the first confidence level, the second confidence level, and the weight coefficients pre-assigned to the first confidence level and the second confidence level.
[0047] In this embodiment, historical dialogue information with a more recent dialogue time generally has a higher first confidence level to reflect the timeliness of the information. For example, a time decay function can be used for calculation. Secondly, dialogues initiated by higher-level officials (such as domain experts or senior administrators) typically have a higher second confidence level to reflect the authoritative value of the information; a mapping relationship between level and confidence level can be preset. Finally, the final confidence level corresponding to each group of historical dialogue information is determined through weighted summation or other fusion methods. The weighting coefficients can be adjusted according to business needs; for example, a higher weight can be assigned to the first confidence level in rapidly changing fields, and a higher weight can be assigned to the second confidence level in specialized fields.
[0048] Step 2: For each set of historical dialogue information, determine the second correlation coefficient based on the confidence level and the degree of matching between the content of the historical dialogue information and the semantic features; Specifically, for each set of historical dialogue information, its confidence level is combined with the degree of matching (such as cosine similarity) between its content vector and the semantic feature vector of the first text to calculate a second association coefficient. The combination method can be product, weighted sum or rule-based comprehensive score to ensure that the retrieval results take into account both content relevance and information quality, that is, the timeliness and authority of the historical dialogue information.
[0049] Step S302: Determine the highest correlation coefficient from the second correlation coefficients as the target second correlation coefficient, and determine the historical dialogue information corresponding to the target second correlation coefficient as the target historical dialogue information corresponding to the first text.
[0050] In some possible embodiments, the highest value of the second correlation coefficient is selected as the target second correlation coefficient, and the historical dialogue information corresponding to the target second correlation coefficient is determined as the target historical dialogue information most relevant to the current first text, which is used for subsequent text answer generation.
[0051] By introducing multi-dimensional attribute information, including the dialogue time and the job level of the dialogue initiator, and calculating the confidence level accordingly to correct the retrieval results of pure semantic matching, it is possible to prioritize the recall of newer and more authoritative historical dialogue information. This effectively simulates the human tendency to refer to the latest instructions and value expert opinions when making decisions, and significantly improves the practicality and intelligence level of long-term memory retrieval.
[0052] In some possible embodiments, the first text, the target basic knowledge, and the target historical dialogue information are simultaneously input into the question-answering model in a structured prompt format. The three are integrated according to a preset logical template to form a unified enhanced context input sequence. Then, the question-answering model is used to generate and output the target text answer corresponding to the first text.
[0053] Optionally, the base model of the question-answering model can be implemented using existing large language models, such as the GPT (Generative Pre-trained Transformer) series, ChatGLM (ChatGenerative Language Model), Claude, LLaMA (Large Language Model Meta AI), and their fine-tuned variants, etc., and is not specifically limited in this embodiment. Based on this base model, a dedicated question-answering system with continuous learning capabilities is constructed using the long-term memory-based question-answering generation method described in this embodiment.
[0054] In some possible embodiments, to make the question-answering model more accurately follow the priority reference to historical dialogues and incorporate basic knowledge-based answering logic, its base model can be fine-tuned using domain-adaptive methods, such as... Figure 4 As shown, adjustments can be made through the following steps: Step S401: Input the training set into the question answering model to be trained. The training set includes multiple sets of interrelated sample first text, sample basic knowledge, sample simulated historical dialogue information, and standard text answers. Step S402: Based on predefined prompt words, guide the question-answering model to generate a predicted text answer by prioritizing the historical dialogue knowledge of the sample and combining it with the basic knowledge of the sample. Step S403: Adjust the parameters of the question-answering model with the goal of minimizing the difference between the predicted text answer and the standard text answer.
[0055] In the embodiments of this application, the adjusted question-answering model can stably follow the decision logic of prioritizing historical dialogues and verifying basic knowledge, significantly improving the accuracy and contextual consistency of the answers, and effectively overcoming the shortcomings of the general base model in professional scenarios, which exhibits randomness and poor controllability.
[0056] In some possible embodiments, in order to build a closed-loop learning system to achieve continuous model optimization, after generating and outputting the target text answer corresponding to the first text, a correction and feedback learning phase is also included, specifically as follows: Figure 5 As shown, it includes: Step S501: Perform intent recognition on the input text, and based on the recognized text intent, if it is determined that the text is a second text carrying a correction text answer, obtain the correction text answer; Step S502: Store the first text answer and the corrected text answer as a set of historical dialogue information in the second knowledge base to update the second knowledge base.
[0057] In this embodiment, a closed loop of generation, feedback, storage, and relearning is constructed through the dynamic update in step S104 and the correction process described above. Benefiting from the dialogue time point attribute information recorded in the historical dialogue information, subsequent retrievals can automatically prioritize correct answers with more recent dialogue times that have already been corrected. This mechanism essentially introduces a timeliness weight to the historical dialogue information, allowing incorrect answers to be naturally covered by subsequent correct feedback. This achieves dynamic optimization and error filtering of the long-term memory, ensuring the accuracy and version advancement of the knowledge learned by the model.
[0058] The following describes a specific process for completing a question-and-answer session using the aforementioned long-term memory-based question-and-answer generation method: Figure 6 As shown, it includes: Step S601: The application performs intent recognition on the input text to determine that the text is a text to be answered carrying a question, and performs semantic extraction on the text to be answered; Step S602: Based on the extracted semantic features, retrieve the associated basic knowledge of the target from the first knowledge base and the associated historical dialogue information of the target from the second knowledge base. Step S603: The text to be answered, the target basic knowledge, and the target historical dialogue information are synchronously input into the question-answering model, and the question-answering model is used to generate and output the target text answer corresponding to the text to be answered. Step S604: Store the text to be answered and the target text answer as a set of historical dialogue information in the second knowledge base.
[0059] The question-answering generation method based on long-term memory described in this application constructs an enhanced input that integrates authoritative facts and historical experience (or real-time corrections) by parallel searching a first knowledge base and a dynamically updated second knowledge base before using the question-answering model. This ensures that the knowledge acquired by the question-answering model is both accurate and context-relevant, enabling the question-answering model to absorb real-time corrections from experts, deposit high-quality feedback into long-term memory, continuously improve the accuracy and reliability of subsequent question-answering, and promote the model's autonomous evolution towards becoming a domain expert.
[0060] Based on the same inventive concept, embodiments of this application also propose a question-and-answer generation device based on long-term memory, such as... Figure 7 As shown, it includes: The intent recognition module 701 is used to perform intent recognition on the input text, and when it is determined that the text is a first text carrying a question based on the recognized text intent, it performs semantic extraction on the first text. The retrieval module 702 is used to retrieve associated target basic knowledge from a first knowledge base and associated target historical dialogue information from a second knowledge base based on the extracted semantic features. The first knowledge base includes at least one pre-stored basic knowledge, and the second knowledge base includes at least one dynamically updated set of historical dialogue information, wherein the historical dialogue information includes a historical first text and the corresponding historical text answer. The answer generation module 703 is used to synchronously input the first text, the target basic knowledge, and the target historical dialogue information into the question answering model, and use the question answering model to generate and output the target text answer corresponding to the first text. The question answering model is pre-trained to generate the text answer based on predefined prompt words, prioritizing historical dialogue information and combining it with basic knowledge. The dynamic storage module 704 is used to store the first text and the target text answer as a set of historical dialogue information in the second knowledge base, so as to dynamically update the second knowledge base.
[0061] In some possible embodiments, the retrieval module is specifically used for: Calculate the first correlation coefficient between the semantic features and each basic knowledge in the first knowledge base; The highest correlation coefficient is determined from the first correlation coefficients and taken as the target first correlation coefficient. The basic knowledge corresponding to the target first correlation coefficient is determined as the target basic knowledge corresponding to the first text.
[0062] In some possible embodiments, the retrieval module is specifically used for: Calculate the second correlation coefficient between the semantic features and each set of historical dialogue information in the second knowledge base; The highest correlation coefficient is determined from the second correlation coefficients and taken as the target second correlation coefficient. The historical dialogue information corresponding to the target second correlation coefficient is determined as the target historical dialogue information corresponding to the first text.
[0063] In some possible embodiments, the retrieval module is further configured to: Obtain the attribute information of each set of historical dialogue information, and determine the confidence level of each set of historical dialogue information based on the attribute information; For each set of historical dialogue information, the second correlation coefficient is determined based on the confidence level and the degree of matching between the content of the historical dialogue information and the semantic features.
[0064] In some possible embodiments, the attribute information includes at least the dialogue time point and the job level of the dialogue initiator, and the retrieval module is further used for: Based on the chronological order of the dialogue timestamps, a first confidence level is generated for each group of historical dialogue information. Based on the job level of the dialogue initiator, a second confidence level is generated for each set of historical dialogue information; The confidence level corresponding to each group of historical dialogue information is determined based on the first confidence level, the second confidence level, and the weight coefficients pre-assigned to the first confidence level and the second confidence level.
[0065] In some possible embodiments, the question-answering model is trained through the following steps: The training set is input into the question-answering model to be trained. The training set includes multiple sets of interrelated sample first text, sample basic knowledge, sample simulated historical dialogue information, and standard text answers. Based on predefined prompts, the question-answering model is guided to prioritize the historical dialogue knowledge of the samples and combine it with the basic knowledge of the samples to generate predicted text answers. The parameters of the question-answering model are adjusted with the goal of minimizing the difference between the predicted text answer and the standard text answer.
[0066] In some possible embodiments, after generating and outputting the target text answer corresponding to the first text, the intent recognition module is further configured to: The input text is subjected to intent recognition, and when the text is determined to be a second text carrying a correction text answer based on the recognized text intent, the correction text answer is obtained; The first text answer and the corrected text answer are stored as a set of historical dialogue information in the second knowledge base to update the second knowledge base.
[0067] The aforementioned device retrieves the most relevant basic knowledge and historical dialogue information from two knowledge bases and integrates them into enhanced prompts. This guides the question-answering model to answer based on a complete knowledge background and past interaction experience, achieving an organic combination of static knowledge retrieval and dynamic experience learning. Through continuous model interaction, the model gradually evolves into a more accurate and personalized domain expert.
[0068] Based on the same inventive concept, embodiments of this application propose an electronic device, including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform steps in a question-and-answer generation method based on long-term memory as described in any of the first aspects of the above embodiments.
[0069] The following reference Figure 8 This application describes an electronic device 800 according to one embodiment of the present application. Figure 8 The device 800 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0070] like Figure 8 As shown, an electronic device 800 is presented in the form of a general-purpose electronic device. The components of an electronic device 800 may include, but are not limited to: at least one processor 801, at least one memory 802, and a bus 803 connecting different system components (including memory 802 and processor 801).
[0071] Bus 803 represents one or more of several bus structures, including a memory bus or memory controller, peripheral bus, processor, or a local bus using any of the various bus structures.
[0072] The memory 802 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 8021 and / or cache memory 8022, and may further include read-only memory (ROM) 8023.
[0073] The memory 802 may also include a program / utility 8025 having a set (at least one) of program modules 8024, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.
[0074] An electronic device 800 can also communicate with one or more external devices 804 (e.g., keyboard, pointing device, etc.), and with one or more devices that enable a user to interact with the electronic device 800, and / or with any device that enables the electronic device 800 to communicate with one or more other electronic devices (e.g., router, modem, etc.). This communication can be performed via an input / output (I / O) interface 805. Furthermore, the electronic device 800 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via a network adapter 806. As shown, the network adapter 806 communicates with other modules used in the electronic device 800 via a bus 803. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0075] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program. The computer program includes program instructions, which, when executed by a computer, cause the computer to perform the steps described in any of the long-term memory-based question-and-answer generation methods previously discussed. Since the principle by which the above-described computer-readable storage medium solves the problem is similar to that of the long-term memory-based question-and-answer generation methods, the implementation of the above-described computer-readable storage medium can be referred to the implementation of the method; repeated details will not be elaborated further.
[0076] Based on the same inventive concept, this application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to perform the steps of any of the long-term memory-based question-and-answer generation methods discussed above. Since the principle of the above-described computer program product in solving problems is similar to that of the long-term memory-based question-and-answer generation method, the implementation of the above-described computer program product can refer to the implementation of the method, and repeated details will not be described again.
[0077] The question-answering generation method, apparatus, device, computer-readable storage medium, and program product based on long-term memory described in this application constructs an enhanced input that integrates authoritative facts and historical experience (or real-time correction) by parallel searching a first knowledge base and a dynamically updated second knowledge base before using the question-answering model. This ensures that the knowledge acquired by the question-answering model is both accurate and context-relevant, enabling the question-answering model to absorb real-time corrections from experts, deposit high-quality feedback into long-term memory, continuously improve the accuracy and reliability of subsequent question-answering, and promote the model's autonomous evolution towards becoming a domain expert.
[0078] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0079] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0080] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0081] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0082] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.
Claims
1. A question-and-answer generation method based on long-term memory, characterized in that, include: The input text is subjected to intent recognition, and when the text is determined to be a first text carrying a question based on the recognized text intent, semantic extraction is performed on the first text. Based on the extracted semantic features, the associated target basic knowledge is retrieved from the first knowledge base, and the associated target historical dialogue information is retrieved from the second knowledge base. The first knowledge base includes at least one pre-stored basic knowledge, and the second knowledge base includes at least one set of dynamically updated historical dialogue information, which includes historical first text and corresponding historical text answers. The first text, the target basic knowledge, and the target historical dialogue information are simultaneously input into the question answering model. The question answering model is used to generate and output the target text answer corresponding to the first text. The question answering model is pre-trained to generate the text answer based on predefined prompt words, prioritizing historical dialogue information and combining it with basic knowledge. The first text and the target text answer are stored as a set of historical dialogue information in the second knowledge base to dynamically update the second knowledge base.
2. The method according to claim 1, characterized in that, The step of retrieving associated target basic knowledge from the first knowledge base based on the extracted semantic features includes: Calculate the first correlation coefficient between the semantic features and each basic knowledge in the first knowledge base; The highest correlation coefficient is determined from the first correlation coefficients and taken as the target first correlation coefficient. The basic knowledge corresponding to the target first correlation coefficient is determined as the target basic knowledge corresponding to the first text.
3. The method according to claim 1, characterized in that, The step of retrieving associated target historical dialogue information from the second knowledge base based on the extracted semantic features includes: Calculate the second correlation coefficient between the semantic features and each set of historical dialogue information in the second knowledge base; The highest correlation coefficient is determined from the second correlation coefficients and taken as the target second correlation coefficient. The historical dialogue information corresponding to the target second correlation coefficient is determined as the target historical dialogue information corresponding to the first text.
4. The method according to claim 3, characterized in that, The calculation of the second association coefficient between the semantic features and each set of historical dialogue information in the second knowledge base includes: Obtain the attribute information of each set of historical dialogue information, and determine the confidence level of each set of historical dialogue information based on the attribute information; For each set of historical dialogue information, the second correlation coefficient is determined based on the confidence level and the degree of matching between the content of the historical dialogue information and the semantic features.
5. The method according to claim 4, characterized in that, The attribute information includes at least the dialogue time point and the job level of the dialogue initiator. Determining the confidence level of each set of historical dialogue information based on the attribute information includes: Based on the chronological order of the dialogue timestamps, a first confidence level is generated for each group of historical dialogue information. Based on the job level of the dialogue initiator, a second confidence level is generated for each set of historical dialogue information; The confidence level corresponding to each group of historical dialogue information is determined based on the first confidence level, the second confidence level, and the weight coefficients pre-assigned to the first confidence level and the second confidence level.
6. The method according to any one of claims 1 to 5, characterized in that, The question-answering model is adjusted through the following steps: The training set is input into the question-answering model to be trained. The training set includes multiple sets of interrelated sample first text, sample basic knowledge, sample simulated historical dialogue information, and standard text answers. Based on predefined prompts, the question-answering model is guided to prioritize the historical dialogue knowledge of the samples and combine it with the basic knowledge of the samples to generate predicted text answers. The parameters of the question-answering model are adjusted with the goal of minimizing the difference between the predicted text answer and the standard text answer.
7. The method according to claim 1, characterized in that, After generating and outputting the target text answer corresponding to the first text, the process also includes: The input text is subjected to intent recognition, and when the text is determined to be a second text carrying a correction text answer based on the recognized text intent, the correction text answer is obtained; The first text answer and the corrected text answer are stored as a set of historical dialogue information in the second knowledge base to update the second knowledge base.
8. A question-and-answer generation device based on long-term memory, characterized in that, include: The intent recognition module is used to recognize the intent of the input text, and when the text is determined to be a first text carrying a question based on the recognized text intent, semantic extraction is performed on the first text. The retrieval module is used to retrieve associated target basic knowledge from a first knowledge base and associated target historical dialogue information from a second knowledge base based on the extracted semantic features. The first knowledge base includes at least one pre-stored basic knowledge, and the second knowledge base includes at least one dynamically updated set of historical dialogue information, which includes a historical first text and the corresponding historical text answer. The answer generation module is used to synchronously input the first text, the target basic knowledge, and the target historical dialogue information into the question answering model, and use the question answering model to generate and output the target text answer corresponding to the first text. The question answering model is pre-trained to generate text answers based on predefined prompt words, prioritizing historical dialogue information and combining basic knowledge. The dynamic storage module is used to store the first text and the target text answer as a set of historical dialogue information in the second knowledge base, so as to dynamically update the second knowledge base.
9. The apparatus according to claim 8, characterized in that, The retrieval module is specifically used for: Calculate the first correlation coefficient between the semantic features and each basic knowledge in the first knowledge base; The highest correlation coefficient is determined from the first correlation coefficients and taken as the target first correlation coefficient. The basic knowledge corresponding to the target first correlation coefficient is determined as the target basic knowledge corresponding to the first text.
10. The apparatus according to claim 8, characterized in that, The retrieval module is specifically used for: Calculate the second correlation coefficient between the semantic features and each set of historical dialogue information in the second knowledge base; The highest correlation coefficient is determined from the second correlation coefficients and taken as the target second correlation coefficient. The historical dialogue information corresponding to the target second correlation coefficient is determined as the target historical dialogue information corresponding to the first text.
11. The apparatus according to claim 10, characterized in that, The retrieval module is also used for: Obtain the attribute information of each set of historical dialogue information, and determine the confidence level of each set of historical dialogue information based on the attribute information; For each set of historical dialogue information, the second correlation coefficient is determined based on the confidence level and the degree of matching between the content of the historical dialogue information and the semantic features.
12. The apparatus according to claim 11, characterized in that, The attribute information includes at least the dialogue time and the job level of the dialogue initiator. The retrieval module is further specifically used for: Based on the chronological order of the dialogue timestamps, a first confidence level is generated for each group of historical dialogue information. Based on the job level of the dialogue initiator, a second confidence level is generated for each set of historical dialogue information; The confidence level corresponding to each group of historical dialogue information is determined based on the first confidence level, the second confidence level, and the weight coefficients pre-assigned to the first confidence level and the second confidence level.
13. The apparatus according to any one of claims 8 to 12, characterized in that, The question-answering model is adjusted through the following steps: The training set is input into the question-answering model to be trained. The training set includes multiple sets of interrelated sample first text, sample basic knowledge, sample simulated historical dialogue information, and standard text answers. Based on predefined prompts, the question-answering model is guided to prioritize the historical dialogue knowledge of the samples and combine it with the basic knowledge of the samples to generate predicted text answers. The parameters of the question-answering model are adjusted with the goal of minimizing the difference between the predicted text answer and the standard text answer.
14. The apparatus according to claim 8, characterized in that, After generating and outputting the target text answer corresponding to the first text, the intent recognition module is further configured to: The input text is subjected to intent recognition, and when the text is determined to be a second text carrying a correction text answer based on the recognized text intent, the correction text answer is obtained; The first text answer and the corrected text answer are stored as a set of historical dialogue information in the second knowledge base to update the second knowledge base.
15. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps in a question-answering generation method based on long-term memory as described in any one of claims 1 to 7.
16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for performing steps in a question-and-answer generation method based on long-term memory as described in any one of claims 1 to 7.
17. A computer program product, characterized in that, The computer program product includes: computer program code, which, when run on a computer, causes the computer to perform the steps of a question-and-answer generation method based on long-term memory as described in any one of claims 1 to 7.