A knowledge encoding and activating method and device based on a large language model
By generating pseudo-queries and encoding document knowledge in a large language model, and then activating internal knowledge through reinforcement learning training, the problems of dependence on external retrieval and semantic mismatch in existing technologies are solved, resulting in more accurate and stable generation results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-19
- Publication Date
- 2026-07-03
AI Technical Summary
Existing large language models suffer from insufficient knowledge and inaccurate content generation when faced with specific domain knowledge or new information tasks. Furthermore, existing retrieval enhancement generation methods are highly dependent on external retrieval modules, making them susceptible to retrieval errors, and there is a semantic mismatch between document representation and user queries.
By generating pseudo-queries to describe document content, encoding from multiple semantic dimensions, document knowledge is encoded into language model parameters, and internal knowledge is activated through reinforcement learning training, reducing dependence on external retrieval modules and improving the accuracy and stability of generated results.
By effectively utilizing the internal knowledge of the language model, the impact of external retrieval errors can be reduced, improving the accuracy and stability of the generated results and enhancing the semantic matching ability between document representation and user queries.
Smart Images
Figure CN122334467A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of natural language processing technology, specifically relating to a knowledge encoding and activation method and apparatus based on a large language model. Background Technology
[0002] With the rapid development of Large Language Models (LLMs), LLM-based text understanding and generation technologies have been widely applied in tasks such as question answering, information retrieval, and knowledge reasoning. However, due to limitations in model parameter size and training data, LLMs may still encounter problems such as insufficient knowledge or inaccurate generated content when faced with tasks requiring specific domain knowledge or new information.
[0003] To address these issues, the Retrieval-Augmented Generation (RAG) framework has been proposed in recent years. RAG introduces an external retrieval module before generation to retrieve information relevant to the user's query from a document repository, and then uses the retrieval results as context input into a language model, thereby assisting in generating a more accurate answer. Existing RAG systems typically employ a two-stage structure: first, the retrieval module obtains relevant documents, and then the generation module generates a response based on the retrieval results.
[0004] However, existing technologies still have some shortcomings. First, the RAG method is highly dependent on the performance of external retrieval modules. When the retrieval results are incomplete or biased, the generative model struggles to compensate for the error, thus affecting the overall system performance. Second, existing retrieval methods typically encode documents using vector representations, but a single vector representation is insufficient to fully express the semantic information of complex documents, easily leading to semantic mismatches between user queries and document content. Furthermore, as language models continue to grow in scale, their parameters implicitly store a large amount of semantic knowledge, but existing methods mainly rely on external retrieval, and the utilization of the model's internal knowledge remains insufficient.
[0005] Therefore, how to reduce reliance on external retrieval modules and more effectively utilize the knowledge already encoded in the language model parameters to improve the accuracy and relevance of generation tasks has become a pressing technical problem that needs to be solved. Summary of the Invention
[0006] To address the problems of existing retrieval enhancement generation methods, such as heavy reliance on external retrieval modules, susceptibility of retrieval results to retrieval errors, and semantic gaps between document representation and user queries, this invention provides a knowledge encoding and activation method and apparatus based on a large language model. This method encodes document knowledge into language model parameters and activates corresponding internal knowledge based on user queries during the inference phase, thereby reducing reliance on external retrieval modules and improving the accuracy and stability of the generated results.
[0007] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:
[0008] A knowledge encoding and activation method based on a large language model includes the following steps:
[0009] Step 1, pseudo-query generation: For the input document set, generate several pseudo-queries for each document. Use pseudo-queries to describe the document content from multiple semantic dimensions, so that the document representation can be better aligned with the user query in the semantic space.
[0010] Step 2, Knowledge Encoding: Using the pseudo-query generated in Step 1 as a supervision signal, the language model is trained, enabling the language model to learn the mapping relationship from the document to the pseudo-query, thereby encoding the knowledge in the document into the model parameters and forming a parameterized knowledge representation;
[0011] Step 3, Knowledge Activation Training: Based on user queries and target answers, a reinforcement learning training process is constructed. The reward function guides the language model to activate internal knowledge related to user queries during the generation process, thereby improving the model's ability to utilize internal knowledge.
[0012] Step 4, Result Generation: Input the user query into the trained language model, and the language model will activate its internal parameterized knowledge based on the query and generate the corresponding output result.
[0013] Furthermore, step 1 specifically includes the following steps:
[0014] Obtain the set of documents to be processed, and generate a set of pseudo queries related to the semantics of each document. By generating pseudo queries, the document content is transformed into an expression that is closer to the user's query form, thereby enhancing the semantic matching ability between the document representation and the user query.
[0015] Document collections are represented as: ;in, Represents a collection of documents. Indicates the first One document, This indicates the number of documents in the document collection;
[0016] Build with each document Corresponding pseudo-query set ; ;in, Document pseudo-query set, Indicates a document The generated first A pseudo query, This indicates the number of pseudo queries generated for each document;
[0017] Pseudo-queries are generated based on document content using a large language model. The specific generation process is as follows: ;in, This represents the generated pseudo-query text. Indicates input document Generating pseudo queries under certain conditions The probability distribution;
[0018] Pseudo-queries are generated from multiple semantic dimensions, where the set of semantic dimensions is represented as follows: ;in, Represents a set of semantic dimensions. Indicates the first One semantic dimension Indicates the number of semantic dimensions;
[0019] For each semantic dimension, a pseudo-query of the corresponding type is generated. The generation process is represented as follows: ;in, In terms of semantic dimension The following is a document The generated first A pseudo-query.
[0020] Furthermore, the semantic dimension includes one or more of entity information, relational information, numerical information, or time information.
[0021] Furthermore, step 2 specifically includes the following steps:
[0022] Assume the parameters of the large language model are In the knowledge encoding stage, the pseudo-query generated in step 1 is used as a supervision signal. The language model is trained by maximizing the generation probability between the document and the pseudo-query. The loss function is minimized. This enables the language model to learn the semantic mapping relationship between documents and pseudo queries, thereby encoding the semantic information in the documents into the language model parameters and forming an internal parameterized knowledge representation;
[0023] The training loss function in the knowledge encoding stage is expressed as: The specific formula is as follows: ;
[0024] in, The parameter is The language model in the input document Generating pseudo queries under certain conditions The probability of.
[0025] Furthermore, step 3 specifically includes the following steps:
[0026] A reinforcement learning training process is constructed based on user queries and target answers, and a reward function guides the language model to activate internal knowledge related to user queries during the generation process;
[0027] Assume the user enters the query as follows: , indicating the user's query text;
[0028] Let the target answer be , indicating the target output corresponding to the user query;
[0029] During the generation process, the language model first selects the internal knowledge sequence to be activated based on the user query, and then generates the final answer based on the activated internal knowledge. This process can be represented as follows: ;
[0030] in, This represents the internal knowledge activation sequence. Indicates in user query Under certain conditions, the model selects internal knowledge activation sequences. strategy, Indicates in user query and activated internal knowledge Generate target answer under conditions The conditional probability;
[0031] During the reinforcement learning training process, queries for each user are performed. The language model generates a set of candidate outputs: ;
[0032] in, Indicates the number of candidate outputs. Indicates the first 10 candidate answers Indicates the relationship with the first The internal knowledge activation sequence corresponding to each candidate answer;
[0033] The optimization objective function for the reinforcement learning phase is: ;
[0034] in, Indicates the first The advantage function corresponding to each candidate output Indicates the reference strategy, Represents the regularization coefficient. This represents the KL divergence.
[0035] Furthermore, the reinforcement learning training employs a composite reward function, which includes a reward for correct answer, a reward for using internal knowledge, and a reward for output format.
[0036] The reward for a correct answer is represented as follows: ;
[0037] in, This indicates the model's generated results. Indicates the target answer. This indicates a function that covers an exact match.
[0038] Internal knowledge usage rewards are represented as follows: ;
[0039] in, This represents the internal knowledge identification and determination function;
[0040] The output format for the reward is as follows: ;
[0041] The composite reward function is expressed as: ;
[0042] in, Represents the total reward function. , and This represents the weighting coefficient for each reward item.
[0043] A knowledge encoding and activation device based on a large language model includes a processor and a memory, wherein the memory stores a computer program, and the computer program executes the aforementioned method when it runs on the processor.
[0044] A knowledge encoding mechanism trains a language model by using documents and their corresponding pseudo-queries as training signals, enabling the semantic knowledge in the documents to be encoded into the model parameters, thereby forming a parameterized knowledge representation.
[0045] An internal knowledge activation mechanism guides a language model to activate relevant internal knowledge based on user queries during the generation process through reinforcement learning training, enabling the model to complete reasoning and generation tasks using internal knowledge without relying on external retrieval modules.
[0046] Compared with the prior art, the present invention has the following advantages:
[0047] This invention generates pseudo-queries for documents from multiple semantic dimensions by utilizing a large language model, enabling document content to be expressed in a way that closely resembles user queries, thereby narrowing the semantic gap between document representation and user queries.
[0048] Compared with traditional retrieval enhancement generation methods, this invention encodes document knowledge into language model parameters and activates relevant internal knowledge based on user queries during the inference stage, thereby reducing reliance on external retrieval modules, minimizing the impact of retrieval errors on the generated results, and improving the stability and accuracy of the generated results.
[0049] Furthermore, by utilizing pseudo-queries to perform multi-dimensional semantic representation of documents, the semantic gap between document representation and user queries can be effectively narrowed, knowledge matching ability can be improved, and language models can more effectively utilize their internal parameterized knowledge for reasoning and generation. Attached Figure Description
[0050] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0051] Figure 1 The present invention provides an overall flowchart of a knowledge encoding and activation method based on a large language model.
[0052] Figure 2 This is a schematic diagram of the pseudo-query generation and knowledge encoding process provided by the present invention.
[0053] Figure 3 This is a schematic diagram of the internal knowledge activation and result generation process provided by the present invention. Detailed Implementation
[0054] To gain a deeper understanding of this invention, we will provide a comprehensive and detailed description. However, this invention has various implementations and is not limited to the specific examples listed herein. These examples are presented to enhance a full understanding of the disclosure of this invention.
[0055] Figure 1 This invention provides an overall flowchart of a knowledge encoding and activation method based on a large language model. (See attached flowchart.) Figure 1 As shown, the knowledge encoding and activation method based on a large language model proposed in this invention mainly includes four steps: pseudo-query generation, knowledge encoding, knowledge activation training, and result generation.
[0056] I. Pseudo-query generation. First, obtain the set of documents to be processed, and generate a set of semantically relevant pseudo-queries for each document. By generating pseudo-queries, the document content can be transformed into an expression that more closely resembles the user's query, thereby enhancing the semantic matching ability between the document representation and the user's query. Assume the document set is: ;in, Represents a collection of documents. Indicates the first One document, This indicates the number of documents in the document collection.
[0057] For each document Construct a pseudo-query set corresponding to it: ;in, Document pseudo-query set, Indicates a document The generated first A pseudo query, This indicates the number of pseudo queries generated for each document.
[0058] The pseudo-query is generated based on document content using a large language model, and its generation process can be represented as follows: ;in, This represents the generated pseudo-query text. Indicates input document Generating pseudo queries under certain conditions The probability distribution.
[0059] To further enhance the semantic expressiveness of documents, this invention generates pseudo-queries from multiple semantic dimensions. Let the set of semantic dimensions be: ;in, Represents a set of semantic dimensions. Indicates the first One semantic dimension This indicates the number of semantic dimensions. Semantic dimensions can include entity information, relational information, numerical information, or time information, etc.
[0060] For each semantic dimension, a corresponding type of pseudo-query can be generated, and the generation process is represented as follows: ;in, In terms of semantic dimension The following is a document The generated first A pseudo-query.
[0061] By generating pseudo-queries from multiple semantic dimensions, the knowledge in a document can be expressed in a way that is closer to the form of a user query, thereby narrowing the gap between the semantics of the document and the user query and improving the semantic matching ability in the subsequent knowledge encoding process.
[0062] II. Knowledge Encoding. After obtaining the pseudo-query, the language model needs to be trained using the pseudo-query so that the semantic knowledge in the document can be encoded into the language model parameters.
[0063] Assume the parameters of the large language model are This parameter set is continuously updated during training and is used to store the semantic knowledge learned from the documents. In the knowledge encoding phase, training is performed by maximizing the generation probability between a document and its pseudo-query; the optimization objective function is expressed as: ;in, This represents the training loss function during the knowledge encoding phase. The parameter is The language model in the input document Generating pseudo queries under certain conditions The probability of.
[0064] By minimizing the loss function described above, the language model can learn the semantic mapping relationship between documents and pseudo queries, thereby encoding the semantic information in the documents into the language model parameters and forming an internal parameterized knowledge representation.
[0065] III. Knowledge Activation Training. After completing knowledge encoding, reinforcement learning training is used to further guide the language model to activate internal knowledge related to user queries during the generation process.
[0066] Assume the user enters the query as follows: ,in This represents the user's query text; let the target answer be... ,in This represents the target output corresponding to the user query. During the generation process, the language model first selects the internal knowledge sequence to be activated based on the user query, and then generates the final answer based on the activated internal knowledge. This process can be represented as: ;in, This represents the internal knowledge activation sequence, used to represent the internal knowledge information invoked by the model during the generation process; Indicates in user query Under certain conditions, the model selects internal knowledge activation sequences. Strategies; Indicates in user query and activated internal knowledge Generate target answer under conditions The probability of.
[0067] Furthermore, during the reinforcement learning training process, for each user query The language model generates a set of candidate outputs. .in, Indicates the number of candidate outputs. Indicates the first 10 candidate answers This represents the corresponding internal knowledge activation sequence. The optimization objective function for the reinforcement learning phase is: ;in, Indicates the first The advantage function corresponding to each candidate output Indicates the reference strategy, Represents the regularization coefficient. This represents the KL divergence.
[0068] In one implementation, to guide the language model to utilize its internal knowledge more effectively during the generation process, the reinforcement learning training employs a composite reward function. The composite reward function includes one or more of the following: a reward for correct answer, a reward for utilizing internal knowledge, and a reward for output formatting.
[0069] The correctness reward is used to measure the consistency between the model-generated result and the target answer. Let the model output be... The target answer is The reward for a correct answer can be expressed as: ;in, Rewards are given for correct answers. This indicates the model's generated results. Indicates the target answer. This indicates a function that covers an exact match. When the target answer... As a substring appearing in the model output When the exact matching function is true, the coverage function is true.
[0070] The internal knowledge usage reward is used to guide the model to invoke pre-encoded internal knowledge during the generation process. The internal knowledge usage reward can be represented as: ;in, This indicates a reward for using internal knowledge. This represents the internal knowledge identification and determination function. When the model outputs... When it contains at least one valid internal knowledge identifier ;otherwise, .
[0071] The output format reward is used to constrain the structural consistency of the generated results. The output format reward can be expressed as: ;in, This indicates the output format reward.
[0072] Furthermore, the composite reward function can be expressed as: ;in, Represents the total reward function. , and This represents the weighting coefficient for each reward item.
[0073] By designing the reward function described above, the language model can be constrained from three aspects: correctness of the answer, utilization of internal knowledge, and standardization of the output structure. This guides the model to more effectively activate and utilize internal knowledge during the generation process, thereby improving the accuracy and relevance of the final generated results.
[0074] IV. Result Generation. After completing the above training process, the trained language model can be used to generate results during the inference phase.
[0075] During the inference phase, the user query is input into the language model. The model then activates its internal parameterized knowledge based on the query and generates the corresponding output. Let's assume the user query is... The generated result is ,in The language model represents the query The generated output content; let the internal knowledge sequence activated during the generation process be... The large language model first determines the user query. Select relevant internal knowledge activation sequence And, under this internal knowledge condition, the output result is generated. The generation process can be represented as follows: ;
[0076] Through the above reasoning process, the language model can utilize its internal parameterized knowledge to generate results, thereby reducing reliance on external retrieval modules and improving the stability and accuracy of the generated results.
[0077] Contents not described in detail in this specification are prior art known to those skilled in the art. Although illustrative specific embodiments of the invention have been described above to facilitate understanding by those skilled in the art, it should be understood that the invention is not limited to the scope of the specific embodiments. Various modifications are readily apparent to those skilled in the art as long as they fall within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of this invention are protected.
Claims
1. A method for knowledge encoding and activation based on a large language model, characterized in that, Includes the following steps: Step 1, pseudo-query generation: For the input document set, generate several pseudo-queries for each document. Use pseudo-queries to describe the document content from multiple semantic dimensions, so that the document representation can be better aligned with the user query in the semantic space. Step 2, Knowledge Encoding: Using the pseudo-query generated in Step 1 as a supervision signal, the language model is trained, enabling the language model to learn the mapping relationship from the document to the pseudo-query, thereby encoding the knowledge in the document into the model parameters and forming a parameterized knowledge representation; Step 3, Knowledge Activation Training: Based on user queries and target answers, a reinforcement learning training process is constructed. The reward function guides the language model to activate internal knowledge related to user queries during the generation process, thereby improving the model's ability to utilize internal knowledge. Step 4, Result Generation: Input the user query into the trained language model, and the language model will activate its internal parameterized knowledge based on the query and generate the corresponding output result.
2. The knowledge encoding and activation method based on a large language model according to claim 1, characterized in that, Step 1 specifically includes the following steps: Obtain the set of documents to be processed, and generate a set of pseudo queries related to the semantics of each document. By generating pseudo queries, the document content is transformed into an expression that is closer to the user's query form, thereby enhancing the semantic matching ability between the document representation and the user query. A document collection is represented as: ; where, represents the th document, represents the number of documents in the document collection; build a set of pseudo queries corresponding to each document corresponding pseudo query set ; ; wherein, represents the number of pseudo queries generated for a document represents the number of pseudo queries generated for each document The pseudo query is generated by a large language model according to document content, and the specific generation process is represented as: ; wherein, represents the generated pseudo query text, represents the input document , and represents the probability distribution of generating the pseudo query Pseudo-queries are generated from multiple semantic dimensions, where the set of semantic dimensions is represented as follows: ;in, Indicates the first One semantic dimension Indicates the number of semantic dimensions; For each semantic dimension, a pseudo-query of the corresponding type is generated. The generation process is represented as follows: ;in, In terms of semantic dimension The following is a document The generated first A pseudo-query.
3. The knowledge encoding and activation method based on a large language model according to claim 2, characterized in that, The semantic dimension includes one or more of entity information, relational information, numerical information, or time information.
4. The knowledge encoding and activation method based on a large language model according to claim 1, characterized in that, Step 2 specifically includes the following steps: Assume the parameters of the large language model are In the knowledge encoding stage, the pseudo-query generated in step 1 is used as a supervision signal. The language model is trained by maximizing the generation probability between the document and the pseudo-query. The loss function is minimized. This enables the language model to learn the semantic mapping relationship between documents and pseudo queries, thereby encoding the semantic information in the documents into the language model parameters and forming an internal parameterized knowledge representation; The training loss function in the knowledge encoding stage is expressed as: The specific formula is as follows: ； in, The parameter is The language model in the input document Generating pseudo queries under certain conditions The probability of.
5. The knowledge encoding and activation method based on a large language model according to claim 2, characterized in that, Step 3 specifically includes the following steps: A reinforcement learning training process is constructed based on user queries and target answers, and a reward function guides the language model to activate internal knowledge related to user queries during the generation process; During the generation process, the language model first selects the internal knowledge sequence to be activated based on the user query, and then generates the final answer based on the activated internal knowledge. This process can be represented as follows: ； in, This represents the internal knowledge activation sequence. Indicates in user query Under certain conditions, the model selects internal knowledge activation sequences. strategy, Indicates in user query and activated internal knowledge Generate target answer under conditions The conditional probability; During the reinforcement learning training process, queries for each user are performed. The language model generates a set of candidate outputs: ； in, Indicates the number of candidate outputs. Indicates the first 10 candidate answers Indicates the relationship with the first The internal knowledge activation sequence corresponding to each candidate answer; The optimization objective function for the reinforcement learning phase is: ； in, Indicates the first The advantage function corresponding to each candidate output Indicates the reference strategy, Represents the regularization coefficient. This represents the KL divergence.
6. The knowledge encoding and activation method based on a large language model according to claim 5, characterized in that, The reinforcement learning training employs a composite reward function, which includes rewards for correct answer, rewards for internal knowledge usage, and rewards for output format. The reward for a correct answer is represented as follows: ； in, This indicates the model's generated results. Indicates the target answer. This indicates a function that covers an exact match; Internal knowledge usage rewards are represented as follows: ； in, This represents the internal knowledge identification and determination function; The output format for the reward is as follows: ； The composite reward function is expressed as: ； in, Represents the total reward function. , and This represents the weighting coefficient for each reward item.
7. A knowledge encoding and activation device based on a large language model, comprising a processor and a memory, characterized in that, The memory stores a computer program, which, when run on the processor, executes the method described in any one of claims 1 to 6.
8. A knowledge encoding mechanism, characterized in that, By using documents and their corresponding pseudo-queries as training signals to train the language model, the semantic knowledge in the documents can be encoded into the model parameters, thereby forming a parameterized knowledge representation.
9. An internal knowledge activation mechanism, characterized in that, By using reinforcement learning to train and guide the language model to activate relevant internal knowledge based on user queries during the generation process, the model can complete reasoning and generation tasks using its internal knowledge without relying on external retrieval modules.