Model-based knowledge base processing method, electronic equipment and intelligent agent
By introducing KV caching and referencing mechanisms into RAG technology, the problems of high model overhead and slow inference speed are solved, achieving efficient knowledge base processing and improving the inference speed and efficiency of the model.
Patent Information
- Application Number
- CN202511075774.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-31
- Publication Date
- 2025-11-04
AI Technical Summary
Existing RAG technology suffers from high model overhead, slow inference speed, and low efficiency, especially due to inefficiency caused by repetitive encoding in the information retrieval and generation stages.
By employing a key-value caching and referencing mechanism, relevant knowledge objects are pre-encoded and cached. The knowledge content of the required knowledge objects is referenced in the input information through specific reference tags, avoiding repeated encoding and improving reasoning efficiency.
By reducing the overhead of repetitive encoding, the model's inference speed and efficiency are significantly improved, the pressure of the model's input information length is reduced, and the accuracy and efficiency of inference are ensured.
Smart Images
Figure CN120893544A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of artificial intelligence, and in particular to a model-based knowledge base processing method, an electronic device, and an agent. BACKGROUND
[0002] RAG (Retrieval-Augmented Generation) is a technology combining information retrieval and text generation. Its core process can be divided into two stages: information retrieval (Retrieval) and generation (Generation). In the retrieval stage, the most relevant knowledge to the input query is retrieved from the knowledge base, and in the generation stage, the retrieved knowledge content and the original query are jointly input into the generation model for reasoning to generate the final reply. However, this technology has problems such as large model overhead, slow reasoning speed, and low efficiency. SUMMARY
[0003] To this end, the present application discloses the following technical solutions:
[0004] An agent capable of being embedded in an application program or an operating system as part of the application program or the operating system for performing the following processing:
[0005] obtaining input information;
[0006] converting the input information into a first encoded sequence;
[0007] sequentially calculating the key values of each encoding in the first encoded sequence to form a first key value set, wherein if there is a target mark in the input information, loading a second key value set of knowledge content of a target object indicated by the target mark, and combining the second key value set with the first key value set to form a third key value set;
[0008] reasoning a second encoded sequence based on the third key value set;
[0009] wherein the second key value set of the knowledge content of the target object includes each encoded key value in the knowledge content stored before the input information is obtained.
[0010] A model-based knowledge base processing method, comprising:
[0011] obtaining input information;
[0012] converting the input information into a first encoded sequence;
[0013] calculating each encoded key value in the first encoding sequence based on the target model in sequence, to form a first key value set, wherein if there is a target mark in the input information, loading a second key value set of knowledge content of a target object indicated by the target mark, and combining the second key value set with the first key value set to form a third key value set;
[0014] reasoning a second encoding sequence from the target model based on the third key value set;
[0015] wherein the second key value set of the knowledge content of the target object comprises each encoded key value in the knowledge content stored before the input information is obtained.
[0016] Optionally, if there is a target mark in the input information, loading a second key value set of knowledge content of a target object indicated by the target mark, comprises:
[0017] If a target mark is identified in the process of calculating each encoded key value in the first encoding sequence based on the target model in sequence, loading a second key value set of knowledge content of a target object indicated by the target mark.
[0018] Optionally, if there is a target mark in the input information, loading a second key value set of knowledge content of a target object indicated by the target mark, comprises:
[0019] If the corresponding to-be-calculated encoding is a target mark when the Nth round of encoded key values is calculated, loading a second key value set of knowledge content of a target object indicated by the target mark in the Nth round;
[0020] wherein the target mark is identified before calculating each encoded key value in the first encoding sequence based on the target model in sequence.
[0021] Optionally, the model-based knowledge base processing method further comprises:
[0022] matching the target object indicated by the target mark with a knowledge base table to obtain a target storage address corresponding to the second key value set of knowledge content of the target object;
[0023] loading the second key value set to the memory based on the target storage address, to at least skip the key value calculation of the target object based on the loaded second key value set;
[0024] wherein the knowledge base table comprises a corresponding relationship between each knowledge object in the knowledge base and the storage address of the key value set of the contained knowledge content.
[0025] Optionally, the input information comprises first information and second information, the second information comprises the target mark and the target object, and the method further comprises:
[0026] Positionally adjusting the second information so that the second information is located at a starting position of the input information or the first encoding sequence.
[0027] Optionally, the model-based knowledge base processing method further comprises:
[0028] In response to the target mark being followed by a character in the position where the input information is located or being followed by an encoding in the position where the first encoding sequence is located, triggering the step of positionally adjusting the second information.
[0029] Optionally, the input information comprises first information and second information, the second information comprises the target mark and the target object, and the method further comprises, before converting the input information into a first encoding sequence:
[0030] In response to identifying that the target mark exists in the input information, determining a second key-value set of the knowledge content of the target object indicated by the target mark, and loading the second key-value set into the memory;
[0031] Converting the input information into a first encoding sequence, comprising:
[0032] Converting the first information in the input information into a first encoding sequence.
[0033] Optionally, the model-based knowledge base processing method further comprises:
[0034] Generating fingerprint information corresponding to each knowledge object in the knowledge base respectively, and storing a mapping relationship between each knowledge object and the corresponding fingerprint information.
[0035] Optionally, the model-based knowledge base processing method further comprises:
[0036] By comparing the current fingerprint information of each knowledge object in the knowledge base with the corresponding stored fingerprint information in the mapping relationship, determining the knowledge object that has changed;
[0037] Based on the current fingerprint information of the knowledge object that has changed, updating the mapping relationship;
[0038] Performing key-value calculation on the encoding of the knowledge content of the knowledge object that has changed, updating the key-value corresponding to the knowledge object that has changed based on the calculation result, and updating the knowledge base table.
[0039] An electronic device, comprising:
[0040] a memory for storing at least a set of computer instructions;
[0041] a processor for implementing the following processes by executing the set of instructions stored in the memory:
[0042] obtaining input information;
[0043] converting the input information into a first encoding sequence;
[0044] calculating key values of each encoding in the first encoding sequence based on a target model in sequence to form a first key value set, wherein if there is a target mark in the input information, loading a second key value set of knowledge content of a target object indicated by the target mark, and combining the second key value set with the first key value set to form a third key value set;
[0045] inference of a second encoding sequence from the target model based on the third key value set;
[0046] wherein the second key value set of the knowledge content of the target object comprises each encoding key value of the knowledge content stored before the input information is obtained.
[0047] A storage medium carrying one or more sets of computer instructions, which can enable an electronic device to implement any one of the model-based knowledge base processing methods provided above when the one or more sets of computer instructions are executed by the electronic device. BRIEF DESCRIPTION OF DRAWINGS
[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the embodiment or related art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.
[0049] Figure 1 is a flowchart of the model-based knowledge base processing method provided by the present application;
[0050] Figure 2 is another flowchart of the model-based knowledge base processing method provided by the present application;
[0051] Figure 3 is a component structure diagram of the model-based knowledge base processing device provided by the present application;
[0052] Figure 4 is a component structure diagram of the electronic device provided by the present application. DETAILED DESCRIPTION
[0053] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application.
[0054] The embodiments of the present application provide a model-based knowledge base processing method and an electronic device, which are used to solve at least part of technical problems of large model reasoning overhead, slow reasoning speed and low efficiency of existing RAG technology. The model-based knowledge base processing method can be applied to, but is not limited to, electronic devices in a plurality of general or special computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, and the like.
[0055] Referring to the method flowchart shown in FIG. 1, Figure 1 The model-based knowledge base processing method provided by the embodiments of the present application can include the following steps 101 to 104, which will be described in detail below.
[0056] Step 101, obtaining input information.
[0057] The input information includes first information and second information.
[0058] The first information is used to represent a user's question (referred to as "user question") or user demand. The second information is additional information of the first information, and is used to represent reference knowledge corresponding to the first information, so that a target model generates a reply corresponding to the first information (such as a user question) based on the reference knowledge.
[0059] Optionally, the input information is text information in natural language form.
[0060] The target model is a neural network model. The neural network model can learn the characteristics and laws of input information by training a large amount of diversified data. Generally, the model has hundreds of millions to tens of billions of model parameters (model parameters are variables that control the behavior of the target model), which can capture complex relationships and patterns in input information. In the embodiments of the present application, the neural network model can be different types of neural network models such as generative models, generative language models (LM), etc. For example, it can be, but is not limited to, a large language model (LLM), a GPT (Generative Pre-trained Transformer), a visual large model, a multi-modal large model, etc.
[0061] The target model can be a general large model, or a specialized large model obtained by fine-tuning based on requirements, and the embodiments of the present application do not limit this.
[0062] RAG combines information retrieval (Retrieval) and generation (Generation) two steps, which first retrieves the document knowledge related to the user's question from a large document library, then takes these knowledge as the context information / reference information of the user's question, and forms the input prompt together with the user's question to provide to the model to make the model produce a reply corresponding to the user's question. However, this method can cause the model to repeatedly encode the same context information / reference information in different inputs, which can cause the model to have large reasoning overhead, slow reasoning speed, low efficiency, etc., and can increase the length of the prompt word of the model.
[0063] To solve the above problems, the embodiments of the present application propose a knowledge base processing method combining KV (Key-Value) cache and reference mechanism, the core idea of which is to pre-process the related knowledge objects that will / may be used as reference knowledge in the offline stage, convert them into key-value pairs that can be used for attention calculation and cache / store them, and in the running stage / model reasoning stage, support users to reference the required knowledge objects as context information / reference information of the user's question through specific reference marks in the input prompt word based on the reference mechanism. The system can automatically load the KV set corresponding to the knowledge content of the referenced knowledge object based on the specific reference mark contained in the input information, without repeatedly encoding the original document content of the referenced knowledge object, to significantly improve the reasoning efficiency and reduce the input token overhead.
[0064] The related knowledge objects that will / may be used as reference knowledge can include, but are not limited to, user local private documents and / or related knowledge documents from the network.
[0065] The specific reference mark can be, but is not limited to, @, #, *, or special symbols such as < >. The referenced knowledge object can be, but is not limited to, represented by an identifier of the knowledge object in the input information, for example, a corresponding file name, an abstract, and / or file identification information such as a storage path to represent the file to be referenced.
[0066] Based on the processing manner provided in the embodiments of the present application, the second information in the input information can specifically include a target mark and a target object indicated by the target mark. The target mark is a specific reference mark for referencing a related knowledge object, such as @, #, *, or special symbols such as < >. The target object is a knowledge object indicated / referenced by the specific reference mark. In the input information, the knowledge object to be referenced can be specifically represented by an identifier (such as a file name) of the knowledge object.
[0067] An example of the input information is provided below.
[0068] "@ xxx company employee handbook how to perform attendance?"
[0069] In this example, the first information is the user question "how to perform attendance?", and the second information includes a specific reference mark "@" and a target object "xxx company employee handbook" indicated by the reference mark.
[0070] Optionally, in actual application, the user can fill in the corresponding identifier of the target mark and the target object indicated by the target mark in the input information (input prompt). Of course, the original knowledge content of the reference knowledge can also be directly filled in, that is, the method of the embodiments of the present application is still compatible with the existing RAG technology, and the user can select the construction manner of the input information based on actual needs.
[0071] Step 102, converting the input information into a first code sequence.
[0072] Specifically, all text content or part of the text content (such as text content corresponding to the first information) included in the input information can be segmented into the smallest semantic units (such as segmented into multiple words) that can be processed by the target model, and then each semantic unit segmented is converted into a corresponding code in sequence, so as to obtain the first code sequence corresponding to the input information.
[0073] Step 103, calculating the key value of each code in the first code sequence in sequence based on the target model to form a first key value set, wherein if the target mark exists in the input information, loading the second key value set of the knowledge content of the target object indicated by the target mark, and combining the second key value set with the first key value set to form a third key value set.
[0074] The second key-value set of the knowledge content of the target object comprises each encoded key-value stored in the knowledge content before the input information is obtained.
[0075] After the first encoded sequence corresponding to the input information is obtained, the step 103 continues to calculate each encoded key-value in the first encoded sequence based on the target model to form a first key-value set.
[0076] If the target mark is identified in the input information, such as the specific reference marks @, #, *, or 《》, the second key-value set of the knowledge content of the target object indicated by the target mark is directly loaded based on the KV set constructed in the offline stage, and the first key-value set and the second key-value set are combined to form a third key-value set.
[0077] When the first key-value set and the second key-value set are combined, the key-value set corresponding to the first information (such as the user question) in the first key-value set is combined with the second key-value set to form an overall key-value set corresponding to the first information (such as the user question) and the knowledge content of the referenced target object, that is, the third key-value set.
[0078] In practical applications, whether the target mark exists in the input information can be identified by matching each character (such as the minimum semantic unit of word segmentation) in the input information with the pre-set specific reference mark. If the character in the input information matches the pre-set specific reference mark, the character is identified as the target mark. Otherwise, if the character in the input information does not match the pre-set specific reference mark, the target mark does not exist in the input information.
[0079] Optionally, the corresponding character matches the pre-set specific reference mark, which means that the corresponding character is the same as the pre-set specific reference mark.
[0080] The number of pre-set specific reference marks can be one or more, which is not limited. For example, multiple specific reference marks such as @, #, *, or 《》 are pre-set, and the user can select any mark for use. As long as at least one character in the input information matches the pre-set corresponding specific reference mark, it means that the input information contains the target mark.
[0081] In an optional embodiment, whether the target mark exists in the input information can be identified in the process of sequentially calculating the key value of each code in the first code sequence based on the target model. In this embodiment, the target mark in the input information can be identified by matching each code in the first code sequence with the code of the pre-set specific reference mark (such as @, #, etc.) in the process of sequentially calculating the key value of each code in the first code sequence. If there is a target code in the first code sequence that matches the code of the pre-set specific reference mark, the target mark exists in the input information, and the target mark is the character represented by the target code. Otherwise, the target mark does not exist in the input information.
[0082] In this embodiment, the second key value set of the knowledge content of the target object indicated by the target mark can be directly loaded at the moment when the target mark is identified in the process of sequentially calculating the key value of each code in the first code sequence based on the target model. For example, the second key value set is first loaded from the knowledge base KV set calculated and stored in the memory in the offline stage to the memory based on the corresponding storage address of the second key value set in the memory, and then the second key value set is loaded from the memory to the register of the processor based on the corresponding address segment of the second key value set in the memory, so that the processor combines the second key value set with the key value set corresponding to the first information (such as user question) in the first key value set, thereby forming the third key value combination.
[0083] The processor can be, but is not limited to, a CPU (Central Processing Unit, central processing unit) or an NPU (Neural Network Processing Unit, neural network processor).
[0084] The knowledge base KV set includes the KV set corresponding to the knowledge content of each knowledge object in the knowledge base, which can be obtained by encoding and KV calculation on the knowledge content of each knowledge object in the knowledge base in the offline stage.
[0085] However, it is not limited thereto, and in other embodiments, the target mark in the input information can also be identified before the key value of each code in the first code sequence is sequentially calculated based on the target model.
[0086] The target mark in the input information can be identified before the first coded sequence is converted from the input information, or can be identified after the first coded sequence is converted from the input information, and no limitation is made to this. In the former implementation, before the input information is converted into the first coded sequence, whether the target mark exists in the input information can be identified by matching the original character (such as a minimum semantic unit such as a word) in the input information with the original mark symbol of the pre-set specific reference mark. In the latter implementation, after the input information is converted into the first coded sequence, whether the target mark exists in the input information can be identified by matching the code of the character in the input information (i.e. the code in the first coded sequence) with the code of the pre-set specific reference mark.
[0087] In the implementation of identifying the target mark in the input information before the key value of each code in the first coded sequence is calculated based on the target model, if the code to be calculated corresponding to the Nth round of calculation of the key value is the target mark, the second key value set of the knowledge content of the target object indicated by the target mark can be loaded in the Nth round. For example, the second key value set is loaded from the memory to the register of the processor based on the corresponding address segment of the second key value set in the memory in the Nth round, so as to combine the second key value set and the key value set corresponding to the first information (such as a user question) in the first key value set into one, thereby forming the third key value combination.
[0088] In this implementation, when the target mark in the input information is identified before the key value of each code in the first coded sequence is calculated based on the target model, the second key value set can be loaded from the knowledge base KV set calculated and stored on the memory in the offline stage to the memory based on the corresponding storage address of the second key value set in the memory, so as to load the second key value set from the memory to the register of the processor in the Nth round.
[0089] Step 104, reasoning a second coded sequence based on the third key value set by the target model.
[0090] The second coded sequence is a coded sequence formed by the code of the first information in the input information and the code of the knowledge content of the referenced target object.
[0091] After obtaining the third key-value set, in this step, the second encoding sequence is continued to be inferred by the target model based on the third key-value set, and by inferring the second encoding sequence based on the third key-value set, the knowledge content of the target object indicated by the target mark is taken as reference information / context information to generate the reply corresponding to the user question in the input information.
[0092] In summary, the method provided by the embodiments of the present application can support the user to reference the required target object as the reference knowledge / context information of the user question through the specific reference mark and other target marks in the input information based on the reference mechanism, by proposing a knowledge base processing method combining KV cache and reference mechanism. Based on the target mark and the target object indicated by the target mark contained in the input information, the system can automatically load the second key-value set of the knowledge content of the referenced target object, without encoding and KV calculation on the original knowledge content of the referenced target object, thereby reducing the repeated encoding overhead, improving the model inference speed and efficiency, and shortening the length of the input information of the model, and reducing the length pressure of the input information of the model.
[0093] In an optional embodiment, the model-based knowledge base processing method provided by the present application can further include the following processing steps 11) and 12):
[0094] 11) Match the target object indicated by the target mark with the knowledge base table to obtain the target storage address corresponding to the second key-value set of the knowledge content of the target object.
[0095] The knowledge base table includes the correspondence between each knowledge object in the knowledge base and the storage address of the key-value set of the contained knowledge content. For example, it can specifically include the correspondence between the file name of each file in the knowledge base and the storage address of the key-value set of the knowledge content of each file.
[0096] In implementation, the target storage address corresponding to the second key-value set of the knowledge content of the target object can be determined by querying the knowledge base table using the identifier such as the file name of the target object indicated by the target mark.
[0097] 12) Load the second key-value set to the memory based on the target storage address, so as to skip the key-value calculation of the target object based on at least the loaded second key-value set.
[0098] On the basis of step 11), a second key-value set of knowledge content of the target object can be located based on the target storage address, and the second key-value set is loaded into the memory, so that the second key-value set is loaded from the memory on demand and combined with the first key-value set in the subsequent process of calculating the encoded key-value in the first encoding sequence by a processor such as CPU or NPU.
[0099] In implementation, after the target mark in the input information is identified, steps 11)-12) can be performed based on the identified target mark. More specifically, steps 11)-12) can be performed based on the identified target mark before the input information is converted into the first encoding sequence, or steps 11)-12) can be performed based on the identified target mark after the input information is converted into the first encoding sequence and before the encoded key-value in the first encoding sequence is calculated, or steps 11)-12) can be performed based on the identified target mark in the process of calculating the encoded key-value in the first encoding sequence. No limitation is imposed thereon, and the actual application condition can be determined.
[0100] In the embodiment, at least the second key-value set loaded into the memory is used to skip the key-value calculation of the target object, so as to further improve the model inference speed.
[0101] Optionally, if the target mark is identified and the second key-value set is loaded into the memory before the input information is converted into the first encoding sequence, the encoding and key-value calculation of the target object can be skipped based on the second key-value set loaded into the memory, and in addition, the encoding and key-value calculation of the target mark can also be skipped. If the target mark is identified and the second key-value set is loaded into the memory after the input information is converted into the first encoding sequence and before the encoded key-value in the first encoding sequence is calculated, the key-value calculation of the target object can be skipped based on the second key-value set loaded into the memory, and the key-value calculation of the target mark can also be skipped. If the target object is identified and the second key-value set is loaded into the memory in the process of calculating the encoded key-value in the first encoding sequence, the key-value calculation of the target object can be skipped based on the second key-value set loaded into the memory, and the key-value calculation of the target mark can also be skipped.
[0102] The embodiment matches the target object indicated by the target mark with the knowledge base table to obtain a target storage address corresponding to a second key-value set of the knowledge content of the target object, and loads the second key-value set into the memory based on the target storage address. On the one hand, in the process of calculating the encoded key values in the first encoding sequence by the CPU or NPU, the second key-value set is directly used by loading from the memory on demand, without the need for real-time encoding and key-value calculation of the referenced knowledge content of the target object, thereby reducing the overhead in the model inference process. On the other hand, the key-value calculation of the target object (such as the file name) in the input information can be skipped based on at least the loaded second key-value set, thereby further accelerating the model inference speed and improving the inference efficiency.
[0103] In an optional embodiment, the model-based knowledge base processing method provided by the present application can further include the following processing: adjusting the position of the second information in the input information so that the second information is located at the starting position of the input information or the first encoding sequence.
[0104] As described above, the second information includes the target mark and the target object indicated by the target mark. For example, the second information is "@xxx company employee handbook".
[0105] In this embodiment, the position adjustment of the second information in the input information can be implemented by adjusting the second information in the input information to the starting position of the input information before converting the input information into the first encoding sequence.
[0106] However, the position adjustment of the second information in the input information can also be implemented by adjusting the second information (such as the encoding of the second information) to the starting position of the first encoding sequence after converting the input information into the first encoding sequence.
[0107] In actual application, it can be detected whether there is a character before the position of the target mark in the input information or whether there is an encoding before the position of the first encoding sequence. If there is a character before the position of the target mark in the input information or there is an encoding before the position of the first encoding sequence, the position adjustment of the second information is triggered so that the second information is located at the starting position of the input information or the first encoding sequence.
[0108] In attention calculation of a target model or the like, the output of the pre-encoding directly affects the KV calculation result of the subsequent encoding. Specifically, the generation of each token needs to be based on the pre-sequencing token (token in the input information or token predicted based on the input information) for self-attention calculation. When the nth token is generated, the model combines the embedding vectors of the first n-1 tokens for self-attention operation to generate the Key and Value of the current token. This process is iterated repeatedly until the complete text is generated, obtaining the reply corresponding to the user question in the input information. Based on this, when the target model calculates the Key and Value of each encoding in the first encoding sequence, the pre-encoding affects the KV calculation result of the subsequent encoding.
[0109] To address the above situation, the embodiment adjusts the position of the second information in the input information to the starting position of the input information or the first encoding sequence, so as to ensure the accuracy of the second Key and Value set of the target object pre-calculated and stored in the offline stage when loaded and applied to the KV calculation of the target model, and accordingly ensure the inference accuracy of the target model, facilitating the target model to quickly and efficiently generate accurate replies for the user question in the input information.
[0110] In an optional embodiment, referring to the method flowchart shown in Figure 2 The model-based knowledge base processing method provided by the present application can further include the following processing before step 102, i.e., before converting the input information into the first encoding sequence:
[0111] Step 201: In response to identifying that the target mark exists in the input information, determining the second Key and Value set of the knowledge content of the target object indicated by the target mark, and loading the second Key and Value set into the memory.
[0112] In the embodiment, whether the target mark exists in the input information can be identified by matching the characters (such as minimum semantic units such as word segmentation) in the input information with the pre-set specific reference mark before converting the input information into the first encoding sequence.
[0113] In response to identifying that the target mark exists in the input information, the target object indicated by the target mark can be matched with the knowledge base table to obtain the target storage address corresponding to the second Key and Value set of the knowledge content of the target object, and then the second Key and Value set is loaded into the memory based on the target storage address.
[0114] In the embodiment, referring to Figure 2 The step 102 of converting the input information into the first encoding sequence can be implemented as:
[0115] Step 202, converting the first information in the input information into a first encoding sequence.
[0116] That is, the embodiment skips the encoding of the second information (including the target mark and the target object indicated by the target mark, such as "@xxx company employee handbook") in the input information based on the second key-value set loaded into the memory, and accordingly can skip the key-value calculation of the second information at the same time, and only needs to encode and calculate the key value of the first information (such as user question) in the input information.
[0117] Subsequently, when calculating the key value of each encoding of the first encoding sequence converted from the input information based on the target model in sequence, if the corresponding encoding to be calculated is the target mark when calculating the key value of the Nth round, the knowledge content of the target object indicated by the target mark can be loaded into the second key-value set of the Nth round, so as to combine the second key-value set and the corresponding key-value set (such as the first key-value set) corresponding to the first encoding sequence into one, forming the third key-value set used for reasoning to generate the final reply.
[0118] The embodiment reduces the overhead in the model reasoning process, accelerates the model reasoning speed, and improves the reasoning efficiency by identifying whether there is a target mark in the input information before converting the input information into a first encoding sequence, and in response to the existence of the target mark in the input information, determining the second key-value set of the knowledge content of the target object indicated by the target mark, loading the second key-value set into the memory, and then skipping the encoding and key-value calculation of the second information in the input information based on the second key-value set loaded into the memory.
[0119] In an optional embodiment, the model-based knowledge base processing method provided by the present application can further include the following processing: generating the fingerprint information corresponding to each knowledge object in the knowledge base respectively, and storing the mapping relationship between each knowledge object and the corresponding fingerprint information.
[0120] Optionally, a unique hash value can be generated for each knowledge object in the knowledge base, such as each document, and the generated hash value can be used as the unique fingerprint information of the knowledge object. On this basis, the mapping relationship between the fingerprint information of each knowledge object and the knowledge object can be established, and the mapping relationship can be stored.
[0121] The mapping relationship between the knowledge object and its fingerprint information can be the mapping relationship between the identifier (such as the file name) of the knowledge object and the fingerprint information of the knowledge object.
[0122] Based on the above processing, in the embodiment, the model-based knowledge base processing method can further include the following steps:
[0123] 21) determining the changed knowledge object by comparing the current fingerprint information of each knowledge object in the knowledge base with the corresponding stored fingerprint information in the mapping relationship.
[0124] If the current fingerprint information of the corresponding knowledge object in the knowledge base is different from the corresponding stored fingerprint information in the mapping relationship, it indicates that the content of the knowledge object has changed; otherwise, if the current fingerprint information of the knowledge object is the same as the corresponding stored fingerprint information in the mapping relationship, it indicates that the content of the knowledge object has not changed.
[0125] 22) updating the mapping relationship based on the current fingerprint information of the changed knowledge object.
[0126] For the changed knowledge object, the stored fingerprint corresponding to the knowledge object in the mapping relationship can be updated to the current fingerprint information of the knowledge object, so as to ensure the consistency of the fingerprint information in the mapping relationship and the actual content of the corresponding knowledge object.
[0127] 23) performing key value calculation on the encoding of the knowledge content of the changed knowledge object, updating the key value corresponding to the changed knowledge object based on the calculation result, and updating the knowledge base table.
[0128] For the changed knowledge object, the knowledge content needs to be re-encoded and the key value calculation is performed, and the stored key value corresponding to the knowledge object is updated to the key value obtained after re-calculation, so that the stored key value information of the knowledge object corresponds to the actual content of the knowledge object, and the knowledge base table can also be updated to ensure the correctness of the corresponding relationship between each knowledge object and the actual storage address of the key value set in the knowledge base table.
[0129] The embodiment can support version comparison or control of the knowledge objects in the knowledge base by constructing and storing the mapping relationship between each knowledge object and its fingerprint information in the knowledge base, and updating the mapping relationship based on the current fingerprint information of the changed knowledge object. In addition, by re-encoding and key value calculation of the knowledge content of the changed knowledge object, updating the stored key value of the knowledge object based on the calculation result, and updating the knowledge base table, it can be ensured that the second key value set corresponding to the actual content of the referenced knowledge object can be accurately loaded during model reasoning, thereby ensuring the accuracy of the model reasoning result.
[0130] In an optional embodiment, the target mark can indicate one or more target objects, that is, the number of referenced target objects can be one or more, which is not limited and can be determined according to actual application requirements.
[0131] In the case where the target mark indicates multiple target objects, for example, in the case where multiple knowledge documents are referenced based on the reference mark "@", optionally, a context fusion mechanism can be provided, based on which the influence of different target objects in the current reasoning (such as the reasoning corresponding to the current dialogue) is controlled through weighted attention or position bias adjustment, and reasoning is performed in combination with the different influences of different target objects to generate a reply corresponding to the user question.
[0132] In the case where the target mark indicates multiple target objects, by controlling the influence of different target objects in the current reasoning and performing reasoning in combination with the different influences of different target objects to generate a reply corresponding to the user question, semantic interference and context conflict caused by different target objects can be effectively prevented, and the accuracy of the model reasoning result is ensured.
[0133] An application example of the method provided in the present application is provided below.
[0134] In this example, a knowledge base management system is constructed based on the method of the present application.
[0135] The system mainly includes the following core modules:
[0136] I. Preprocessing module
[0137] Responsible for receiving structured or unstructured text files (such as Word, PDF, Markdown, HTML, etc.) and other knowledge objects, and performing data cleaning, standardization, and other preprocessing on them.
[0138] II. KV cache construction module
[0139] Responsible for using the target model to perform forward propagation on each knowledge object such as each text file after preprocessing, extracting the key (K) and value (V) vectors of the intermediate layer, obtaining the key-value set of each knowledge object, and saving the obtained key-value set to the KV cache pool.
[0140] III. Fingerprint and mapping module
[0141] Responsible for generating a unique fingerprint identifier (such as a hash value) for each knowledge object and establishing a mapping relationship with the corresponding knowledge object, to support version control functions for knowledge objects based on the mapping relationship, and ensure that after the knowledge object such as a text document is changed, the affected cache area can be quickly identified based on the mapping relationship and in combination with the knowledge base table (which contains the mapping relationship between the cache addresses of the key-value set corresponding to the knowledge object and its knowledge content) to perform synchronous updating of the key-value.
[0142] IV. Context reference analysis module
[0143] Responsible for parsing the reference mark (such as @) in the user input to identify the target object of the required reference, and calling the KV cache loading module to load the KV set corresponding to the target object.
[0144] Five, KV cache loading module
[0145] Responsible for loading the KV set of the referenced target object from the KV cache pool according to the parsing result of the context reference parsing module, and completing attention calculation combined with the Query vector of the current dialogue.
[0146] Six, cache update management module
[0147] Support the KV cache synchronization update mechanism after the change of knowledge objects. When detecting that the knowledge content such as document content of the knowledge object has changed, automatically trigger the KV cache reconstruction of the corresponding knowledge object, and update the knowledge base table to ensure the consistency and timeliness of the cache data.
[0148] Seven, context fusion and priority control module
[0149] When multiple target objects such as multiple knowledge documents are referenced at the same time, provide a context fusion mechanism to control the influence of different target objects in the current reasoning through weighted attention or position bias adjustment, to prevent semantic interference and context conflict.
[0150] The operation flow of the system provided by the present example mainly includes:
[0151] One, knowledge object preprocessing and KV cache construction (offline stage, such as model training / deployment stage)
[0152] Step S11: Obtain each knowledge object that can be used as reference knowledge, such as FAQ (Frequently Asked Questions), manual, policy document, etc.
[0153] Step S12: Use the target model to forward propagate each knowledge object to extract the key (K) and value (V) vectors of the intermediate layer, and obtain the KV set of the knowledge content of each knowledge object;
[0154] Step S13: Store the obtained KV set in the cache pool in units of knowledge objects (such as documents), and establish an index mapping (such as the mapping relationship between the file name and the KV address);
[0155] Step S14 (optional): Compress or quantize the stored KV set to reduce storage space occupancy;
[0156] Step S15 (optional): Generate a unique fingerprint identifier for each document and other knowledge object, which is used for subsequent version comparison and cache update.
[0157] II. User question stage (running stage)
[0158] Step S21: Receive user input information and determine whether it contains a reference mark (such as @);
[0159] Step S22: If there is a reference mark, call the cache loading module to load the KV set corresponding to the referenced target object;
[0160] Step S23: Encode the user input information into a query vector (Q) and perform attention calculation in combination with the loaded KV set;
[0161] Step S24: Generate a response content and return it to the user.
[0162] III. Cache update mechanism (optional)
[0163] Step S31: Listen to the update event of the knowledge base;
[0164] Step S32: Identify the changed knowledge object by comparing the fingerprints of the knowledge objects in the knowledge base;
[0165] Step S33: Trigger KV cache reconstruction of the changed knowledge object;
[0166] Step S34: Update the cache index to ensure consistency.
[0167] Compared with existing RAG technology, the present application focuses on the preprocessing stage, i.e., performing KV calculation on related knowledge objects (such as the user's local private files) that may be used as reference knowledge / context in advance, to convert them into key-value pairs that can be used for attention calculation and cached, so that when the user asks questions, there is no need to repeatedly encode the knowledge content of the original knowledge object, thereby significantly improving the reasoning efficiency and reducing the input token overhead.
[0168] Compared with existing RAG technology, the present application has at least the following technical advantages:
[0169] 1. Faster reasoning speed
[0170] Reason: Since the KV set corresponding to the knowledge content of the context file (i.e., the referenced target object) has been calculated and cached in advance, when the user asks questions, only loading is required to directly perform attention calculation.
[0171] Comparison with RAG technology: RAG needs to retrieve the document at runtime and concatenate it to the input information before encoding, which increases the additional delay.
[0172] 2. Reduce model input length pressure
[0173] Reason: User prompt words (user input information) do not need to contain complete or partial context content, and only specific reference marks such as @ can be used for reference, which greatly reduces the token quantity of user prompt words.
[0174] Comparison with RAG technology: RAG needs to explicitly splice the retrieved document fragment or the content of the document into the input, which is easy to reach the maximum context limit of the model (such as 32k or 128k tokens), affecting scalability.
[0175] 3. Reduce the overhead of repeated coding
[0176] Reason: The KV set of each knowledge object that has been cached can be reused, avoiding the need to re-code each time the same target object is referenced.
[0177] Comparison with RAG technology: Even if the same target object is used multiple times, RAG still needs to reprocess and encode each time.
[0178] 4. More suitable for local private fixed or limited knowledge source scenarios
[0179] Reason: When the source of the knowledge object is a preset knowledge base, FAQ or structured data, all knowledge object KV caches can be processed offline at one time.
[0180] Comparison with RAG technology: RAG is more suitable for open domain, dynamic data sources, and needs to be retrieved in real time, but at a higher cost.
[0181] Corresponding to the above model-based knowledge base processing method, the embodiments of the present application also provide a model-based knowledge base processing device, the component structure of which is as shown in Figure 3 , comprising:
[0182] The acquisition module 301 is configured to obtain input information.
[0183] The encoding module 302 is configured to convert the input information into a first encoding sequence.
[0184] The calculation module 303 is configured to sequentially calculate the key values of each encoding in the first encoding sequence based on a target model to form a first key value set, wherein if there is a target mark in the input information, load the second key value set of the knowledge content of the target object indicated by the target mark, and combine the second key value set with the first key value set to form a third key value set.
[0185] The reasoning module 304 is configured to reason a second encoding sequence based on the third key value set by the target model.
[0186] The second key-value set of the knowledge content of the target object includes each encoded key-value in the knowledge content stored before the input information is obtained.
[0187] In an optional implementation, the computing module 303 is specifically configured to:
[0188] If the target mark is identified in the process of sequentially computing each encoded key-value in the first encoding sequence based on the target model, the second key-value set of the knowledge content of the target object indicated by the target mark is loaded.
[0189] In an optional implementation, the computing module 303 is specifically configured to:
[0190] If the corresponding to-be-computed encoding is the target mark when the Nth round of encoded key-value computation is performed, the second key-value set of the knowledge content of the target object indicated by the target mark is loaded in the Nth round.
[0191] The target mark is identified before each encoded key-value in the first encoding sequence is sequentially computed based on the target model.
[0192] In an optional implementation, the apparatus further includes a matching module configured to match the target object indicated by the target mark with a knowledge base table to obtain a target storage address corresponding to the second key-value set of the knowledge content of the target object, load the second key-value set into the memory based on the target storage address, and skip the key-value computation of the target object based at least on the loaded second key-value set.
[0193] The knowledge base table includes a correspondence between each knowledge object in the knowledge base and a storage address of a key-value set of contained knowledge content.
[0194] In an optional implementation, the input information includes first information and second information, and the second information includes the target mark and the target object.
[0195] The apparatus further includes an adjusting module configured to adjust the position of the second information so that the second information is located at a starting position of the input information or the first encoding sequence.
[0196] In an optional implementation, the adjusting module is specifically configured to trigger the position adjustment of the second information in response to the target mark being followed by a character in the position where the input information is located or being followed by an encoding in the position where the first encoding sequence is located.
[0197] In an optional implementation, the input information includes first information and second information, the second information includes the target mark and the target object; the apparatus further includes a determination module, configured to, before converting the input information into the first encoded sequence, in response to identifying that the target mark exists in the input information, determine a second key-value set of knowledge content of the target object indicated by the target mark, and load the second key-value set into the memory.
[0198] The encoding module 302 is specifically configured to: convert the first information in the input information into a first encoded sequence.
[0199] In an optional implementation, the apparatus further includes a fingerprint generation and mapping construction module, configured to generate fingerprint information corresponding to each knowledge object in the knowledge base respectively, and store a mapping relationship between each knowledge object and the corresponding fingerprint information.
[0200] In an optional implementation, the apparatus further includes an update processing module, configured to:
[0201] determine the knowledge object that has changed by comparing the current fingerprint information of each knowledge object in the knowledge base with the corresponding stored fingerprint information in the mapping relationship;
[0202] update the mapping relationship based on the current fingerprint information of the knowledge object that has changed;
[0203] perform key-value calculation on the encoding of the knowledge content of the knowledge object that has changed, update the key-value corresponding to the knowledge object that has changed based on the calculation result, and update the knowledge base table.
[0204] Embodiments of the present application further disclose an electronic device, and a component structure of the electronic device, as shown in Figure 4 at least includes:
[0205] a memory 10 configured to store a computer instruction set;
[0206] The computer instruction set can be implemented in the form of a computer program.
[0207] a processor 20 configured to implement the model-based knowledge base processing method provided in any of the method embodiments above by executing the computer instruction set in the memory.
[0208] The processor 20 can be a central processing unit (CPU), a graphics processor, an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a neural network processor (NPU), a deep learning processor (DPU), or other programmable logic devices, etc.
[0209] Optionally, the electronic device further includes a memory, a cache, and the like storage resources.
[0210] The memory can also store the knowledge base, the knowledge base table, and the knowledge base KV set provided in the present application, and the like information, so as to facilitate loading the second key value set corresponding to the knowledge content of the target object based on the storage address of the second key value set corresponding to the knowledge content of the target object indicated by the target mark in the input information from the memory to the memory when the target mark in the input information is identified, so as to facilitate the subsequent processor to directly load the second key value combination from the memory for use in the KV calculation of the corresponding round (for example, the Nth round of calculation of the code corresponding to the target mark when the code is calculated).
[0211] Of course, in implementation, the knowledge base, the knowledge base table, and the knowledge base KV set and the like information can also be stored on a peripheral such as a mobile hard disk.
[0212] Optionally, the electronic device further includes a camera assembly, and / or is connected with an external camera assembly.
[0213] In addition, the electronic device can also include a communication interface, a communication bus, and the like components. The memory, the processor, and the communication interface complete mutual communication through the communication bus.
[0214] The communication interface is used for communication between the electronic device and other devices. The communication bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, and the like.
[0215] An agent capable of being embedded in an application program or an operating system to provide intelligent functions for the application program or the operating system as part of the application program or the operating system. The intelligent functions provided are used to implement the model-based knowledge base processing method provided by any method embodiment of the present application. Herein, no further elaboration is made.
[0216] In actual applications, the model-based knowledge base processing method provided in any method embodiment of the present application can be implemented by referencing the target model in the agent. However, it is not limited thereto. In the case that the target model is small in size and relatively low in data volume, the target model can also be integrated in the agent, so that the agent can execute the processing procedure of the model-based knowledge base processing method provided in the present application based on the integrated target model.
[0217] The agent (Agent) is an application of artificial intelligence technology, which can be implemented based on a target model (such as a large language model). The behavior of the agent is determined by the target model in the agent according to the current state and external input, and the agent can be used to solve a certain type of problem. The target model provides a decision basis through learning and training of a large amount of data; the agent can also call tools, plug-ins and knowledge bases to provide reasoning, decision-making and execution capabilities. In short, the target model provides decision support, and the agent continuously optimizes the performance of the target model in the agent through feedback data (such as a prompt word prompt) in actual applications.
[0218] The agent in the embodiments of the present application can determine that the key value (KV value) corresponding to the knowledge content is directly loaded into the memory through a special identifier, and the target model directly uses the key value (KV value) corresponding to the knowledge content for reasoning or / and calculation. This reduces the knowledge content vector encoding matching process and the process of assembling the retrieved knowledge content vector code into a prompt word prompt, and also reduces the token quantity of the prompt word.
[0219] The embodiments of the present application also disclose a storage medium carrying one or more computer instruction sets, which can enable an electronic device to implement the model-based knowledge base processing method provided in any method embodiment described above when the one or more computer instruction sets are executed by the electronic device.
[0220] It should be noted that each embodiment in the present specification adopts a progressive manner for description, and each embodiment focuses on the different places from other embodiments. The same and similar parts between the embodiments can be referred to each other.
[0221] For the convenience of description, the above system or device is described in various modules or units in terms of functions. Of course, the functions of each unit can be implemented in the same or multiple software and / or hardware in the implementation of the present application.
[0222] Those skilled in the art can clearly understand the application by the description of the above embodiments. The technical solutions of the application can be implemented by means of software and necessary universal hardware platforms. Based on such an understanding, the technical solutions of the application can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the application.
[0223] Finally, it should be noted that the terms such as first, second, third, and fourth are used herein to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.
[0224] The above description is only the preferred embodiments of the application, and it should be pointed out that those skilled in the art can make some improvements and refinements without departing from the principles of the application, and these improvements and refinements should also be regarded as the protection scope of the application.
Claims
1.An agent capable of being embedded in an application program or an operating system as a part of the application program or the operating system for performing the following processes: obtaining input information; converting the input information into a first coded sequence; sequentially computing key values of each code in the first coded sequence to form a first key value set, wherein if there is a target mark in the input information, loading a second key value set of knowledge content of a target object indicated by the target mark, the second key value set being combined with the first key value set to form a third key value set; inferring a second coded sequence based on the third key value set; wherein the second key value set of the knowledge content of the target object comprises each coded key value of the knowledge content stored before the obtaining input information. 2.A model-based knowledge base processing method comprising: obtaining input information; converting the input information into a first coded sequence; sequentially computing key values of each code in the first coded sequence based on a target model to form a first key value set, wherein if there is a target mark in the input information, loading a second key value set of knowledge content of a target object indicated by the target mark, the second key value set being combined with the first key value set to form a third key value set; inferring a second coded sequence based on the third key value set by the target model; wherein the second key value set of the knowledge content of the target object comprises each coded key value of the knowledge content stored before the obtaining input information. 3.The model-based knowledge base processing method of claim 2, wherein if there is a target mark in the input information, loading a second key value set of knowledge content of a target object indicated by the target mark comprises: if the target mark is identified in the process of sequentially computing key values of each code in the first coded sequence based on the target model, loading the second key value set of the knowledge content of the target object indicated by the target mark. 4.The model-based knowledge base processing method of claim 2, wherein if there is a target mark in the input information, loading a second key value set of knowledge content of a target object indicated by the target mark comprises: if the corresponding code to be computed is the target mark in the Nth round of computing coded key values, loading the second key value set of the knowledge content of the target object indicated by the target mark in the Nth round; wherein the target mark is identified before sequentially computing key values of each code in the first coded sequence based on the target model. 5.The model-based knowledge base processing method of claim 2, further comprising: matching the target object indicated by the target mark with a knowledge base table to obtain a target storage address corresponding to the second key value set of the knowledge content of the target object; loading the second key value set into a memory based on the target storage address to skip the key value computation of the target object based on at least the loaded second key value set; wherein the knowledge base table comprises a correspondence between each knowledge object in a knowledge base and a storage address of a key value set of contained knowledge content. 6.The method of claim 5, wherein the input information comprises a first information and a second information, and the second information comprises the target mark and the target object, and the method further comprises: adjusting a position of the second information, so that the second information is located at a starting position of the input information or the first encoded sequence. In response to a character existing before the target mark in the position of the input information or a code existing before the target mark in the position of the first encoded sequence, the step of adjusting the position of the second information is triggered. 7.The method of claim 2, wherein the input information comprises a first information and a second information, and the second information comprises the target mark and the target object, and before converting the input information into the first encoded sequence, the method further comprises: in response to identifying that the target mark exists in the input information, determining a second key-value set of the knowledge content of the target object indicated by the target mark, and loading the second key-value set into the memory; and converting the input information into the first encoded sequence, comprising: converting the first information in the input information into the first encoded sequence. 8.The method of claim 5, further comprising: generating a fingerprint information corresponding to each knowledge object in the knowledge base, and storing a mapping relationship between each knowledge object and the corresponding fingerprint information. 9.The method of claim 8, further comprising: determining a changed knowledge object by comparing a current fingerprint information of each knowledge object in the knowledge base with a corresponding stored fingerprint information in the mapping relationship; updating the mapping relationship based on the current fingerprint information of the changed knowledge object; and updating a key-value of the changed knowledge object based on a key-value calculation of a code of the knowledge content of the changed knowledge object. 10.An electronic device, comprising: a memory configured to store at least a set of computer instructions; and a processor configured to implement the following processes by executing the set of instructions stored in the memory: obtaining input information; converting the input information into a first encoded sequence; sequentially calculating a key-value of each code in the first encoded sequence based on a target model to form a first key-value set, wherein if a target mark exists in the input information, a second key-value set of a knowledge content of a target object indicated by the target mark is loaded, and the second key-value set is combined with the first key-value set to form a third key-value set; inferring a second encoded sequence from the target model based on the third key-value set; and wherein the second key-value set of the knowledge content of the target object comprises a key-value of each code in the knowledge content stored before the input information is obtained.