Document and interactive feedback combined large language model knowledge base self-adaptive updating method

Abstract

The invention discloses a large language model knowledge base self-adaptive updating method combining documents and interactive feedback, and relates to the technical field of knowledge base updating, and the method comprises the steps that a real-time change detection module monitors streaming documents and interactive feedback, and captures and marks a single fact change event; the decision module calculates a shunting decision coefficient according to the heat density index and the linkage sweep index, and the event is shunted to a Micro-Edit or Incremental-Tune path; if shunting to the Micro-Edit path, the positioning module executes attention backtracking and gradient sensitivity analysis, and determines a to-be-updated weight address; the rank-one editing module implements rank-one updating and records an audit log; the verification module generates an online token or a rollback instruction through consistency comparison; the version management module manages a version, activates a new version route and cleans out expired branches. Through event granularity identification, intelligent shunting, local updating and multi-version management, the timeliness of knowledge base updating is remarkably improved, and resource consumption and maintenance cost are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge base updating, and in particular to a method for adaptively updating a large language model knowledge base by combining documents and interactive feedback. Background Art

[0002] In an environment where knowledge-intensive businesses are continuously launched, the latest technical manuals and user corrections are continuously streamed into large-scale language model-driven knowledge service systems. These contents need to be vectorized before being written into the retrieval index, but real-time writing is often delayed due to throughput bottlenecks and index merging delays, resulting in system responses that are inconsistent with the latest documents, weakening user trust. With the prevalence of retrieval-enhanced generative architectures, if the model cannot capture the latest fragments at the moment of conversation, outdated references will be noticed by users. At the same time, if corrections and additional instructions from multiple channels lack efficient aggregation and priority screening, they are easily diluted by noise and adversarial prompts, causing model drift and maintenance burdens.

[0003] When document flows and interactive feedback jointly shape a rapidly changing operational scenario, the traditional approach of relying on retraining or rebuilding the index of the entire model exposes the contradiction of wasted computing power and delayed response.

[0004] After searching, a Chinese invention patent with application publication number CN118535887A discloses a method and system for efficient fine-tuning and evaluation of large language models, including: a file storage module, which is used to store files and supports adding, deleting, querying and modifying the stored file contents through file paths. The file storage module includes a basic model area, a fine-tuning model area, a baseline data area, a process storage area and a result storage area; a structured storage module, which uses a relational database to store structured data related to the evaluation. The structured data includes model information, baseline data information, template information, pipeline information and task information; an evaluation program module, which is used to call the file storage module and the structured storage module to execute the evaluation system. The evaluation program module includes system information configuration function, execution and scheduling function and evaluation information management function.

[0005] In this context, if the change only involves a single fact, such as a personnel position adjustment or an update to a regulation number, the existing process often still triggers a full fine-tuning or complete index rewrite, which not only consumes resources but also increases the risk of downtime. In addition, there is a lack of visual assessment of the scope of the modification and traceable logs, making it difficult for subsequent audits to explain the ins and outs of the knowledge change. Methods such as rank-one model editing and large-scale memory editing have been proven to be able to directly write new facts while freezing the main parameters, achieving rapid effectiveness with controllable side effects, making it possible to make real-time knowledge patching possible. However, the system has not yet established automatic judgment logic for when to use local editing and when to perform incremental training, and lacks a multi-version evidence storage mechanism for safe rollback. As a result, repeated changes and rollbacks are prone to oscillations in continuous interactions, further weakening consistency and response speed.

[0006] To this end, the present invention provides a method for adaptively updating a large language model knowledge base by combining documents and interactive feedback. Summary of the Invention

[0007] (1) Technical problems solved

[0008] In response to the deficiencies of the prior art, the present invention provides a method for adaptively updating a large language model knowledge base that combines documents and interactive feedback. The method uses a real-time change detection module to monitor streaming documents and interactive feedback, capture and mark single fact change events; the decision module calculates the diversion decision coefficient based on the heat density index and the chain reaction index, and diverts the event to the Micro-Edit or Incremental-Tune path; if it is diverted to the Micro-Edit path, the positioning module performs attention backtracking and gradient sensitivity analysis to determine the weight address to be updated; the rank-one editing module implements the rank-one update and records the audit log; the verification module generates an online token or a rollback instruction through consistency comparison; the version management module manages versions, activates the new version route and cleans up expired branches. Through event granularity recognition, intelligent diversion, local update and multi-version management, the timeliness of knowledge base updates is significantly improved, thereby solving the technical problems recorded in the background technology.

[0009] (2) Technical solution

[0010] To achieve the above objectives, the present invention is implemented through the following technical solutions: a method for adaptively updating a large language model knowledge base that combines documents and interactive feedback, including: a real-time change detection module that monitors streaming documents and interactive feedback, captures and marks change events involving a single fact, and outputs event labels and location keywords;

[0011] The decision module calculates the diversion decision coefficient based on the heat density index and the chain reaction index and diverts the event to the Micro-Edit path or the Incremental-Tune path accordingly;

[0012] If the event is diverted to the Micro-Edit path, the positioning module performs attention backtracking based on the positioning keyword and combines it with gradient sensitivity analysis to determine the set of weight addresses to be updated within the model and generate a mapping list;

[0013] The rank-one editing module performs rank-one updates on the target weights according to the mapping list, and generates an audit log to record the modification source, weight coordinates, and timestamp;

[0014] The verification module performs a consistency comparison based on the automatically generated differential question-answer group before and after the modification. If the output offset is within the acceptable range, an online token is generated; if it exceeds the threshold, the event is rolled back to the incremental-tune path.

[0015] After receiving the launch token, the version management module writes the new model shard and the corresponding incremental index into the multi-version repository, activates the new version routing, and cleans up expired branches to complete the process.

[0016] Furthermore, the real-time change detection module monitors streaming documents and interactive feedback data streams through a subscription-publishing model, monitors content updates, and records change events and timestamps. It uses named entity recognition and relationship extraction techniques to analyze the content of change events to determine whether they are single factual changes, and generates structured event tags containing the change type and content summary.

[0017] Combined with dependency syntax analysis technology, core entity and attribute names are extracted as positioning keywords, and finally event labels and positioning keywords are output in the form of key-value pairs through a queue mechanism.

[0018] Furthermore, the decision engine module calculates the heat density index and chain reaction index and uses the Transformer-Path-Selector model to generate the diversion decision coefficient to divert the change event to the Micro-Edit path or the Incremental-Tune path.

[0019] Furthermore, the heat density index is calculated, which is obtained by multiplying the number of clusters of the change event by the normalized query heat;

[0020] Calculate the chain reaction index, which is obtained by multiplying the number of adjacent nodes of the change event in the concept graph by the centrality weight;

[0021] The Transformer-Path-Selector model receives the heat density index and the chain reaction index as input and outputs the diversion decision coefficient. Based on this coefficient, it determines whether the change event is diverted to the Micro-Edit path or the Incremental-Tune path.

[0022] Furthermore, the positioning keywords are used to backtrack in the attention mechanism of the model, the attention score of each attention head is calculated and the average is taken to form a comprehensive attention distribution, and the weighted addresses with higher attention values ​​are extracted to form a preliminary weighted address set.

[0023] Furthermore, the Integrated-Gradients method is used to perform gradient sensitivity analysis on the weights in the preliminary weight address set, calculate the cumulative gradient of the weights on the model output, and screen the top k weights with the greatest impact to form an optimized weight address set; the optimized weight address set is organized into a JSON format mapping list containing weight addresses, current values, and target values.

[0024] Furthermore, the target weights in the mapping list are locally modified using the rank-one update technique, where

[0025] Precise adjustment of the weight matrix is ​​achieved by calculating the change in weight and constructing an update matrix with non-zero values ​​only at specified positions; an audit log containing the source of the modification, weight coordinates, timestamp, value before modification, and value after modification is generated for each weight modification and stored in JSON format.

[0026] Furthermore, based on the event tags and positioning keywords generated by the real-time change detection module, a differential question and answer group related to the change event is generated, and the pre-modification model and the post-modification model are used to generate answers to the questions in the differential question and answer group. The semantic similarity of the pre-modification answer and the post-modification answer is calculated and the average value is taken as the output offset. By comparing the output offset with the preset threshold, an online token or a fallback instruction is generated.

[0027] Furthermore, after receiving the online token, the version management module generates a unique version identifier, writes the unique version identifier into the multi-version repository, activates the new version route, and cleans up expired branches.

[0028] Furthermore, a unique version identifier is generated by the version management module to identify the new version. The multi-version repository stores all version data. The new version route points to the version corresponding to the activated unique version identifier. The expired branch is a version branch that is no longer used in the multi-version repository.

[0029] (3) Beneficial effects

[0030] The present invention provides a method for adaptively updating a large language model knowledge base by combining documents and interactive feedback, which has the following beneficial effects:

[0031] The real-time change detection module monitors streaming documents and interactive feedback in real time, captures and tags individual factual change events, and outputs event labels and location keywords. This allows for rapid perception of external changes and ensures the immediacy of knowledge base content. The decision engine module then calculates the diversion decision coefficient based on the heat density index and chain reaction index, intelligently diverting change events to either the Micro-Edit path or the Incremental-Tune path. This mechanism enables precise classification and efficient processing of change events, optimizing resource allocation and enhancing the system's adaptability to high-frequency change scenarios.

[0032] In the Micro-Edit path, the localization module uses attention backtracking and gradient sensitivity analysis to precisely locate the set of weight addresses to be updated within the model and generate a mapping list. The rank-one editing module then performs a rank-one update based on this list, making local modifications to the model weights. This avoids the high resource consumption and downtime risks associated with traditional full fine-tuning or index rewriting. This local update mechanism significantly reduces computational resource consumption and improves system energy efficiency while maintaining update speed, fully demonstrating the technological advancement of this solution in resource optimization.

[0033] The verification module performs a consistency comparison based on the automatically generated differential question-and-answer sets before and after the modification. It calculates the output offset and compares it with a preset threshold to generate a launch token or a rollback instruction. This step ensures the consistency and accuracy of the updated model output, effectively preventing knowledge base content distortion caused by update errors. Furthermore, an intelligent rollback mechanism quickly restores to a stable version in the event of an update failure, ensuring service continuity and user trust, demonstrating the collaborative design of this solution for stability and security.

[0034] After receiving the launch token, the version management module writes the new model shards and incremental indexes to the multi-version repository, activates the new version routing, and cleans up expired branches, enabling rapid version switching and optimizing storage space. The multi-version repository design supports rapid version rollbacks, provides reliable data support for audits and compliance, and ensures the traceability and security of knowledge base updates.

[0035] In summary, through the organic combination of technical features such as event granularity identification, intelligent diversion, local updates, consistency verification, and multi-version management, an efficient knowledge base update link has been formed. The synergistic effect of each module significantly improves the overall performance of the system, especially in high-frequency change scenarios, achieving continuous adaptive evolution of the knowledge base. Through local updates and intelligent diversion mechanisms, the present invention subverts the resource-intensive drawbacks of traditional update methods and significantly improves response speed and resource utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1A flowchart of a method for updating a large language model knowledge base adaptively. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work are within the scope of protection of the present application.

[0038] Please refer to Figure 1 The present application provides a large language model knowledge base adaptive updating method combining documents and interactive feedback, which comprises the following steps: Step 1, a real-time change detection module listens to the data stream of streaming documents and interactive feedback through a subscription-published mode, monitors content updates, records change events and time stamps, analyzes the change event content using named entity recognition and relationship extraction technology to determine whether it is a single fact change, generates a structured event label containing change type and content summary, extracts core entities and attribute names as positioning keywords using dependency syntax analysis technology, and finally transmits the event label and positioning keywords in the form of key-value pairs to the next step through a queue mechanism.

[0039] The step 1 comprises the following contents:

[0040] Step 101, data source listening and event capture

[0041] The real-time change detection module first continuously listens to the data source, which includes two parts: one is streaming documents, such as regulatory updates and technical manual revisions, etc. These contents are input into the knowledge service system in the form of continuous data stream; the other is interactive feedback, such as user-submitted error correction information and additional explanations, etc. These information are input as supplementary data stream. The listening mechanism is realized by using a subscription-published mode, specifically, the system subscribes to the data channels of streaming documents and interactive feedback, to ensure that the listening module can immediately receive the notification and obtain the relevant data whenever new content is generated in the data stream. This way can capture the change events in the data stream in real time, providing input basis for subsequent processing.

[0042] On the basis of monitoring the data source, the real-time change detection module continuously monitors the streaming documents and interactive feedback to capture the change events contained therein. A change event refers to a specific content update that occurs in the data stream, such as a modification of a regulatory provision or a user-identified erroneous description in a technical manual. The system analyzes each input data and, when a change in content is detected, identifies it as a change event and records a timestamp of the time when it occurred for each change event. The recording of the timestamp ensures that subsequent processing can trace the order and time point of the occurrence of the change event, thereby supporting the ordered management and verification of changes.

[0043] Step 102, single fact change identification

[0044] For each captured change event, the real-time change detection module analyzes whether it belongs to a single fact change. A single fact change refers to an update that only involves a single entity or attribute, such as a change in the position of a certain person from manager to director or a change in the number of a regulation from A-123 to A-124. The analysis process uses named entity recognition technology and relation extraction technology, with the specific steps being: first, extract the core entities from the content of the change event, such as the name of a person or the number of a regulation, using named entity recognition technology; then determine the relationship between these entities and their associated attributes, such as position or number, using relation extraction technology; finally, determine whether the change event only affects one entity and its single attribute. If the update only involves a single entity and its single attribute, it is determined to be a single fact change; if it involves multiple entities or complex relationship updates, it is marked as a non-single fact change and left for subsequent steps.

[0045] Step 103, event labeling and keyword extraction

[0046] For change events determined to be single fact changes, the real-time change detection module generates event labels. The event label contains two parts of content: one is the change type, which describes the category of the change, such as personnel position adjustment or regulation number update; the other is the content summary, which briefly summarizes the specific content of the change, such as the position change of Zhang San to director or the regulation number change to A-124. The event label is stored in a structured format, such as using key-value pairs to organize data, to facilitate the parsing and calling of the label content in subsequent steps. The process of generating event labels ensures that the information of change events is clearly recorded, providing structured data that can be directly used for subsequent processing.

[0047] After generating the event label, the real-time change detection module extracts the positioning keywords from the content of the single fact change to support accurate positioning and retrieval in subsequent steps. The positioning keywords include the core entities involved in the change event, such as Zhang San or A-124, and the related attribute names, such as position or number. The extraction process combines dependency syntax analysis techniques, specifically, the system analyzes the syntactic structure of the change event text, identifies the words that represent the core entities and attributes, and ensures that the extracted keywords accurately reflect the key elements of the change. In this way, the positioning keywords can provide accurate retrieval basis for subsequent steps.

[0048] The real-time change detection module passes the generated event label and positioning keywords as the output results of step one to the next step. The output data is organized in the form of key-value pairs, for example, the event label may be recorded as structured data containing the change type and content summary, and the positioning keywords are recorded as a list containing the core entities and attribute names. The output results are transmitted through a queue mechanism, for example, using a message queue system, to ensure that the data can be efficiently and seamlessly transmitted to step two for subsequent shunt decision calculation and event processing.

[0049] When used, the real-time change detection module uses the subscribe-publish mode to listen to the data source because this mode can efficiently handle high-concurrency data streams and ensure that the system can still capture change events in real time in high-frequency change scenarios. The advantage is that it improves the response speed and timeliness of data processing of the system, which is suitable for application scenarios with frequent regulatory updates or large amounts of user feedback.

[0050] In single fact change recognition, the use of named entity recognition and relationship extraction can accurately analyze the content of the change event, extract entities and their attribute relationships, and thus correctly distinguish between single fact changes and complex changes. The advantage is that it avoids misjudgment of complex changes and improves the accuracy and processing efficiency of recognition.

[0051] The use of dependency syntax analysis in positioning keyword extraction can deeply analyze the syntactic structure of the text to ensure that the extracted keywords accurately reflect the core content of the change event. The advantage is that it enhances the representativeness and accuracy of the keywords, providing reliable support for positioning and retrieval in subsequent steps. Structured data facilitates system parsing and processing, and the queue mechanism ensures the stability and orderliness of data transmission. The advantage is that it improves the overall efficiency and maintainability of the system.

[0052] Step one is the starting point of the entire process, responsible for capturing change events from streaming documents and interactive feedback and performing preliminary processing without relying on pre-step. The output results of step one, namely the event label and positioning keywords, will be directly passed to step two for calculating the shunt decision coefficient and performing event shunt processing, thereby realizing the complete process from data capture to intelligent shunting.

[0053] The technical features of the specific real-time process of step two, the decision engine module, for diverting the change event to the Micro-Edit path or the Incremental-Tune path by calculating the heat density index and the chain contagion index and using the Transformer-Path-Selector model to generate the diversion decision coefficient, are as follows: First, calculate the heat density index by multiplying the number of clustering clusters of the change event by the normalized query heat; second, calculate the chain contagion index by multiplying the number of adjacent nodes of the change event in the concept graph by the centrality weight; finally, the Transformer-Path-Selector model receives the heat density index and the chain contagion index as input and outputs the diversion decision coefficient, which determines whether the change event is diverted to the Micro-Edit path or the Incremental-Tune path.

[0054] The step two includes the following contents:

[0055] Step 201, heat density index calculation

[0056] First, perform density clustering analysis on the change event to evaluate the degree of aggregation of the change event in the knowledge graph. Use the density-based spatial clustering algorithm (DBSCAN), input the coordinates of the change event in the knowledge graph according to the positioning keyword mapping provided in step one, and output the number of clustering clusters. The number of clustering clusters represents the size of the event group spatially adjacent to the change event in the knowledge graph, reflecting the local aggregation characteristics of the change event.

[0057] Next, count the real-time query heat of each change event within a predetermined time window. The query heat is obtained by analyzing the number of requests in the user query log that match the change event label, with the unit being times / time window.

[0058] Then, perform minimum-maximum normalization on the query heat of all change events. The normalization process is performed by subtracting the minimum value of the query heat of all change events from the query heat of each change event, and then dividing by the difference between the maximum and minimum values of the query heat of all change events, to obtain the normalized query heat. The normalized query heat has a value range of 0 to 1, eliminating the dimensional influence and facilitating subsequent comprehensive calculation.

[0059] Finally, multiply the number of clustering clusters by the normalized query heat to obtain the heat density index. The heat density index comprehensively reflects the degree of aggregation of the change event in the knowledge graph and the frequency of user attention, with a larger value indicating a more popular and concentrated change event.

[0060] Step 202, chain contagion index calculation

[0061] First, maintain a concept graph, where nodes represent knowledge entities and edges represent semantic relationships between knowledge entities. Using the positioning keywords provided in step 1, map change events to specific nodes in the concept graph. For each change event, count the number of adjacent nodes directly connected to the corresponding node in the concept graph. The number of adjacent nodes reflects the scale of the direct connections between the change event in the knowledge graph.

[0062] Next, the PageRank algorithm is applied to each node in the concept graph to calculate its centrality weight. This centrality weight represents the global importance of a node in the concept graph and ranges from 0 to 1. It is directly derived from the iterative convergence of the PageRank algorithm. The number of adjacent nodes is multiplied by the centrality weight to obtain the chain reaction index. This chain reaction index quantifies the potential impact of a change event on other knowledge entities in the knowledge graph, with larger values ​​indicating more significant chain reactions.

[0063] Step 203: Calculate the diversion decision coefficient

[0064] First, the heat density index and chain reaction index are used as feature vectors to form an input pair.

[0065] The pre-trained Transformer-Path-Selector model then processes the input pairs. The Transformer-Path-Selector model is a classification network based on the Transformer architecture. Its input layer receives the heat density index and chain reaction index, encodes feature relationships through a multi-layer attention mechanism, and the output layer uses a sigmoid activation function to generate a diversion decision coefficient. The diversion decision coefficient ranges from 0 to 1 and represents the probability that a change event will be diverted to the Micro-Edit path. The decision rule is: if the diversion decision coefficient is greater than 0.5, the Micro-Edit path is selected; if the diversion decision coefficient is less than or equal to 0.5, the Incremental-Tune path is selected.

[0066] Path selection is performed based on the diversion decision coefficient. If the diversion decision coefficient is greater than 0.5, the change event is diverted to the Micro-Edit path, where it is processed by the positioning module in step 3. This is suitable for scenarios with fast, local updates. If the diversion decision coefficient is less than or equal to 0.5, the change event is diverted to the Incremental-Tune path, where subsequent steps perform a full update as needed. This is suitable for scenarios with a larger impact range.

[0067] The heat density index combines density clustering with query heat statistics. The density-based spatial clustering algorithm (DBSCAN) effectively reveals the clustering characteristics of change events in the knowledge graph, while user query logs truly reflect the frequency of change event attention. Normalization ensures that the number of clusters and query heat are calculated using a unified scale. The heat density index accurately identifies popular and concentrated change events, providing data support for the rationality of path selection.

[0068] The calculation of the chain reaction index combines the number of adjacent nodes and the centrality weight, which can directly reflect the correlation scale of the change event in the knowledge graph; the PageRank algorithm can quantify the importance of nodes in the global graph, and the chain reaction index can comprehensively evaluate the potential impact range of the change event, making it easier to judge the priority and risk of update operations.

[0069] The diversion decision coefficient is generated through the Transformer-Path-Selector model. Its multi-layer attention mechanism can capture the complex nonlinear relationship between the heat density index and the chain reaction index, ensuring the accuracy of decision-making. The diversion decision coefficient output by the model improves the intelligence of path selection, enabling change events to be efficiently diverted to appropriate processing paths.

[0070] In step 2, based on the event tags and location keywords provided in step 1, the clustering characteristics and impact range of the change events are quantified by calculating the heat density index and chain reaction index. Next, the Transformer-Path-Selector model is used to generate diversion decision coefficients, enabling intelligent diversion of change events. After diversion, the change events are passed to the Micro-Edit path in step 3 or the Incremental-Tune path in subsequent steps for processing, depending on the path selection results. This ensures efficient and accurate knowledge base updates.

[0071] Step 3: Use the positioning keywords to backtrack in the model's attention mechanism, calculate the attention score of each attention head and take the average to form a comprehensive attention distribution, extract the weighted addresses with higher attention values ​​to form a preliminary weighted address set;

[0072] The Integrated-Gradients method is used to perform gradient sensitivity analysis on the weights in the preliminary weight address set, calculate the cumulative gradient of the weights to the model output, and screen the top k weights with the greatest influence to form an optimized weight address set; the optimized weight address set is organized into a JSON format mapping list containing weight addresses, current values, and target values.

[0073] The step three includes the following:

[0074] Step 301: Attention Retrieval

[0075] The attention backtracking process uses the positioning keywords provided in step 1 to analyze them in the model's attention mechanism to identify the model's internal weights related to the positioning keywords. Specifically, attention backtracking analyzes the model's attention weights when generating outputs related to the positioning keywords and identifies the weights that contribute most to these keywords. In the multi-head attention mechanism, the attention score is calculated for each attention head. The calculation process of the attention score is:

[0076] First, a dot product is performed on the query vector and the key vector to obtain an initial score. This initial score is then scaled by the key vector's dimensions to avoid excessively large values. Finally, a softmax normalization process is applied to the scaled scores to obtain the attention score for each attention head. Next, the attention scores of all attention heads are averaged to generate a comprehensive attention distribution associated with the targeted keyword. Based on the high-attention components of the comprehensive attention distribution, the corresponding weighted addresses are extracted to form a preliminary set of weighted addresses.

[0077] Step 302: Gradient sensitivity analysis

[0078] Based on the preliminary set of weight addresses, a gradient sensitivity analysis is performed to assess the impact of each weight on the model output. Specifically, the Integrated-Gradients method is used to calculate the cumulative gradients of the weights with respect to the model output. The Integrated-Gradients method's calculation process is as follows: First, the baseline input is defined as a zero vector. Then, the path integral of the weight from the baseline value to the current value is calculated. The path integral is the cumulative result of the partial derivatives of the weight and the output with respect to the weight over the path interpolation parameter, where the path interpolation parameter gradually varies from 0 to 1. The partial derivative reflects the sensitivity of the output to the weight. An Integrated-Gradients value is calculated for each weight in the preliminary set of weight addresses, and the weights are sorted from most influential to least influential. The top k weights with the greatest impact are selected, where k is a preset number of selections that can be adjusted based on update requirements. Once the selection is complete, an optimized set of weight addresses is formed.

[0079] Step 303: Generate mapping list

[0080] The set of weight addresses selected through gradient sensitivity analysis is organized into a mapping list. Each entry in the mapping list contains the weight address, current value, and updated target value. The weight address represents a specific index in the model parameter matrix, the current value represents the value of the weight before the update, and the target value is calculated based on the new facts corresponding to the positioning keyword. The specific calculation process is handled by the rank-one editing module in step 4. Only the target value field is reserved here. The mapping list is stored in JSON format to facilitate system parsing and subsequent processing.

[0081] Attention backtracking utilizes the model's attention mechanism to identify weights associated with positioning keywords. Attention weights intuitively reflect the model's focus on input features when generating outputs and are highly interpretable. Analyzing attention weights reveals which weights are directly related to the output of positioning keywords, enabling rapid identification of weights associated with change events, improving the targetedness and efficiency of updates and reducing interference with irrelevant parts of the model. Gradient sensitivity analysis uses the Integrated-Gradients method to assess the impact of weights on outputs. Path integrals quantify the contribution of weights to the output, providing accurate sensitivity assessments. By screening the weights with the greatest impact, gradient sensitivity analysis ensures the accuracy of update operations, avoids unnecessary changes to the outputs of other parts of the model, and thus maintains the overall stability of the model.

[0082] Step 3 uses attention backtracking to identify a preliminary set of weighted addresses associated with the targeted keywords. Gradient sensitivity analysis is then used to optimize this set of weighted addresses, ultimately generating a mapping list that provides precise input data for the rank-one editing module in step 4. In high-frequency change scenarios, step 3 localizes changes to individual facts, avoiding the resource consumption associated with full model retraining. Furthermore, through this structured mapping list, step 3 provides data support for subsequent updates and audits, ensuring efficient and consistent knowledge base updates.

[0083] Step 4. The technical features of the specific real-time process of the rank-one editing module in step 4 can be summarized as follows: using the rank-one update technology to locally modify the target weights in the mapping list, specifically by calculating the change in weight and constructing an update matrix with non-zero values ​​only at specified positions to achieve precise adjustment of the weight matrix; at the same time, an audit log containing the modification source, weight coordinates, timestamp, value before modification and value after modification is generated for each weight modification and stored in JSON format.

[0084] The step 4 includes the following contents:

[0085] Step 401: Rank-one update

[0086] Rank-one update is a technique for locally modifying a matrix through the outer product of two vectors, widely used in scenarios requiring precise adjustment of matrix elements. In this context, rank-one update is used to quickly and locally modify the weight matrix of the model to adapt to single fact change events. The weight matrix is the core parameter of the model, directly affecting the results of the model's output.

[0087] The specific process is as follows: First, obtain the detailed information of the weight to be updated from the mapping list generated in step three, including the address of the weight, the current value and the target value; Then, calculate the change of the weight, which is the result of the target value minus the current value; Then, construct two vectors, the first vector represents the amplitude and row position of the change, which is the change multiplied by a unit vector, the value of the unit vector at the corresponding row position is 1, and the value of the rest position is 0; The second vector represents the column position, which is a unit vector, the value of the corresponding column position is 1, and the value of the rest position is 0.

[0088] By calculating the outer product of the two vectors, a matrix is generated, which has non-zero values only at the intersection of the specified weight row position and column position, and the non-zero value is equal to the change of the weight; Finally, add this matrix to the original weight matrix to complete the update of the specified weight, while other elements in the weight matrix remain unchanged. This method realizes efficient and accurate modification of the weight matrix, avoiding the need for comprehensive adjustment of the entire matrix.

[0089] Step 402, audit log generation

[0090] Audit log is a key tool for recording system operation details, used to ensure the traceability of operations, and audit log is used to record specific information of each weight update, including modification source, weight coordinates, timestamp, pre-modification value and post-modification value. These information is stored in JSON format for easy parsing and external audit.

[0091] The specific generation process is as follows: the modification source is determined by the event label of the change event provided in step one, the weight coordinates are the address of the updated weight in the weight matrix, the timestamp is the accurate time of the update operation execution, the pre-modification value is the weight value before the update, and the post-modification value is the weight value after the update. The generation of audit log is executed synchronously with the weight update operation, ensuring that each modification has a complete record. In this way, the audit log provides reliable data support for the transparency, controllability and subsequent audit and security rollback of the system.

[0092] Rank-one updates can make local and precise modifications to the weight matrix, which is particularly efficient for processing large matrices. In high-frequency change scenarios, changes to a single fact need to be quickly reflected in the model, and rank-one updates can adjust only specific weights without retraining the entire model. First, rank-one updates significantly reduce resource consumption and avoid the high computational cost of full-matrix updates; second, by modifying only specified weights, rank-one updates ensure the stability of other parts of the weight matrix and reduce potential interference with the overall performance of the model. This method is particularly suitable for scenarios where the knowledge base needs to be updated frequently, improving the system's responsiveness and operational efficiency.

[0093] The audit log provides a detailed record of each weight update, facilitating subsequent audits and compliance checks. Second, the audit log supports secure rollback. When problems occur during an update, the system can be quickly restored to its pre-modification state based on the log. In addition, the audit log enhances system transparency and reduces the risk of misoperation or unauthorized changes, thereby improving the stability and reliability of the overall system.

[0094] Step four is the core link in the knowledge base update process of the present invention, and the operation is performed directly based on the mapping list generated in step three. The mapping list contains the address, current value and target value of the weight to be updated, providing accurate data input for the rank-one update. Through the rank-one update, step four realizes the rapid modification of the target weight in the weight matrix, ensuring that the model can reflect the changes of a single fact in a timely manner. At the same time, the generation of the audit log records the detailed information of each update, which provides the necessary data support for the consistency check of the verification module in step five, and also provides the operational basis for the version management module in step six to perform version management.

[0095] Step 5. The technical features of the specific real-time process of the verification module in step 5 can be summarized as follows: based on the event labels and positioning keywords generated by the real-time change detection module, a differential question and answer group related to the change event is generated, and the pre-modification model and the post-modification model are used to generate answers to the questions in the differential question and answer group. The semantic similarity of the pre-modification answer and the post-modification answer is calculated and the average value is taken as the output offset. By comparing the output offset with the preset threshold, an online token or a fallback instruction is generated.

[0096] The step five includes the following:

[0097] Step 501: Generate differential question-answer groups

[0098] The verification module first uses the event label and positioning keywords generated in step one to automatically construct question-and-answer pairs related to the change event. The specific process is as follows: the verification module designs question templates based on the event label and positioning keywords. For example, if the event label is "personnel position adjustment" and the positioning keywords are "Zhang San" and "position," the generated question is "What is Zhang San's position?" The verification module then extracts the pre- and post-modification answers from the change event content. For example, the pre-modification answer is "Manager," while the post-modification answer is "Director." Next, the verification module uses the pre- and post-modification models trained in step four to generate answers to the same question. The pre-modification model generates the answer "Zhang San's position is Manager," while the post-modification model generates the answer "Zhang San's position is Director." These questions and their corresponding pre- and post-modification answers together constitute the differential question-and-answer set, which provides the necessary data support for subsequent consistency comparisons.

[0099] Step 502: consistency comparison

[0100] After generating the differential question-answer group, the verification module performs semantic similarity analysis on each pair of answers in the differential question-answer group, i.e., the answer before modification and the answer after modification.

[0101] The specific process is as follows: The verification module first uses a pre-trained sentence embedding model, such as BERT, to convert the pre-modified answer and the post-modified answer into vector representations. Then, the cosine similarity of the two vectors is calculated. The cosine similarity is calculated by first calculating the dot product of the two vectors, then calculating the square root of the sum of the squares of each vector's own elements, multiplying the two square roots to obtain the denominator, and finally dividing the dot product by the denominator to obtain a similarity value between -1 and 1. The closer the value is to 1, the more similar the semantics. Next, the verification module averages the cosine similarities of all question and answer pairs in the differential question and answer group to obtain the average similarity. After that, the output offset is calculated. The output offset is calculated by subtracting the average similarity from 1 to obtain a value between 0 and 2. The smaller the value, the smaller the change in the model output before and after the modification.

[0102] The verification module compares the calculated output offset with a preset threshold to determine whether to allow the model update to go online. For example, the preset threshold is set to 0.05. The specific decision-making process is as follows: If the output offset is less than or equal to the preset threshold, it indicates that the change in the model output before and after the modification is within an acceptable range. The verification module generates an online token, allowing the modified model to be put into use. If the output offset is greater than the preset threshold, it indicates that the model output change exceeds the acceptable range. The verification module generates a rollback instruction and returns the change event to the Incremental-Tune path for further adjustment to ensure the consistency and accuracy of the model output.

[0103] The generation of differential question-and-answer pairs relies on event labels and positioning keywords. This information accurately points to the core content of the change event, ensuring that the generated questions are highly relevant to the change event. The question-and-answer pairs intuitively reflect the model's ability to answer specific questions, consistent with the actual application scenarios of knowledge service systems. Cosine similarity is a mature and common similarity metric in text analysis that effectively captures changes at the semantic level. Using a pre-trained sentence embedding model (such as BERT) for vectorization can generate high-quality semantic vector representations.

[0104] Step five is the core verification link in the knowledge base update process. Its operation is directly based on the event tags and positioning keywords generated in step one, as well as the pre-modification and post-modification models trained in step four. The verification module systematically evaluates the output consistency of the model before and after the update by generating differential question-answer groups, calculating the output offset, and comparing it with the preset threshold. Based on the comparison results, the verification module generates an online token or a rollback instruction, providing a clear execution basis for the version management module in step six. In the scenario of high-frequency changes to the knowledge base, step five ensures the quality of the model update through a rigorous verification process, providing technical support for the stable operation of the system and the continuous optimization of the user experience.

[0105] Step 6. The technical features of the specific real-time process of the version management module in step 6 are summarized as follows: after receiving the online token, the version management module generates a unique version identifier, writes the unique version identifier into the multi-version repository, activates the new version route, and cleans up expired branches. The unique version identifier is generated by the version management module and is used to identify the new version. The multi-version repository stores all version data. The new version route points to the version corresponding to the activated unique version identifier. The expired branch is a version branch that is no longer used in the multi-version repository.

[0106] The step six includes the following contents:

[0107] Step six is ​​the version management and deployment phase of the knowledge base update process, which is performed by the version management module. After receiving the launch token generated by the verification module in step five, the version management module writes the new model shards and corresponding incremental indexes to the multi-version repository, activates the new version routing, and cleans up expired branches to ensure the security and traceability of knowledge base updates.

[0108] Step 601: Generate a version identifier

[0109] The version management module first generates a unique version identifier for the updated model shard and incremental index.

[0110] The generation process is as follows: obtain the Unix timestamp of the current time and the event label generated by the real-time change detection module for the change event in step 1, combine the two and input them into the hash function to obtain a unique identifier of fixed length.

[0111] A hash function, such as SHA-256, is an algorithm that converts arbitrary-length input into a fixed-length output. This calculation ensures that the resulting version identifier is globally unique and unpredictable. The timestamp reflects the specific time the update occurred, while the event tag identifies the source of the change. The combination of the two, processed by a hash function, produces a result that distinguishes each update while also being tamper-resistant.

[0112] Step 602: Write to the multi-version repository

[0113] Subsequently, the version management module submits the updated model weight matrix, the newly generated incremental index, and the version identifier to the multi-version repository. The multi-version repository is a storage system that supports version control and can save multiple historical versions of model shards and indexes. The submission operation stores the model weight matrix, incremental index, and version identifier as a complete data unit, ensuring that the data content of each version is consistent and traceable. The submission process is similar to transaction processing in a database, requiring all data to be consistent when written and persisted after the write is completed. In this way, the multi-version repository can record the complete update history of the knowledge base.

[0114] After data is written, the version management module updates the version pointer in the global routing configuration to point to the newly generated version identifier. The version pointer is a key setting in the global routing configuration, used to specify the model shard and incremental index version currently being served. Updating the version pointer involves replacing the value of the version pointer with the new version identifier. Once this is done, all new user requests will be served based on the updated model weight matrix and incremental index.

[0115] Finally, the version management module identifies and deletes old model shards and indexes that are no longer needed in the multi-version repository based on predefined retention policies. Retention policies are preconfigured rules for the system, such as retaining the last five versions or versions from the past thirty days. The cleanup operation first checks each version in the multi-version repository to determine whether it is referenced by the current service or rollback mechanism, and only deletes versions that are not relied upon. The deletion process removes expired model weight matrices and incremental indexes one by one, ensuring that storage space is freed up while retaining the necessary version history.

[0116] When used, the hash function can convert the combination of the timestamp and the event label into a fixed-length unique identifier, avoiding version duplication or conflict. The timestamp and the event label provide information in the time and content dimensions, respectively, and the hash function ensures the uniqueness and security of the identifier through irreversible calculation.

[0117] The multi-version repository supports version control and can save multiple historical versions of the knowledge base. In the event of an update failure or exception, it can quickly switch to the previous stable version, reducing the risk of service interruption. By recording complete historical versions, the system can support version auditing and quick switching without the need to regenerate models or indexes, reducing computational costs.

[0118] By updating the version pointer, the latest model shard and incremental index can be enabled immediately, ensuring that user requests reflect the current knowledge content. The update of the version pointer avoids the service delay or downtime problem caused by model switching in traditional methods. The new version can take effect seamlessly, and the user experience is not interrupted. At the same time, the system response speed is improved, meeting the requirements of real-time application scenarios.

[0119] Step six, as the final step of the knowledge base update process, closely connects with the previous steps. Based on the online token generated after the verification module passes the verification in step five, the version management module generates a version identifier, writes it to the multi-version repository, activates the new version route, and cleans up the expired branches, completing the deployment and management of the new model shard and incremental index. These operations collectively ensure the safety, real-time performance, and traceability of the knowledge base update. In a high-frequency change environment, step six supports the continuous evolution of the knowledge base through an efficient version management mechanism, ensuring the stable operation of the system and the continuity of user services, while providing technical support for version auditing and resource optimization.

[0120] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0121] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0122] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the division of the above-described device embodiment is only a logical function division, and there can be another division manner for actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, or the among different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0123] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0124] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for adaptively updating a large language model knowledge base by combining documents and interactive feedback, characterized by: include, The real-time change detection module monitors streaming documents and interactive feedback, captures and marks change events involving a single fact, and outputs event labels and location keywords; The decision module calculates the diversion decision coefficient based on the heat density index and the chain reaction index and diverts the event to the Micro-Edit path or the Incremental-Tune path accordingly; If the event is diverted to the Micro-Edit path, the positioning module performs attention backtracking based on the positioning keyword and combines it with gradient sensitivity analysis to determine the set of weight addresses to be updated within the model and generate a mapping list; The rank-one editing module performs rank-one updates on the target weights according to the mapping list and generates an audit log to record the modification source, weight coordinates, and timestamp; The verification module performs a consistency comparison based on the automatically generated differential question-answer group before and after the modification. If the output offset is within the acceptable range, an online token is generated. If it exceeds the threshold, the event is rolled back to the incremental-tune path. After receiving the online token, the version management module writes the new model shard and the corresponding incremental index into the multi-version warehouse, activates the new version routing and cleans up expired branches.

2. The method for adaptively updating a large language model knowledge base according to claim 1, characterized in that: The real-time change detection module listens to streaming documents and interactive feedback data streams in a subscription-publishing model, monitors content updates, and records change events and timestamps. It uses named entity recognition and relationship extraction techniques to analyze the content of change events to determine whether they are single factual changes, and generates structured event tags that include the change type and content summary. Combined with dependency syntax analysis technology, core entity and attribute names are extracted as positioning keywords, and finally event labels and positioning keywords are output in the form of key-value pairs through a queue mechanism.

3. The method for adaptively updating a large language model knowledge base according to claim 2, characterized in that: The decision engine module calculates the heat density index and chain reaction index and uses the Transformer-Path-Selector model to generate the diversion decision coefficient to divert change events to the Micro-Edit path or Incremental-Tune path.

4. The method for adaptively updating a large language model knowledge base according to claim 3, characterized in that: Calculate the heat density index, which is obtained by multiplying the number of clusters of the change event by the normalized query heat; Calculate the chain reaction index, which is obtained by multiplying the number of adjacent nodes of the change event in the concept graph by the centrality weight; The Transformer-Path-Selector model receives the heat density index and the chain reaction index as input and outputs the diversion decision coefficient. Based on this coefficient, it determines whether the change event is diverted to the Micro-Edit path or the Incremental-Tune path.

5. The method for adaptively updating a large language model knowledge base according to claim 4, characterized in that: The positioning keywords are used to backtrack in the attention mechanism of the model, the attention score of each attention head is calculated and the average is taken to form a comprehensive attention distribution, and the weighted addresses with higher attention values ​​are extracted to form a preliminary weighted address set.

6. The method for adaptively updating a large language model knowledge base according to claim 5, characterized in that: The Integrated-Gradients method is used to perform gradient sensitivity analysis on the weights in the preliminary weight address set, calculate the cumulative gradient of the weights to the model output, and screen the top k weights with the greatest influence to form an optimized weight address set; the optimized weight address set is organized into a JSON format mapping list containing weight addresses, current values, and target values.

7. The method for adaptively updating a large language model knowledge base according to claim 6, characterized in that: The target weights in the mapping list are locally modified using the rank-one update technique, where Precise adjustment of the weight matrix is ​​achieved by calculating the change in weight and constructing an update matrix with non-zero values ​​only at specified positions; an audit log containing the source of the modification, weight coordinates, timestamp, value before modification, and value after modification is generated for each weight modification and stored in JSON format.

8. The method for adaptively updating a large language model knowledge base according to claim 7, characterized in that: Based on the event labels and positioning keywords generated by the real-time change detection module, a differential question and answer group related to the change event is generated. The pre-modification model and the post-modification model are used to generate answers to the questions in the differential question and answer group. The semantic similarity of the pre-modification answer and the post-modification answer is calculated and the average is taken as the output offset. By comparing the output offset with the preset threshold, an online token or a rollback instruction is generated.

9. The method for adaptively updating a large language model knowledge base according to claim 8, characterized in that: After receiving the online token, the version management module generates a unique version identifier, writes the unique version identifier into the multi-version repository, activates the new version route, and cleans up expired branches.

10. The method for adaptively updating a large language model knowledge base according to claim 9, characterized in that: The unique version identifier is generated by the version management module and is used to identify the new version. The multi-version repository stores all version data. The new version route points to the version corresponding to the activated unique version identifier. The expired branch is the version branch that is no longer used in the multi-version repository.

Images