Method for updating knowledge base of large language model in combination with document and interactive feedback

Abstract

The application discloses a large language model knowledge base adaptive updating method combining documents and interactive feedback, relates to the technical field of knowledge base updating, and comprises the following steps: a real-time change detection module listens to streaming documents and interactive feedback, captures and marks a single fact change event; a decision module calculates a shunt decision coefficient according to a heat density index and a chain ripple index, and shunts the event to a Micro-Edit or Incremental-Tune path; if the event is shunted to the Micro-Edit path, a positioning module performs attention backtracking and gradient sensitivity analysis to determine a weight address to be updated; a rank-one editing module implements rank-one updating and records an audit log; a verification module generates an online token or a rollback instruction through consistency comparison; and a version management module manages versions, activates a new version route and cleans up expired branches. Through event granularity identification, intelligent shunting, local updating and multi-version management, the timeliness of knowledge base updating is significantly improved, and resource consumption and maintenance costs are reduced.

Description

Technical Field

[0001] This invention relates to the field of knowledge base update technology, specifically to an adaptive update method for a large language model knowledge base that combines document and interactive feedback. Background Technology

[0002] In an environment where knowledge-intensive services are continuously being launched, the latest technical manuals and user corrections are constantly being streamed into a large language model-driven knowledge service system. This content needs to be vectorized before being written into the retrieval index, but real-time writing is often delayed due to throughput bottlenecks and index merging latency, resulting in inconsistencies between the system's responses and the latest documents, undermining user trust. With the widespread adoption of retrieval-enhanced generative architectures, if the model cannot capture the latest snippets in a conversation, outdated references will be noticed by users. At the same time, corrections and supplementary explanations from multiple channels, if lacking efficient aggregation and priority filtering, are easily diluted by noise and adversarial prompts, leading to model drift and increased maintenance burden.

[0003] When document flow and interactive feedback together shape a rapidly changing operating scenario, the traditional approach of relying on retraining the entire model or rebuilding the index exposes the contradiction between wasted computing power and slow response.

[0004] A search revealed a Chinese invention patent with publication number CN118535887A, which discloses an efficient fine-tuning evaluation method and system for large language models. The system includes: a file storage module for storing files and supporting operations such as adding, deleting, querying, and modifying file content via file paths; the file storage module comprising a basic model area, a fine-tuning model area, a baseline data area, a process storage area, and a result storage area; a structured storage module using a relational database to store evaluation-related structured data, including model information, baseline data information, template information, pipeline information, and task information; and an evaluation program module for calling the file storage module and the structured storage module to execute the evaluation system; the evaluation program module including system information configuration functions, execution and scheduling functions, and evaluation information management functions.

[0005] In this context, if a change involves only a single fact, such as a personnel reassignment or a regulatory number update, existing processes often still trigger full fine-tuning or complete index rewriting, which consumes resources, increases downtime risks, and lacks a visual assessment of the scope of modification and traceable logs, making it difficult for subsequent audits to explain the origins and development of knowledge changes. Methods such as rank-one model editing and large-scale memory editing have proven that new facts can be written directly while freezing the main parameters, achieving rapid implementation with controllable side effects, making real-time knowledge patching possible. However, the system has not yet established automatic judgment logic for when to use partial editing and when to perform incremental training, and it also lacks a multi-version evidence storage mechanism for safe rollback, causing oscillations of repeated modifications and rollbacks during continuous interaction, further weakening consistency and response speed.

[0006] To address this, the present invention provides an adaptive update method for a large language model knowledge base that combines document and interactive feedback. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] To address the shortcomings of existing technologies, this invention provides an adaptive update method for a large language model knowledge base that combines document and interactive feedback. A real-time change detection module monitors streaming documents and interactive feedback, capturing and marking individual fact change events. A decision-making module calculates a routing decision coefficient based on a heat density index and a cascading effect index, routing the event to either a Micro-Edit or Incremental-Tune path. If routed to a Micro-Edit path, a location module performs attention backtracking and gradient sensitivity analysis to determine the address to be updated. A rank-one editing module performs a rank-one update and records an audit log. A verification module generates a launch token or rollback command through consistency comparison. A version management module manages versions, activates new version routes, and cleans up expired branches. Through event granularity identification, intelligent routing, partial updates, and multi-version management, the timeliness of knowledge base updates is significantly improved, thus solving the technical problems described in the background section.

[0009] (II) Technical Solution

[0010] To achieve the above objectives, the present invention is implemented through the following technical solution: an adaptive update method for a large language model knowledge base that combines documents and interactive feedback, including a real-time change detection module that listens to streaming documents and interactive feedback, captures and marks change events involving a single fact, and outputs event tags and location keywords;

[0011] The decision module calculates the diversion decision coefficient based on the heat density index and the cascading effect index, and diverts the event to the Micro-Edit path or the Incremental-Tune path accordingly.

[0012] If the event is routed to the Micro-Edit path, the localization module performs attention backtracking based on the localization keywords and combines gradient sensitivity analysis to determine the set of weight addresses to be updated within the model and generate a mapping list;

[0013] The rank-one editing module performs rank-one updates on the target weights according to the mapping list, and generates an audit log to record the source of the modification, weight coordinates and timestamp;

[0014] The verification module performs a consistency comparison based on the differential question-and-answer groups automatically generated before and after the modification. If the output offset is within an acceptable range, an online token is generated; if it exceeds the threshold, the event is rolled back to the Incremental-Tune path.

[0015] After receiving the launch token, the version management module writes the new model shards and corresponding incremental indexes to the multi-version repository, activates the new version route, and cleans up expired branches, thus completing the process.

[0016] Furthermore, the real-time change detection module listens to the data stream of streaming documents and interactive feedback through a subscription-publishing model, monitors content updates and records change events and timestamps, uses named entity recognition and relation extraction technology to analyze the content of change events to determine whether it is a single fact change, and generates structured event tags containing change type and content summary;

[0017] By combining dependency parsing techniques to extract core entity and attribute names as location keywords, the event tags and location keywords are finally output as key-value pairs through a queue mechanism.

[0018] Furthermore, the decision engine module calculates the heat density index and the cascading effect index and uses the Transformer-Path-Selector model to generate triage decision coefficients, thereby triaging change events to the Micro-Edit path or the Incremental-Tune path.

[0019] Furthermore, the heat density index is calculated, which is obtained by multiplying the number of clusters of change events by the normalized query heat.

[0020] The cascading effect index is calculated by multiplying the number of adjacent nodes of the change event in the concept graph by the centrality weight.

[0021] The Transformer-Path-Selector model takes the heat density index and cascading effect index as inputs and outputs a triage decision coefficient. Based on this coefficient, it determines whether a change event is triaged to the Micro-Edit path or the Incremental-Tune path.

[0022] Furthermore, by using the location keywords to backtrack in the model's attention mechanism, the attention score of each attention head is calculated and averaged to form a comprehensive attention distribution, and the weight addresses with higher attention values ​​are extracted to form a preliminary set of weight addresses.

[0023] Furthermore, the Integrated-Gradients method is used to perform gradient sensitivity analysis on the weights in the initial weight address set, calculate the cumulative gradient of the weights with respect to the model output, and select the top k weights with the greatest influence to form the optimized weight address set; the optimized weight address set is organized into a JSON format mapping list containing weight address, current value, and target value.

[0024] Furthermore, the rank-one update technique is used to locally modify the target weights in the mapping list, where,

[0025] The weight matrix is ​​precisely adjusted by calculating the change in weights and constructing an update matrix that has non-zero values ​​only at specified positions; an audit log is generated for each weight modification, containing the source of the modification, weight coordinates, timestamp, value before modification, and value after modification, and stored in JSON format.

[0026] Furthermore, based on the event tags and location keywords generated by the real-time change detection module, differential question-and-answer groups related to the change event are generated. The model before and after modification are used to generate answers to the questions in the differential question-and-answer groups. The semantic similarity between the answers before and after modification is calculated and the average value is taken as the output offset. By comparing the output offset with a preset threshold, an online token or rollback instruction is generated.

[0027] Furthermore, after receiving the launch token, the version management module generates a unique version identifier, writes the unique version identifier into the multi-version repository, activates the new version route, and cleans up expired branches.

[0028] Furthermore, a unique version identifier is generated by the version management module to identify a new version. The multi-version repository stores all version data, and the new version route points to the version corresponding to the activated unique version identifier. Expired branches are version branches that are no longer used in the multi-version repository.

[0029] (III) Beneficial Effects

[0030] This invention provides an adaptive update method for a large language model knowledge base that combines document and interactive feedback, which has the following beneficial effects:

[0031] The real-time change detection module monitors streaming documents and interactive feedback in real time, capturing and tagging individual fact change events, and outputting event tags and location keywords. This enables rapid detection of external changes, ensuring the timeliness of the knowledge base content. Next, the decision engine module calculates a triage decision coefficient based on the heat density index and the cascading effect index, intelligently routing change events to either the Micro-Edit path or the Incremental-Tune path. This mechanism achieves accurate classification and efficient processing of change events, optimizing resource allocation and enhancing the system's adaptability to high-frequency change scenarios.

[0032] In the Micro-Edit path, the localization module uses attention backtracking and gradient sensitivity analysis to accurately locate the set of weight addresses to be updated within the model, generating a mapping list. Subsequently, the rank-one editing module performs a rank-one update based on this list, making local modifications to the model weights. This avoids the high resource consumption and downtime risks associated with traditional full-scale fine-tuning or index rewriting. This local update mechanism significantly reduces computational resource consumption and improves system energy efficiency while maintaining update speed, fully demonstrating the technological advancement of this solution in resource optimization.

[0033] The verification module performs a consistency comparison based on the automatically generated differential question-and-answer groups before and after the modification, calculates the output offset and compares it with a preset threshold, generating an online token or a rollback instruction. This step ensures the consistency and accuracy of the updated model output, effectively preventing knowledge base content distortion due to update errors. Furthermore, the intelligent rollback mechanism can quickly restore to a stable version in case of update failure, ensuring service continuity and user trust, demonstrating the synergistic design of this solution in terms of stability and security.

[0034] After receiving the deployment token, the version management module writes the new model shards and incremental indexes to the multi-version repository, activates the new version route, and cleans up expired branches, enabling rapid version switching and optimized storage space. The multi-version repository design supports rapid version rollback and provides reliable data support for auditing and compliance, ensuring the traceability and security of knowledge base updates.

[0035] In summary, by organically combining technical features such as event granularity recognition, intelligent traffic routing, partial updates, consistency verification, and multi-version management, a highly efficient knowledge base update chain is formed. The synergistic effect of each module significantly improves the overall performance of the system, especially enabling continuous adaptive evolution of the knowledge base in high-frequency change scenarios. This invention, through partial updates and intelligent traffic routing mechanisms, overturns the resource-intensive drawbacks of traditional update methods, significantly improving response speed and resource utilization efficiency. Attached Figure Description

[0036] Figure 1This is a schematic diagram of the adaptive update method for the large language model knowledge base of this invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] Please see Figure 1 This invention provides an adaptive update method for a large language model knowledge base that combines documents and interactive feedback. The method includes the following steps: Step 1: A real-time change detection module listens to the data stream of streaming documents and interactive feedback through a subscription-publishing model, monitors content updates and records change events and timestamps, analyzes the content of change events using named entity recognition and relation extraction techniques to determine whether it is a single fact change, generates structured event tags containing change type and content summary, extracts core entity and attribute names as location keywords using dependency parsing techniques, and finally passes the event tags and location keywords to the next step in the form of key-value pairs through a queue mechanism.

[0039] Step one includes the following:

[0040] Step 101: Data Source Listening and Event Capture

[0041] The real-time change detection module first continuously monitors the data source, which consists of two parts: first, streaming documents, such as regulatory updates and technical manual revisions, which are input into the knowledge service system as a continuous data stream; and second, interactive feedback, such as error corrections and supplementary explanations submitted by users through various channels, which serve as supplementary data stream input. The monitoring mechanism employs a publish-subscribe pattern. Specifically, the system subscribes to the data channels for streaming documents and interactive feedback, ensuring that the monitoring module immediately receives notifications and retrieves relevant data whenever new content is generated in the data stream. This approach enables real-time capture of change events in the data stream, providing an input basis for subsequent processing.

[0042] Building upon the monitoring of data sources, the real-time change detection module continuously monitors streaming documents and interactive feedback to capture change events. A change event refers to a specific content update occurring in the data stream, such as a modification of regulatory clauses or a user-reported error in a technical manual. The system analyzes each piece of input data, identifying each detected content change as a change event and recording a timestamp for each event. This timestamp recording ensures that subsequent processing can trace the sequence and timing of change events, thus supporting the orderly management and verification of changes.

[0043] Step 102: Identification of Single Fact Changes

[0044] For each captured change event, the real-time change detection module analyzes it to determine whether it is a single-fact change. A single-fact change refers to an update involving only a single entity or attribute, such as a person's job title changing from manager to director, or a regulation number changing from A-123 to A-124. The analysis process uses named entity recognition (NAME) and relation extraction techniques. The specific steps are: first, NAME is used to extract core entities from the content of the change event, such as the person's name or regulation number; then, relation extraction techniques are used to determine the relationships between these entities and their associated attributes, such as job title or number; finally, it is determined whether the change event affects only one entity and its single attribute. If it involves only an update of a single entity and its single attribute, it is determined to be a single-fact change; if it involves multiple entities or complex relationship updates, it is marked as a non-single-fact change and left for further processing.

[0045] Step 103: Event Tagging and Keyword Extraction

[0046] For change events determined to be single-fact changes, the real-time change detection module generates event tags. Event tags contain two parts: a change type, describing the category of the change, such as a personnel position adjustment or a regulation number update; and a content summary, briefly summarizing the specific content of the change, such as Zhang San's position changing to director or the regulation number changing to A-124. Event tags are stored in a structured format, such as using key-value pairs to organize the data, facilitating the parsing and retrieval of tag content in subsequent steps. The process of generating event tags ensures that the information of change events is clearly recorded, providing directly usable structured data for subsequent processing.

[0047] After generating event tags, the real-time change detection module extracts location keywords from the content of individual factual changes to support precise location and retrieval in subsequent steps. Location keywords include the core entities involved in the change event, such as Zhang San or A-124, and related attribute names, such as job title or ID number. The extraction process incorporates dependency parsing technology; specifically, the system analyzes the syntactic structure of the change event text, identifies words representing core entities and attributes, and ensures that the extracted keywords accurately reflect the key elements of the change. In this way, location keywords provide precise retrieval basis for subsequent steps.

[0048] The real-time change detection module outputs the generated event tags and location keywords as the first step's results, which are then passed to the next step. The output data is organized in key-value pairs; for example, event tags might be recorded as structured data containing change type and content summary, while location keywords are recorded as a list containing core entity and attribute names. The output is transmitted via a queue mechanism, such as a message queue system, to ensure efficient and seamless data transfer to the second step for subsequent traffic allocation decision calculations and event processing.

[0049] The real-time change detection module uses a publish-subscribe pattern to listen to the data source because this pattern can efficiently handle high-concurrency data streams, ensuring that the system can still capture change events in real time even in scenarios with frequent changes. This improves the system's response speed and the timeliness of data processing, making it suitable for applications with frequent regulatory updates or a large volume of user feedback.

[0050] In identifying single fact changes, named entity recognition (NAME) technology and relation extraction can accurately parse the content of the change event, extract entities and their attribute relationships, and thus correctly distinguish between single fact changes and complex changes. This avoids misjudging complex changes and improves the accuracy and processing efficiency of identification.

[0051] Combining dependency parsing with keyword extraction allows for in-depth analysis of the text's syntactic structure, ensuring that extracted keywords accurately reflect the core content of the change event. This enhances the representativeness and accuracy of the keywords, providing reliable support for subsequent location and retrieval steps. Structured data facilitates system parsing and processing, while a queuing mechanism guarantees the stability and order of data transmission. This improves the overall efficiency and maintainability of the system.

[0052] Step one is the starting point of the entire process, responsible for capturing and initially processing change events from streaming documents and interactive feedback, without relying on previous steps. The output of step one, namely event tags and location keywords, will be directly passed to step two for calculating the triage decision coefficient and performing event triage processing, thus realizing a complete process from data capture to intelligent triage.

[0053] Step 2: The decision engine module calculates the heat density index and the cascading effect index, and uses the Transformer-Path-Selector model to generate triage decision coefficients. The technical features of this real-time process for triaging change events to either the Micro-Edit path or the Incremental-Tune path are as follows: First, the heat density index is calculated by multiplying the number of clusters of the change event by the normalized query heat. Second, the cascading effect index is calculated by multiplying the number of adjacent nodes of the change event in the concept graph by the centrality weight. Finally, the Transformer-Path-Selector model receives the heat density index and the cascading effect index as input, outputs the triage decision coefficients, and determines whether the change event is triaged to the Micro-Edit path or the Incremental-Tune path based on these coefficients.

[0054] Step two includes the following:

[0055] Step 201: Calculation of Thermal Consistency Index

[0056] First, density clustering analysis is performed on the change events to assess their clustering degree in the knowledge graph. A density-based spatial clustering algorithm (DBSCAN) is used. The input is the coordinates of the change events in the knowledge graph mapped according to the location keywords provided in step one, and the output is the number of clusters. The number of clusters represents the size of the event group spatially adjacent to the change event in the knowledge graph, reflecting the local clustering characteristics of the change events.

[0057] Next, the real-time query popularity of each change event within a predetermined time window is calculated. Query popularity is obtained by analyzing the number of requests matching the change event tag in the user query logs, measured in times per time window.

[0058] Then, the query popularity of all change events is subjected to minimum-maximum normalization. The normalization process is to subtract the minimum query popularity of all change events from the query popularity of each change event, and then divide by the difference between the maximum and minimum query popularity of all change events to obtain the normalized query popularity. The normalized query popularity ranges from 0 to 1, eliminating the influence of units and facilitating subsequent comprehensive calculations.

[0059] Finally, the number of clusters is multiplied by the normalized query popularity to obtain the popularity density index. The popularity density index comprehensively reflects the degree of clustering of change events in the knowledge graph and the frequency of user attention. The larger the value, the more popular and concentrated the change event is.

[0060] Step 202: Calculation of the cascading effect index

[0061] First, maintain a concept graph, where nodes represent knowledge entities and edges represent semantic relationships between them. Using the location keywords provided in step one, map change events to specific nodes in the concept graph. For each change event, count the number of directly connected neighboring nodes of its corresponding node in the concept graph. The number of neighboring nodes reflects the scale of direct associations of the change event in the knowledge graph.

[0062] Next, the PageRank algorithm is applied to each node in the concept graph to calculate its centrality weight. The centrality weight represents the global importance of a node in the concept graph, ranging from 0 to 1, and is directly obtained from the iterative convergence result of the PageRank algorithm. Multiplying the number of adjacent nodes by the centrality weight yields the cascading effect index. The cascading effect index quantifies the potential impact of a change event on other knowledge entities in the knowledge graph; a higher value indicates a more significant cascading effect.

[0063] Step 203: Calculation of Diversion Decision Coefficients

[0064] First, the heat density index and the cascading effect index are used as feature vectors to form input pairs.

[0065] Then, a pre-trained Transformer-Path-Selector model is used to process the input pairs. The Transformer-Path-Selector model is a classification network based on the Transformer architecture. The input layer receives the heat density index and the chain reaction index, and encodes feature relationships through a multi-layer attention mechanism. The output layer uses a sigmoid activation function to generate triage decision coefficients. The triage decision coefficients range from 0 to 1; they represent the probability that a change event is triaged to the Micro-Edit path. The decision rule is: if the triage decision coefficient is greater than 0.5, the Micro-Edit path is selected; if the triage decision coefficient is less than or equal to 0.5, the Incremental-Tune path is selected.

[0066] The path selection is based on the value of the triage decision coefficient: if the triage decision coefficient is greater than 0.5, the change event is triaged to the Micro-Edit path and handled by the positioning module in step three, which is suitable for scenarios with fast and partial updates; if the triage decision coefficient is less than or equal to 0.5, the change event is triaged to the Incremental-Tune path, and subsequent steps will perform a full update as needed, which is suitable for scenarios with a large impact.

[0067] In practice, the density index calculation combines density clustering and query popularity statistics. The density-based spatial clustering algorithm (DBSCAN) effectively reveals the clustering characteristics of change events in the knowledge graph; user query logs accurately reflect the frequency of attention given to change events. Normalization ensures that the number of clusters and query popularity are calculated under a unified dimension. The density index accurately identifies popular and concentrated change events, providing data support for the rationality of path selection.

[0068] The cascading effect index combines the number of neighboring nodes and centrality weights, which can directly reflect the scale of association of a change event in the knowledge graph. The PageRank algorithm can quantify the importance of a node in the global graph, while the cascading effect index can comprehensively assess the potential impact of a change event, making it easier to determine the priority and risk of update operations.

[0069] The diversion decision coefficients are generated by the Transformer-Path-Selector model. Its multi-layer attention mechanism can capture the complex nonlinear relationship between the heat density index and the cascading effect index, ensuring the accuracy of the decision. The diversion decision coefficients output by the model improve the intelligence of path selection, enabling change events to be efficiently diverted to appropriate processing paths.

[0070] Step two, based on the event tags and location keywords provided in step one, quantifies the clustering characteristics and impact scope of change events by calculating the heat density index and cascading effect index. Next, the Transformer-Path-Selector model is used to generate triage decision coefficients, enabling intelligent triage of change events. After triage, the change events are then passed to the Micro-Edit path in step three or the Incremental-Tune path in subsequent steps for processing, based on the path selection results, ensuring the efficiency and accuracy of knowledge base updates.

[0071] Step 3: Use the localized keywords to backtrack in the model's attention mechanism, calculate the attention score of each attention head and take the average to form a comprehensive attention distribution, and extract the weight addresses with higher attention values ​​to form a preliminary set of weight addresses.

[0072] The Integrated-Gradients method is used to perform gradient sensitivity analysis on the weights in the initial weight address set, calculate the cumulative gradient of the weights with respect to the model output, and select the top k weights with the greatest influence to form the optimized weight address set. The optimized weight address set is then organized into a JSON format mapping list containing the weight address, current value, and target value.

[0073] Step three includes the following:

[0074] Step 301, Attention Regression

[0075] The attention backtracking process utilizes the localized keywords provided in Step 1 to analyze the model's attention mechanism and identify the internal model weights related to these keywords. Specifically, attention backtracking identifies the weights that contribute most to these keywords by analyzing the attention weights when the model generates outputs related to them. In the multi-head attention mechanism, the attention score for each attention head is calculated. The calculation process for the attention score is as follows:

[0076] First, the query vector and key vector are multiplied by a dot product to obtain an initial score. Then, the initial score is scaled by the dimension of the key vector to avoid excessively large values. Finally, softmax normalization is applied to the scaled score to obtain the attention score for each attention head. Next, the attention scores of all attention heads are averaged to generate a comprehensive attention distribution associated with the targeted keywords. Based on the higher attention values ​​in the comprehensive attention distribution, the corresponding weight addresses are extracted to form a preliminary set of weight addresses.

[0077] Step 302, Gradient Sensitivity Analysis

[0078] Based on the initial set of weight addresses, gradient sensitivity analysis is performed to assess the impact of each weight on the model output. Specifically, the Integrated-Gradients method is used to calculate the cumulative gradient of the weights with respect to the model output. The calculation process of the Integrated-Gradients method is as follows: First, the baseline input is defined as the zero vector; then, the path integral of the weights from the baseline value to the current value is calculated. The path integral is the cumulative result of the partial derivatives of the weights with respect to the output with respect to the weights over the path interpolation parameters, where the path interpolation parameters gradually change from 0 to 1, and the partial derivatives reflect the sensitivity of the output to the weights. An Integrated-Gradients value is calculated for each weight in the initial set of weight addresses, and they are sorted from largest to smallest according to their impact. The top k weights with the greatest impact are selected, where k is a preset selection number that can be adjusted according to update requirements. After selection, an optimized set of weight addresses is formed.

[0079] Step 303: Generating the mapping list

[0080] The set of weight addresses filtered through gradient sensitivity analysis is organized into a mapping list. Each entry in the mapping list contains a weight address, its current value, and the updated target value. The weight address represents a specific index in the model parameter matrix, the current value represents the weight's value before the update, and the target value is calculated based on the new facts corresponding to the localized keywords. The specific calculation process is handled by the rank-one editing module in subsequent step four; only the target value field is reserved here. The mapping list is stored in JSON format for easy system parsing and subsequent processing.

[0081] Attention backtracking utilizes the model's attention mechanism to identify weights related to the located keywords. Attention weights intuitively reflect the model's focus on input features when generating output, offering strong interpretability. By analyzing attention weights, it becomes clear which weights are directly related to the output of the located keywords, enabling rapid identification of weights relevant to changing events, improving the targeting and efficiency of updates, and reducing interference with irrelevant parts of the model. Gradient sensitivity analysis employs the Integrated-Gradients method to evaluate the impact of weights on the output. Quantifying the contribution of weights to the output through path integrals provides accurate sensitivity assessments. By selecting the weights with the greatest impact, gradient sensitivity analysis ensures the accuracy of update operations, avoiding unnecessary changes to the output of other parts of the model, thereby maintaining the overall stability of the model.

[0082] Step three identifies a preliminary set of weighted addresses related to the identified keywords through attention backtracking. This set is then optimized using gradient sensitivity analysis, ultimately generating a mapping list that provides precise input data for the rank-one editing module in step four. In high-frequency change scenarios, step three achieves localized processing of single fact changes, avoiding the resource consumption of retraining the entire model. Simultaneously, the structured mapping list provides data support for subsequent update operations and auditing processes, ensuring the efficiency and consistency of knowledge base updates.

[0083] Step 4: The technical features of the rank-one editing module in the specific real-time process of Step 4 can be summarized as follows: The target weights in the mapping list are locally modified using the rank-one update technology. Specifically, the weight matrix is ​​precisely adjusted by calculating the change in weights and constructing an update matrix that has non-zero values ​​only at specified positions. At the same time, an audit log containing the source of modification, weight coordinates, timestamp, value before modification, and value after modification is generated for each weight modification and stored in JSON format.

[0084] Step four includes the following:

[0085] Step 401, Rank 1 Update

[0086] Rank-one update is a technique that locally modifies a matrix using the outer product of two vectors. It's widely used in scenarios requiring precise adjustments to matrix elements. Here, rank-one update is used to quickly and locally modify the model's weight matrix to adapt to changes in a single fact. The weight matrix is ​​a core parameter of the model, directly affecting the model's output.

[0087] The specific processing steps are as follows: First, obtain the detailed information of the weights that need to be updated from the mapping list generated in step three, including the address of the weight, its current value, and its target value; then, calculate the change in the weight, which is the result of subtracting the current value from the target value; subsequently, construct two vectors. The first vector represents the magnitude and row position of the change, specifically the change multiplied by a unit vector, where the value of the unit vector is 1 at the corresponding row position and 0 at the other positions; the second vector represents the column position, which is a unit vector, where the value of the unit vector is 1 at the corresponding column position and 0 at the other positions.

[0088] By calculating the outer product of these two vectors, a matrix is ​​generated that has non-zero values ​​only at the intersection of the row and column positions of the specified weights, and these non-zero values ​​are equal to the change in weight. Finally, this matrix is ​​added to the original weight matrix to update the specified weights, while the other elements in the weight matrix remain unchanged. This method achieves efficient and accurate modification of the weight matrix, avoiding the need for a comprehensive adjustment of the entire matrix.

[0089] Step 402: Generate Audit Logs

[0090] Audit logs are a crucial tool for recording system operation details to ensure traceability. They record specific information for each weight update, including the source of the modification, weight coordinates, timestamp, value before modification, and value after modification. This information is stored in JSON format for easy parsing and external auditing.

[0091] The specific generation process is as follows: the source of the modification is determined by the event tag of the change event provided in step one; the weight coordinate is the address of the updated weight in the weight matrix; the timestamp is the exact time the update operation was executed; the value before modification is the weight value before the update; and the value after modification is the weight value after the update. The generation of audit logs is performed synchronously with the weight update operation, ensuring that each modification has a corresponding complete record. In this way, audit logs provide reliable data support for the system's transparency, controllability, and subsequent auditing and secure rollback.

[0092] Rank-one updates enable localized and precise modifications to the weight matrix, which is particularly efficient for handling large matrices. In high-frequency change scenarios, changes to individual facts need to be quickly reflected in the model, and rank-one updates can adjust only specific weights without retraining the entire model. First, rank-one updates significantly reduce resource consumption, avoiding the high computational cost of full matrix updates; second, by modifying only specified weights, rank-one updates ensure the stability of other parts of the weight matrix, reducing potential interference with the overall model performance. This method is particularly suitable for scenarios requiring frequent knowledge base updates, improving system response speed and operational efficiency.

[0093] The audit log provides detailed records of each weight update, facilitating subsequent audits and compliance checks. Secondly, the audit log supports secure rollback, allowing the system to quickly revert to its pre-modification state if an update encounters a problem. Furthermore, the audit log enhances system transparency, reduces the risk of accidental or unauthorized changes, and thus improves the overall stability and reliability of the system.

[0094] Step four is the core step in the knowledge base update process of this invention, directly executing operations based on the mapping list generated in step three. The mapping list contains the address, current value, and target value of the weight to be updated, providing precise data input for the rank-one update. Through the rank-one update, step four achieves rapid modification of the target weight in the weight matrix, ensuring that the model can promptly reflect changes in individual facts. Simultaneously, the generation of audit logs records detailed information for each update, providing necessary data support for consistency verification by the verification module in step five and operational basis for version management by the version management module in step six.

[0095] Step 5: The technical features of the verification module in the specific real-time process of Step 5 can be summarized as follows: Based on the event tags and location keywords generated by the real-time change detection module, a differential question-and-answer group related to the change event is generated. The model before modification and the model after modification are used to generate answers to the questions in the differential question-and-answer group. The semantic similarity between the answers before modification and the answers after modification is calculated and the average value is taken as the output offset. The online token or rollback instruction is generated by comparing the output offset with a preset threshold.

[0096] Step five includes the following:

[0097] Step 501: Generation of Differential Question-Answer Groups

[0098] The verification module first uses the event tags and location keywords generated in step one to automatically construct question-and-answer pairs related to the change event. Specifically, the verification module designs question templates based on the event tags and location keywords. For example, if the event tag is "personnel position adjustment" and the location keywords are "Zhang San" and "position," the generated question is "What is Zhang San's position?". Then, the verification module extracts the answers before and after the modification from the change event content; for example, the answer before modification is "manager," and the answer after modification is "director." Next, the verification module uses the pre-modification and post-modification models trained in step four to generate answers to the same question. The pre-modification model generates the answer "Zhang San's position is manager," and the post-modification model generates the answer "Zhang San's position is director." These questions and their corresponding pre-modification and post-modification answers together constitute a differential question-and-answer set, which provides necessary data support for subsequent consistency comparisons.

[0099] Step 502, Consistency Comparison

[0100] After generating the differential question-answer pairs, the verification module performs semantic similarity analysis on each pair of answers in the differential question-answer pairs, namely the answer before modification and the answer after modification.

[0101] The specific process is as follows: The validation module first uses a pre-trained sentence vector model, such as BERT, to convert the answers before and after modification into vector representations. Then, it calculates the cosine similarity between these two vectors. The cosine similarity is calculated as follows: first, the dot product of the two vectors is calculated; then, the square root of the sum of the squares of the elements of each vector is calculated; the two square roots are multiplied together to obtain the denominator; finally, the dot product is divided by the denominator to obtain a similarity value ranging from -1 to 1. The closer the value is to 1, the more semantically similar the meaning. Next, the validation module averages the cosine similarities of all question-answer pairs in the differential question-answer group to obtain the average similarity. After that, the output offset is calculated. The output offset is calculated by subtracting the average similarity from 1 to obtain a value ranging from 0 to 2. The smaller the value, the smaller the change in the model output before and after modification.

[0102] The verification module compares the calculated output offset with a preset threshold to determine whether to allow the model update to go live. The preset threshold is, for example, set to 0.05. The specific decision-making process is as follows: if the output offset is less than or equal to the preset threshold, it indicates that the change in model output before and after the modification is within an acceptable range, and the verification module generates a go-live token, allowing the modified model to be used; if the output offset is greater than the preset threshold, it indicates that the change in model output exceeds an acceptable range, and the verification module generates a rollback instruction, returning the change event to the Incremental-Tune path for further adjustment to ensure the consistency and accuracy of the model output.

[0103] The generation of differential question-answer pairs relies on event labels and keyword location. This information accurately points to the core content of the changed event, ensuring that the generated questions are highly relevant to the changed event. The question-answer pairs can intuitively reflect the model's ability to answer specific questions, which is consistent with the actual application scenarios of knowledge service systems. Cosine similarity is a mature and universal similarity measurement method in the field of text analysis, which can effectively capture semantic changes. Choosing a pre-trained sentence vector model (such as BERT) for vectorization can generate high-quality semantic vector representations.

[0104] Step five is the core verification step in the knowledge base update process. Its operation is directly based on the event tags and location keywords generated in step one, as well as the pre-modification and post-modification models trained in step four. The verification module generates differential question-answer sets, calculates the output offset, and compares it with a preset threshold, systematically evaluating the consistency of the model's output before and after the update. Based on the comparison results, the verification module generates an online token or a rollback instruction, providing a clear execution basis for the version management module in step six. In scenarios with frequent changes to the knowledge base, step five, through a rigorous verification process, ensures the quality of model updates, providing technical support for the stable operation of the system and continuous optimization of the user experience.

[0105] Step Six: The technical features of the specific real-time process in Step Six can be summarized as follows: After receiving the online token, the version management module generates a unique version identifier, writes the unique version identifier into the multi-version repository, activates the new version route, and cleans up expired branches. The unique version identifier is generated by the version management module and is used to identify the new version. The multi-version repository stores all version data. The new version route points to the version corresponding to the activated unique version identifier. Expired branches are version branches that are no longer used in the multi-version repository.

[0106] Step six includes the following:

[0107] Step six is ​​the version management and deployment phase of the knowledge base update process, which is executed by the version management module. After receiving the online token generated by the verification module in step five, the version management module writes the new model shards and corresponding incremental indexes to the multi-version repository, activates the new version route, and cleans up expired branches to ensure the security and traceability of knowledge base updates.

[0108] Step 601: Generate version identifier

[0109] The version management module first generates a unique version identifier for the model shards and incremental indexes updated in this update.

[0110] The generation process is as follows: obtain the Unix timestamp of the current time and the event label generated by the real-time change detection module for the change event in step one, combine the two and input them into a hash function to obtain a fixed-length unique identifier.

[0111] A hash function is an algorithm that converts an input of arbitrary length into a fixed-length output. For example, SHA-256 ensures that the output version identifier is globally unique and unpredictable. A timestamp reflects the specific time the update occurred, and an event tag identifies the source of the change event. Combining these two elements and processing them with a hash function generates a result that distinguishes each update while also possessing tamper-proof properties.

[0112] Step 602: Write to the multi-version repository

[0113] Subsequently, the version management module commits the updated model weight matrix, the newly generated incremental index, and the version identifier to the multi-version repository. A multi-version repository is a version-controlled storage system capable of storing multiple historical versions of model shards and indexes. The commit operation stores the model weight matrix, incremental index, and version identifier as a complete data unit, ensuring data consistency and traceability for each version. The commit process is similar to transaction processing in a database, requiring all data to maintain consistency during writing and be persistently saved after writing. In this way, the multi-version repository can record the complete update history of the knowledge base.

[0114] After the data writing is complete, the version management module updates the version pointer in the global routing configuration, making it point to the newly generated version identifier. The version pointer is a key setting in the global routing configuration, used to specify the model shard and incremental index version used by the current service. The process of updating the version pointer is as follows: the value of the version pointer is replaced with the new version identifier. After the replacement is completed, all newly received user requests will be served based on the updated model weight matrix and incremental index.

[0115] Finally, the version management module identifies and deletes outdated model shards and indexes that are no longer needed in the multi-version repository based on a predefined retention policy. The retention policy consists of pre-configured rules, such as retaining the five most recent versions or versions from the past thirty days. The cleanup operation first checks each version in the multi-version repository to determine if it is referenced by the current service or rollback mechanism, deleting only versions that are not depended upon. The deletion process removes expired model weight matrices and incremental indexes one by one, ensuring that storage space is freed up while retaining necessary version history.

[0116] In use, a hash function transforms a combination of timestamps and event tags into a fixed-length unique identifier, preventing version duplication or conflicts. Timestamps and event tags provide information on time and content, respectively, and the hash function ensures the uniqueness and security of the identifier through irreversible computation.

[0117] Multi-version repositories support version control, enabling the storage of multiple historical versions of the knowledge base. In the event of update failures or anomalies, it allows for rapid switching to a previous stable version, reducing the risk of service interruption. By recording complete historical versions, the system supports version auditing and rapid switching without the need to regenerate models or indexes, thus reducing computational costs.

[0118] By updating the version pointer, the latest model shards and incremental indexes are immediately enabled, ensuring that user requests reflect the current knowledge content. Updating the version pointer avoids service delays or downtime caused by model switching in traditional methods. New versions take effect seamlessly, providing a seamless user experience without interruption, while also improving system response speed, meeting the needs of applications with high real-time requirements.

[0119] Step six, as the final step in the knowledge base update process, is closely linked to the preceding steps. Based on the deployment token generated after successful verification by the verification module in step five, the version management module completes the deployment and management of the new model shards and incremental indexes by generating version identifiers, writing to multi-version repositories, activating new version routes, and cleaning up expired branches. These operations collectively ensure the security, real-time nature, and traceability of knowledge base updates. In a high-frequency change environment, step six, through its efficient version management mechanism, supports the continuous evolution of the knowledge base, ensures the stable operation of the system and the continuity of user services, and provides technical support for version auditing and resource optimization.

[0120] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0121] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0122] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0123] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0124] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An adaptive update method for a large language model knowledge base that combines document and interactive feedback, characterized by: include, The real-time change detection module listens to streaming documents and interactive feedback, captures and marks change events involving individual facts, and outputs event tags and location keywords; The decision-making module calculates the heat density index and cascading effect index based on the change event, and generates diversion decision coefficients accordingly, wherein: The heat density index is obtained by multiplying the number of clusters of the change event by the normalized query heat; the cascading effect index is obtained by multiplying the number of adjacent nodes of the change event in the concept graph by the centrality weight. Path selection is performed based on the traffic splitting decision coefficient and a preset threshold: when the traffic splitting decision coefficient is greater than the preset threshold, the change event is split to the Micro-Edit path; when the traffic splitting decision coefficient is less than or equal to the preset threshold, the change event is split to the Incremental-Tune path. Among them, the Micro-Edit path is a path that performs local parameter editing and updating on the target weights related to the localized keywords within the model, and the local parameter editing and updating includes at least performing rank-one updates on the target weights; the Incremental-Tune path is a path that performs a full update on the model; If the event is routed to the Micro-Edit path, the localization module performs attention backtracking based on the localization keywords and combines gradient sensitivity analysis to determine the set of weight addresses to be updated within the model and generate a mapping list; The rank-one editing module performs rank-one updates on the target weights according to the mapping list, and generates an audit log to record the source of the modification, weight coordinates and timestamp; The verification module performs a consistency comparison based on the differential question and answer groups automatically generated before and after the modification. If the output offset is within an acceptable range, an online token is generated. If it exceeds the threshold, the event is rolled back to the Incremental-Tune path. After receiving the launch token, the version management module writes the new model shards and corresponding incremental indexes to the multi-version repository, activates the new version route, and cleans up expired branches.

2. The adaptive update method for a large language model knowledge base according to claim 1, characterized in that: The real-time change detection module listens to the data stream of streaming documents and interactive feedback through a subscription-publishing model, monitors content updates and records change events and timestamps, uses named entity recognition and relation extraction technology to analyze the content of change events to determine whether it is a single fact change, and generates structured event tags containing change type and content summary; By combining dependency parsing techniques to extract core entity and attribute names as location keywords, the event tags and location keywords are finally output as key-value pairs through a queue mechanism.

3. The adaptive update method for a large language model knowledge base according to claim 2, characterized in that: The decision engine module calculates the heat density index and the cascading effect index and uses the Transformer-Path-Selector model to generate triage decision coefficients, thereby triaging change events to the Micro-Edit path or the Incremental-Tune path.

4. The adaptive update method for a large language model knowledge base according to claim 3, characterized in that: The heat density index is calculated by multiplying the number of clusters of change events by the normalized query heat. The cascading effect index is calculated by multiplying the number of adjacent nodes of the change event in the concept graph by the centrality weight. The Transformer-Path-Selector model takes the heat density index and cascading effect index as inputs and outputs a triage decision coefficient. Based on this coefficient, it determines whether a change event is triaged to the Micro-Edit path or the Incremental-Tune path.

5. The adaptive update method for a large language model knowledge base according to claim 4, characterized in that: By using keywords to backtrack through the model's attention mechanism, the attention score of each attention head is calculated and averaged to form a comprehensive attention distribution. The weight addresses with higher attention values ​​are extracted to form a preliminary set of weight addresses.

6. The adaptive update method for a large language model knowledge base according to claim 5, characterized in that: The Integrated-Gradients method is used to perform gradient sensitivity analysis on the weights in the initial weight address set, calculate the cumulative gradient of the weights with respect to the model output, and select the top k weights with the greatest influence to form the optimized weight address set. The optimized weight address set is then organized into a JSON format mapping list containing the weight address, current value, and target value.

7. The adaptive update method for a large language model knowledge base according to claim 6, characterized in that: The rank-one update technique is used to locally modify the target weights in the mapping list, where... The weight matrix is ​​precisely adjusted by calculating the change in weights and constructing an update matrix that has non-zero values ​​only at specified positions; an audit log is generated for each weight modification, containing the source of the modification, weight coordinates, timestamp, value before modification, and value after modification, and stored in JSON format.

8. The adaptive update method for a large language model knowledge base according to claim 7, characterized in that: Based on the event tags and location keywords generated by the real-time change detection module, differential question-and-answer groups related to the change event are generated. The pre-modification model and the post-modification model are used to generate answers to the questions in the differential question-and-answer groups. The semantic similarity between the pre-modification answer and the post-modification answer is calculated and the average value is taken as the output offset. By comparing the output offset with a preset threshold, an online token or rollback instruction is generated.

9. The adaptive update method for a large language model knowledge base according to claim 8, characterized in that: After receiving the launch token, the version management module generates a unique version identifier, writes the unique version identifier to the multi-version repository, activates the new version route, and cleans up expired branches.

10. The adaptive update method for a large language model knowledge base according to claim 9, characterized in that: The unique version identifier is generated by the version management module and is used to identify the new version. The multi-version repository stores all version data. The new version route points to the version corresponding to the activated unique version identifier. Expired branches are version branches that are no longer used in the multi-version repository.

Images