A multi-hop knowledge editing structure perception processing method for large language models

Abstract

This invention discloses a structure-aware processing method for multi-hop knowledge editing in large language models, relating to the field of artificial intelligence technology. The method includes: acquiring the question to be answered, the basic large language model, the editing request, and the editing memory; predicting the target hop count and forming a structural prior; generating and filtering candidate reasoning chains under the constraint of the target hop count; performing conflict-aware filling on the candidate reasoning chains; constructing structural diagnostic features and performing pairwise sorting; and outputting multi-hop question-answering results that conform to the edited knowledge state. This invention can reduce shortcut reasoning, early stopping, and redundant loops caused by old parameter knowledge in the model, making it easier for edited facts to propagate along the correct reasoning skeleton. It has good modular deployment advantages, reduces the cost of modifying existing question-answering systems, and can reduce the dependence on the surface quality of chained thinking text, making the final result more dependent on verifiable structure rather than the fluency of natural language narration.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a multi-hop knowledge editing structure perception processing method for large language models. Background Technology

[0002] Large language models excel in knowledge-intensive tasks, but their parameterized knowledge can become outdated as real-world facts change. While continuous pre-training or fine-tuning can update model knowledge, it is computationally expensive, carries the risk of forgetting information, and is difficult to roll back frequently. Knowledge editing techniques aim to modify specific factual relationships in the model at a lower cost, thereby improving the model's reliability when faced with new facts, updated databases, or personalized knowledge.

[0003] Existing knowledge editing methods can be broadly categorized into parametric editing and non-parametric editing. Parametric editing methods typically modify the model's internal weights directly, incorporating new facts into the model parameters. While effective for some single-fact updates, these methods may lead to forgetting, impaired reasoning ability, and difficulties in rollback. Non-parametric editing methods, also known as contextual knowledge editing, do not alter model weights but instead inject edited facts into the model by retrieving or inputting context. Therefore, they are more suitable for closed-source models, personalized updates, and frequent update scenarios.

[0004] The advantage of parameter editing is that the edited results can be internalized into the model weights, and additional context retrieval is not necessarily required in subsequent calls. However, its disadvantages are also quite obvious: on the one hand, weight modifications may affect unedited adjacent knowledge or language capabilities; on the other hand, the cost of recovery is high when facts change again or the edited results need to be undone. For commercial closed-source models or remote interface models, users usually cannot directly access and modify the weights, so parameter editing is not deployable in many practical application environments.

[0005] Nonparametric editing methods typically treat editable facts as external memory, influencing the output of large language models during inference through retrieval, hints, or contextual injection. These methods are non-intrusive to model weights, facilitating the addition, deletion, and updating of facts, and allowing for the maintenance of different editable memories for different users. However, the effectiveness of nonparametric editing is highly dependent on whether the model can actually use the injected facts during generation. For single-hop problems, the relationship between injected facts and the answer is direct, making it relatively easy for the model to utilize; for multi-hop problems, injected facts may only affect a single intermediate node in the inference chain, and their influence must continue to propagate to subsequent nodes, significantly increasing the problem's difficulty.

[0006] In multi-hop question-answering knowledge editing, the answer to a question typically relies on a reasoning chain consisting of multiple adjacent fact triples. Therefore, the core of multi-hop knowledge editing is not simply replacing an entity name, but rather maintaining an effective fact transformation chain of edited facts.

[0007] Existing context-based multi-hop editing, explicit sub-problem decomposition, chain-based reasoning rewriting, and graph retrieval methods typically attempt to enable models to use new knowledge by retrieving edited facts, decomposing multi-hop problems, or rewriting reasoning chains. Their common characteristic is that they still primarily rely on the model to organize the reasoning topology itself, or to locally modify the answer content after generation, lacking unified control over the target reasoning structure, the number of candidate chain hops, the location of conflicts with old knowledge, and the structural consistency of candidate answers.

[0008] The main drawback of existing technologies is that multi-hop knowledge editing failures do not entirely stem from unretrieved or unfilled factual content, but often occur at an earlier structural level. Under the influence of strong parametric priors, models may perform rapid reasoning along statistically more common old knowledge paths, resulting in edited facts appearing in the context but not on the factual transformation skeleton upon which the final answer depends. Furthermore, existing methods often lack records of intermediate states in candidate reasoning chains. For example, whether the candidate chain was generated forward or backward, whether it perfectly matches the target hop count, whether it comes from a relaxed backup set, which entities in the candidate chain are filled by edit memory, and which are freely generated by the model—if this information is not recorded, it is difficult to determine the reliability of candidate answers in the final selection stage. The final ranking can only rely on textual surface quality or answer confidence, failing to fully utilize structural diagnostic information.

[0009] Existing methods tend to implicitly incorporate experimental configurations, prior datasets, or specific model capabilities into their effectiveness. For example, when the base model happens to be able to answer certain questions based on prior knowledge, the apparent accuracy may be high, but the answer does not actually utilize edited facts; as the questions become longer or the edited facts are located at earlier intermediate nodes, the accuracy will drop significantly. For knowledge update systems that require stable deployment, relying solely on such implicit capabilities is unreliable.

[0010] Therefore, the core technical problem this invention aims to solve is: how to explicitly constrain and diagnose the inference chain structure during multi-hop knowledge editing in a large language model, so that candidate inference paths can match the target number of hops and fact transformation relationships, and fill the edited facts into the correct inference position when there is interference from old parameter knowledge, thereby outputting multi-hop question-and-answer results that conform to the edited knowledge state. Specifically, this invention intends to solve the following technical problems.

[0011] First, it addresses the mismatch between the number of hops in the model's inference chain and the transformation structure of the target fact in multi-hop knowledge editing. Second, it addresses the issue that when the edited fact conflicts with old parameter knowledge, the model tends to generate answers along shortcuts from old knowledge. Third, it addresses the lack of joint diagnosis of structural consistency, knowledge injection consistency, and path planning determinism in the candidate answer selection stage.

[0012] To address the aforementioned shortcomings, the present invention aims to provide a structure-aware processing method for multi-hop knowledge editing in large language models. This method predicts the target number of hops and forms a structural prior before candidate answer generation, performs hop count and type constraints during the candidate reasoning chain generation stage, detects and handles conflicts of old knowledge during the entity filling stage, and utilizes structural diagnosis for pairwise sorting during the candidate selection stage. This improves the matching of reasoning structures, the stability of propagation of edited facts, and the reliability of the final answer in multi-hop knowledge editing. Summary of the Invention

[0013] In view of the problems existing in the structure-aware processing method for multi-hop knowledge editing of large language models, this invention is proposed. It addresses the problems of structure degradation, interruption of edited fact propagation, and lack of structure diagnosis of candidate selection that are prone to occur in the existing multi-hop knowledge editing methods for large language models. It provides a structure-aware processing method for multi-hop knowledge editing of large language models to improve the matching of reasoning structure, the stability of edited fact utilization, and the reliability of final answer in multi-hop knowledge editing scenarios.

[0014] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0015] In a first aspect, the present invention provides a multi-hop knowledge editing structure-aware processing method for large language models, which includes: receiving a question to be answered, a basic large language model, an editing request and an editing memory, and initializing an allowed hop count set, a candidate sampling budget, a final candidate budget, a minimum threshold for retrieval relevance and a first-item leading magnitude threshold, and writing it into a task record;

[0016] The target hop count estimator performs stepwise structure analysis and target hop count prediction on the question to be answered, and obtains the stepwise structure analysis results and the target hop count. The stepwise structure analysis results and the target hop count are then written back to the task record.

[0017] The original candidates are generated and parsed according to the candidate sampling budget. They are restored and deduplicated in the order of the first recovery state, the second recovery state, and the third recovery state. The candidates that do not exceed the final candidate budget, along with their step number, type, state, generation direction, and word probability trajectory, are written into the final candidate set.

[0018] Read each candidate in the final candidate set one by one, construct a query and retrieve the edit facts from the edit memory, and append the fill value, fill source and conflict determination to the candidate record to form the filled candidate input;

[0019] For each candidate, construct a structural diagnostic feature vector, construct candidate pairs within the candidate set of the same question and calculate the pairwise preference probability, select the highest-scoring candidate by accumulating candidate scores, and write the answer field of the highest-scoring candidate into the output record;

[0020] Read the answer field of the highest-scoring candidate from the output record, output the multi-hop question and answer results that match the edit knowledge status, and extract intermediate information from the task record to form an audit record.

[0021] As a preferred embodiment of the multi-hop knowledge editing structure-aware processing method for large language models described in this invention, the question to be answered includes a natural language question and a query text preprocessed by the application system; the editing request consists of the subject, relation, and updated target object of the fact to be edited; the editing memory is a set of one or more editing requests; and the basic large language model includes a local deployment model and a remote interface model.

[0022] As a preferred embodiment of the multi-hop knowledge editing structure-aware processing method for large language models described in this invention, the stepwise structure analysis and target hop count prediction of the question to be answered via the target hop count estimator include:

[0023] Standardize the responses to questions by identifying the starting entities and relational expressions;

[0024] Input the normalization problem into the target hop count estimator to obtain a structured result containing step structure analysis fields and predicted hop count segments, and parse the explicit step markers and hop count segments;

[0025] When the predicted hop count does not belong to the allowed hop count set, it is mapped or truncated to a value in the allowed hop count set to obtain the target hop count;

[0026] Write the questions to be answered, the step-by-step structure analysis, the target number of hops, and the parsing status into the task record; if the output cannot be parsed, use the preset number of hops from the allowed number of hops set and record the rollback status.

[0027] As a preferred embodiment of the multi-hop knowledge editing structure-aware processing method for large language models described in this invention, the step of generating and parsing the original candidates according to the candidate sampling budget includes:

[0028] Generate a corresponding number of masked original candidate inference chains according to the candidate sampling budget. Each candidate must contain at least a step marker, an intermediate entity mask, a final answer mask, and an entity type block. Parse the step markers to obtain the actual number of hops of the candidate, and record the mask position, type constraints, generation direction, and lexical probability trajectory.

[0029] Establish a final candidate set and a recovery buffer. The first recovery state receives candidates that are exactly the same as the target number of hops; the second recovery state receives candidates that are no more than one hop different from the target number of hops; the third recovery state receives candidates that contain the final answer mask and whose type constraints are valid.

[0030] During the recovery process, the number of candidates in the first recovery state is counted first. If it is less than the final candidate budget, it is supplemented from the second recovery state. If it is still less than the final candidate budget, it is supplemented from the third recovery state. The process stops when the final candidate budget is reached or there are no more qualified candidates.

[0031] Candidates are obtained through forward or backward generation. Back sampling is turned off when the direct answer is empty, too short, rejected, or has insufficient confidence.

[0032] The final candidate set is deduplicated and the number of steps, generation direction, state, and lexical probability trajectory are frozen. The output retains candidates that do not exceed the final candidate budget.

[0033] As a preferred embodiment of the multi-hop knowledge editing structure-aware processing method for large language models described in this invention, the step of forming filled candidate inputs includes:

[0034] Traverse the mask to be filled in the order of candidate steps, construct a query for the current step, and retrieve relevant editing facts from the editing memory;

[0035] When the conflict detection preset conditions are met, the current step is determined to be highly correlated with the edited fact corresponding to the edited item with the highest similarity; otherwise, the old fact support judgment is entered.

[0036] When the score gate cannot be determined, the basic large language model is called to determine whether the discarded original facts support the current step; if they support them, it is determined that there is an old knowledge conflict. When an old knowledge conflict is detected, the basic large language model is prevented from generating freely and the updated object of the edit triple is used to fill it; if no old knowledge conflict is detected, the restricted model completion is performed according to the type constraint.

[0037] When the same mask appears again, the initial padding value is reused; when the current intermediate entity is used as a query condition for the next step, it is appended to the next query step.

[0038] After processing the final answer mask, save the candidate answers; when the filling fails, the type does not match, or the search result is empty, mark the candidate as low confidence or filling failed, and output the candidate after all mask processing is completed;

[0039] Record the source of the fill and output it along with the entity type check results to form filled candidate inputs.

[0040] As a preferred embodiment of the multi-hop knowledge editing structure-aware processing method for large language models described in this invention, the step of writing the answer field of the highest-scoring candidate into the output record includes:

[0041] For each candidate construct, a structural diagnostic feature vector is used to calculate knowledge injection consistency and path planning determinism. The structural diagnostic feature vector includes knowledge injection consistency score, path planning determinism score, hop count matching feature, structural state feature, generation direction feature, state confidence prior, candidate pool context feature, pool ranking feature, and answer-level aggregation feature.

[0042] Within the candidate set of the same problem, construct two candidate pairs composed of different candidates, calculate the dimension-wise difference of the structural diagnostic feature vectors of the candidate pairs, and then use the structural diagnostic pairwise sorter to calculate the pairwise preference probability and accumulate the score.

[0043] Initialize the cumulative score of each candidate, traverse the candidate pairs and accumulate the corresponding preference probability and its complement probability respectively, select the candidate with the highest cumulative score, and directly select and record the single candidate selection when there is only one valid candidate in the candidate set.

[0044] Write the final candidate, answer, cumulative score, and selection features into the output record.

[0045] As a preferred embodiment of the multi-hop knowledge editing structure awareness processing method for large language models described in this invention, the knowledge injection consistency score is calculated by checking whether the candidate filled entities meet the type constraints of the corresponding position. For each entity that needs to be checked, the basic large language model is called to determine whether the entity meets the preset type, and the proportion of the number of entities that pass the check to the total number of checked entities is used as the knowledge injection consistency score.

[0046] The path planning deterministic score is calculated based on the information entropy of the lexical prediction probability distribution during the candidate skeleton generation process, taking the negative of the mean of the prediction entropy at each position of the generated skeleton.

[0047] In a second aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of a multi-hop knowledge editing structure-aware processing method for large language models.

[0048] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements the steps of a multi-hop knowledge editing structure-aware processing method for large language models.

[0049] The beneficial effects of this invention are as follows: Instead of relying solely on retrieving editable facts and allowing the model to freely organize the inference chain, this invention explicitly predicts the target hop count before generation and constrains the fact transformation structure during the candidate chain generation stage. This reduces shortcut reasoning, premature stopping, and redundant loops caused by old parameter knowledge, making it easier for editable facts to propagate along the correct inference framework.

[0050] The candidate chain parsing results, hop count matching, mask type, generation direction, and state confidence are explicitly recorded and used for subsequent sorting. This not only generates readable reasoning text but also provides a computable diagnosis of whether the reasoning structure conforms to the target problem.

[0051] The pairwise sorting of this invention can compare conflicts between structural integrity, knowledge consistency and path confidence. In addition, this invention belongs to a non-parametric, context-based knowledge editing framework, which does not require direct rewriting of the weights of large language models. Therefore, it is more suitable for closed-source models, personalized fact updates and application scenarios that require frequent rollback or updates.

[0052] This invention also has the advantage of modular deployment. Target hop count estimation, candidate chain generation, conflict-aware filling, and pairwise sorting can be implemented, replaced, and optimized separately, reducing the cost of modifying existing question-answering systems.

[0053] This invention provides a clearer explanation of failures. If the final answer is incorrect, the error can be determined to stem from a deviation in the target hop count estimation, insufficient candidate skeleton generation, failure to edit fact retrieval, failure to trigger conflict detection, entity filling errors, or candidate sorting errors. Compared to obtaining only one incorrect answer, the intermediate structure records retained by this invention are more beneficial for subsequent parameter tuning, manual review, and continuous system improvement.

[0054] This invention can also reduce the reliance on the surface quality of chain-thinking texts by performing machine-readable checks on candidate chains through the number of steps, mask type, padding source, and structural state, making the final result more dependent on the verifiable structure rather than the fluency of natural language narration. Attached Figure Description

[0055] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 A flowchart of a multi-hop knowledge editing structure-aware processing method for large language models; Figure 2 This represents the case-level accuracy under different inference jump counts in the embodiments of the present invention. Detailed Implementation

[0057] To make the above-mentioned objects, features, and advantages of the present invention more readily understood, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0058] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0059] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it a single embodiment or an embodiment selectively excluded from other embodiments.

[0060] Reference Figure 1 and Figure 2 This is the first embodiment of the present invention, which provides a multi-hop knowledge editing structure-aware processing method for large language models, including:

[0061] Step S1: Obtain the question to be answered, the basic large language model, the edit request, and the edit memory; receive the question to be answered, the basic large language model, the edit request, and the edit memory; initialize the allowed hop count set, the candidate sampling budget, the final candidate budget, the minimum threshold for retrieval relevance, and the first item lead magnitude threshold, and write them into the task record; this task record serves as the input for S2, and intermediate results are continuously appended in S3 to S6;

[0062] Step S2: Predict the target hop count and form a structural prior; read the unanswered questions and the set of allowed hop counts from the task record, and obtain the stepwise structural analysis and target hop count through the target hop count estimator; write the stepwise structural analysis, target hop count and parsing state back to the task record, where the target hop count is directly used as the structural constraint for candidate generation and recovery in S3, and is reserved for hop count matching features to be calculated in S5;

[0063] Step S3: Generate and filter candidate inference chains under the target hop count constraint; read the written target hop count and the unanswered questions, basic large language model, candidate sampling budget and final candidate budget in the task record; generate and parse the original candidates according to the candidate sampling budget; restore and deduplicate them in order according to state A, state B and state C; write the candidates that do not exceed the final candidate budget and their step count, type, state, generation direction and word probability trajectory into the final candidate set; freeze and output the final candidate set.

[0064] Step S4: Perform conflict-aware filling on the candidate inference chain; read the final candidate set of the output one by one, construct the current step query for each mask and retrieve the edited facts from the edit memory; when the highest similarity exceeds the minimum relevance threshold and the lead of the highest similarity relative to the second highest similarity exceeds the minimum lead threshold, the edit memory takes over the filling; otherwise, perform old fact support judgment or type-restricted model completion; append the filling value, filling source and conflict judgment to the original candidate record to form the filled candidate input;

[0065] Step S5: Construct structural diagnostic features and perform pairwise sorting; read the filled candidates and reuse the target hop count and frozen structural fields; calculate the structural diagnostic feature vector for each candidate, construct candidate pairs within the same question and calculate the pairwise preference probability, accumulate candidate scores through a tournament to obtain the highest-scoring candidate; write the highest-scoring candidate, answer field, accumulated score and selection features into the output record;

[0066] Step S6: Output multi-hop question-and-answer results that conform to the knowledge editing status. Read the highest-scoring candidate and its answer field, and output multi-hop question-and-answer results that conform to the knowledge editing status; at the same time, extract the target hop count, candidate status, filling source, conflict judgment and sorting features from the same task record to form an audit record.

[0067] Specifically, in step S1, the question to be answered can be a natural language question or a query text preprocessed by the application system. Each edit request consists of the subject, relation, and updated target object of the fact to be edited. The edit memory is a collection of one or more edit requests, which can be maintained by user, business domain, time window, or knowledge base version. The basic large language model can be a locally deployed model or a remote interface model; this invention does not require modification of model parameters.

[0068] In addition to receiving questions to be answered, the underlying large language model, and edit memory, a parameter record for a single processing task is initialized. This parameter record includes at least the allowed hop count set, final candidate budget, candidate sampling budget, minimum threshold for retrieval relevance, first-item lead magnitude threshold, state confidence prior, and a trained or rule-configured Structure Diagnostic Pairwise Ranker (SDPR). These parameters can be pre-defined or configured according to real-time question answering or offline batch processing scenarios, but must remain consistent throughout the same processing task.

[0069] The steps pass machine-readable structured records, not just natural language answers: S2 outputs a step-by-step structure analysis and the target hop count; S3 outputs candidate records with steps, masks, types, generation directions, actual hop counts, states, and lexical probability trajectories; S4 appends fill values, fill sources, and conflict judgments to this structured record; S5 appends structural diagnostic features, pairwise preference probabilities, and cumulative scores; S6 reads the answer field of the highest-scoring candidate and forms the output. Therefore, any candidate can be traced back from the final answer to its structure prediction, generation, fill, and selection process.

[0070] When an intermediate stage cannot produce a normal output, the system continues processing according to preset branches: if the target hop count cannot be resolved, it is constrained to the allowed hop count set or a preset hop count is used; if there are insufficient precise hop count candidates, nearest neighbor recovery and backup recovery in S3 are initiated; if a single mask filling fails, the candidate is marked as low confidence or restricted completion is triggered; if all candidates are unavailable, the candidate budget is reallocated and candidate generation is performed once, or the system returns to the state of not obtaining a reliable answer. Thus, S1 to S6 form a complete closed loop from input, structure planning, candidate processing to answer output.

[0071] The input to step S2 is the unanswered questions, the set of allowed hops, and the target hop count estimator recorded in the task record of step S1. The output is the stepwise structure analysis, the target hop count, and the parsing status. The specific execution is as follows.

[0072] S2-1, standardize the questions to be answered by identifying the starting entity and relational expressions.

[0073] S2-2, input the normalization problem into the target hop count estimator, obtain the structured result containing step structure analysis fields and predicted hop count segments, and parse the explicit step markers and hop count segments.

[0074] S2-3, when the predicted hop count does not belong to the allowed hop count set, map or truncate it to a value in the allowed hop count set to obtain the target hop count.

[0075] The target hop count estimator learns the joint conditional distribution of problem structure and hop count during the training phase using the following formula:

[0076]

[0077] In the formula, The training loss for the target hop count estimator is used to measure the joint structure prediction error. The input problem serves as the input to the target hop count estimator; For stepwise structural analysis, The target number of hops during the training phase is the joint output of the stepwise structural analysis and the target number of hops during the training phase. The probability that the target hop count estimator generates this joint output given the input problem; These are trainable parameters used to characterize the target hop count estimator.

[0078] Negative logarithmic operations result in a higher generation probability for the correct structure analysis and the correct hop count combination. This loss is not calculated during the inference phase; instead, the trained estimator is used to generate the target hop count.

[0079] S2-4: Write the question to be answered, the step-by-step structure analysis, the target hop count, and the parsing status into the task record; S3: Read the target hop count and perform candidate generation and recovery; S5: Read the same target hop count and calculate the hop count matching feature. If the output cannot be parsed, use the preset hop count from the allowed hop count set and record the rollback status.

[0080] Step S3 reads the question to be answered, the target hop count, the basic large language model, the candidate sampling budget, and the final candidate budget, and outputs the final candidate set to be filled and the structure record of each candidate. The specific execution is as follows:

[0081] S3-1: Generate a corresponding number of masked original candidate inference chains according to the candidate sampling budget. Each candidate contains at least a step marker, an intermediate entity mask, a final answer mask, and an entity type block; the system parses the step marker to obtain the actual number of hops for the candidate and records the mask position, type constraints, generation direction, and lexical probability trajectory.

[0082] S3-2, establish the final candidate set and the recovery buffer. State A receives candidates that are exactly the same as the target hop count; State B receives candidates that differ from the target hop count by no more than one hop; State C receives candidates that are still parseable, contain the final answer mask, and have valid type constraints. The recovery states include State A, State B, and State C.

[0083] S3-3: For each parsed candidate, save the text, step sequence, mask and type mapping, actual hop count, generation direction, lexical probability trajectory, and parsing state. Candidates whose actual hop count equals the target hop count and whose type block is valid enter the final candidate set and are marked as state A; the remaining parseable candidates enter the recovery buffer; candidates that are unparseable, lack a final answer mask, or have an invalid type mapping do not enter subsequent padding.

[0084] The recovery process loosens unidirectionally in the order of state A, state B, and state C. First, the number of state A candidates in the final candidate set is counted. If this number is less than the final candidate budget, candidates whose actual hop count differs from the target hop count by no more than one hop are selected from the recovery buffer, moved or copied to the final candidate set, and marked as state B. If this number is still less than the final candidate budget, candidates that are parsable, contain the final answer mask, and have a valid type are selected from the remaining recovery buffer and added to the final candidate set, marked as state C. The process stops when the final candidate budget is reached or no more qualified candidates are available. Candidates already in a stricter state will not be downgraded due to subsequent loosening. States A, B, and C each have progressively decreasing structural reliability priors, but these priors only indicate the degree of recovery strictness and do not directly determine the correctness of the answer.

[0085] In one non-limiting embodiment, the prior structural reliability of states A, B, and C decreases sequentially, and can be set to 1.0, 0.5, and 0.25, respectively. Specific values ​​can be adjusted according to the application scenario, but the order of structural reliability among the three states should be maintained.

[0086] S3-4: Candidates can be obtained through forward or backward generation. Forward generation starts from the initial entity of the question and plans to the answer, while backward generation searches for intermediate entity paths from the direct answer or weak anchor points. Backsampling is disabled when the direct answer is empty, too short, rejected, or has insufficient confidence.

[0087] S3-5: Perform deduplication on the final candidate set and freeze structural fields such as step number, generation direction, state and lexical probability trajectory. After retaining candidates that do not exceed the final candidate budget, output them to S4. S4 only adds padding values ​​and sources. S5 reads the frozen fields to construct structural diagnostic features.

[0088] Step S4 reads the final candidate set, edit memory, and basic large language model, and outputs the filled candidate set; each candidate retains the original structure fields and appends the query, retrieval results, conflict determination, filling entity, and filling source.

[0089] S4-1, traverse the mask to be filled in the order of candidate steps, construct a query for the current step, and retrieve relevant editing facts from the edit memory; conflict detection first performs the fractional gating shown in the following formula:

[0090]

[0091] In the formula, This is a similarity function used to calculate the relevance between the query and the edited item in the current step. For the highest similarity edit item, The second highest similarity edit item is the one with the highest similarity edit item, while the first two items are the one with the highest similarity edit item and the one with the second highest similarity edit item. This is the lowest relevance threshold used to exclude irrelevant edits. This is the minimum leading margin threshold, used to determine whether the first term is significantly better than the second term.

[0092] When the conflict detection preset conditions are met, it is determined that the current step is highly related to the edited fact corresponding to the edited item with the highest similarity; otherwise, it proceeds to the old fact support judgment.

[0093] In S4-2, when the fractional gating cannot be determined, the basic large language model is called to determine whether the discarded original facts support the current step; if they do, it is determined that there is an old knowledge conflict. When an old knowledge conflict is detected, the basic large language model is prevented from generating freely and the updated object of the edit triple is used to fill it; when no old knowledge conflict is detected, restricted model completion is performed according to type constraints.

[0094] S4-3: When the same mask appears again, the initial fill value is reused; when the current intermediate entity is used as the next query condition, it is appended to the next query step, so that subsequent fill depends on the determined preceding entity.

[0095] S4-4 processes the final answer mask and saves candidate answers; if the filling fails, the type does not match, or the search result is empty, the candidate is marked as low confidence or the filling failed, and is then passed to S5 for downweighting or exclusion. The candidate is output only after all mask processing is completed.

[0096] S4-5 records the source of the fill as "edit memory takeover", "type-restricted model completion" or "fill failure", and outputs it along with the entity type check results to S5 for calculating knowledge injection consistency and answer-level aggregation features.

[0097] Step S5 reads the filled candidates, target hop count, structural state, generation direction and word probability trajectory, and outputs the cumulative score of each candidate and the candidate with the highest score; Step S6 reads the answer field of the candidate and forms the final output.

[0098] S5-1, Diagnostic feature vector for each candidate construction structure :

[0099]

[0100] In the formula, Inject consistency scores into knowledge to measure the degree of consistency between populated entities and types, as well as edited facts; The path planning determinism score is used to measure the uncertainty in candidate skeleton generation. The hop count matching feature indicates whether the candidate actual hop count matches the target hop count; The structural state feature represents the recovery state to which the candidate belongs; The generation direction feature indicates whether the generation is in the forward or reverse direction. The state confidence prior represents the initial value of the structural reliability; For candidate pool context features, For pooling ranking features, These are answer-level aggregation features, which describe the size and distribution of the candidate pool, the relative positions of candidates, and the multi-path support obtained by the same answer.

[0101] S5-2, calculate the consistency of knowledge injection and the determinism of path planning according to the following three formulas.

[0102]

[0103]

[0104]

[0105] In the formula, The entities that have already been filled are the candidate entities that need to be checked; The entity type is a type predefined by the mask; This is an indicator function that takes the value 1 if the condition is true, and 0 otherwise. It serves as the foundation for a large language model, used to perform consistency checks. For the type to satisfy the probability, The probability of type non-compliance corresponds to the conditional probabilities of "yes" and "no" output by the basic large language model, respectively. The system calculates entity-level knowledge injection consistency for all entities that need to be checked in the candidate list, and then obtains the knowledge injection consistency score by dividing the number of entities that passed the check by the total number of entities checked. When there are no entities to inspect, a preset neutral value is used and a missing marker is recorded to avoid division by zero. The lexical position indicates the position in the candidate skeleton generation process; This is the model vocabulary, containing generative lexical units; "Candidate word" indicates one of the word elements; The lexical prediction probability represents the probability of generating that lexical at the corresponding position; Predict the entropy for the location; The set of lexical positions included in the statistics records the positions that participated in the calculation when generating the candidate skeleton, and the size of the set represents the number of positions that participated in the statistics. To determine the path planning determinism score, the negative of the mean of the predicted entropy of these locations is taken. The larger the value, the lower the average uncertainty when generating the skeleton. When the location set is empty, a preset missing value is used and a missing label is recorded.

[0106] S5-3: Within the candidate set for the same problem, construct candidate pairs composed of two different candidates, calculate the dimension-wise difference between their structural diagnostic feature vectors, and then use the structural diagnostic pairwise sorter to calculate the pairwise preference probability and accumulate the score.

[0107]

[0108]

[0109]

[0110] In the formula, A candidate set for the same problem, used to provide a range for comparison; As the first candidate, The second candidate is one of two distinct candidates in the candidate set. This is the corresponding feature vector of the first candidate. This is the corresponding feature vector of the second candidate, used to describe the structural diagnostic information of the two candidates; Let be a dimensionally differentiable vector, and be the difference between two eigenvectors; For pairwise preference functions, Let the conditional probability that the first candidate is better than the second candidate be denoted as the pairwise preference probability. ; First candidate The tournament's cumulative score records the candidate's cumulative results across all pairwise comparisons; As the second candidate The tournament's cumulative points; The candidate with the highest cumulative score in the candidate set is the final candidate. The maximum value selection operation is used to complete the selection.

[0111] S5-4: Initialize the cumulative score of each candidate to 0, iterate through the candidate pairs and accumulate the corresponding preference probability and its complement probability, and select the candidate with the highest cumulative score; when scores are tied, use the knowledge injection consistency score, hop count matching status and preset stability rule in turn to break the tie; when there is only one valid candidate in the candidate set, select it directly and record "single candidate selection". S6: Write the final candidate, answer, cumulative score and selection features into the output record.

[0112] In one specific training method, supervised data for the SDPR can be constructed from the candidate generation logs. For two candidates for the same question, if one candidate answer is correct and the other is incorrect, they form a pair of preference samples; if both candidates are correct or incorrect, the candidate pair can be skipped to avoid introducing unstable preferences. This training method does not require additional manual sorting and labeling, but instead uses the final answer matching results of existing question-and-answer data to generate sorting supervision.

[0113] In summary, this invention no longer relies solely on allowing the model to freely organize the inference chain after retrieving edited facts. Instead, it explicitly predicts the target hop count before generation and constrains the fact transformation structure during the candidate chain generation stage. This reduces shortcut reasoning, premature stopping, and redundant loops caused by old parameter knowledge, making it easier for edited facts to propagate along the correct inference framework.

[0114] The candidate chain parsing results, hop count matching, mask type, generation direction, and state confidence are explicitly recorded and used for subsequent sorting. This not only generates readable reasoning text but also provides a computable diagnosis of whether the reasoning structure conforms to the target problem.

[0115] The pairwise sorting of this invention can compare conflicts between structural integrity, knowledge consistency and path confidence. In addition, this invention belongs to a non-parametric, context-based knowledge editing framework, which does not require direct rewriting of the weights of large language models. Therefore, it is more suitable for closed-source models, personalized fact updates and application scenarios that require frequent rollback or updates.

[0116] This invention also has the advantage of modular deployment. Target hop count estimation, candidate chain generation, conflict-aware filling, and pairwise sorting can be implemented, replaced, and optimized separately, reducing the cost of modifying existing question-answering systems.

[0117] This invention provides a clearer explanation of failures. If the final answer is incorrect, the error can be determined to stem from a deviation in the target hop count estimation, insufficient candidate skeleton generation, failure to edit fact retrieval, failure to trigger conflict detection, entity filling errors, or candidate sorting errors. Compared to obtaining only one incorrect answer, the intermediate structure records retained by this invention are more beneficial for subsequent parameter tuning, manual review, and continuous system improvement.

[0118] This invention can also reduce the reliance on the surface quality of chain-thinking texts by performing machine-readable checks on candidate chains through the number of steps, mask type, padding source, and structural state, making the final result more dependent on the verifiable structure rather than the fluency of natural language narration.

[0119] To verify the feasibility and effectiveness of the technical solution of this invention, multiple sets of experiments were conducted covering multi-hop question-answering knowledge editing datasets such as MQuAKE-CF-3k-v2, MQuAKE-T, RippleEdits-Popular, and RippleEdits-Random. The results were validated on large language models such as Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Qwen2.5-14B-Instruct. The evaluation metric was multi-hop question-answering accuracy; a higher value indicated that the model was more capable of providing the correct answer in the knowledge editing state.

[0120] The dataset names, model names, and comparison method names in the following table are the original identifiers of the experimental objects, such as MQuAKE, RippleEdits, Llama, Qwen, MEMIT, RAE, etc. Therefore, the original text is retained in the experimental data. These English identifiers are only used to indicate the source of the experiment and the comparison objects.

[0121] Experiments were conducted to validate the target hop count estimator (StructEst). These experiments compared the accuracy of the target hop count estimator with several general large models on the target hop count prediction task, demonstrating that a specially trained lightweight structure estimator can provide reliable structural priors for subsequent candidate generation.

[0122] Table 1 Target Jump Count Prediction Accuracy

[0123]

[0124] As shown in Table 1, the target hop count estimator achieves a target hop count prediction accuracy of 93.34% with a 7B parameter scale, significantly higher than several larger-scale general models. This indicates that target hop count estimation is not merely a matter of model size, but requires specialized training and format constraints for multi-hop structures. This result supports the use of the target hop count estimator as a pre-module for subsequent structural constraint generation in this invention.

[0125] Table 2. Main Experiment of Multi-Hop Knowledge Editing

[0126] Table 2 shows that StructKE maintains a stable advantage across different model backbones and datasets. For example, on MQuAKE-CF-3k-v2, Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct achieve accuracies of 63.5% and 59.7%, respectively, both higher than the comparative methods under the same settings. This indicates that structural constraint candidate generation and structural diagnostic selection can improve the accuracy of answers after the edited facts propagate along multi-hop paths.

[0127] This invention demonstrates stable performance across different model sizes and dataset types. For datasets like MQuAKE-CF-3k-v2, which include counterfactual updates and cover 2- to 4-hop problems, StructKE's advantages are particularly evident. Even on datasets like RippleEdits, which primarily consist of 2-hop problems, StructKE can still improve or maintain high accuracy. This indicates that structure-aware processing is not only applicable to long-chain problems but can also provide assistance in shorter problems that still require propagation of edit facts.

[0128] Under the Qwen2.5-7B-Instruct settings, the case-level accuracy of 2-hop, 3-hop, and 4-hop problems was statistically analyzed. StructKE remained competitive in all hop arrays and demonstrated a more significant structural control advantage on longer inference chains. It still maintained an accuracy of over 50% on the most difficult 4-hop subset, indicating that this method can maintain the effective propagation of edited knowledge as the number of fact transformations increases.

[0129] Table 3 Final Output Jump Count Matching Rate

[0130] Table 3 shows that StructKE achieved the highest hop count matching rate across correct samples, incorrect samples, and all outputs, with an overall matching rate of 62.77%, higher than RAE's 19.40%, PokeMQA's 18.30%, EditCoT's 7.63%, and MeLLo's 41.57%. This result indicates that the target hop count constraint and structural diagnostic selection not only improve answer accuracy but also enhance the consistency between the final output and the target fact transformation structure.

[0131] In summary, the target hop count estimator can reliably predict the target hop count; the complete method improves the accuracy of multi-hop question answering on multiple models and multiple datasets; it remains effective on 2- to 4-hop questions, and the final output hop count matching rate is significantly improved. The above results demonstrate the feasibility and technical effectiveness of this method from four aspects: structural prior reliability, overall answer correctness, long chain applicability, and output structure consistency.

[0132] This embodiment also provides a computer device applicable to a multi-hop knowledge editing structure-aware processing method for large language models, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement all or part of the steps of the method described in the above embodiments of the present invention.

[0133] This embodiment also provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, it performs the method in any optional implementation of the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0134] The storage medium proposed in this embodiment and the data storage method proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0135] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A structure-aware processing method for multi-hop knowledge editing of large language models, characterized in that: include: Receive the questions to be answered, the basic large language model, the edit request and the edit memory, and initialize the allowed hop count set, the candidate sampling budget, the final candidate budget, the minimum threshold for retrieval relevance and the first item lead magnitude threshold, and write them to the task record; The target hop count estimator performs stepwise structure analysis and target hop count prediction on the question to be answered, and obtains the stepwise structure analysis results and the target hop count. The stepwise structure analysis results and the target hop count are then written back to the task record. The original candidates are generated and parsed according to the candidate sampling budget. They are restored and deduplicated in the order of the first recovery state, the second recovery state, and the third recovery state. The candidates that do not exceed the final candidate budget, along with their step number, type, state, generation direction, and word probability trajectory, are written into the final candidate set. Generate a corresponding number of masked original candidate inference chains according to the candidate sampling budget. Each candidate must contain at least a step marker, an intermediate entity mask, a final answer mask, and an entity type block. Parse the step markers to obtain the actual number of hops of the candidate, and record the mask position, type constraints, generation direction, and lexical probability trajectory. Establish a final candidate set and a recovery buffer. The first recovery state receives candidates that are exactly the same as the target hop count. The second recovery state receives candidates whose hop count differs from the target hop count by no more than one hop. The third recovery state receives candidates that contain the final answer mask and whose type constraints are valid; Read each candidate in the final candidate set one by one, construct a query and retrieve the edit facts from the edit memory, and append the fill value, fill source and conflict determination to the candidate record to form the filled candidate input; For each candidate, construct a structural diagnostic feature vector, construct candidate pairs within the candidate set of the same question and calculate the pairwise preference probability, select the highest-scoring candidate by accumulating candidate scores, and write the answer field of the highest-scoring candidate into the output record; Read the answer field of the highest-scoring candidate from the output record, output the multi-hop question and answer results that match the edit knowledge status, and extract intermediate information from the task record to form an audit record.

2. The method for multi-hop knowledge editing structure awareness processing for large language models as described in claim 1, characterized in that: The questions to be answered include natural language questions and query text preprocessed by the application system. The edit request consists of the subject, relation, and updated target object of the fact to be edited; the edit memory is a collection of one or more edit requests; and the basic large language model includes a local deployment model and a remote interface model.

3. The method for multi-hop knowledge editing structure awareness processing for large language models as described in claim 1, characterized in that: The stepwise structural analysis and target hop count prediction of the question to be answered using the target hop count estimator include: Standardize the responses to questions by identifying the starting entities and relational expressions; Input the normalization problem into the target hop count estimator to obtain a structured result containing step structure analysis fields and predicted hop count segments, and parse the explicit step markers and hop count segments; When the predicted hop count does not belong to the allowed hop count set, it is mapped or truncated to a value in the allowed hop count set to obtain the target hop count; Write the questions to be answered, the step-by-step structure analysis, the target number of hops, and the parsing status into the task record; if the output cannot be parsed, use the preset number of hops from the allowed number of hops set and record the rollback status.

4. The method for multi-hop knowledge editing structure awareness processing for large language models as described in claim 1, characterized in that: The step of generating and parsing the original candidates according to the candidate sampling budget includes: Generate a corresponding number of masked original candidate inference chains according to the candidate sampling budget. Each candidate must contain at least a step marker, an intermediate entity mask, a final answer mask, and an entity type block. Parse the step markers to obtain the actual number of hops of the candidate, and record the mask position, type constraints, generation direction, and lexical probability trajectory. Establish a final candidate set and a recovery buffer. The first recovery state receives candidates that are exactly the same as the target number of hops; the second recovery state receives candidates that are no more than one hop different from the target number of hops; the third recovery state receives candidates that contain the final answer mask and whose type constraints are valid. During the recovery process, the number of candidates in the first recovery state is counted first. If it is less than the final candidate budget, it is supplemented from the second recovery state. If it is still less than the final candidate budget, it is supplemented from the third recovery state. The process stops when the final candidate budget is reached or there are no more qualified candidates. Candidates are obtained through forward or backward generation. Back sampling is turned off when the direct answer is empty, too short, rejected, or has insufficient confidence. The final candidate set is deduplicated and the number of steps, generation direction, state, and lexical probability trajectory are frozen. The output retains candidates that do not exceed the final candidate budget.

5. The method for multi-hop knowledge editing structure awareness processing for large language models as described in claim 1, characterized in that: The process of forming filled candidate inputs includes: Traverse the mask to be filled in the order of candidate steps, construct a query for the current step, and retrieve relevant editing facts from the editing memory; When the conflict detection preset conditions are met, the current step is determined to be highly correlated with the edited fact corresponding to the edited item with the highest similarity; otherwise, the old fact support judgment is entered. When the score gate cannot be determined, the basic large language model is called to determine whether the discarded original facts support the current step; if they support them, it is determined that there is an old knowledge conflict. When an old knowledge conflict is detected, the basic large language model is prevented from generating freely and the updated object of the edit triple is used to fill it; if no old knowledge conflict is detected, the restricted model completion is performed according to the type constraint. When the same mask appears again, the initial padding value is reused; when the current intermediate entity is used as a query condition for the next step, it is appended to the next query step. After processing the final answer mask, save the candidate answers; when the filling fails, the type does not match, or the search result is empty, mark the candidate as low confidence or filling failed, and output the candidate after all mask processing is completed; Record the source of the fill and output it along with the entity type check results to form filled candidate inputs.

6. The method for multi-hop knowledge editing structure-aware processing for large language models as described in claim 1, characterized in that: The step of writing the answer field of the highest-scoring candidate to the output record includes: For each candidate construct, a structural diagnostic feature vector is used to calculate knowledge injection consistency and path planning determinism. The structural diagnostic feature vector includes knowledge injection consistency score, path planning determinism score, hop count matching feature, structural state feature, generation direction feature, state confidence prior, candidate pool context feature, pool ranking feature, and answer-level aggregation feature. Within the candidate set of the same problem, construct two candidate pairs composed of different candidates, calculate the dimension-wise difference of the structural diagnostic feature vectors of the candidate pairs, and then use the structural diagnostic pairwise sorter to calculate the pairwise preference probability and accumulate the score. Initialize the cumulative score of each candidate, traverse the candidate pairs and accumulate the corresponding preference probability and its complement probability respectively, select the candidate with the highest cumulative score, and directly select and record the single candidate selection when there is only one valid candidate in the candidate set. Write the final candidate, answer, cumulative score, and selection features into the output record.

7. The method for multi-hop knowledge editing structure awareness processing for large language models as described in claim 6, characterized in that: The knowledge injection consistency score is calculated by checking whether the candidate filled entities meet the type constraints of the corresponding position. For each entity that needs to be checked, the basic large language model is called to determine whether the entity meets the preset type. The proportion of the number of entities that pass the check to the total number of checked entities is used as the knowledge injection consistency score. The path planning deterministic score is calculated based on the information entropy of the word prediction probability distribution during the candidate skeleton generation process, taking the negative of the mean of the prediction entropy at each position of the generated skeleton.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the multi-hop knowledge editing structure-aware processing method for large language models as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the multi-hop knowledge editing structure-aware processing method for large language models as described in any one of claims 1 to 7.

Images