Stable knowledge editing method and system based on attention drift constraint

Abstract

The invention provides a stable knowledge editing method and system based on attention drift constraint, and relates to the technical field of natural language processing, and the method comprises the steps: introducing a selective attention drift limitation strategy in a knowledge editing process of a large language model; by comparing model attention output before and after editing, positioning an attention head with a serious attention drifting phenomenon; and based on the new optimization target function, training the model to obtain an optimal model parameter, and realizing stable knowledge editing. According to the method, the editing accuracy is remarkably improved, the probability of wrong answers is reduced, experimental verification shows that the specific task performance is effectively improved, and theoretical support and practical guidance are provided for knowledge editing of a large language model.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a stable knowledge editing method and system based on attention drift constraints. Background Art

[0002] Large language models have shown excellent performance in a wide range of tasks, however, due to the internal defects of the model, many unpredictable errors will inevitably occur, which originate from negative samples or outdated samples in a large number of pre-training datasets. The knowledge editing task addresses the above problems by effectively modifying a very small proportion of parameters to accurately edit specific model knowledge. Existing knowledge editing methods can be divided into three main paradigms: parameter preservation, localization-post-editing, and meta-learning methods. Parameter preservation methods explicitly store the modified knowledge in memory and use methods such as classifiers, hint engineering, or external parameter retrieval knowledge. Localization-post-editing methods update specific parameters by identifying the storage locations of the target knowledge and directly editing these locations. Meta-learning methods involve training hypernetworks to edit the parameters of the model.

[0003] Despite the promising prospects of knowledge editing, existing methods still face major challenges in terms of specificity. Editing a specific piece of knowledge can lead to a chain reaction within the knowledge graph, affecting the performance of other parts of the model, which is called specificity failure. A typical example is that after editing a certain geographical knowledge in the model (such as a city is the capital of a country), the model will still make mistakes in other contexts involving the city. In order to limit these side effects, the edited model may be under-edited or over-edited, affecting the model's ability to handle a wide range of tasks. In recent research, Hoelscher-Obermaier explored this issue in depth, proposed an improved benchmark CounterFact+ by introducing dynamic components to evaluate the specificity failure of models in knowledge editing, and proposed a metric to measure specificity based on KL divergence. ROME focuses on the problem of the damage to the neighboring knowledge of the edited entity and alleviates the damage to local knowledge by repairing these chain reactions. However, it does not fully consider the impact of the edited entity on the broader context, which may lead to incorrect inferences in other unrelated knowledge areas. Although RETS proposes a knowledge editing method based on relational central interpretation to solve the problem of specificity failure, this method significantly sacrifices the accuracy of editing. Therefore, there is an urgent need for a method that can balance specificity and generalization when dealing with knowledge editing tasks. Summary of the invention

[0004] To this end, an embodiment of the present invention provides a stable knowledge editing method and system based on attention drift constraints, which are used to solve the specific failure problem that occurs in the current large language model (LLMs) knowledge editing process in the prior art.

[0005] In order to solve the above problem, an embodiment of the present invention provides a stable knowledge editing method based on attention drift constraint, the method comprising:

[0006] In the knowledge editing process of large language models, a selective attention drift limitation strategy is introduced, which includes:

[0007] By comparing the model attention output before and after editing, we can locate the attention heads with serious attention drift.

[0008] According to the set constraint criteria, select the attention heads that pay too much attention to the edited object for constraint;

[0009] Align the attention output of the selected attention head with the output before editing to prevent over-editing;

[0010] Constructing the objective function of selective attention drift limitation, combining the objective function of selective attention drift limitation with the original loss function to form a new optimization objective function;

[0011] Based on the new optimization target objective function, the model is trained to obtain the optimal model parameters and achieve stable knowledge editing.

[0012] Preferably, the method also includes: testing the optimized model using test data, and using multiple evaluation indicators to evaluate its performance, wherein the evaluation indicators include performance score, performance level, interpretation score, interpretation level, neighborhood score, neighborhood level, relationship score, relationship level, interference neighborhood score, and interference neighborhood level.

[0013] Preferably, the set constraint standard is: when the attention weight of a certain attention head to the last word of the subject exceeds the maximum attention weight of all attention heads in the original model, the attention head is selected for constraint.

[0014] Preferably, the method for constructing the objective function of selective attention drift limitation is:

[0015]

[0016] In the formula, represents the objective function of selective attention drift limitation; W l,h (S) represents the attention weight of the h-th attention head in layer l when processing prompt S, where l represents the number of layers of the attention mechanism, h represents the index of the attention head in a specific layer l, and S represents the input prompt or sentence; represents the attention weight of the h-th attention head in the l-th layer when the edited model with z processes the prompt S, where Indicates that during the editing process, for the specific model or method selected at level l, :=z indicates that the model or method is edited or modified with a specific parameter or value z; H l (S j ) indicates that in the jth sample S j In the above example, the set of attention heads in the lth layer that meet the specific condition, where the specific condition is that the attention weight of the edited model on the edited object s at the last word is greater than the maximum attention weight of the original model on the object at this layer; S j represents the jth input sample; D kL is the Kullback-Leibler divergence, which is used to measure the difference between two probability distributions; N represents the total number of input samples.

[0017] Preferably, the H l (S j ) is expressed as:

[0018]

[0019] in

[0020] M l (S j )=max h W l,h (S j )[-1,s];

[0021] Where M l (S j ) represents the maximum attention weight of the last token on the edited object s among all the attention heads in the lth layer of the original model.

[0022] Preferably, the method of combining the objective function of selective attention drift limitation with the original loss function to form a new optimization objective function is:

[0023]

[0024] In the formula, v * Represents the new optimization objective function; represents the original loss function; γ represents the control weight.

[0025] The embodiment of the present invention further provides a stable knowledge editing system based on attention drift constraint, which is used to implement the above-mentioned stable knowledge editing method based on attention drift constraint, and specifically includes:

[0026] The selective attention drift limitation strategy introduction module is used to introduce the selective attention drift limitation strategy in the knowledge editing process of a large language model. The selective attention drift limitation strategy includes:

[0027] By comparing the model attention output before and after editing, we can locate the attention heads with serious attention drift.

[0028] According to the set constraint criteria, select the attention heads that pay too much attention to the edited object for constraint;

[0029] Align the attention output of the selected attention head with the output before editing to prevent over-editing;

[0030] The objective function construction and combination module is used to construct the objective function of selective attention drift limitation, and combine the objective function of selective attention drift limitation with the original loss function to form a new optimization objective function;

[0031] The model training and optimization module is used to train the model based on the new optimization target objective function to obtain the optimal model parameters and achieve stable knowledge editing.

[0032] Preferably, the system also includes a testing and evaluation module for testing the optimized model using test data, and using multiple evaluation indicators to evaluate its performance, wherein the evaluation indicators include performance score, performance level, interpretation score, interpretation level, neighborhood score, neighborhood level, relationship score, relationship level, interference neighborhood score, and interference neighborhood level.

[0033] An embodiment of the present invention also provides an electronic device, which includes a processor, a memory and a bus system, wherein the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the above-mentioned stable knowledge editing method based on attention drift constraints.

[0034] An embodiment of the present invention also provides a computer storage medium, which stores a computer software product. The computer software product includes a number of instructions for enabling a computer device to execute the above-mentioned stable knowledge editing method based on attention drift constraints.

[0035] It can be seen from the above technical solutions that the present invention has the following beneficial effects:

[0036] (1) By introducing the selective attention drift restriction (SADR) strategy, the present invention optimizes the knowledge storage and retrieval mechanism within the large language model, effectively reducing the specificity failure problem that occurs during the knowledge editing process, thereby improving the accuracy and stability of the model.

[0037] (2) The present invention can dynamically impose constraints on different attention heads as needed to avoid over-editing caused by attention heads that over-focus on entities related to edited knowledge. This dynamic adjustment strategy not only improves the accuracy of editing, but also enables the method to flexibly adapt to various editing methods without major modifications to the model structure.

[0038] (3) The present invention comprehensively evaluates the reliability, generalization, and specificity of knowledge editing through multiple experimental indicators, and significantly improves the performance of specific tasks. These experimental results not only verify the effectiveness of the SADR strategy, but also provide theoretical support and practical guidance for research and practice in related fields, which has important scientific value and application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the implementation cases of the present invention or the technical solutions in the prior art, the following is a brief description of the drawings required for use in the embodiments. By referring to the drawings, the features and advantages of the present invention will be more clearly understood. The drawings are schematic and should not be understood as limiting the present invention in any way. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0040] Figure 1 A flowchart of a stable knowledge editing method based on attention drift constraint provided in an embodiment;

[0041] Figure 2 It is a schematic diagram of positioning specificity in the embodiment;

[0042] Figure 3 Schematic diagram of the positive correlation between attention weight drift and the probability of wrong answers in the embodiment;

[0043] Figure 4 A schematic diagram of alleviating specificity by fixing attention drift in an embodiment;

[0044] Figure 5 Schematic diagram of an embodiment in which selectively limiting the attention heads that over-focus on the edit token outperforms limiting all attention heads in terms of editing success and specificity under different γ settings;

[0045] Figure 6 It is a schematic diagram of the trade-off comparison between the present invention and other methods in the embodiments;

[0046] Figure 7 A block diagram of a stable knowledge editing system based on attention drift constraints provided in an embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Embodiment 1

[0049] In order to solve the problem of specificity failure in the knowledge editing process of large language models (LLMs) in the prior art. Figure 1 As shown, an embodiment of the present invention proposes a stable knowledge editing method based on attention drift constraint, the method comprising:

[0050] S1: Introducing a selective attention drift limitation strategy in the knowledge editing process of large language models;

[0051] S2: Locate the attention heads with serious attention drift by comparing the model attention output before and after editing;

[0052] S3: Based on the new optimization target objective function, the model is trained to obtain the optimal model parameters and achieve stable knowledge editing.

[0053] From the above technical solution, it can be seen that the present invention provides a stable knowledge editing method based on attention drift constraints. By comparing the model attention output before and after editing, the attention heads with serious attention drift are located and constrained, the internal knowledge storage of the model is optimized, and the constraint strategy is dynamically adjusted to adapt to various editing methods. It significantly improves the editing accuracy and reduces the probability of wrong answers. Experimental verification shows that this method effectively improves the performance of specific tasks and provides theoretical support and practical guidance for knowledge editing of large language models.

[0054] The technical solution of the present invention is based on the existing knowledge editing method and aims to optimize the knowledge storage and retrieval within the model. Previous studies have represented factual associations as knowledge tuples t = (s, r, o), where s is the subject, r is the relation, and o is the object. The goal of knowledge editing is to use new factual associations (s, r, o edit ) replaces the original fact association stored in the model parameters, where o edit is the counterfactual target object (e.g., replacing “Paris” with “New York”), represented by o true Represents a real answer from the real world.

[0055] The core idea of ​​the ROME method is to modify the "value" (v) of the second layer of MLP so that when the "key" (k) is associated with the target subject, the target knowledge can be correctly predicted. When the "key" is not associated, the original knowledge of the model is kept unaffected to ensure the stability of the model in irrelevant contexts. Its loss function is as follows:

[0056]

[0057] However, this goal may cause attention drift, which may lead to specificity failure. In order to optimize the ideal vector v * In order to enhance the specificity of knowledge editing, the present invention introduces the Selective Attention Drift Limitation (SADR) strategy, which is a regularization term based on the above formula. It is worth noting that the SADR strategy dynamically imposes constraints on different attention heads as needed, because the Transformer model contains multiple knowledge-specific attention heads that capture different factual associations. In addition, the SADR strategy is a simple and efficient method that can flexibly adapt to various editing methods.

[0058] More specifically, since the excessive attention of some attention heads to the edited object is closely related to the failure of specificity, the SADR strategy is applied to those attention heads that over-focus on the edited object on the last word. The SADR strategy includes: locating the attention heads with serious attention drift by comparing the model attention output before and after editing; selecting the attention heads that over-focus on the edited object for constraint according to the set constraint criteria (for example, when the attention weight of an attention head to the last word of the subject exceeds the maximum attention weight of all attention heads in the original model, the attention head is selected for constraint); aligning the attention output of the selected attention head with the output before editing to prevent over-editing.

[0059] In step S2, an objective function of selective attention drift limitation is constructed, and the objective function of selective attention drift limitation is combined with the original loss function to form a new optimization objective function.

[0060] Furthermore, the objective function for selective attention drift limitation is constructed as follows:

[0061]

[0062] In the formula, represents the objective function of selective attention drift limitation; W l,h (S) represents the attention weight of the h-th attention head in layer l when processing prompt S, where l represents the number of layers of the attention mechanism, h represents the index of the attention head in a specific layer l, and S represents the input prompt or sentence; represents the attention weight of the h-th attention head in the l-th layer when the edited model with z processes the prompt S, where Indicates that during the editing process, for the specific model or method selected at level l, :=z indicates that the model or method is edited or modified with a specific parameter or value z; H l (S j ) indicates that in the jth sample S j In the above example, the set of attention heads in the lth layer that meet the specific condition, where the specific condition is that the attention weight of the edited model on the edited object s at the last word is greater than the maximum attention weight of the original model on the object at this layer; S j represents the jth input sample; D KL is the Kullback-Leibler divergence, which is used to measure the difference between two probability distributions; N represents the total number of input samples.

[0063] The above H l (S j ) is expressed as:

[0064]

[0065] in

[0066] M l (S j )=max h W l,h (S j )[-1,s];

[0067] Where M l (S j ) represents the maximum attention weight of the last token on the edited object s among all the attention heads in the lth layer of the original model.

[0068] The new optimization objective function is:

[0069]

[0070] In the formula, v * Represents the new optimization objective function (ideal vector v * ); represents the original loss function (the loss function of the ROME method); γ represents the control weight.

[0071] In step S3, based on the new optimization target objective function, the model is trained to obtain the optimal model parameters and achieve stable knowledge editing.

[0072] In addition, in order to demonstrate the rationality of the stable knowledge editing method based on attention drift constraints proposed in this invention, we conducted a series of experiments to explore the root causes of specificity failure.

[0073] Existing work mainly evaluates the reliability, generalization, and specificity of knowledge editing through the following evaluation indicators:

[0074] Efficacy Score (ES) and Efficacy Magnitude (EM) represent the p(o edit |s,r)>P(O true |s,r) and p(o edit |s,r)’s average value.

[0075] Paraphrase Score (PS) and Paraphrase Magnitude (PM) evaluation models in paraphrase prompts (s para ,r para ) under the generalization performance (e.g., “the Eiffel Tower is located at”). These two indicators are denoted as p(o edit |s para ,r para )>P(o true |s para ,r para ) and P(o edit |s para ,r para ).

[0076] Neighborhood Score (NS) and Neighborhood Magnitude (NM) evaluate specificity by providing a neighboring but different subject s′ (e.g., “Louvre”). These two metrics are denoted as P(o true |s′,r)>P(o edit |s′,r) and P(o true |s′,r).

[0077] Recent studies have shown that the presence of edited subjects in reasoning contexts impairs specificity performance, and thus, the present invention introduces two additional specificity indicators that include edited subjects in the test cues.

[0078] Relation Score (RS) and Relation Magnitude (RM) are used to evaluate how the model handles attributes of the edited subject that are irrelevant to the edit, defined as relation r′. Empirically, the edited model tends to predict o even under irrelevant relations.edit (For example, “The color of the Eiffel Tower is New York”). So we pass P(o true |s,r′)>P(o edit |s,r′) to calculate the relationship score and pass P(o true |s,r′) to calculate the magnitude of the relationship.

[0079] Distract Neighborhood Score (DNS) and Distract Neighborhood Magnitude (DNM) are similar to NS and NM, but the edited sentences (s, r, o) are concatenated first in the test prompt of the neighborhood task. edit These indicators are expressed as and

[0080] 1. Failure of calibrating specificity on counterfactual benchmarks: To measure the performance of relational and distractor domain tasks, we used the counterfact The dataset consists of 1683 factual statements.

[0081] Table 1 below illustrates the significant specificity failure when the edited subject appears in context, with the edited model incorrectly outputting the edited object in more than 50% of the test cases. In addition, the wrong answer o edit The average probability is much higher than the correct answer o true , 48.4% vs. 3.3% in the relation task, and 24.9% vs. 10.5% in the distractor neighborhood task.

[0082] Table 1

[0083]

[0084] 2. Localizing Specificity Failure: In the forward pass of an autoregressive language model, the information flow can be viewed as a causal graph. When a model with L layers makes predictions based on a prompt containing T tokens, each module (such as attention modules, MLPs, and transformer blocks) produces T×L outputs. Each of these outputs is affected by the previous output of the previous layer and the previous token position. Inspired by causal tracing, the present invention tracks different states in the causal graph to determine which parts contaminate the information flow in specificity failure.

[0085] First, a forward pass is performed on the edited model using the test prompt and the outputs of various network modules from different layers and token positions are recorded. Then, a forward pass is performed using the original model, copying the representation of a specific module from the stored output to the corresponding position without changing other computations. Each module at layer l and token position t is traversed within a window of layer k. We call this approach "contaminated substitution" and quantify its impact on the final output probability as:

[0086]

[0087] in, It means replacing the module result at the token t position of layer l with z.

[0088] like Figure 2 As shown, the light-colored areas represent the main states that lead to incorrect answers. We observe that replacing the six-layer MLP activations or attention activations can reduce the probability of the correct answer by 4.59% and 3.74%, respectively, while the total reduction caused by editing the model is 5.26%. When we edit the MLP module at layer 5, it is expected that the contamination replacement of the MLP activations near the edited layer will significantly affect the final prediction. However, modifying the attention activations in the middle and upper layers also has a similar impact on the correct output, indicating that the main reason for the failure of specificity is that the attention module incorrectly processes the information on the last token due to the editing. These findings are consistent on the relational task, which further demonstrates the rationality of our constraints on attention.

[0089] 3. Identifying attention drift as a trigger for specificity failure: As mentioned above, attention activation is one of the main causes of specificity failure. Previous studies have shown that attention modules in the middle and upper layers extract factual attributes during prediction. This suggests that attention modules may mistakenly focus on edited information, thereby ignoring other information when predicting the final token. Therefore, we use the Pearson coefficient to quantify the relationship between attention weight drift and failure of distractor domain and relation tasks. Given that the editing model overestimates the edited object o edit Relative to the real object o true We analyzed the probability and P(o edit ), where W l,h and denote the attention weights of the last token on the previous token in layer l and attention head h before and after editing, respectively.

[0090] Figure 3 Shows the attention weight drift and the wrong answer o editThis suggests that the wrong attention to previous information is a key factor in the failure of specificity. Further analysis of attention drift from the perspective of attended tokens and attention heads naturally raises the following two questions:

[0091] (1) The last token misattributes attention to previous tokens, leading to a failure of specificity. Among these previous tokens, which one would have a significant effect when misattended?

[0092] (2) Which has a greater impact on prediction: excessive local attention drift of a specific attention head or the cumulative attention drift of all heads?

[0093] To address these issues, we calculated the relationship between various factors and P(o edit ). Table 2 below shows that the drift Pearson coefficient of the last token of the edited object s is higher than that of other tokens (Formula 1 vs Formula 2). In addition, the maximum drift of the attention weights between attention heads has a greater impact on the final result than the cumulative drift of all attention heads (Formula 1 vs Formula 2), which indicates that the excessive attention of some attention heads to the last token is the main reason for triggering specificity failure. Therefore, we only constrain the attention heads whose attention weight of the last token of the subject exceeds the maximum attention weight of all heads in the original model.

[0094] Table 2

[0095]

[0096] 4. Mitigating Specificity Failure by Patching Attention Drift: To further verify the significant impact of attention drift on specificity failure, we quantified the change in predicted probability after patching the attention weights at each layer. We first performed a forward pass using the original model with specificity task prompts and stored the intermediate attention weights. Then, we tested the edited model on the same prompts, replacing its attention weights with the previously stored weights.

[0097] We found that patching the attention weights in the upper and middle layers can lead to significant improvements on specific tasks. Figure 4 As shown, 10 consecutive layers of patching attention weights lead to the correct answer P(o true ) increased by 28.6% and 739.2% respectively. The wrong answer P(o edit ) decreased by 54.0% and 89.6% respectively. This shows that preventing attention drift can effectively alleviate specificity failure.

[0098] Based on the analysis of the above experiments, we can conclude that: (1) the attention activation at the last token position significantly contaminates the forward pass of the editing model, leading to specificity failure; (2) the maximum attention drift between attention heads on the edit token position is the error output o edit (3) Fixing attention drift can largely alleviate specificity failure. It can be concluded that our proposed method of introducing regularization terms in the editing process to dynamically constrain the weights of partially selected attention heads and prevent excessive attention to edited entities is reasonable and effective.

[0099] The advantages of the present invention are described below in conjunction with specific experiments.

[0100] Due to the limited availability of datasets that meet the requirements of our task, we combined COUNTERFACT and WikiData counterfact 1683 fact statements of are combined as test data. In addition, we extend the experiments to a wider range of datasets, including QA formats and recent knowledge editing tasks. The specificity failure phenomenon and the performance of SADR remain consistent across these datasets. We evaluate the performance of our method on three mainstream localization-post-editing knowledge editing baselines: ROME, MEMIT, and PMET. Specifically, we focus on knowledge editing where all baselines have one fact association. We implement our SADR method on three editing baselines on GPT-J-6b, Llama3-8b, and GPT-NeoX-20b.

[0101] In addition to the evaluation metrics mentioned above, we also use a fluency score (FL) to evaluate the generative ability of the editing model, and use prompts related to the editing entity, calculated by the weighted average of bigrams and trigrams. Results that are closer to the original model indicate better performance. To further test the generalization-specificity trade-off, we report the harmonic mean of ES, PS, NS, RS, and DNS as the average score (Avg.S).

[0102] Table 3

[0103]

[0104] Specificity failure is widespread in existing knowledge editing methods: As shown in Table 3 above, relational and interference domain tasks show significant drops in all editing methods, even though these methods perform editing at different layers and modules. Specifically, in relational tasks, the accuracy of all editing models drops to less than half of the original performance. This shows that specificity failure is a serious problem that is prevalent in knowledge editing.

[0105] The SADR strategy significantly mitigates specificity failure: In relation and interference neighborhood tasks, the SADR strategy consistently improves the specificity of all editing methods. Notably, in more than half of the settings, our method improves the original specificity metric by more than 50% (green marks). The SADR strategy also stabilizes the performance of neighborhood tasks and improves the fluency of generated text, e.g., the fluency score is significantly improved when using ROME and PMET on GPT-NeoX. SADR also achieves better performance on TinyLlama-1.1b and Llama2-13b compared to the baseline, indicating that our method is effective on models of different scales and knowledge density.

[0106] The SADR strategy has the least impact on the editing success rate: the SADR strategy causes less than 3% performance degradation on the rewriting and generation tasks. It is worth noting that it is very difficult to significantly mitigate specificity failure while fully maintaining the rewriting and generalization performance. This is because the specificity failure highlighted in our paper is often overlooked in many previous model editing evaluation frameworks. As a result, previous methods tend to prioritize generalization and rewriting scores while ignoring the risk of specificity failure. Under such evaluation criteria, high scores can be obtained by simply identifying the subject and greedily predicting the object, even if the relationship between them is completely ignored. We believe that ensuring stable knowledge editing is more important than achieving close to 100% accuracy on generalization. In real-world scenarios, methods that achieve 97% generalization through stable and safe editing are usually more acceptable than those with 100% generalization but significant specificity failures (e.g., severe knowledge errors caused by attention drift after editing the subject).

[0107] In addition, we also conducted many ablation experiments.

[0108] Effect of Suppressing Head Selection: We first explore the effect of selectively suppressing attention heads with significant attention drift versus suppressing all attention heads on ROME with GPT-J by various control weights γ. Editing success is quantified by the average of PS and ES, while specificity is calculated by the average of NS, RS, and DNS.

[0109] like Figure 5 As shown, selectively limiting the attention heads that over-focus on the edit token outperforms limiting all attention heads in terms of editing success and specificity across different γ settings. This suggests that not all attention drift is harmful; rather, it is the over-attention problem that should be addressed compared to the original model.

[0110] Tradeoff between generalization and specificity: The tradeoff between editing success and specificity of knowledge editing methods can be visualized by adjusting the optimization step, learning rate, and hyperparameters such as ω that control intrinsic drift. We analyze the tradeoff of our method by varying γ and compare it with the tradeoffs of other hyperparameters of the ROME method implemented on GPT-J. edit ) changes more than P(o edit )>P(o true ) cases, we use the average of EM and PM to measure editing success, and use RM and DNM to evaluate the relation and interference domain tasks, respectively.

[0111] like Figure 6 As shown, our method shows a better trade-off compared with the tuning of hyperparameters in the original ROME method, indicating that SADR can obtain v * , enabling the model to more effectively distinguish when to output edited knowledge.

[0112] Embodiment 2

[0113] like Figure 7 As shown, the present invention provides a stable knowledge editing system based on attention drift constraint, which is used to implement the stable knowledge editing method based on attention drift constraint of the above embodiment 1, and specifically includes:

[0114] The selective attention drift limitation strategy introduction module is used to introduce the selective attention drift limitation strategy in the knowledge editing process of a large language model. The selective attention drift limitation strategy includes:

[0115] By comparing the model attention output before and after editing, we can locate the attention heads with serious attention drift.

[0116] According to the set constraint criteria, select the attention heads that pay too much attention to the edited object for constraint;

[0117] Align the attention output of the selected attention head with the output before editing to prevent over-editing;

[0118] The objective function construction and combination module is used to construct the objective function of selective attention drift limitation, and combine the objective function of selective attention drift limitation with the original loss function to form a new optimization objective function;

[0119] The model training and optimization module is used to train the model based on the new optimization target objective function to obtain the optimal model parameters and achieve stable knowledge editing.

[0120] In addition, the system also includes a testing and evaluation module, which is used to test the optimized model using test data and use multiple evaluation indicators to evaluate its performance, wherein the evaluation indicators include performance score, performance level, interpretation score, interpretation level, neighborhood score, neighborhood level, relationship score, relationship level, interference neighborhood score, and interference neighborhood level.

[0121] A stable knowledge editing system based on attention drift constraints in this embodiment is used to implement the aforementioned stable knowledge editing method based on attention drift constraints. Therefore, the specific implementation method of the stable knowledge editing system based on attention drift constraints can be found in the implementation example part of the stable knowledge editing method based on attention drift constraints in the previous text. In order to avoid redundancy, it will not be repeated here.

[0122] Embodiment 3

[0123] An embodiment of the present invention provides an electronic device, which includes a processor, a memory and a bus system. The processor and the memory are connected through the bus system. The memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the above-mentioned stable knowledge editing method based on attention drift constraints.

[0124] Embodiment 4

[0125] An embodiment of the present invention provides a computer storage medium, which stores a computer software product. The computer software product includes a number of instructions for enabling a computer device to execute the above-mentioned stable knowledge editing method based on attention drift constraints.

[0126] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0127] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0128] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0129] Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from them are still within the protection scope of the invention.

Claims

1. A stable knowledge editing method based on attention drift constraint, characterized in that: include: In the knowledge editing process of large language models, a selective attention drift limitation strategy is introduced, which includes: By comparing the model attention output before and after editing, we can locate the attention heads with serious attention drift. According to the set constraint criteria, select the attention heads that pay too much attention to the edited object for constraint; Align the attention output of the selected attention head with the output before editing to prevent over-editing; Constructing the objective function of selective attention drift limitation, combining the objective function of selective attention drift limitation with the original loss function to form a new optimization objective function; Based on the new optimization target objective function, the model is trained to obtain the optimal model parameters and achieve stable knowledge editing.

2. The stable knowledge editing method based on attention drift constraint according to claim 1 is characterized in that: The method also includes: testing the optimized model using test data and using multiple evaluation indicators to evaluate its performance, wherein the evaluation indicators include performance score, performance level, interpretation score, interpretation level, neighborhood score, neighborhood level, relationship score, relationship level, interference neighborhood score, and interference neighborhood level.

3. The stable knowledge editing method based on attention drift constraint according to claim 1 is characterized in that: The constraint standard set is: when the attention weight of a certain attention head to the last word of the subject exceeds the maximum attention weight of all attention heads in the original model, the attention head is selected for constraint.

4. The stable knowledge editing method based on attention drift constraint according to claim 1 is characterized in that: The method for constructing the objective function of selective attention drift limitation is: In the formula, represents the objective function for limiting the drift of selective attention; W l，h (S) represents the attention weight of the h-th attention head in layer l when processing prompt S, where l represents the number of layers of the attention mechanism, h represents the index of the attention head in a specific layer l, and S represents the input prompt or sentence; represents the attention weight of the h-th attention head in the l-th layer when the edited model with z processes the prompt S, where Indicates that during the editing process, for the specific model or method selected at level l, :=z indicates that the model or method is edited or modified with a specific parameter or value z; H l (S j ) indicates that in the jth sample S j In the above example, the set of attention heads in the lth layer that meet the specific condition, where the specific condition is that the attention weight of the edited model on the edited object s at the last word is greater than the maximum attention weight of the original model on the object at this layer; S j represents the jth input sample; D KL is the Kullback-Leibler divergence, which is used to measure the difference between two probability distributions; N represents the total number of input samples.

5. The stable knowledge editing method based on attention drift constraint according to claim 4 is characterized in that: The H l (S j ) is expressed as: in M l (S j )=max h W l，h (S j )[-1,s]; Where M l (S j ) represents the maximum attention weight of the last token on the edited object s among all the attention heads in the lth layer of the original model.

6. The stable knowledge editing method based on attention drift constraint according to claim 4 is characterized in that: The method of combining the objective function of selective attention drift limitation with the original loss function to form a new optimization objective function is: In the formula, v * Represents the new optimization objective function; represents the original loss function; γ represents the control weight.

7. A stable knowledge editing system based on attention drift constraint, characterized in that: The system is used to implement the stable knowledge editing method based on attention drift constraint according to any one of claims 1 to 6, specifically comprising: The selective attention drift limitation strategy introduction module is used to introduce the selective attention drift limitation strategy in the knowledge editing process of a large language model. The selective attention drift limitation strategy includes: By comparing the model attention output before and after editing, we can locate the attention heads with serious attention drift. According to the set constraint criteria, select the attention heads that pay too much attention to the edited object for constraint; Align the attention output of the selected attention head with the output before editing to prevent over-editing; The objective function construction and combination module is used to construct the objective function of selective attention drift limitation, and combine the objective function of selective attention drift limitation with the original loss function to form a new optimization objective function; The model training and optimization module is used to train the model based on the new optimization target objective function to obtain the optimal model parameters and achieve stable knowledge editing.

8. The stable knowledge editing system based on attention drift constraint according to claim 7 is characterized in that: The system also includes a testing and evaluation module for testing the optimized model using test data and using multiple evaluation indicators to evaluate its performance, wherein the evaluation indicators include performance score, performance level, interpretation score, interpretation level, neighborhood score, neighborhood level, relationship score, relationship level, interference neighborhood score, and interference neighborhood level.

9. An electronic device, characterized in that: The electronic device includes a processor, a memory and a bus system, the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to implement the stable knowledge editing method based on attention drift constraints as described in any one of claims 1 to 6.

10. A computer storage medium, characterized in that: The computer storage medium stores a computer software product, which includes a number of instructions for enabling a computer device to execute the stable knowledge editing method based on attention drift constraints as described in any one of claims 1 to 6.

Images