A stable knowledge editing method and system based on attention drift constraint

Abstract

The application provides a stable knowledge editing method and system based on attention drift constraint, and relates to the technical field of natural language processing. The method comprises the following steps: introducing a selective attention drift limiting strategy in the knowledge editing process of a large language model; positioning the attention head with serious attention drift phenomenon by comparing the model attention output before and after editing; training the model based on a new optimization target function to obtain optimal model parameters and realize stable knowledge editing. The application significantly improves editing accuracy and reduces the probability of incorrect answers. Experimental verification shows that the method effectively improves the performance of specific tasks and provides theoretical support and practical guidance for knowledge editing of large language models.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a stable knowledge editing method and system based on attention drift constraint. BACKGROUND

[0002] Large language models have shown excellent performance in a wide range of tasks, however, due to the internal defects of the model, many unpredictable errors will inevitably occur, which are caused by negative samples or outdated samples in a large pre-training dataset. The knowledge editing task solves the above problems by precisely editing specific model knowledge by effectively modifying a very small proportion of parameters. Existing knowledge editing methods can be divided into three main paradigms: parameter preservation, localization-post-editing and meta-learning methods. The parameter preservation method explicitly stores the modified knowledge in the memory and uses methods such as classifiers, prompt engineering, or external parameter retrieval knowledge. The localization-post-editing method updates specific parameters by identifying the storage location of the target knowledge and directly editing these locations. The meta-learning method includes training a hypernetwork to edit the parameters of the model.

[0003] Although the knowledge editing has a promising prospect, the existing methods still have major challenges in specificity. Editing specific knowledge fragments will cause a chain reaction within the knowledge graph, affecting the performance of other parts of the model, which is called specificity failure, a typical example is that after editing a certain geographical knowledge (such as a city is the capital of a country) in the model, the model will still make mistakes in other contexts related to the city. In order to limit these side effects, the edited model may be under-edited or over-edited, affecting the model's ability to handle a wide range of tasks. In a recent study, Hoelscher-Obermaier in-depth explored this problem, introduced a dynamic component to propose an improved benchmark CounterFact+, which is used to evaluate the specificity failure problem of the model in knowledge editing, and proposed a KL divergence-based metric to measure specificity. ROME focuses on the problem that the knowledge near the edited entity is damaged, and alleviates the damage of local knowledge by repairing these chain reactions. However, it does not fully consider the impact of the edited entity on a wider context, which may lead to incorrect inferences in other unrelated knowledge areas. RETS, although it proposes a knowledge editing method based on relationship-centric explanation to solve the specificity failure problem, significantly sacrifices the accuracy of editing. Therefore, there is an urgent need for a method that can balance specificity and generalizability when handling knowledge editing tasks. SUMMARY

[0004] To this end, an embodiment of the present application provides a stable knowledge editing method and system based on attention drift constraint, which is used to solve the specificity failure problem in the current large language model (LLM) knowledge editing process in the prior art.

[0005] To solve the above problems, the embodiment of the present application provides a stable knowledge editing method based on attention drift constraint, which comprises the following steps:

[0006] In the knowledge editing process of a large language model, a selective attention drift restriction strategy is introduced, which comprises the following steps:

[0007] By comparing the model attention outputs before and after editing, the attention head with serious attention drift phenomenon is located;

[0008] According to the set constraint standard, the attention head that pays excessive attention to the editing object is selected for constraint;

[0009] The attention output of the selected attention head is aligned with the output before editing to prevent excessive editing;

[0010] A target function of selective attention drift restriction is constructed, and the target function of selective attention drift restriction is combined with the original loss function to form a new optimization target function;

[0011] Based on the new optimization target function, the model is trained to obtain the optimal model parameters, and stable knowledge editing is realized.

[0012] Preferably, the method further comprises testing the optimized model using test data and evaluating its performance using a plurality of evaluation indicators, wherein the evaluation indicators include performance score, performance level, paraphrase score, paraphrase level, neighborhood score, neighborhood level, relationship score, relationship level, interference neighborhood score, and interference neighborhood level.

[0013] Preferably, the set constraint standard is that when the attention weight of a certain attention head to the last word of the subject exceeds the maximum attention weight of all attention heads in the original model, the attention head is selected for constraint.

[0014] Preferably, the method for constructing the target function of selective attention drift restriction is as follows:

[0015]

[0016] In the formula, The target function of selective attention drift restriction is represented by W l,h (S) represents the attention weight of the lth layer hth attention head when processing prompt S, wherein l represents the number of layers of the attention mechanism, h represents the index of the attention head in a specific layer l, and S represents the input prompt or sentence; represents the attention weight of the lth layer hth attention head of the edited model with z when processing prompt S, wherein represents a specific model or method selected for the l-th layer during the editing process: = z represents that the model or method is edited or modified with a specific parameter or value z; H l (S j ) represents the set of attention heads of the l-th layer that meet a specific condition in the j-th sample S j , wherein the specific condition refers to the attention weight of the edited model on the last token on the editing object s being greater than the maximum attention weight of the original model on the object at this layer; S j represents the j-th input sample; D kL is the Kullback-Leibler divergence used to measure the difference between two probability distributions; N represents the total number of input samples.

[0017] Preferably, the H l (S j ) is represented as:

[0018]

[0019] wherein

[0020] M l (S j ) = max h W l,h (S j )[-1,s];

[0021] In the formula, M l (S j ) represents the maximum attention weight of the last token on the editing object s in all attention heads of the l-th layer of the original model.

[0022] Preferably, the method for combining the selective attention drift restriction target function with the original loss function to form a new optimization target function is:

[0023]

[0024] In the formula, v * represents the new optimization target function; represents the original loss function; and γ represents the control weight.

[0025] The embodiment of the present application also provides a stable knowledge editing system based on attention drift constraint, which is used to implement the stable knowledge editing method based on attention drift constraint described above, and specifically comprises:

[0026] The selective attention drift restriction strategy introduction module is used in a knowledge editing process of a large language model, and introduces a selective attention drift restriction strategy, which includes:

[0027] By comparing the model attention output before and after editing, the attention head with serious attention drift phenomenon is located;

[0028] According to the set constraint standard, the attention head that pays excessive attention to the editing object is selected for constraint;

[0029] The attention output of the selected attention head is aligned with the output before editing to prevent excessive editing;

[0030] The target function construction and combination module is used to construct the target function of the selective attention drift restriction, combine the target function of the selective attention drift restriction with the original loss function, and form a new optimization target function;

[0031] The model training and optimization module is used to train the model based on the new optimization target function to obtain optimal model parameters and realize stable knowledge editing.

[0032] Preferably, the system further comprises a test and evaluation module for testing the optimized model using test data and evaluating its performance using a plurality of evaluation indicators, wherein the evaluation indicators include performance score, performance level, paraphrase score, paraphrase level, neighborhood score, neighborhood level, relationship score, relationship level, interference neighborhood score and interference neighborhood level.

[0033] The embodiment of the application also provides an electronic device, which comprises a processor, a memory and a bus system, the processor and the memory are connected through the bus system, the memory is used to store instructions, and the processor is used to execute the instructions stored in the memory to realize the stable knowledge editing method based on attention drift restriction.

[0034] The embodiment of the application also provides a computer storage medium, which stores a computer software product, and the computer software product comprises a plurality of instructions for enabling a computer device to execute the stable knowledge editing method based on attention drift restriction.

[0035] From the above technical solutions, the present application has the following beneficial effects:

[0036] (1) By introducing the selective attention drift restriction (SADR) strategy, the present application optimizes the knowledge storage and retrieval mechanism in the large language model, effectively reduces the specific failure problem in the knowledge editing process, and improves the accuracy and stability of the model.

[0037] (2) The application can dynamically apply constraints to different attention heads as needed, avoiding excessive attention to knowledge-related entities that may lead to over-editing. This dynamic adjustment strategy not only improves editing accuracy, but also makes the method flexible and adaptable to various editing methods without major modifications to the model structure.

[0038] (3) The application comprehensively evaluates the reliability, generalizability and specificity of knowledge editing through multiple experimental indicators, and significantly improves the performance of specific tasks. These experimental results not only verify the effectiveness of the SADR strategy, but also provide theoretical support and practical guidance for related research and practice, with important scientific value and application prospects. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly described below. The features and advantages of the present application can be more clearly understood by referring to the drawings. The drawings are schematic and should not be construed as any limitation on the present application. For those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings. Among them:

[0040] Figure 1 A flowchart of a stable knowledge editing method based on attention drift constraint provided in the embodiments;

[0041] Figure 2 A positioning specificity diagram in the embodiments;

[0042] Figure 3 A diagram showing the positive correlation between attention weight drift and probability of incorrect answers in the embodiments;

[0043] Figure 4 A diagram showing that attention drift is mitigated by repairing in the embodiments;

[0044] Figure 5 A diagram showing that in different gamma settings, attention heads that excessively focus on editing tokens are selectively limited, which is superior to limiting all attention heads in terms of editing success and specificity in the embodiments;

[0045] Figure 6 A diagram showing the trade-off comparison between the present application and other methods in the embodiments;

[0046] Figure 7 A block diagram of a stable knowledge editing system based on attention drift constraint provided in the embodiments. DETAILED DESCRIPTION

[0047] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0048] Embodiment one

[0049] In order to solve the specificity failure problem in the current large language model (LLM) knowledge editing process in the prior art. As shown in Figure 1 , the embodiment of the present application proposes a stable knowledge editing method based on attention drift constraint, which comprises:

[0050] S1: In the knowledge editing process of the large language model, a selective attention drift restriction strategy is introduced;

[0051] S2: By comparing the model attention outputs before and after editing, the attention head with serious attention drift phenomenon is located;

[0052] S3: Based on the new optimization target function, the model is trained to obtain the optimal model parameters, and the stable knowledge editing is realized.

[0053] From the above technical solution, the present application provides a stable knowledge editing method based on attention drift constraint, which locates and restricts the attention head with serious attention drift by comparing the model attention outputs before and after editing, optimizes the internal knowledge storage of the model, dynamically adjusts the constraint strategy to adapt to various editing methods, significantly improves the editing accuracy and reduces the probability of incorrect answers. Experimental verification shows that this method effectively improves the specificity task performance, and provides theoretical support and practical guidance for the knowledge editing of large language models.

[0054] The technical solution of the present application is based on the existing knowledge editing method, aiming to optimize the internal knowledge storage and retrieval of the model. Previous studies represent the fact association as a knowledge tuple t=(s,r,o), where s is the subject, r is the relationship, and o is the object. The goal of knowledge editing is to replace the original fact association stored in the model parameters with a new fact association (s,r,o edit ), where o edit is the counterfactual target object (for example, replacing "Paris" with "New York"), and o true represents the real answer in the real world.

[0055] The core idea of the ROME method is to modify the "values" (v) of the second layer of the MLP so that when the "keys" (k) are associated with the target subject, the target knowledge can be correctly predicted. When the "keys" are not associated, the original knowledge of the model is not affected, ensuring the stability of the model in unrelated contexts. The loss function is as follows:

[0056]

[0057] However, this goal can cause attention drift, leading to specificity failure. To enhance the specificity of knowledge editing when optimizing the ideal vector v * , the present application introduces a selective attention drift restriction (SADR) strategy, which is a regularization term based on the above formula. It is worth noting that the SADR strategy dynamically imposes constraints on different attention heads as needed, because the Transformer model contains multiple knowledge-specific attention heads that capture different factual associations. In addition, the SADR strategy is a simple and efficient method that can be flexibly adapted to various editing methods.

[0058] More specifically, since excessive attention to the editing object by some attention heads is closely related to specificity failure, the SADR strategy is applied to those attention heads that pay excessive attention to the editing object on the last word. The SADR strategy includes: locating the attention heads that exhibit severe attention drift by comparing the model's attention output before and after editing; selecting attention heads that pay excessive attention to the editing object according to the set constraint criteria (for example: when the attention weight of a certain attention head on the last word of the subject exceeds the maximum attention weight of all attention heads in the original model, the attention head is selected for constraint.); Aligning the attention output of the selected attention heads with the output before editing to prevent excessive editing.

[0059] In step S2, the objective function of the selective attention drift restriction is constructed, and the objective function of the selective attention drift restriction is combined with the original loss function to form a new optimization objective function.

[0060] Further, the method for constructing the objective function of the selective attention drift restriction is:

[0061]

[0062] In the formula, the objective function of the selective attention drift restriction; W l,h (S) represents the attention weight of the lth layer and the hth attention head when processing the prompt S, where l represents the number of layers of the attention mechanism, h represents the index of the attention head in a specific layer l, and S represents the input prompt or sentence; denotes the attention weight of the h-th attention head of the l-th layer of the model with z when processing the prompt S denotes the specific model or method selected for the l-th layer during the editing process l (S j ) denotes the set of attention heads of the l-th layer that meet the specific condition in the j-th sample S j , where the specific condition refers to the edited model having a greater attention weight on the last token of the editing object s than the maximum attention weight of the original model on the object at that layer j denotes the j-th input sample; D KL is the Kullback-Leibler divergence, used to measure the difference between two probability distributions; N represents the total number of input samples.

[0063] The above H l (S j ) is represented as:

[0064]

[0065] where

[0066] M l (S j ) = max h W l,h (S j )[-1,s];

[0067] In the formula, M l (S j ) denotes the maximum attention weight of the last token on the editing object s among all attention heads of the l-th layer of the original model.

[0068] The new optimization objective function is:

[0069]

[0070] In the formula, v * denotes the new optimization objective function (ideal vector v * ); denotes the original loss function (the loss function of the ROME method); γ represents the control weight.

[0071] In step S3, the model is trained based on the new optimization objective function to obtain the optimal model parameters, achieving stable knowledge editing.

[0072] In addition, in order to demonstrate the rationality of the stable knowledge editing method based on attention drift constraint proposed in the present application, we carried out a series of experiments to explore the root cause of specificity failure.

[0073] The existing work mainly evaluates the reliability, generalization and specificity of knowledge editing through the following evaluation indexes:

[0074] Efficacy Score (ES) and Efficacy Magnitude (EM) represent p(o edit |s,r)>P(O true |s,r) and the proportion of the average value of p(o edit |s,r) respectively.

[0075] Paraphrase Score (PS) and Paraphrase Magnitude (PM) evaluate the generalization performance of the model under paraphrase prompt (s para ,r para ) (for example, “Eiffel Tower is located in”). These two indexes are represented as p(o edit |s para ,r para )>P(o true |s para ,r para ) and P(o edit |s para ,r para ) respectively.

[0076] Neighborhood Score (NS) and Neighborhood Magnitude (NM) evaluate specificity by providing a neighboring but different subject s′ (for example, “Louvre Museum”). These two indexes are represented as P(o true |s′,r)>P(o edit |s′,r) and P(o true |s′,r) respectively.

[0077] Recent studies have shown that the presence of the edited subject in the reasoning context will weaken the specificity performance, therefore, the present application introduces two additional specificity indexes, which contain the edited subject in the test prompt.

[0078] Relation Score (RS) and Relation Magnitude (RM) are used to evaluate how the model handles the attributes of the edited subject that are irrelevant to the editing, defined as relation r′. From experience, the edited model tends to predict oedit (e.g., "The color of the Eiffel Tower is New York"). Thus we compute the relation score by P(o true |s,r′)>P(o edit |s,r′) and the relation magnitude by P(o true |s,r′).

[0079] Distract Neighborhood Score (DNS) and Distract Neighborhood Magnitude (DNM) are similar to NS and NM, but concatenate the edited sentence (s, r, o edit ) in the test prompt of the distract neighborhood task. These metrics are denoted as and

[0080] 1. Calibration of specificity failure on counterfactual benchmark: To measure the performance of the relation and distract neighborhood tasks, we use the dataset composed of COUNTERFACT and WikiData counterfact . This dataset contains 1683 factual statements.

[0081] The following Table 1 illustrates the significant specificity failure when the edited subject appears in the context, the edited model incorrectly outputs the edited object in more than 50% of the test cases. Moreover, the average probability of the incorrect answer o edit is much higher than the correct answer o true , 48.4% vs. 3.3% in the relation task, and 24.9% vs. 10.5% in the distract neighborhood task.

[0082] Table 1

[0083]

[0084] 2. Localization of specificity failure: In the forward pass of autoregressive language models, the flow of information can be seen as a causal graph. When a model with L layers makes a prediction based on a prompt containing T tokens, each module (e.g., attention module, MLP, and transformer block) produces a T x L output. Each of these outputs is influenced by the previous outputs of the preceding layer and the preceding token position. Inspired by causal tracing, the present invention traces different states in the causal graph to determine which parts pollute the flow of information in the specificity failure.

[0085] The edited model is first passed forward using the test prompts, and the outputs from various network modules at different layers and token positions are recorded. Then, a forward pass is performed using the original model, copying the representation of a particular module from the stored output to the corresponding position without altering other computations. We traverse the modules at each layer l and token position t within a window of k layers. We refer to this approach as “contamination substitution” and quantify its effect on the final output probability, formulated as:

[0086]

[0087] where, represents replacing the module result at layer l and token position t with z.

[0088] As Figure 2 shown, the light-colored regions represent the primary states that lead to incorrect answers. We observe that substituting the six-layer MLP activations or attention activations can reduce the probability of the correct answer by 4.59% and 3.74%, respectively, while the total reduction rate caused by the edited model is 5.26%. When we edit the MLP modules at the 5th layer, we expect that the contamination substitution of MLP activations near the edited layer will significantly affect the final prediction. However, modifying the attention activations in the upper layers also has a similar impact on the correct output, indicating that the primary cause of the specificity failure is that the attention modules incorrectly process the information on the last token due to the edit, and these findings are consistent on the relation task, which further justifies our rationality for constraining attention.

[0089] 3. Identifying attention drift as a trigger for specificity failure: As mentioned above, attention activations are one of the primary causes of specificity failure. Previous studies have shown that the attention modules in the upper-middle layers extract factual attributes during prediction. This suggests that the attention modules may incorrectly focus on the edited information, thereby ignoring other information when predicting the final token. Therefore, we use the Pearson coefficient to quantify the relationship between attention weight drift and interference domain and relation task failure. Given that the edited model overestimates the probability of the edited object o edit relative to the true object o true , we analyze the correlation between and P(o edit ), where W l,h and represent the attention weights of the last token on the precedent tokens at layer l and attention head h before and after editing, respectively.

[0090] Figure 3 shows the attention weight drift and the incorrect answer o editThe positive correlation between the probabilities suggests that misfocusing on prior information is a key factor in specificity failure. Further analysis of attention drift from the perspective of the tokens being focused on and the attention head naturally raises the following two questions:

[0091] (1) The last token incorrectly allocates attention to the previous tokens, resulting in a failure of specificity. Of these previous tokens, which one would have had a significant impact if it were incorrectly focused on?

[0092] (2) Which has a greater impact on prediction: excessive local attention drift of a specific attention head or cumulative attention drift of all heads?

[0093] To address these issues, we calculated various factors and P(o) edit The Pearson coefficients between the attention heads are shown in Table 2 below. Table 2 indicates that the drift Pearson coefficient for the last token of the edited object s is higher compared to other tokens (Equation 1 vs. Equation 2). Furthermore, the maximum drift of attention weights among attention heads has a greater impact on the final result than the cumulative drift of all attention heads (Equation 1 vs. Equation 2), suggesting that excessive attention to the last token by some attention heads is the primary cause of specificity failures. Therefore, we only constrain attention heads whose attention weight for the last token of the subject exceeds the maximum attention weight of all heads in the original model.

[0094] Table 2

[0095]

[0096] 4. Mitigating Specificity Failure by Plugging Attention Shift: To further verify the significant impact of attention shift on specificity failure, we quantified the change in predicted probabilities after plucking attention weights at each layer. We first performed a forward pass using the original model with specific task cues and stored the intermediate attention weights. Then, we tested the edited model on the same cues, replacing its attention weights with the previously stored weights.

[0097] We found that modifying the attention weights in the upper-middle layers can lead to significant improvements in specific tasks. (As above) Figure 4 As shown, 10 consecutive layers of repaired attention weights lead to a decrease in the correct answer P(o) for two specific tasks. true The probabilities of correct answers (P(o)) increased by 28.6% and 739.2% respectively, while the probabilities of incorrect answers (P(o)) increased by 739.2%. edit The probabilities of these events decreased by 54.0% and 89.6%, respectively. This indicates that preventing attention drift can effectively alleviate specific failures.

[0098] Based on the analysis of the above experiments, we can draw the following conclusions: (1) the attention activation of the last token position significantly contaminates the forward pass of the edit model, leading to specificity failure; (2) the maximum attention drift of the edit token position between attention heads is the main trigger of the error output o edit ; (3) fixing the attention drift can largely alleviate the specificity failure. Therefore, we can conclude that our proposed method of introducing a regularization term in the editing process to dynamically constrain the weight of the partially selected attention head and prevent excessive attention to the edited entity is reasonable and effective.

[0099] The advantages of the present application will be illustrated below in combination with specific experiments.

[0100] Due to the limited availability of data sets that meet our task requirements, we combined the 1683 fact statements of COUNTERFACT and WikiData counterfact as test data. In addition, we extended the experiment to a wider data set, including QA format and recent knowledge editing tasks. The specificity failure phenomenon and the performance of SADR remain consistent in these data sets. We evaluated the performance of our method on three mainstream position-post-editing knowledge editing baselines: ROME, MEMIT and PMET. Specifically, we focused on knowledge editing with a fact relation for all baselines. We implemented our SADR method on three editing baselines on GPT-J-6b, Llama3-8b and GPT-NeoX-20b.

[0101] In addition to the evaluation indicators mentioned in the foregoing, the present application also uses fluency score (FL) to evaluate the generation ability of the editing model, and uses the prompt related to the edited entity to calculate the weighted average of binary syntax and ternary syntax. The results closer to the original model indicate better performance. In order to further test the generalization-specificity trade-off, we report the harmonic mean of ES, PS, NS, RS and DNS as the average score (Avg.S).

[0102] Table 3

[0103]

[0104] Among existing knowledge editing methods, the specificity failure phenomenon is very common: as shown in Table 3 above, the relationship and interference domain tasks show significant decline in all editing methods, even if these methods perform editing on different layers and modules. Specifically, in the relationship task, the precision of all editing models drops to less than half of the original performance. This shows that specificity failure is a serious problem that is prevalent in knowledge editing.

[0105] SADR strategy significantly mitigates specificity failure: In relation and interference neighborhood tasks, SADR strategy consistently improves the specificity of all editing methods. Notably, in more than half of the settings, our methods improve the original specificity metric by more than 50% (marked in green). SADR strategy also stabilizes the performance of neighborhood tasks and improves the fluency of generated text, for example, when using ROME and PMET on GPT-NeoX, the fluency scores are significantly improved. SADR also achieves better performance on TinyLlama-1.1b and Llama2-13b compared to the baseline, which shows that our method is effective on models of different scales and knowledge densities.

[0106] SADR strategy has minimal impact on editing success rate: SADR strategy results in less than 3% performance degradation in rewrite and generation tasks. Notably, it is very difficult to significantly mitigate specificity failure while completely maintaining rewrite and generalization performance. This is because in many previous model editing evaluation frameworks, the specificity failure highlighted in our paper is often overlooked, so previous methods tend to prioritize generalization and rewrite scores, while ignoring the risk of specificity failure. Under such evaluation criteria, even if we completely ignore the relationship between them, we can still get a high score by simply identifying the subject and greedily predicting the object. We believe that ensuring stable knowledge editing is more important than achieving near 100% accuracy in generalization. In real-world scenarios, methods that achieve 97% generalization through stable and safe editing are often more acceptable than those that have 100% generalization but have significant specificity failure (e.g., severe knowledge errors caused by attention drift after editing the topic).

[0107] In addition, we also conducted a number of ablation experiments.

[0108] Effect of suppressing head selection: We first explored the effect of selectively suppressing attention heads with significant attention drift and GPT-J on ROME by various control weights γ. Editing success is quantified by the average of PS and ES, while specificity is calculated by the average of NS, RS and DNS.

[0109] As Figure 5 shown, selectively limiting attention heads that pay excessive attention to edited tokens outperforms limiting all attention heads in editing success and specificity in different γ settings. This shows that not all attention drift is harmful; on the contrary, the problem to be addressed compared to the original model is excessive attention.

[0110] Trade-off between generalization and specificity: The trade-off between editing success and specificity made by the knowledge editing method can be visualized by adjusting the hyperparameters that control the drift in essence, such as optimization steps, learning rate, and ω. We analyze the trade-off of our method by changing γ and compare it with the trade-off of other hyperparameters of the ROME method implemented on GPT-J. Since the change of P(o edit ) is more obvious than the proportion of P(o edit )>P(o true ), we use the average of EM and PM to measure the editing success, and use RM and DNM to evaluate the relationship and interference domain tasks, respectively.

[0111] As shown in Figure 6 , compared with the adjustment of hyperparameters in the original ROME method, our method shows better trade-off, indicating that SADR can obtain v * , enabling the model to more effectively distinguish when to output edited knowledge.

[0112] Embodiment two

[0113] As shown in Figure 7 , the present application provides a stable knowledge editing system based on attention drift constraint, which is used to implement the stable knowledge editing method based on attention drift constraint of the above-mentioned embodiment one, and specifically comprises:

[0114] The selective attention drift restriction strategy introduction module is used in the knowledge editing process of a large language model, and introduces a selective attention drift restriction strategy, wherein the selective attention drift restriction strategy comprises:

[0115] By comparing the model attention output before and after editing, the attention head with serious attention drift phenomenon is located;

[0116] According to the set constraint standard, the attention head that pays excessive attention to the editing object is selected for constraint;

[0117] The attention output of the selected attention head is aligned with the output before editing to prevent excessive editing;

[0118] The target function construction and combination module is used to construct the target function of the selective attention drift restriction, combine the target function of the selective attention drift restriction with the original loss function, and form a new optimization target function;

[0119] The model training and optimization module is used to train the model based on the new optimization target function, obtain the optimal model parameters, and realize stable knowledge editing.

[0120] In addition, the system further comprises a test and evaluation module configured to test the optimized model by using test data and evaluate the performance of the optimized model by using a plurality of evaluation indexes, wherein the evaluation indexes comprise an efficiency score, an efficiency level, an interpretation score, an interpretation level, a neighborhood score, a neighborhood level, a relationship score, a relationship level, an interference neighborhood score, and an interference neighborhood level.

[0121] The embodiment of the stable knowledge editing system based on attention drift constraint is used to implement the stable knowledge editing method based on attention drift constraint, and therefore the specific embodiments of the stable knowledge editing system based on attention drift constraint can be seen from the foregoing embodiment part of the stable knowledge editing method based on attention drift constraint. In order to avoid redundancy, details are not described herein again.

[0122] Embodiment three

[0123] The embodiment of the present application provides an electronic device, which comprises a processor, a memory and a bus system, the processor and the memory are connected through the bus system, the memory is used for storing instructions, and the processor is used for executing the instructions stored in the memory to realize the stable knowledge editing method based on attention drift constraint.

[0124] Embodiment four

[0125] The embodiment of the present application provides a computer storage medium, which stores a computer software product, the computer software product comprises a plurality of instructions, and is used for enabling a computer device to execute the stable knowledge editing method based on attention drift constraint.

[0126] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can adopt a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can adopt a computer program product in the form of a computer program product embodied on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0127] The present application is described with reference to flowcharts and / or block diagrams according to the method, device (system) and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device for implementing the functions described in the flowcharts and / or block diagrams. Figure 1apparatuses that implement the functions specified in the flowchart or flowchart Figure 1

[0128] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart or flowchart Figure 1 Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart or flowchart Figure 1 Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart or flowchart

[0129] Obviously, the above-described embodiments are merely examples and are not intended to limit the embodiments. Based on the above description, one of ordinary skill in the art can make other different forms of changes or modifications. Here, it is not necessary and impossible to enumerate all the embodiments. The obvious changes or modifications derived therefrom are still within the protection scope of the present application.​​​

Claims

1. A stable knowledge editing method based on attention drift constraint, characterized in that, The method comprises the following steps: In the knowledge editing process of a large language model, a selective attention drift limiting strategy is introduced, the selective attention drift limiting strategy comprising: By comparing the model attention outputs before and after editing, the attention heads with serious attention drift phenomena are located; According to the set constraint standard, the attention heads that pay excessive attention to the editing object are selected for constraint; The attention outputs of the selected attention heads are aligned with the outputs before editing to prevent excessive editing; A target function of selective attention drift limiting is constructed, the target function of selective attention drift limiting is combined with the original loss function to form a new optimization target function; wherein the method for constructing the target function of selective attention drift limiting is: ； wherein, represents the target function of selective attention drift restriction; represents the prompt when processing the prompt , the attention weight of the th attention head in the th layer, wherein represents the number of layers of the attention mechanism, represents the index of the attention head in the th layer, represents the input prompt or sentence; represents the edited and model when processing the prompt , the attention weight of the th attention head in the th layer, wherein represents the specific model or method selected for the th layer during the editing process, represents that the model or method is edited or modified with a specific parameter or value z; represents the set of attention heads in the th layer that meet the specific condition in the th sample, the specific condition being that the attention weight of the edited model on the last token on the edited object is greater than the maximum attention weight of the original model on the object at this layer; represents the th input sample; represents the total number of input samples; The method for combining the target function of selective attention drift limiting with the original loss function to form a new optimization target function is: ； wherein denotes the new optimization objective function; denotes the original loss function; denotes the control weight; Based on the new optimization target function, the model is trained to obtain optimal model parameters, and stable knowledge editing is realized.

2. The stable knowledge editing method based on attention drift constraint according to claim 1, wherein, The method further comprises: testing the optimized model by using test data, and evaluating the performance thereof by using multiple evaluation indexes, wherein the evaluation indexes comprise performance score, performance level, paraphrase score, paraphrase level, neighborhood score, neighborhood level, relationship score, relationship level, interference neighborhood score, and interference neighborhood level.

3. The stable knowledge editing method based on attention drift constraint according to claim 1, wherein, The constraint standard is that when the attention weight of a certain attention head to the last word element of the subject exceeds the maximum attention weight of all attention heads in the original model, the attention head is selected for constraint.

4. The method of claim 1, wherein, The is represented as: ； The system is used to realize the stable knowledge editing method based on attention drift constraint according to any one of claims 1 to 4, and specifically comprises: ； In the formula, indicates the maximum attention weight of the last token pair in all attention heads of the original model layer to the editing object. layer to the editing object.

5. A stable knowledge editing system based on attention drift constraints, characterized in that, A selective attention drift limiting strategy introduction module is used to introduce a selective attention drift limiting strategy in the knowledge editing process of a large language model, the selective attention drift limiting strategy comprising: By comparing the model attention outputs before and after editing, the attention heads with serious attention drift phenomena are located; According to the set constraint standard, the attention heads that pay excessive attention to the editing object are selected for constraint; The attention outputs of the selected attention heads are aligned with the outputs before editing to prevent excessive editing; A target function construction and combination module is used to construct a target function of selective attention drift limiting, and combine the target function of selective attention drift limiting with the original loss function to form a new optimization target function; A model training and optimization module is used to train the model based on the new optimization target function to obtain optimal model parameters, and realize stable knowledge editing. The system further comprises a test and evaluation module, which is used to test the optimized model by using test data, and evaluate the performance thereof by using multiple evaluation indexes, wherein the evaluation indexes comprise performance score, performance level, paraphrase score, paraphrase level, neighborhood score, neighborhood level, relationship score, relationship level, interference neighborhood score, and interference neighborhood level.

6. The stable knowledge editing system based on attention drift constraint of claim 5, wherein, ​ 7. An electronic device, comprising: The electronic device comprises a processor, a memory and a bus system, the processor and the memory are connected through the bus system, the memory is used for storing instructions, and the processor is used for executing the instructions stored in the memory to realize the stable knowledge editing method based on attention drift constraint in any one of claims 1 to 4.

8. A computer storage medium, characterized in that The computer storage medium stores a computer software product, and the computer software product comprises a plurality of instructions for enabling a computer device to execute the stable knowledge editing method based on attention drift constraint in any one of claims 1 to 4.

Images