Conflict attention head pruning based knowledge conflict elimination method and device

Abstract

The application provides a knowledge conflict elimination method and device based on conflict attention head pruning, wherein the method comprises the following steps: obtaining an original input and an interference input; running a target language model based on the original input to determine the original activation state of each attention head; running the target language model based on the interference input to determine the interference activation state of each attention head; replacing the original activation state of each attention head with the interference activation state respectively to determine the output change of each attention head; then determining the proxy importance score of each attention head; sorting based on the proxy importance score of each attention head, taking the attention head with the target serial number as the conflict attention head, wherein the conflict attention head is a context head; and performing attention pruning on the conflict attention head to eliminate the knowledge conflict of the target language model; and the application can improve the flexibility of the model in processing internal memory and external context.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a knowledge conflict elimination method and device based on conflicting attention head pruning. Background Art

[0002] With the rapid development of natural language processing (NLP) technology, language models have demonstrated tremendous potential in handling complex tasks. During the pre-training phase, language models are trained on large amounts of text data, accumulating and storing a wealth of factual knowledge. However, the internal memory capacity of language models is limited and cannot capture all possible facts and knowledge.

[0003] To compensate for the shortcomings of internal memory, recent research has proposed the use of retrieval augmentation and tool augmentation methods. These methods expand the knowledge boundaries of the model by providing external context (i.e., non-parametric knowledge). However, while this external context can help the model provide more accurate information in certain situations, it also introduces new problems: conflicts between internal memory and external context can occur, resulting in the model being unable to correctly generate the expected output when faced with conflicting information.

[0004] This shows that the language models in related technologies cannot correctly generate expected outputs when faced with contradictory information. Summary of the Invention

[0005] The present invention provides a knowledge conflict elimination method and device based on conflicting attention head pruning, which is used to solve the defect in the prior art that language models cannot correctly generate expected outputs when faced with contradictory information, and to improve the flexibility of the model in processing internal memory and external context.

[0006] The present invention provides a knowledge conflict elimination method based on conflicting attention head pruning, comprising the following steps: obtaining original input and interference input, wherein a first output result of a target language model based on the original input and a second output result based on the interference input have a knowledge conflict; running the target language model based on the original input, and determining the original activation state of each attention head among all attention heads in the target language model; running the target language model based on the interference input, and determining the interference activation state of each attention head among all attention heads in the target language model; replacing the original activation state of each attention head with the interference activation state, and determining the output change of each attention head; determining the proxy importance score of each attention head based on the output change of each attention head; sorting based on the proxy importance score of each attention head, and taking the attention head with the target sequence number as the conflicting attention head, wherein the conflicting attention head is a context head; performing attention pruning on the conflicting attention heads to eliminate the knowledge conflict of the target language model.

[0007] According to a knowledge conflict elimination method based on conflicting attention head pruning provided by the present invention, before obtaining the original input and the interference input, the method also includes: knocking out the activation values ​​of the multi-head attention module and the feedforward network module of each layer in the target language model, and determining the performance of the multi-head attention module and the feedforward network model respectively.

[0008] According to a knowledge conflict elimination method based on conflict attention head pruning provided by the present invention, the performance of the feedforward network module includes: constructing a semantic representation of the input sequence; the performance of the multi-head attention module includes: passing the processed information to the last text unit.

[0009] According to a knowledge conflict elimination method based on conflicting attention head pruning provided by the present invention, after respectively knocking out the activation values ​​of the multi-head attention module and the feedforward network module of each layer in the target language model and respectively determining the performance of the multi-head attention module and the feedforward network model, the method further includes: determining the time point when the multi-head attention module transmits the processed information to the last text unit and the position of the last text unit.

[0010] According to a knowledge conflict elimination method based on conflicting attention head pruning provided by the present invention, the original activation state of each attention head is replaced with an interference activation state, and the output change of each attention head is determined, including: running the target language model based on the original input to obtain the original output result output by the target language model; when the original activation state of each attention head is replaced with the interference activation state one by one, running the target language model based on the original input to obtain the interference output result corresponding to each attention head output by the target language model; based on the interference output result corresponding to each attention head, comparing the interference output result with the original output result, and determining the output change of each attention head.

[0011] According to a knowledge conflict elimination method based on conflicting attention head pruning provided by the present invention, attention pruning is performed on the conflicting attention heads to eliminate the knowledge conflicts of the target language model, including: setting the weights of the conflicting attention heads to predetermined values ​​to ignore the influence of the conflicting attention heads in the target language model.

[0012] The present invention also provides a knowledge conflict elimination device based on conflicting attention head pruning, comprising the following modules: an acquisition module for acquiring original input and interference input, wherein a first output result of a target language model based on the original input and a second output result based on the interference input have knowledge conflicts; an original activation module for running the target language model based on the original input and determining the original activation state of each attention head among all attention heads in the target language model; an interference activation module for running the target language model based on the interference input and determining the interference activation state of each attention head among all attention heads in the target language model; a replacement module for replacing the original activation state of each attention head with the interference activation state and determining the output change of each attention head; a determination module for determining the proxy importance score of each attention head based on the output change of each attention head; a sorting module for sorting based on the proxy importance score of each attention head and taking the attention head with the target sequence number as the conflicting attention head, wherein the conflicting attention head is a context head; and a pruning module for performing attention pruning on the conflicting attention heads to eliminate the knowledge conflicts of the target language model.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the method for eliminating knowledge conflicts based on conflicting attention head pruning as described above is implemented.

[0014] The application further provides a non-transitory computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the knowledge conflict elimination method based on conflict attention head pruning according to any one of the above.

[0015] The application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the knowledge conflict elimination method based on conflict attention head pruning according to any one of the above.

[0016] The application provides the knowledge conflict elimination method and device based on conflict attention head pruning, which records original activation states and interference activation states of all attention heads by respectively running a target language model under original input and interference input, then replaces the activation states of specific attention heads with the activation states of the interference input, observes the change of the output, and calculates the output change amount under the original input and the interference input for each attention head. By comparing the output changes under the two conditions, the proxy importance scores of each attention head can be obtained; the attention heads (context heads) with target serial numbers are sorted as conflict attention heads according to the proxy importance scores, and the conflict attention heads are pruned, so that the context heads which have a negative impact on memory attribute prediction are pruned, thereby reducing the knowledge conflict, and eliminating the knowledge conflict of the target language model; and the defects that the language model in the prior art cannot correctly generate expected output when facing mutually contradictory information are solved. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description one by one. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is a flowchart of the knowledge conflict elimination method based on conflict attention head pruning provided by the application.

[0019] Figure 2 is a process diagram of the knowledge conflict in the language model provided by the application.

[0020] Figure 3 is an architecture diagram of the knowledge conflict elimination method based on conflict attention head pruning provided by the application.

[0021] Figure 4 is a result diagram of the activation value knockout of the FFN component and the MHA component provided by the application.

[0022] Figure 5The present invention provides an internal memory prediction map and an external environment prediction map based on different input elements.

[0023] Figure 6 It is a result diagram of the attention head extraction rate of the memory head and the context head provided by the present invention.

[0024] Figure 7 It is a schematic diagram comparing the effects of the existing method and the present method provided by the present invention.

[0025] Figure 8 It is a schematic diagram of the applicability results of the PH3 method provided by the present invention in different models.

[0026] Figure 9 It is a structural diagram of the knowledge conflict elimination device based on conflict attention head pruning provided by the present invention.

[0027] Figure 10 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0028] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0029] With the rapid development of natural language processing technology, language models have demonstrated tremendous potential in handling complex tasks. During the pre-training phase, language models are trained on large amounts of text data, accumulating and storing a wealth of factual knowledge. This knowledge is primarily stored in the model in the form of parameters, known as internal memory. However, internal memory has several major issues:

[0030] Knowledge update lag: The knowledge in internal memory is fixed during the pre-training phase and cannot be updated dynamically. When the world changes or new information emerges, the model cannot update its knowledge base in a timely manner.

[0031] Limited memory capacity: Although the model is trained with a large amount of data, its internal memory capacity is limited and cannot cover all possible facts and knowledge.

[0032] Factual errors: Due to noise or errors in the training data, the model may remember some inaccurate facts, resulting in errors when generating answers.

[0033] To compensate for the shortcomings of internal memory, recent research has proposed the use of retrieval augmentation and tool augmentation methods. These methods expand the knowledge boundaries of the model by providing external context (i.e., non-parametric knowledge), enabling the model to reference the latest and relevant external information when generating answers. External context can include:

[0034] Real-time information retrieval: By retrieving the latest information and data in real time, the model can refer to it when generating answers.

[0035] Tools and databases: Use external tools and databases to provide additional knowledge support, such as knowledge graphs, search engines, and specialized databases.

[0036] However, while this external context can help the model provide more accurate information in some cases, it also introduces a new problem: conflicts may arise between internal memory and external context. This knowledge conflict can cause the model to fail to generate output correctly when faced with conflicting information, severely impacting the model's reliability and performance.

[0037] The fundamental problem with knowledge conflict is that current research has largely focused on understanding the behavior and preferences of models in conflicting situations, attempting to determine whether models tend to rely more on internal memory or external context when faced with knowledge conflicts. However, academic understanding of the specific mechanisms of knowledge conflict and how to effectively mitigate these conflicts is still very limited. The embodiments of the present invention explain the mechanisms of knowledge conflict from the perspective of information flow and implement precise interventions at key points to mitigate conflicts.

[0038] In embodiments of the present invention, it was discovered that in the later layers of a language model, some attention heads have opposing effects: memory heads can recall knowledge from internal memory, while context heads can retrieve knowledge from external context. When these attention heads process information with inconsistent information flows, this can lead to knowledge conflicts. Specifically, the key to knowledge conflicts lies in the difficulty these attention heads encounter when integrating inconsistent information flows.

[0039] Based on the above findings, this paper proposes a new method called Pruning Head via PatH PatcHing (PH3), which can efficiently alleviate knowledge conflicts by pruning conflicting attention heads without updating model parameters.

[0040] The PH3 method has the following main advantages: Flexible control of the model's use of internal memory or external context: PH3 can significantly improve the model's ability to use internal memory (↑44.0%) or external context (↑38.5%), thereby flexibly adjusting the model's behavior according to task requirements. Improve the model's performance in open-domain question-answering tasks: By reducing knowledge conflicts, PH3 can help the model more accurately generate answers based on retrieved external context, thereby improving the performance of question-answering tasks. Wide applicability: The embodiments of the present invention conducted a large number of experiments, demonstrating the wide applicability of the PH3 method across different models, different relationship types, and different data formats.

[0041] In order to overcome the defects of the existing technology, the present invention designs a knowledge conflict elimination method based on conflicting attention heads pruning to address the knowledge conflict problem in the retrieval enhancement model. This method uses path patching technology to identify the key memory heads and context heads in the multi-head attention module, and prunes the context heads that have a negative impact on the memory attribute prediction based on the calculated importance scores, thereby reducing knowledge conflicts.

[0042] Optionally, the knowledge conflict elimination method based on conflicting attention head pruning in the embodiment of the present application can be executed by a server, or by a terminal device, or jointly by a server and a terminal device, taking the example of the knowledge conflict elimination method based on conflicting attention head pruning in this embodiment executed by a server.

[0043] refer to Figure 1 , Figure 1 : is a flow chart of the knowledge conflict elimination method based on conflict attention head pruning provided by the present invention, such as Figure 1 As shown, the method includes the following steps.

[0044] Step 101: obtaining an original input and an interference input, wherein a first output result of a target language model based on the original input and a second output result based on the interference input have knowledge conflicts.

[0045] In the embodiment of the present invention, the original input may be internal knowledge in the target language model, while the interference input is external knowledge obtained outside the target language model. Meanwhile, the internal knowledge is contradictory to the external knowledge.

[0046] refer to Figure 2 , Figure 21 is a schematic diagram of the process of knowledge conflict in the language model provided by the present invention, which includes input (Context: The capital of France is Rome, Question: What is the capital of France), language model, external knowledge (Paris) and internal knowledge (Rome).

[0047] For example, when the input question is "What is the capital of France", the expected output of the target language model based on internal knowledge (original input "The capital of France is Paris") is "Paris", while the expected output of the target language model based on external knowledge (interference input "The capital of France is Rome") is "Rome", that is, there is a knowledge conflict between the first output result of the target language model based on the original input and the second output result based on the interference input.

[0048] It should be noted that the original input is internal knowledge pre-set in the target language model (for example, pre-stored factual knowledge, etc.), and the interference output can be external knowledge obtained by real-time retrieval by the target language model based on the input question, or it can be external knowledge directly input by the user.

[0049] The target language model is an abstract mathematical model of language based on objective linguistic facts. It is a deep learning model trained using large amounts of text data and can generate natural language text or understand the meaning of text. The target language model can handle a variety of natural language tasks, such as text classification, question answering, and conversation. For example, the target language model can be a large language model (LLM).

[0050] Step 102: Run the target language model based on the original input to determine the original activation state of each attention head in all attention heads in the target language model.

[0051] Step 103: Run the target language model based on the interference input to determine the interference activation state of each attention head in all attention heads in the target language model.

[0052] In processing natural language processing tasks, especially when using Transformer-based models (such as BERT, GPT, etc.), the attention mechanism is one of the core parts of the model. In the Transformer model, the attention mechanism works through multiple attention heads in parallel to capture different aspects of the input data. Each attention head generates an attention weight matrix, which represents the correlation or importance between different positions in the sequence. In the embodiments of the present application, the attention weight matrix of each attention head can be used as the activation state.

[0053] In the embodiments of the present application, the model is run under the original input and the interference input (replacing the key information in the external context with special symbols), and the activation states of all attention heads are recorded. By comparing the activation states under the two input conditions, it can be preliminarily determined which attention heads have significant changes when processing different information streams.

[0054] Step 104, respectively replacing the original activation state of each attention head with the interference activation state, and determining the output change amount of each attention head.

[0055] In the embodiments of the present application, the target language model is run under the original input, but the activation state of a specific attention head is replaced with the activation state of the interference input, and the change in the output is observed. Through this method, the output change amount of each attention head can be quantified to determine which heads play a key role in knowledge conflict.

[0056] Step 105, determining the proxy importance score of each attention head based on the output change amount of each attention head.

[0057] In the embodiments of the present application, for each attention head, the output change amount under the original input and the interference input is calculated, and by comparing the output changes under the two conditions, the proxy importance score of each attention head can be obtained.

[0058] Step 106, sorting based on the proxy importance score of each attention head, and taking the attention head of the target sequence number as the conflict attention head, wherein the conflict attention head is the context head.

[0059] Step 107, pruning the conflict attention head to eliminate the knowledge conflict of the target language model.

[0060] In the embodiments of the present application, according to the proxy importance score of each attention head, the context head that has a negative impact on the prediction of the memory attribute is pruned, thereby reducing the knowledge conflict and improving the ability of the model to use internal memory or external context.

[0061] For example, all attention heads are sorted according to their proxy importance scores. The context heads with the highest scores are selected for pruning. The selected context heads are pruned by setting their weights to zero, thus ignoring their influence during model execution.

[0062] By pruning context heads that negatively impact memory attribute prediction, the model can more effectively process conflicting information, reduce knowledge conflicts, and improve the efficiency of internal memory and external context.

[0063] Through the above steps of the embodiment of the present invention, an original input and an interference input are obtained, wherein there is a knowledge conflict between a first output result of the target language model based on the original input and a second output result based on the interference input; the target language model is run based on the original input to determine the original activation state of each attention head in all attention heads in the target language model; the target language model is run based on the interference input to determine the interference activation state of each attention head in all attention heads in the target language model; the original activation state of each attention head is replaced with the interference activation state, and the output change of each attention head is determined; based on the output change of each attention head, the proxy importance score of each attention head is determined; sorting is performed based on the proxy importance score of each attention head, and the attention head with the target sequence number is used as the conflicting attention head, wherein the conflicting attention head is a context head; attention pruning is performed on the conflicting attention heads to eliminate the knowledge conflict of the target language model, thereby pruning the context head that has a negative impact on the memory attribute prediction, thereby reducing the knowledge conflict and eliminating the knowledge conflict of the target language model; thereby solving the defect in the prior art that the language model cannot correctly generate the expected output when faced with contradictory information.

[0064] According to a knowledge conflict elimination method based on conflict attention head pruning provided by the present invention, before obtaining the original input and the interference input, the method further includes:

[0065] The activation values ​​of the multi-head attention module and feedforward network module of each layer in the target language model are knocked out respectively to determine the performance of the multi-head attention module and the feedforward network model.

[0066] In an embodiment of the present invention, an activation value knockout experiment is conducted to explore the functions of the multi-head attention (MHA) module and the feedforward neural network (FFN) module in knowledge conflict.

[0067] Specifically, we knock out the activation values ​​of FFN and MHA in each layer, and then evaluate the performance of the model in specific tasks after knocking out these components.

[0068] For example, activation knockout (also known as ablation experiments) can be used to observe the impact of these modules on model performance. MHA knockout involves temporarily setting the activation value of the MHA module to zero (or removing the MHA module) for each layer in the model, while keeping the other parts unchanged; re-evaluating the model's performance on the test set; and recording and analyzing the performance changes. FFN knockout involves temporarily setting the activation value of the FFN module to zero (or removing the FFN module) for each layer in the model; re-evaluating the model's performance on the test set; and recording and analyzing the performance changes.

[0069] Considering that the Transformer model usually contains multiple encoder layers and decoder layers (for generation tasks), we can further analyze the impact of MHA and FFN in each layer, and observe whether the importance of these modules in different layers varies by knocking them out layer by layer.

[0070] Use task-appropriate evaluation metrics, such as accuracy, BLEU score (for machine translation), F1 score, etc. Compare the performance differences before and after the knockout, and quantify the contribution of the multi-head attention module and the feedforward network module to the overall performance of the model.

[0071] For example, the model performance dropped significantly after MHA was knocked out, indicating that MHA plays a key role in capturing dependencies and contextual information in sequences. The knockout of FFN may weaken the model's ability in nonlinear transformation and feature extraction, thereby affecting model performance. MHA and FFN in different layers may have different effects on model performance, reflecting the model's emphasis when processing information at different levels.

[0072] According to a knowledge conflict elimination method based on conflict attention head pruning provided by the present invention, the performance of the feedforward network module includes: constructing a semantic representation of the input sequence; the performance of the multi-head attention module includes: passing the processed information to the last text unit.

[0073] In this embodiment of the present invention, the activation values ​​of the FFN and MHA in each layer were removed, and the performance of the model after removing these components on specific tasks was evaluated. It was found that the FFN plays an important role in enriching the semantic information of input elements in the early layers of the model, helping the model to build a semantic representation of the input in the initial stages. In contrast, the MHA plays a crucial role in the middle and late layers of the model, especially in transmitting processed information to the last token. This shows that the MHA plays a key role in the information transmission process.

[0074] According to a knowledge conflict elimination method based on conflicting attention head pruning provided by the present invention, after respectively performing activation value knockout on the multi-head attention module and the feedforward network module of each layer in the target language model and respectively determining the performance of the multi-head attention module and the feedforward network model, the method further comprises:

[0075] Determine the time point at which the multi-head attention module passes the processed information to the last text unit and the position of the last text unit.

[0076] In this embodiment of the present invention, we further explore the specific timing and location of MHA in transmitting information to the last text unit (token). To better understand the information flow transmission process, we use an attention weight removal experiment to analyze how different input elements (such as context, context relations, context attributes, question subject, and question relations) transmit information to the last token.

[0077] The method used was to set the attention weights at specific layers to zero and observe the effect on the model output. The researchers found that question information is first transmitted to the last token, which then extracts attribute information from the context. Specifically, the attention mechanism at early layers primarily conveys relational information about the question, while at mid- and late-stage layers, the attention mechanism gradually shifts its focus to the subject and attributes of the context, ultimately integrating this information at the last token.

[0078] According to a knowledge conflict elimination method based on conflicting attention head pruning provided by the present invention, the original activation state of each attention head is replaced with an interference activation state, and the output change of each attention head is determined, including:

[0079] Run the target language model based on the original input to obtain the original output result of the target language model;

[0080] When the original activation state of each attention head is replaced with the interference activation state one by one, the target language model is run based on the original input to obtain the interference output corresponding to each attention head output by the target language model;

[0081] The interference output results corresponding to each attention head are compared with the original output results to determine the output change of each attention head.

[0082] In an embodiment of the present invention, path patching technology is used to analyze the causal relationship of attention heads to determine which heads play a key role in knowledge conflicts. Path patching technology can accurately identify which attention heads have a significant impact on knowledge conflicts.

[0083] Run the model with both the original input and the perturbation input (where key information in the external context is replaced with special symbols), recording the activation states of all attention heads. By comparing the activation states under the two input conditions, we can gain a preliminary understanding of which attention heads significantly change when processing different information streams. Run the model with the original input, but replace the activation states of specific attention heads with those of the perturbation input and observe how the output changes. This method quantifies the impact of each attention head on the output, determines the amount of output change for each attention head, and thus identifies which heads play a key role in knowledge conflicts.

[0084] According to the present invention, a knowledge conflict elimination method based on conflicting attention head pruning is provided, which performs attention pruning on conflicting attention heads to eliminate knowledge conflicts of a target language model, including:

[0085] The weights of conflicting attention heads are set to predetermined values ​​to ignore the influence of conflicting attention heads in the target language model.

[0086] In this embodiment of the present invention, context heads that negatively impact memory attribute prediction are pruned based on the change in the output of each attention head, thereby reducing knowledge conflicts and improving the model's ability to utilize internal memory or external context. All attention heads are ranked according to their proxy importance scores. Those with the highest scores are selected for pruning. These selected context heads are pruned by setting their weights to zero, thereby ignoring their influence during model execution.

[0087] The following describes an example of a knowledge conflict elimination method based on conflicting attention head pruning provided by the present invention in a practical application scenario.

[0088] refer to Figure 3 , Figure 3 1 is an architecture diagram of the knowledge conflict elimination method based on conflict attention head pruning provided by the present invention, including (1) a gradient-based method, and (2) a path patching method of an embodiment of the present invention, wherein (2) the path patching method includes: (a) the operation of the original input (Original Input), (b) the operation of the interference input (Corrupted run), and (c) the operation of the replacement (Run with replacing activations).

[0089] The knowledge conflict resolution method based on conflicting attention head pruning provided by the present invention mainly consists of three parts: a path repair module, which runs the model under original input and interference input, records the activation states of all attention heads, then replaces the activation state of a specific attention head with the activation state of the interference input and observes the change in output; an importance score calculation module, which calculates the output change of each attention head under original input and interference input. By comparing the output changes under these two conditions, a proxy importance score for each head can be obtained; and an attention head pruning module: Based on the calculated importance scores, context heads that have a negative impact on memory attribute prediction are pruned to reduce knowledge conflicts.

[0090] This paper proposes a knowledge conflict resolution method based on conflicting attention head pruning, which mainly includes four steps: activation value knockout, attention weight knockout, path repair, and attention head pruning. The following is a detailed description of each step of this method.

[0091] Step 1: By knocking out the activation value, we explored the functions of the multi-head attention module and the feedforward network module in knowledge conflict and found that multi-head attention plays a key role in information transmission.

[0092] In this embodiment of the present invention, activation value knockout experiments were conducted to explore the functions of the multi-head attention (MHA) module and the feedforward network (FFN) module in knowledge conflict. These experiments aim to understand the specific role and influence of different modules in the information processing process.

[0093] Experimental Design: We remove the FFN and MHA components from the model layer by layer to observe their impact on the final prediction results. Specifically, we remove the activation values ​​of the FFN and MHA in each layer and then evaluate the performance of the model on specific tasks after removing these components.

[0094] refer to Figure 4 , Figure 4Schematic diagram of the results of activation value knockout of the FFN component and the MHA component provided by the present invention, including: (a) Effect of FFNs on internal memory, (b) Effect of MHAs on internal memory, (c) Extraction rate of internal memory, (d) Effect of FFNs one external context, (e) Effect of MHAs on external context, and (f) Extraction rate of external context.

[0095] Result analysis: Figure 4 As shown, FFN plays an important role in enriching the semantic information of input elements in the early layers of the model, helping the model build a semantic representation of the input in the initial stages. MHA, on the other hand, plays a crucial role in the middle and later layers of the model, particularly in transferring processed information to the last token. This demonstrates that MHA plays a key role in the information transfer process.

[0096] Step 2: By analyzing the information flow of multi-head attention in passing information to the last word through attention weight knockout, it is found that the problem information is first passed to the last token, and then the token extracts attribute information from the context.

[0097] Based on the above findings, we further explored the specific timing and location of MHA in delivering information to the last token.

[0098] In order to better understand the information flow transmission process, an experiment of attention weight knockout was designed in the embodiment of the present invention.

[0099] Experimental Design: By removing attention weights layer by layer, we analyzed how different input elements (such as the context, context relations, context attributes, question subject, and question relations) convey information to the final token. Specifically, we set the attention weights at specific layers to zero and observed the impact on the model output.

[0100] refer to Figure 5 , Figure 5Prediction based on internal memory, (b) Prediction based on external context.

[0101] Results analysis: As shown in 6, in the middle and later layers, some attention heads make significant contributions to the prediction of memory attributes, while others make significant contributions to the prediction of context attributes. The present invention further verifies the above findings by analyzing the role of attention heads identified as memory heads and context heads in the case of knowledge conflict. The results show that these heads show significant differences in extracting different attribute information, confirming that the key point of knowledge conflict lies in the inconsistent information flow integrated by these heads. Figure 5

[0102] Step 3: Analyze the causal relationship of attention heads using path patch technology to get the importance score of each attention head in knowledge conflict, determine which heads play a key role in knowledge conflict, and locate the memory head and context head.

[0103] The present invention further studies the specific role of attention heads in knowledge conflict. In particular, attention is paid to attention heads that have opposite effects in the later layers of the model.

[0104] Experimental design: Calculate the gradient importance score of each attention head in the case of knowledge conflict to determine which heads are crucial to the prediction of memory attributes or context attributes. The specific method is to analyze the behavior and contribution of different attention heads when processing internal memory and external context.

[0105] Reference Figure 6 , Figure 6 The present invention provides a schematic diagram of the extraction rate of memory heads and context heads, which includes: (a) memory head, (b) context head, (c) extraction rate of attention heads.

[0106] Results analysis: As shown in 6, in the middle and later layers, some attention heads make significant contributions to the prediction of memory attributes, while others make significant contributions to the prediction of context attributes. The present invention further verifies the above findings by analyzing the role of attention heads identified as memory heads and context heads in the case of knowledge conflict. The results show that these heads show significant differences in extracting different attribute information, confirming that the key point of knowledge conflict lies in the inconsistent information flow integrated by these heads.

[0107] ​Based on a deeper understanding of the mechanisms of knowledge conflict, this paper proposes a new method, Pruning Head via PatH PatcHing (PH3). The core idea of ​​PH3 is to improve the model's ability to handle conflicting information by pruning attention heads that cause knowledge conflict without changing model parameters. First, path patching is used to analyze the causal relationships among attention heads to determine which heads play a key role in knowledge conflict. Path patching can precisely identify which attention heads have a significant impact on knowledge conflict. The model is run with both original input and perturbation input (where key information in the external context is replaced with special symbols), and the activation states of all attention heads are recorded. By comparing the activation states under these two input conditions, we can initially understand which attention heads significantly change when processing different information streams. The model is run with the original input, but the activation states of specific attention heads are replaced with the activation states of the perturbation input, and the changes in the output are observed. This method quantifies the impact of each attention head on the output, thereby identifying which heads play a key role in knowledge conflict.

[0108] Step 4: Sort all attention heads and select and prune context heads that have a negative impact on memory attribute prediction. After pruning, the model can more effectively handle conflicting information, reduce knowledge conflicts, and improve the efficiency of internal memory and external context.

[0109] In this embodiment of the present invention, to further clarify which attention heads are the key to causing knowledge conflicts, a proxy importance score is calculated for each head in the knowledge conflict situation. The proxy importance score can quantify the contribution of each head to the prediction of memory attributes or context attributes.

[0110] The calculation method is as follows: For each attention head, the output change between the original and perturbed inputs is calculated. By comparing the output changes under these two conditions, a proxy importance score is obtained for each head. Based on the calculated proxy importance scores, it is possible to determine which attention heads contribute significantly to the prediction of the memory attribute (memory heads) and which heads contribute significantly to the prediction of the context attribute (context heads). Inconsistent information flow when these heads process information can lead to knowledge conflicts. Finally, based on the calculated importance scores, context heads that negatively impact the prediction of the memory attribute are pruned, thereby reducing knowledge conflicts and improving the model's ability to utilize internal memory or external context. The specific steps are as follows: All attention heads are sorted according to their proxy importance scores. Context heads with the highest scores (those with the greatest negative impact) are selected for pruning. These selected context heads are pruned by setting their weights to zero, thereby ignoring their influence during model execution.

[0111] In order to illustrate the effectiveness of the system, the present invention conducted experiments to verify the performance of the method.

[0112] The embodiments of the present invention are experimented on multiple datasets, including a world capital dataset, an official language dataset, a country and continent dataset, etc.

[0113] refer to Figure 7 , Figure 7 It is a schematic diagram comparing the effects of the existing method and the present method provided by the present invention.

[0114] The effectiveness of this method is illustrated by comparing the effects of existing methods with those of this method. Figure 7 Experimental results show that the PH3 method can increase the average proportion of the model's internal memory usage by 44.0%. This indicates that by pruning context heads that negatively impact the prediction of memory attributes, the model can more effectively utilize internal memory when faced with conflicting information. The PH3 method also significantly increases the average proportion of the model's external context usage, reaching 38.5%. This demonstrates that by pruning memory heads that negatively impact external context, the model is able to better integrate and utilize external context information.

[0115] refer to Figure 8 , Figure 8 It is a schematic diagram of the applicability results of the PH3 method provided by the present invention in different models.

[0116] In open-domain question answering tasks, the PH3 method can significantly improve the accuracy of the model's answer generation based on retrieved external context. Specifically, the PH3 method enables the model to be more faithful to the retrieved information when generating answers based on external context, reducing errors caused by knowledge conflicts.

[0117] As Figure 8 shown, the PH3 method also demonstrates its wide applicability on different models, different relationship types, and different data formats. The experimental results prove that the PH3 method is not only suitable for small models such as GPT-2 and GPT-J, but also can be effectively applied to large models such as LLaMA2. In addition, whether it is dealing with the relationship between world capitals or dealing with different types of data sets such as official languages, the PH3 method performs well, verifying its universality across models, relationships, and formats.

[0118] The conflict attention head pruning-based knowledge conflict elimination device provided by the present application is described below. The conflict attention head pruning-based knowledge conflict elimination device described below can be mutually corresponding to the conflict attention head pruning-based knowledge conflict elimination method described above.

[0119] Reference Figure 9 , Figure 9 is a structural schematic diagram of the conflict attention head pruning-based knowledge conflict elimination device provided by the present application, which includes an acquisition module 901, an original activation module 902, an interference activation module 903, a replacement module 904, a determination module 905, an ordering module 906, and a pruning module 907.

[0120] The acquisition module 901 is configured to acquire an original input and an interference input, wherein a target language model has a knowledge conflict between a first output result based on the original input and a second output result based on the interference input.

[0121] The original activation module 902 is configured to run the target language model based on the original input to determine an original activation state of each attention head in the target language model.

[0122] The interference activation module 903 is configured to run the target language model based on the interference input to determine an interference activation state of each attention head in the target language model.

[0123] The replacement module 904 is configured to replace the original activation state of each attention head with the interference activation state to determine an output change amount of each attention head.

[0124] The determination module 905 is configured to determine a proxy importance score of each attention head based on the output change amount of each attention head.

[0125] The ordering module 906 is configured to order based on the proxy importance score of each attention head, and take an attention head with a target serial number as a conflict attention head, wherein the conflict attention head is a context head.

[0126] The pruning module 907 is used to perform attention pruning on the conflicting attention heads to eliminate the knowledge conflict of the target language model.

[0127] Specifically, the knowledge conflict elimination device based on conflicting attention head pruning provided by the present invention can implement all the method steps implemented in the above-mentioned knowledge conflict elimination method embodiment based on conflicting attention head pruning, and can achieve the same technical effect. The parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0128] refer to Figure 10 , Figure 10 This is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as Figure 10 As shown, the electronic device may include: a processor (processor) 1010 , a communication interface (Communications Interface) 1020 , a memory (memory) 1030 and a communication bus 1040 , wherein the processor 1010 , the communication interface 1020 , and the memory 1030 communicate with each other via the communication bus 1040 . The processor 1010 can call the logic instructions in the memory 1030 to execute a knowledge conflict elimination method based on conflicting attention head pruning, which includes: obtaining original input and interference input, wherein there is a knowledge conflict between a first output result of the target language model based on the original input and a second output result based on the interference input; running the target language model based on the original input to determine the original activation state of each attention head in all attention heads in the target language model; running the target language model based on the interference input to determine the interference activation state of each attention head in all attention heads in the target language model; replacing the original activation state of each attention head with the interference activation state, and determining the output change of each attention head; determining the proxy importance score of each attention head based on the output change of each attention head; sorting based on the proxy importance score of each attention head, and taking the attention head with the target sequence number as the conflicting attention head, wherein the conflicting attention head is the context head; and performing attention pruning on the conflicting attention heads to eliminate the knowledge conflict of the target language model.

[0129] Furthermore, the logic instructions in the aforementioned memory 1030 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0130] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the knowledge conflict elimination method based on conflicting attention head pruning provided by the above methods, the method including: obtaining original input and interference input, wherein there is a knowledge conflict between the first output result of the target language model based on the original input and the second output result based on the interference input; running the target language model based on the original input to determine the original activation state of each attention head among all attention heads in the target language model; running the target language model based on the interference input to determine the interference activation state of each attention head among all attention heads in the target language model; replacing the original activation state of each attention head with the interference activation state, and determining the output change of each attention head; determining the proxy importance score of each attention head based on the output change of each attention head; sorting based on the proxy importance score of each attention head, and taking the attention head with the target sequence number as the conflicting attention head, wherein the conflicting attention head is the context head; performing attention pruning on the conflicting attention heads to eliminate the knowledge conflict of the target language model.

[0131] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the knowledge conflict elimination method based on conflicting attention head pruning provided by the above-mentioned methods, the method comprising: obtaining original input and interference input, wherein there is a knowledge conflict between a first output result of a target language model based on the original input and a second output result based on the interference input; running the target language model based on the original input, and determining the original activation state of each attention head among all attention heads in the target language model; running the target language model based on the interference input, and determining the interference activation state of each attention head among all attention heads in the target language model; replacing the original activation state of each attention head with the interference activation state, and determining the output change of each attention head; determining the proxy importance score of each attention head based on the output change of each attention head; sorting based on the proxy importance score of each attention head, and taking the attention head with the target sequence number as the conflicting attention head, wherein the conflicting attention head is the context head; performing attention pruning on the conflicting attention heads to eliminate the knowledge conflict of the target language model.

[0132] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0133] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A knowledge conflict elimination method based on conflict attention head pruning, characterized in that: include: Obtaining an original input and an interference input, wherein a first output result of the target language model based on the original input and a second output result based on the interference input have knowledge conflicts; Running the target language model based on the original input, determining an original activation state of each attention head in all attention heads in the target language model; running the target language model based on the interference input to determine an interference activation state for each of all attention heads in the target language model; Replace the original activation state of each attention head with the interference activation state and determine the output change of each attention head; Determining a proxy importance score for each attention head based on the output change of each attention head; Sort the attention heads based on the proxy importance scores of the attention heads, and use the attention heads with target numbers as conflicting attention heads, wherein the conflicting attention heads are context heads; Attention pruning is performed on the conflicting attention heads to eliminate the knowledge conflict of the target language model.

2. The knowledge conflict elimination method based on conflicting attention head pruning according to claim 1 is characterized in that: Before obtaining the original input and the interference input, the method further includes: The activation values ​​of the multi-head attention module and feedforward network module of each layer in the target language model are knocked out respectively to determine the performance of the multi-head attention module and the feedforward network model.

3. The knowledge conflict elimination method based on conflicting attention head pruning according to claim 2 is characterized in that: The performance of the feedforward network module includes: constructing a semantic representation of the input sequence; the performance of the multi-head attention module includes: passing the processed information to the last text unit.

4. The knowledge conflict elimination method based on conflicting attention head pruning according to claim 3 is characterized in that: After respectively performing activation value knockout on the multi-head attention module and the feedforward network module of each layer in the target language model and determining the performance of the multi-head attention module and the feedforward network model, the method further includes: Determine the time point at which the multi-head attention module transfers the processed information to the last text unit and the position of the last text unit.

5. The knowledge conflict elimination method based on conflicting attention head pruning according to claim 1 is characterized in that: The method replaces the original activation state of each attention head with the interference activation state and determines the output change of each attention head, including: Running the target language model based on the original input to obtain an original output result output by the target language model; When replacing the original activation state of each attention head with the interference activation state one by one, running the target language model based on the original input, obtaining an interference output result corresponding to each attention head output by the target language model; Based on the interference output results corresponding to each attention head and the original output results, the output change of each attention head is determined.

6. The knowledge conflict elimination method based on conflicting attention head pruning according to claim 1 is characterized in that The performing attention pruning on the conflicting attention heads to eliminate the knowledge conflict of the target language model includes: The weight of the conflicting attention head is set to a predetermined value to ignore the influence of the conflicting attention head in the target language model.

7. A knowledge conflict elimination device based on conflict attention head pruning, characterized in that: include: an acquisition module, configured to acquire an original input and an interference input, wherein a first output result of the target language model based on the original input and a second output result based on the interference input have knowledge conflicts; an original activation module, configured to run the target language model based on the original input and determine an original activation state of each attention head in all attention heads in the target language model; an interference activation module, configured to run the target language model based on the interference input and determine an interference activation state of each attention head in all attention heads in the target language model; The replacement module is used to replace the original activation state of each attention head with the interference activation state and determine the output change of each attention head; A determination module, configured to determine a proxy importance score of each attention head based on a change in the output of each attention head; a sorting module, configured to sort the attention heads based on the proxy importance scores of the attention heads, and use the attention heads with target sequence numbers as conflicting attention heads, wherein the conflicting attention heads are context heads; A pruning module is used to perform attention pruning on the conflicting attention heads to eliminate the knowledge conflict of the target language model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the knowledge conflict elimination method based on conflicting attention head pruning as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for eliminating knowledge conflicts based on conflicting attention head pruning according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for eliminating knowledge conflicts based on conflicting attention head pruning according to any one of claims 1 to 6 is implemented.

Images