Knowledge conflict elimination method and device based on conflict attention head pruning

Abstract

The invention provides a knowledge conflict elimination method and device based on conflict attention head pruning, and the method comprises the steps: obtaining original input and interference input; running a target language model based on the original input, and determining an original activation state of each attention head; running a target language model based on interference input, and determining an interference activation state of each attention head; the original activation state of each attention head is replaced with an interference activation state, and the output variable quantity of each attention head is determined; determining the proxy importance score of each attention head; sorting is carried out based on the proxy importance score of each attention head, the attention head of the target sequence number is used as a conflict attention head, and the conflict attention head is a context head; performing attention pruning on the conflict attention head to eliminate knowledge conflicts of the target language model; according to the invention, the flexibility of the model in processing the internal memory and the external context can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a knowledge conflict elimination method and device based on conflict attention head pruning. Background Art

[0002] With the rapid development of natural language processing (NLP) technology, language models have shown great potential in handling complex tasks. In the pre-training stage, language models are trained with a large amount of text data to accumulate and store rich factual knowledge; however, the internal memory capacity of language models is limited and cannot cover all possible facts and knowledge.

[0003] In order to make up for the shortcomings of internal memory, recent studies have proposed the use of retrieval enhancement and tool enhancement methods. These methods expand the knowledge boundary of the model by providing external context (i.e., non-parametric knowledge). However, although these external contexts can help the model provide more accurate information in some cases, they also bring new problems, namely, conflicts may occur between internal memory and external context, resulting in the model being unable to correctly generate the expected output when faced with contradictory information.

[0004] This shows that the language model in the related art cannot correctly generate the expected output when faced with contradictory information. Summary of the invention

[0005] The present invention provides a knowledge conflict elimination method and device based on conflict attention head pruning, which is used to solve the defect in the prior art that the language model cannot correctly generate the expected output when facing contradictory information, and improve the flexibility of the model in processing internal memory and external context.

[0006] The present invention provides a knowledge conflict elimination method based on conflicting attention head pruning, comprising the following steps. Obtaining original input and interference input, wherein there is a knowledge conflict between a first output result of a target language model based on the original input and a second output result based on the interference input; running the target language model based on the original input, and determining the original activation state of each attention head among all attention heads in the target language model; running the target language model based on the interference input, and determining the interference activation state of each attention head among all attention heads in the target language model; replacing the original activation state of each attention head with the interference activation state, and determining the output change of each attention head; determining the proxy importance score of each attention head based on the output change of each attention head; sorting based on the proxy importance score of each attention head, and taking the attention head with the target sequence number as the conflicting attention head, wherein the conflicting attention head is a context head; performing attention pruning on the conflicting attention head to eliminate the knowledge conflict of the target language model.

[0007] According to a knowledge conflict elimination method based on conflicting attention head pruning provided by the present invention, before obtaining the original input and the interference input, the method also includes: performing activation value knockout on the multi-head attention module and the feedforward network module of each layer in the target language model, and determining the multi-head attention module performance and the feedforward network model performance respectively.

[0008] According to a knowledge conflict elimination method based on conflict attention head pruning provided by the present invention, the feedforward network module performance includes: constructing a semantic representation of the input sequence; the multi-head attention module performance includes: passing the processed information to the last text unit.

[0009] According to a knowledge conflict elimination method based on conflicting attention head pruning provided by the present invention, after respectively performing activation value knockout on the multi-head attention module and the feedforward network module of each layer in the target language model and respectively determining the multi-head attention module performance and the feedforward network model performance, the method further includes: determining the time point when the multi-head attention module transmits the processed information to the last text unit and the position of the last text unit.

[0010] According to a knowledge conflict elimination method based on conflicting attention head pruning provided by the present invention, the original activation state of each attention head is replaced with an interference activation state to determine the output change of each attention head, including: running the target language model based on the original input to obtain the original output result of the target language model; when the original activation state of each attention head is replaced with an interference activation state one by one, running the target language model based on the original input to obtain the interference output result corresponding to each attention head output by the target language model; based on the interference output result corresponding to each attention head, comparing the interference output result with the original output result, to determine the output change of each attention head.

[0011] According to a knowledge conflict elimination method based on conflicting attention head pruning provided by the present invention, attention pruning is performed on the conflicting attention heads to eliminate the knowledge conflicts of the target language model, comprising: setting the weights of the conflicting attention heads to predetermined values ​​to ignore the influence of the conflicting attention heads in the target language model.

[0012] The present invention also provides a knowledge conflict elimination device based on conflicting attention head pruning, comprising the following modules: an acquisition module, used to acquire original input and interference input, wherein a first output result of a target language model based on the original input and a second output result based on the interference input have knowledge conflicts; an original activation module, used to run the target language model based on the original input, and determine the original activation state of each attention head among all attention heads in the target language model; an interference activation module, used to run the target language model based on the interference input, and determine the interference activation state of each attention head among all attention heads in the target language model; a replacement module, used to replace the original activation state of each attention head with the interference activation state, and determine the output change of each attention head; a determination module, used to determine the proxy importance score of each attention head based on the output change of each attention head; a sorting module, used to sort based on the proxy importance score of each attention head, and use the attention head with the target sequence number as the conflicting attention head, wherein the conflicting attention head is a context head; and a pruning module, used to perform attention pruning on the conflicting attention head to eliminate the knowledge conflict of the target language model.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for eliminating knowledge conflicts based on conflicting attention head pruning as described above is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described knowledge conflict elimination methods based on conflict attention head pruning.

[0015] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-described knowledge conflict elimination methods based on conflict attention head pruning.

[0016] The knowledge conflict elimination method and device based on conflicting attention head pruning provided by the present invention respectively runs the target language model under the original input and interference input, records the original activation state and interference activation state of all attention heads, then replaces the activation state of a specific attention head with the activation state of the interference input, observes the change of the output, and calculates the output change amount under the original input and interference input conditions for each attention head. By comparing the output changes under these two conditions, the proxy importance score of each attention head can be obtained; sorting is performed according to the proxy importance score, and the attention head (context head) of the target sequence number is used as the conflicting attention head, and the conflicting attention head is pruned, thereby pruning the context head that has a negative impact on the memory attribute prediction, thereby reducing knowledge conflicts, so as to eliminate the knowledge conflicts of the target language model; thereby solving the defect that the language model in the prior art cannot correctly generate the expected output when facing contradictory information. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the knowledge conflict elimination method based on conflict attention head pruning provided by the present invention.

[0019] Figure 2 It is a schematic diagram of the process of knowledge conflict in the language model provided by the present invention.

[0020] Figure 3 It is an architectural diagram of the knowledge conflict elimination method based on conflict attention head pruning provided by the present invention.

[0021] Figure 4 It is a schematic diagram of the results of activation value knockout of the FFN component and the MHA component provided by the present invention.

[0022] Figure 5The present invention provides an internal memory prediction map and an external environment prediction map based on different input elements.

[0023] Figure 6 It is a result schematic diagram of the attention head extraction rate of the memory head and the context head provided by the present invention.

[0024] Figure 7 It is a schematic diagram comparing the effects of the existing method and the method provided by the present invention.

[0025] Figure 8 It is a schematic diagram of the applicability results of the PH3 method provided by the present invention in different models.

[0026] Fig. 9 It is a structural schematic diagram of a knowledge conflict elimination device based on conflict attention head pruning provided by the present invention.

[0027] Fig.10 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] With the rapid development of natural language processing technology, language models have shown great potential in handling complex tasks. In the pre-training stage, language models are trained with a large amount of text data, accumulating and storing rich factual knowledge. This knowledge is mainly stored in the model in the form of parameters, which is called internal memory. However, internal memory has the following main problems: Knowledge update lag: The knowledge in the internal memory is fixed during the pre-training phase and cannot be updated dynamically. When the world changes or new information emerges, the model cannot update its knowledge base in time.

[0030] Limited memory capacity: Although the model is trained with a large amount of data, its internal memory has a limited capacity and cannot cover all possible facts and knowledge.

[0031] Factual errors: Due to noise or errors in the training data, the model may remember some inaccurate facts, resulting in errors when generating answers.

[0032] In order to make up for the shortcomings of internal memory, recent studies have proposed the use of retrieval enhancement and tool enhancement methods. These methods expand the knowledge boundary of the model by providing external context (i.e., non-parametric knowledge), enabling the model to refer to the latest and relevant external information when generating answers. External context can include: Real-time information retrieval: By retrieving the latest information and data in real time, the model can refer to it when generating answers.

[0033] Tools and databases: Use external tools and databases to provide additional knowledge support, such as knowledge graphs, search engines, and specialized databases.

[0034] However, although these external contexts can help the model provide more accurate information in some cases, they also bring new problems: conflicts may occur between internal memory and external context. This knowledge conflict will cause the model to fail to generate output correctly when faced with contradictory information, seriously affecting the reliability and performance of the model.

[0035] The fundamental problem with knowledge conflict is that most current research focuses on understanding the behavior and preferences of models in conflict situations, trying to determine whether models tend to rely more on internal memory or external context when faced with knowledge conflicts. However, the academic community's understanding of the specific mechanisms of knowledge conflicts and how to effectively mitigate these conflicts is still very limited. The embodiments of the present invention explain the mechanism of knowledge conflict from the perspective of information flow and perform precise intervention at key points to mitigate conflicts.

[0036] In the embodiment of the present invention, it is found that in the later layers of the language model, some attention heads have opposite effects, where the memory heads can recall knowledge from the internal memory, while the context heads can retrieve knowledge from the external context. When these attention heads process information, if the information flow is inconsistent, it will lead to knowledge conflict. Specifically, the key point of knowledge conflict lies in the problem that these attention heads encounter when integrating inconsistent information flows.

[0037] Based on the above findings, the present invention proposes a new method, called Pruning Head via PatH PatcHing (PH3), which can efficiently alleviate knowledge conflicts by pruning conflicting attention heads without updating model parameters.

[0038] The PH3 method has the following main advantages: Flexible control of the model's use of internal memory or external context: PH3 can significantly improve the model's ability to use internal memory (↑44.0%) or external context (↑38.5%), thereby flexibly adjusting the model's behavior according to task requirements. Improve the performance of the model in open domain question-answering tasks: By reducing knowledge conflicts, PH3 can help the model more accurately generate answers based on retrieved external context, thereby improving the performance of question-answering tasks. Wide versatility: The embodiments of the present invention have conducted a large number of experiments to demonstrate the wide applicability of the PH3 method on different models, different relationship types, and different data formats.

[0039] In order to overcome the defects of the prior art, the present invention designs a knowledge conflict elimination method based on conflicting attention head pruning to address the knowledge conflict problem in the retrieval enhancement model. The method utilizes path patching technology to identify the key memory heads and context heads in the multi-head attention module, and prunes the context heads that have a negative impact on the memory attribute prediction according to the calculated importance scores, thereby reducing knowledge conflicts.

[0040] Optionally, the knowledge conflict elimination method based on conflicting attention head pruning in the embodiment of the present application can be executed by a server, or by a terminal device, or jointly by a server and a terminal device, taking the example of the knowledge conflict elimination method based on conflicting attention head pruning in the present embodiment being executed by a server.

[0041] refer to Figure 1 , Figure 1 is a flow chart of the knowledge conflict elimination method based on conflict attention head pruning provided by the present invention, such as Figure 1 As shown, the method includes the following steps.

[0042] Step 101: obtaining an original input and an interference input, wherein a first output result of a target language model based on the original input and a second output result based on the interference input have knowledge conflicts.

[0043] In the embodiment of the present invention, the original input may be the internal knowledge in the target language model, while the interference input is the external knowledge acquired outside the target language model. Meanwhile, the internal knowledge contradicts the external knowledge.

[0044] refer to Figure 2 , Figure 21 is a schematic diagram of the process of knowledge conflict in the language model provided by the present invention, which includes input (Context: The capital of France is Rome, Question: What is the capital of France), language model, external knowledge (Paris) and internal knowledge (Rome).

[0045] For example, when the input question is "What is the capital of France", the expected output of the target language model based on internal knowledge (original input "The capital of France is Paris") is "Paris", while the expected output of the target language model based on external knowledge (interference input "The capital of France is Rome") is "Rome", that is, there is a knowledge conflict between the first output result of the target language model based on the original input and the second output result based on the interference input.

[0046] It should be noted that the original input is internal knowledge pre-set in the target language model (for example, pre-stored factual knowledge, etc.), and the interference output can be external knowledge obtained by real-time retrieval by the target language model based on the input question, or it can be external knowledge directly input by the user.

[0047] Here, the target language model is a language abstract mathematical model based on objective language facts. It is a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. The target language model can handle a variety of natural language tasks, such as text classification, question answering, dialogue, etc. For example, the target language model can be a large language model (LLM, Large Language Model).

[0048] Step 102, running the target language model based on the original input, and determining the original activation state of each attention head in all attention heads in the target language model.

[0049] Step 103, running the target language model based on the interference input, and determining the interference activation state of each attention head in all attention heads in the target language model.

[0050] When dealing with natural language processing tasks, especially when using Transformer-based models (such as BERT, GPT, etc.), the attention mechanism is one of the core parts of the model. In the Transformer model, the attention mechanism captures different aspects of the input data by working in parallel with multiple attention heads. Each attention head generates an attention weight matrix, which represents the correlation or importance between different positions in the sequence. In an embodiment of the present invention, the attention weight matrix of each attention head can be used as an activation state.

[0051] In the embodiment of the present invention, the model is run under the original input and the interference input (key information in the external context is replaced with special symbols), and the activation status of all attention heads is recorded. By comparing the activation status under the two input conditions, it is possible to preliminarily determine which attention heads have significant changes when processing different information streams.

[0052] Step 104, replace the original activation state of each attention head with the interference activation state, and determine the output change of each attention head.

[0053] In the embodiment of the present invention, the target language model is used under the original input, but the activation state of a specific attention head is replaced with the activation state of the interference input, and the change in the output is observed. In this way, the impact of each attention head on the output result can be quantified to obtain the output change of each attention head, thereby identifying which heads play a key role in the knowledge conflict.

[0054] Step 105, based on the output change of each attention head, determine the proxy importance score of each attention head.

[0055] In an embodiment of the present invention, for each attention head, the output change under the original input and interference input conditions is calculated, and the proxy importance score of each attention head can be obtained by comparing the output changes under these two conditions.

[0056] Step 106, sorting based on the proxy importance score of each attention head, and taking the attention head with the target sequence number as the conflicting attention head, wherein the conflicting attention head is the context head.

[0057] Step 107, perform attention pruning on the conflicting attention heads to eliminate the knowledge conflict of the target language model.

[0058] In an embodiment of the present invention, context heads that have a negative impact on memory attribute prediction are pruned based on the proxy importance score of each attention head, thereby reducing knowledge conflicts and improving the model's ability to use internal memory or external context.

[0059] For example, all attention heads are sorted according to their proxy importance scores. The context heads with the highest scores are selected for pruning. The selected context heads are pruned by setting their weights to zero, thus ignoring their influence during model execution.

[0060] Through the embodiment of the present invention, the context heads that have a negative impact on memory attribute prediction are selected for pruning. After pruning, the model can more effectively process conflicting information, reduce knowledge conflicts, and improve the efficiency of using internal memory and external context.

[0061] Through the above steps of the embodiment of the present invention, an original input and an interference input are obtained, wherein there is a knowledge conflict between a first output result of the target language model based on the original input and a second output result based on the interference input; the target language model is run based on the original input to determine the original activation state of each attention head in all attention heads in the target language model; the target language model is run based on the interference input to determine the interference activation state of each attention head in all attention heads in the target language model; the original activation state of each attention head is replaced with the interference activation state to determine the output change of each attention head; based on the output change of each attention head, the proxy importance score of each attention head is determined; sorting is performed based on the proxy importance score of each attention head, and the attention head with the target sequence number is used as the conflicting attention head, wherein the conflicting attention head is a context head; attention pruning is performed on the conflicting attention head to eliminate the knowledge conflict of the target language model, thereby pruning the context head that has a negative impact on the memory attribute prediction, thereby reducing the knowledge conflict and eliminating the knowledge conflict of the target language model; thereby solving the defect in the prior art that the language model cannot correctly generate the expected output when facing contradictory information.

[0062] According to a knowledge conflict elimination method based on conflict attention head pruning provided by the present invention, before obtaining the original input and the interference input, the method further includes: The activation values ​​of the multi-head attention module and the feedforward network module of each layer in the target language model are knocked out respectively to determine the performance of the multi-head attention module and the feedforward network model.

[0063] In an embodiment of the present invention, an activation value knockout experiment is performed to explore the functions of a multi-head attention (MHA) module and a feedforward neural network (FFN) module in knowledge conflict.

[0064] Specifically, we knock out the activation values ​​of FFN and MHA in each layer, and then evaluate the performance of the model in specific tasks after knocking out these components.

[0065] For example, the impact of these modules on model performance can be observed by performing activation value knockout (also known as ablation experiments). MHA knockout includes temporarily setting the activation value of the MHA module to zero (or removing the MHA module) for each layer in the model, and keeping other parts unchanged; re-evaluating the performance of the model on the test set; recording and analyzing performance changes. FFN knockout includes temporarily setting the activation value of the FFN module to zero (or removing the FFN module) for each layer in the model; re-evaluating the performance of the model on the test set; recording and analyzing performance changes.

[0066] Considering that the Transformer model usually contains multiple encoder layers and decoder layers (for generation tasks), we can further analyze the impact of MHA and FFN in each layer, and observe whether the importance of these modules in different layers is different by knocking them out layer by layer.

[0067] Use evaluation metrics suitable for the task, such as accuracy, BLEU score (for machine translation), F1 score, etc. Compare the performance difference before and after knockout, and quantify the contribution of the multi-head attention module and the feedforward network module to the overall performance of the model.

[0068] For example, the model performance drops significantly after MHA is knocked out, indicating that MHA plays a key role in capturing dependencies and contextual information in sequences. The knockout of FFN may weaken the model's ability in nonlinear transformation and feature extraction, thereby affecting model performance. MHA and FFN in different layers may have different effects on model performance, reflecting the model's focus when processing information at different levels.

[0069] According to a knowledge conflict elimination method based on conflict attention head pruning provided by the present invention, the performance of the feedforward network module includes: constructing a semantic representation of the input sequence; the performance of the multi-head attention module includes: passing the processed information to the last text unit.

[0070] In the embodiment of the present invention, the activation values ​​of FFN and MHA in each layer are knocked out respectively, and then the performance of the model in a specific task after knocking out these components is evaluated, and it is found that FFN plays an important role in enriching the semantic information of the input elements in the early layers of the model, helping the model to build a semantic representation of the input in the initial stage. MHA plays a vital role in the middle and late layers of the model, especially responsible for passing the processed information to the last token. This shows that MHA plays a key role in the information transmission process.

[0071] According to a knowledge conflict elimination method based on conflict attention head pruning provided by the present invention, after respectively performing activation value knockout on the multi-head attention module and the feedforward network module of each layer in the target language model, and respectively determining the multi-head attention module performance and the feedforward network model performance, the method further includes: Determine the time point at which the multi-head attention module passes the processed information to the last text unit and the position of the last text unit.

[0072] In the embodiment of the present invention, the specific timing and position of MHA in transmitting information to the last text unit (token) are further explored. In order to better understand the transmission process of information flow, the present invention adopts the experiment of attention weight knockout to analyze how different input elements (such as context subject, context relationship, context attribute, question subject and question relationship) transmit information to the last token.

[0073] The specific method is to set the attention weight of a specific layer to zero and observe its impact on the model output, and then find that the question information is first transmitted to the last token, and then the token extracts attribute information from the context. In particular, the attention mechanism in the early layers is mainly responsible for transmitting the relational information of the question, while in the middle and late layers, the attention mechanism gradually shifts the focus to the subject and attributes of the context, and finally integrates this information on the last token.

[0074] According to a knowledge conflict elimination method based on conflict attention head pruning provided by the present invention, the original activation state of each attention head is replaced with an interference activation state, and the output change amount of each attention head is determined, including: Run the target language model based on the original input to obtain the original output result of the target language model; When the original activation state of each attention head is replaced with the interference activation state one by one, the target language model is run based on the original input to obtain the interference output result corresponding to each attention head output by the target language model; The interference output results corresponding to each attention head are compared with the original output results to determine the output change of each attention head.

[0075] In an embodiment of the present invention, path patching technology is used to analyze the causal relationship of attention heads, so as to determine which heads play a key role in knowledge conflicts. The path patching technology can accurately identify which attention heads have a significant impact on knowledge conflicts.

[0076] Run the model under the original input and the interference input (replacing the key information in the external context with special symbols) respectively, and record the activation status of all attention heads. By comparing the activation status under the two input conditions, we can get a preliminary understanding of which attention heads have significant changes when processing different information streams. Run the model under the original input, but replace the activation state of a specific attention head with the activation state of the interference input, and observe the changes in the output. In this way, we can quantify the impact of each attention head on the output result, determine the output change of each attention head, and thus identify which heads play a key role in knowledge conflicts.

[0077] According to a knowledge conflict elimination method based on conflict attention head pruning provided by the present invention, attention pruning is performed on conflict attention heads to eliminate knowledge conflicts of a target language model, including: The weights of conflicting attention heads are set to predetermined values ​​to ignore the influence of conflicting attention heads in the target language model.

[0078] In an embodiment of the present invention, according to the output change of each attention head, the context heads that have a negative impact on the memory attribute prediction are pruned, thereby reducing knowledge conflicts and improving the model's ability to use internal memory or external context. All attention heads are sorted according to the proxy importance score. Those context heads with the highest scores are selected for pruning. The selected context heads are pruned by setting the weights of these attention heads to zero, thereby ignoring the influence of these heads during the model operation.

[0079] The following describes an example of a knowledge conflict elimination method based on conflict attention head pruning provided by the present invention in a practical application scenario.

[0080] refer to Figure 3 , Figure 3 1 is an architecture diagram of a knowledge conflict elimination method based on conflict attention head pruning provided by the present invention, including (1) a gradient-based method, and (2) a path patching method of an embodiment of the present invention, wherein (2) the path patching method includes: (a) the operation of the original input, (b) the operation of the interference input, and (c) the operation of the replacement.

[0081] The knowledge conflict elimination method based on conflict attention head pruning provided by the present invention mainly includes three parts: a path repair module, which runs the model under the original input and interference input respectively, records the activation state of all attention heads, and then replaces the activation state of a specific attention head with the activation state of the interference input, and observes the change in output; an importance score calculation module, for each attention head, calculates the output change under the original input and interference input conditions. By comparing the output changes under these two conditions, the proxy importance score of each head can be obtained; an attention head pruning module: according to the calculated importance score, those context heads that have a negative impact on memory attribute prediction are pruned, thereby reducing knowledge conflicts.

[0082] The present invention proposes a knowledge conflict elimination method based on conflict attention head pruning, which mainly includes four steps: activation value knockout, attention weight knockout, path repair and attention head pruning. The following is a detailed description of each step included in the method.

[0083] Step 1: By knocking out the activation value, we explored the functions of the multi-head attention module and the feedforward network module in knowledge conflict and found that multi-head attention plays a key role in information transmission.

[0084] In the embodiment of the present invention, activation value knockout experiments are performed to explore the functions of the multi-head attention (MHA) module and the feed-forward network (FFN) module in knowledge conflict. These experiments are intended to understand the specific roles and influences of different modules in the information processing process.

[0085] Experimental design: By knocking out the FFN and MHA components in the model layer by layer, we observe their impact on the final prediction results. Specifically, we knock out the activation values ​​of FFN and MHA in each layer, and then evaluate the performance of the model in specific tasks after knocking out these components.

[0086] refer to Figure 4 , Figure 4Schematic diagram of the results of activation value knockout of the FFN component and the MHA component provided by the present invention, including: (a) Effect of FFNs on internal memory, (b) Effect of MHAs on internal memory, (c) Extraction rate of internal memory, (d) Effect of FFNs one external context, (e) Effect of MHAs on external context, and (f) Extraction rate of external context.

[0087] Result analysis: Figure 4 As shown in the figure, FFN plays an important role in enriching the semantic information of input elements in the early layers of the model, helping the model to build a semantic representation of the input in the initial stage. MHA plays a crucial role in the middle and late layers of the model, especially responsible for passing the processed information to the last token. This shows that MHA plays a key role in the information transmission process.

[0088] Step 2: By analyzing the information flow of multi-head attention in transmitting information to the last word through attention weight knockout, it is found that the problem information is first transmitted to the last token, and then the token extracts attribute information from the context.

[0089] Based on the above findings, we further explored the specific timing and location of MHA in delivering information to the last token.

[0090] In order to better understand the information flow transmission process, an experiment of attention weight knockout is designed in the embodiment of the present invention.

[0091] Experimental design: By knocking out the attention weights layer by layer, we analyzed how different input elements (such as context body, context relationship, context attribute, question body, and question relationship) pass information to the last token. The specific method is to set the attention weights of specific layers to zero and observe their impact on the model output.

[0092] refer to Figure 5 , Figure 5The present invention provides an internal memory prediction map and an external context prediction map based on different input elements, including (a) prediction based on internal memory and (b) prediction based on external context.

[0093] Result analysis: Figure 5 As shown in Figure 1, the question information is first passed to the last token, which then extracts attribute information from the context. In particular, the attention mechanism in the early layers is mainly responsible for passing the relational information of the question, while in the middle and late layers, the attention mechanism gradually shifts its focus to the subject and attributes of the context, and finally integrates this information on the last token.

[0094] Step 3: Use the path patching technology to analyze the causal relationship of the attention heads, obtain the importance score of each attention head in the knowledge conflict, determine which heads play a key role in the knowledge conflict, and locate the memory head and context head.

[0095] The embodiments of the present invention further study the specific role of attention heads in knowledge conflicts, especially focusing on the attention heads that have opposite effects in the later layers of the model.

[0096] Experimental design: We compute the gradient importance scores of each attention head under knowledge conflict to determine which heads are critical to the prediction of memory attributes or context attributes by analyzing the behaviors and contributions of different attention heads when processing internal memory and external context.

[0097] refer to Figure 6 , Figure 6 Schematic diagram of the results of the attention head extraction rate of the memory head and the context head provided by the present invention, including: (a) memory head, (b) context head, and (c) extraction rate of attention heads.

[0098] Results Analysis: As shown in Figure 6, in the middle and late layers, some attention heads contribute significantly to the prediction of memory attributes, while others contribute significantly to the prediction of context attributes. The present invention further verifies the above findings and analyzes the role of attention heads identified as memory heads and context heads in the case of knowledge conflict. The results show that these heads show significant differences in extracting different attribute information, confirming that the key point of knowledge conflict is that these heads integrate inconsistent information flows.

[0099] Based on a deep understanding of the knowledge conflict mechanism, this paper proposes a new method called Pruning Head via PatH PatcHing (PH3). The core idea of ​​the PH3 method is to improve the model's ability to handle conflicting information by pruning the attention heads that cause knowledge conflicts without changing the model parameters. First, the path patching technique is used to analyze the causal relationship of the attention heads to determine which heads play a key role in knowledge conflicts. The path patching technique can accurately identify which attention heads have a significant impact on knowledge conflicts. The model is run under the original input and the interference input (the key information in the external context is replaced by special symbols), and the activation states of all attention heads are recorded. By comparing the activation states under the two input conditions, we can get a preliminary understanding of which attention heads have significant changes when processing different information streams. Run the model under the original input, but replace the activation state of a specific attention head with the activation state of the interference input, and observe the changes in the output. In this way, the impact of each attention head on the output result can be quantified, thereby identifying which heads play a key role in knowledge conflicts.

[0100] Step 4: Sort all attention heads and select the context heads that have a negative impact on memory attribute prediction for pruning. After pruning, the model can handle conflicting information more effectively, reduce knowledge conflicts, and improve the efficiency of using internal memory and external context.

[0101] In the embodiment of the present invention, in order to further clarify which attention heads are the key to causing knowledge conflicts, the proxy importance score of each head in the case of knowledge conflicts is calculated. The proxy importance score can quantify the contribution of each head to the prediction of memory attributes or context attributes.

[0102] The specific calculation method is as follows: For each attention head, calculate the output change in the original input and the perturbed input. By comparing the output change in these two cases, the proxy importance score of each head can be obtained. Based on the calculated proxy importance scores, it is possible to determine which attention heads contribute significantly to the prediction of memory attributes (memory heads) and which heads contribute significantly to the prediction of contextual attributes (context heads). When these heads process information, if the information flow is inconsistent, it will cause knowledge conflicts. Finally, according to the calculated importance scores, those context heads that have a negative impact on the prediction of memory attributes are pruned, thereby reducing knowledge conflicts and improving the model's ability to use internal memory or external context. The specific steps are as follows: Sort all attention heads according to the proxy importance scores. Select those context heads with the highest scores (the most negative impact) for pruning. Prune the selected context heads by setting the weights of these attention heads to zero, so that the influence of these heads is ignored during the model operation.

[0103] In order to illustrate the effectiveness of the system, the present invention conducted experiments to verify the performance of the method.

[0104] The embodiments of the present invention are experimented on multiple data sets, including a world capital data set, an official language data set, a country and continent data set, etc.

[0105] refer to Figure 7 , Figure 7 It is a schematic diagram comparing the effects of the existing method and the method provided by the present invention.

[0106] The effectiveness of this method is illustrated by comparing the effects of existing methods with those of this method. Figure 7 ,The experimental results show that the PH3 method can increase the average proportion of the model's use of internal memory by 44.0%. This shows that by pruning those context heads that have a negative impact on the prediction of memory attributes, the model can more effectively use internal memory when facing conflicting information. The PH3 method also significantly increases the average proportion of the model's use of external context, reaching 38.5%. This shows that by pruning memory heads that have a negative impact on external context, the model can better integrate and utilize external context information.

[0107] refer to Figure 8 , Figure 8 It is a schematic diagram of the applicability results of the PH3 method provided by the present invention in different models.

[0108] In the open-domain question answering task, the PH3 method can significantly improve the accuracy of the model in generating answers based on the retrieved external context. Specifically, the PH3 method makes the model more faithful to the retrieved information when generating answers based on the external context, reducing the generation of errors due to knowledge conflicts.

[0109] like Figure 8 As shown, the present invention also demonstrates the wide applicability of the PH3 method in different models, different relationship types, and different data formats. Experimental results show that the PH3 method is not only applicable to small models such as GPT-2 and GPT-J, but can also be effectively applied to large models such as LLaMA2. In addition, whether it is processing the relationship between world capitals or processing different types of data sets such as official languages, the PH3 method performs well, verifying its versatility across models, relationships, and formats.

[0110] The knowledge conflict elimination device based on conflicting attention head pruning provided by the present invention is described below. The knowledge conflict elimination device based on conflicting attention head pruning described below and the knowledge conflict elimination method based on conflicting attention head pruning described above can be referenced to each other.

[0111] refer to Fig. 9 , Fig. 9 It is a structural diagram of the knowledge conflict elimination device based on conflict attention head pruning provided by the present invention, which includes an acquisition module 901, an original activation module 902, an interference activation module 903, a replacement module 904, a determination module 905, a sorting module 906 and a pruning module 907.

[0112] An acquisition module 901 is used to acquire an original input and an interference input, wherein a first output result of a target language model based on the original input and a second output result based on the interference input have knowledge conflicts; An original activation module 902 is used to run the target language model based on the original input to determine the original activation state of each attention head in all attention heads in the target language model; An interference activation module 903 is used to run the target language model based on the interference input to determine the interference activation state of each attention head in all attention heads in the target language model; A replacement module 904 is used to replace the original activation state of each attention head with the interference activation state, and determine the output change of each attention head; A determination module 905 is used to determine the proxy importance score of each attention head based on the output change of each attention head; A sorting module 906 is used to sort the attention heads based on the proxy importance scores of each attention head, and use the attention head with the target sequence number as the conflicting attention head, wherein the conflicting attention head is a context head; The pruning module 907 is used to perform attention pruning on the conflicting attention heads to eliminate the knowledge conflict of the target language model.

[0113] Specifically, the knowledge conflict elimination device based on conflicting attention head pruning provided by the present invention can implement all the method steps implemented by the above-mentioned knowledge conflict elimination method embodiment based on conflicting attention head pruning, and can achieve the same technical effect. The parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0114] refer to Fig.10 , Fig.10 is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as Fig.10 As shown, the electronic device may include: a processor (processor) 1010 , a communication interface (Communications Interface) 1020 , a memory (memory) 1030 and a communication bus 1040 , wherein the processor 1010 , the communication interface 1020 , and the memory 1030 communicate with each other via the communication bus 1040 . The processor 1010 can call the logic instructions in the memory 1030 to execute a knowledge conflict elimination method based on conflicting attention head pruning, the method comprising: obtaining an original input and an interference input, wherein a first output result of a target language model based on the original input and a second output result based on the interference input have a knowledge conflict; running the target language model based on the original input to determine an original activation state of each attention head among all attention heads in the target language model; running the target language model based on the interference input to determine an interference activation state of each attention head among all attention heads in the target language model; replacing the original activation state of each attention head with an interference activation state to determine an output change of each attention head; determining a proxy importance score of each attention head based on the output change of each attention head; sorting based on the proxy importance score of each attention head, and taking the attention head with the target sequence number as a conflicting attention head, wherein the conflicting attention head is a context head; and performing attention pruning on the conflicting attention heads to eliminate the knowledge conflict of the target language model.

[0115] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0116] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the knowledge conflict elimination method based on conflicting attention head pruning provided by the above methods, the method including: obtaining original input and interference input, wherein there is a knowledge conflict between a first output result of a target language model based on the original input and a second output result based on the interference input; running the target language model based on the original input to determine the original activation state of each attention head among all attention heads in the target language model; running the target language model based on the interference input to determine the interference activation state of each attention head among all attention heads in the target language model; replacing the original activation state of each attention head with the interference activation state respectively, and determining the output change of each attention head; determining the proxy importance score of each attention head based on the output change of each attention head; sorting based on the proxy importance score of each attention head, and taking the attention head with the target sequence number as the conflicting attention head, wherein the conflicting attention head is the context head; performing attention pruning on the conflicting attention heads to eliminate the knowledge conflict of the target language model.

[0117] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the knowledge conflict elimination method based on conflicting attention head pruning provided by the above methods, the method comprising: obtaining an original input and an interference input, wherein a first output result of a target language model based on the original input and a second output result based on the interference input have a knowledge conflict; running the target language model based on the original input, and determining an original activation state of each attention head among all attention heads in the target language model; running the target language model based on the interference input, and determining an interference activation state of each attention head among all attention heads in the target language model; replacing the original activation state of each attention head with an interference activation state, and determining an output change of each attention head; determining a proxy importance score of each attention head based on the output change of each attention head; sorting based on the proxy importance score of each attention head, and taking the attention head with the target sequence number as a conflicting attention head, wherein the conflicting attention head is a context head; and performing attention pruning on the conflicting attention heads to eliminate the knowledge conflict of the target language model.

[0118] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0119] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0120] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A knowledge conflict elimination method based on conflict attention head pruning, characterized in that: include: Obtaining an original input and an interference input, wherein a first output result of the target language model based on the original input and a second output result based on the interference input have a knowledge conflict; Running the target language model based on the original input, determining an original activation state of each attention head in all attention heads in the target language model; Running the target language model based on the interference input, determining an interference activation state of each attention head in all attention heads in the target language model; Replace the original activation state of each attention head with the interference activation state, and determine the output change of each attention head; Determining a proxy importance score for each attention head based on the output change of each attention head; Sorting based on the proxy importance score of each attention head, taking the attention head with the target sequence number as the conflicting attention head, wherein the conflicting attention head is a context head; Attention pruning is performed on the conflicting attention heads to eliminate the knowledge conflict of the target language model.

2. The knowledge conflict elimination method based on conflicting attention head pruning according to claim 1 is characterized in that: Before obtaining the original input and the interference input, the method further includes: The activation values ​​of the multi-head attention module and the feedforward network module of each layer in the target language model are knocked out respectively to determine the performance of the multi-head attention module and the feedforward network model.

3. The knowledge conflict elimination method based on conflicting attention head pruning according to claim 2 is characterized in that: The performance of the feedforward network module includes: constructing a semantic representation of the input sequence; the performance of the multi-head attention module includes: passing the processed information to the last text unit.

4. The knowledge conflict elimination method based on conflicting attention head pruning according to claim 3 is characterized in that: After respectively performing activation value knockout on the multi-head attention module and the feedforward network module of each layer in the target language model and respectively determining the multi-head attention module performance and the feedforward network model performance, the method further includes: Determine the time point at which the multi-head attention module transfers the processed information to the last text unit and the position of the last text unit.

5. The knowledge conflict elimination method based on conflicting attention head pruning according to claim 1, characterized in that: The method replaces the original activation state of each attention head with the interference activation state to determine the output change of each attention head, including: Running the target language model based on the original input to obtain an original output result output by the target language model; When the original activation state of each attention head is replaced with the interference activation state one by one, the target language model is run based on the original input to obtain the interference output result corresponding to each attention head output by the target language model; Based on comparing the interference output results corresponding to each attention head with the original output results, the output change amount of each attention head is determined.

6. The knowledge conflict elimination method based on conflicting attention head pruning according to claim 1, characterized in that: The performing attention pruning on the conflicting attention heads to eliminate the knowledge conflict of the target language model includes: The weight of the conflicting attention head is set to a predetermined value to ignore the influence of the conflicting attention head in the target language model.

7. A knowledge conflict elimination device based on conflict attention head pruning, characterized in that: include: An acquisition module, configured to acquire an original input and an interference input, wherein a first output result of a target language model based on the original input and a second output result based on the interference input have a knowledge conflict; an original activation module, configured to run the target language model based on the original input and determine an original activation state of each attention head in all attention heads in the target language model; an interference activation module, configured to run the target language model based on the interference input and determine an interference activation state of each attention head in all attention heads in the target language model; A replacement module is used to replace the original activation state of each attention head with the interference activation state, and determine the output change of each attention head; A determination module, configured to determine a proxy importance score of each attention head based on an output change of each attention head; A sorting module, used for sorting based on the proxy importance score of each attention head, and taking the attention head with the target sequence number as the conflicting attention head, wherein the conflicting attention head is a context head; A pruning module is used to perform attention pruning on the conflicting attention heads to eliminate the knowledge conflict of the target language model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for eliminating knowledge conflicts based on conflicting attention head pruning as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for eliminating knowledge conflicts based on conflicting attention head pruning as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for eliminating knowledge conflicts based on conflicting attention head pruning as described in any one of claims 1 to 6 is implemented.

Images