Fine-grained knowledge editing method based on neuron attribution and dynamic mask

Abstract

The invention provides a fine-grained knowledge editing method based on neuron attribution and dynamic masks, and the method comprises the steps: obtaining a large language model and target knowledge needing to be newly learned, and editing a plurality of knowledge templates for the target knowledge; based on the plurality of knowledge templates, performing neuron-level attribution quantization on the multi-layer feed-forward neural network of the large language model to obtain an importance score of each neuron in each layer of feed-forward neural network for prediction of each knowledge template; dynamically constructing a mask vector of each layer of feed-forward neural network according to the importance score predicted by each neuron for each knowledge template in each layer of feed-forward neural network, the mask vector indicating the neuron needing knowledge updating and the neuron not needing knowledge updating in the feed-forward neural network; a plurality of knowledge templates and mask vectors are utilized to train the large language model, and during training, only neurons needing knowledge updating in each layer of feedforward neural network in the large language model are updated, so that the precision of lifelong knowledge editing of the model is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of natural language processing, specifically to the field of large language model knowledge editing, and more specifically to a fine-grained knowledge editing method based on neuron attribution and dynamic masking. BACKGROUND

[0002] In recent years, large language models (LLMs) have become the core technology paradigm in the field of artificial intelligence, providing an effective way for model maintenance and iteration through continuous dynamic knowledge injection, error correction, and sensitive content removal, as described in references [1]-[3]. For example, the knowledge of "the year of the next Olympic Games" in the LLM can be updated from 2024 to 2028 without complete retraining. An ideal lifelong knowledge editing method needs to meet the dual requirements of high editing accuracy and preservation of model generalization ability, where lifelong knowledge editing refers to continuously, flexibly, and efficiently updating the internal knowledge of the model without retraining the entire large language model.

[0003] Existing knowledge editing methods can be mainly divided into the following two categories:

[0004] Methods that integrate external parameters (i.e., external parameter editing methods): These methods require additional parameter modules for storing new knowledge without modifying the internal parameters of the current large language model. External parameter editing methods are widely used due to their structural flexibility and ease of implementation. Typical examples include the SERAC method in reference [4], which implements input routing through a cache counterfactual model and scope classifier, the GRACE method in reference [5], which introduces a discrete key-value adapter, the Melo method in reference [6], which proposes a semantic clustering low-rank adaptation module, and the WISE method in reference [7], which uses a dual-memory architecture and a trainable routing mechanism. However, these methods rely on memory indexing or parameter duplication, and as the number of edits increases, they face increasing resource overhead and gradually declining editing accuracy.

[0005] Methods that directly modify internal parameters (internal parameter editing methods): These methods modify the internal weight parameters of the model directly through constrained updates to achieve knowledge updates. From early methods such as the MEND method in reference [8] and the KE method in reference [9], which use meta-learning methods to generate task-specific parameter updates using super networks, to the development of "position-editing" paradigms in references

[10] -

[11] and the AlphaEdit method in reference

[12] , which uses zero-space projection to achieve precise updates, these methods have gradually improved the explainability and architectural simplicity of editing. However, these methods generally operate at the layer or parameter block level, which can easily interfere with unrelated neurons, leading to degradation of model generalization ability.

[0006] Based on the above analysis, as the number of knowledge updates increases, the existing internal parameter editing method will damage the LLM internal parameter structure and cause LLM knowledge forgetting due to its coarse-grained updating method for LLM internal parameters. Moreover, in the lifelong editing scenario, the errors of both the existing external parameter editing method and the internal parameter editing method will accumulate exponentially with continuous editing operations. Therefore, there is an urgent need for a method that can alleviate the decline in editing accuracy and generalization ability caused by error accumulation in the lifelong editing process of large language models, in order to improve their continuous editing ability.

[0007] It should be noted that the background art is only used to introduce the relevant information of the present application, in order to help understand the technical solutions of the present application, but does not mean that the relevant information must be prior art. The relevant information is submitted and disclosed together with the present application scheme, and in the absence of evidence showing that the relevant information has been publicly disclosed before the filing date of the present application, the relevant information should not be considered as prior art.

[0008] The references are as follows:

[0009] [1] Zhang N, Yao Y, Tian B, et al. A comprehensive study of knowledgeediting for large language models[J]. arXiv preprint arXiv:2401.01286, 2024.

[0010] [2] Yao Y, Wang P, Tian B, et al. Editing large language models:Problems, methods, and opportunities[J]. arXiv preprint arXiv:2305.13172,2023.

[0011] [3] Wang S, Zhu Y, Liu H, et al. Knowledge editing for large languagemodels: A survey[J]. ACM Computing Surveys, 2024, 57(3): 1-37.

[0012] [4] Mitchell E, Lin C, Bosselut A, et al. Memory-based model editing at scale[C] / / International Conference on Machine Learning. PMLR, 2022: 15817-15831.

[0013] [5] Hartvigsen T, Sankaranarayanan S, Palangi H, et al. Aging with grace: Lifelong model editing with discrete key-value adaptors[J]. Advances in Neural Information Processing Systems, 2023, 36: 47934-47959.

[0014] [6] Yu L, Chen Q, Zhou J, et al. Melo: Enhancing model editing with neuron-indexed dynamic lora[C] / / Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(17): 19449-19457.

[0015] [7] Wang P, Li Z, Zhang N, et al. Wise: Rethinking the knowledge memory for lifelong model editing of large language models[J]. Advances in Neural Information Processing Systems, 2024, 37: 53764-53797.

[0016] [8] Mitchell E, Lin C, Bosselut A, et al. Fast model editing at scale[J]. arXiv preprint arXiv:2110.11309, 2021.

[0017] [9] De Cao N, Aziz W, Titov I. Editing factual knowledge in language models[J]. arXiv preprint arXiv:2104.08164, 2021.

[0018]

[10] Meng K, Bau D, Andonian A, et al. Locating and editing factual associations in gpt[J]. Advances in neural information processing systems, 2022, 35: 17359-17372.

[0019]

[11] Meng K, Sharma A S, Andonian A, et al. Mass-editing memory in a transformer[J]. arXiv preprint arXiv:2210.07229, 2022.

[0020]

[12] Fang J, Jiang H, Wang K, et al. Alphaedit: Null-space constrained knowledge editing for language models[J]. arXiv preprint arXiv:2410.02355, 2024. SUMMARY

[0021] Therefore, the purpose of the present application is to overcome the defects of the prior art described above, and to provide a fine-grained knowledge editing method based on neuron attribution and dynamic mask.

[0022] The purpose of the present application is achieved by the following technical solutions:

[0023] According to a first aspect of the present invention, a fine-grained knowledge editing method based on neuron attribution and dynamic masking is provided, comprising: S1, acquiring a large language model and the target knowledge to be newly learned therein, and editing multiple knowledge templates for the target knowledge; S2, performing neuron-level attribution quantization on a multi-layer feedforward neural network of the large language model based on the multiple knowledge templates to obtain an attribution score matrix, which includes the importance score of each neuron in each layer of the feedforward neural network for each knowledge template; S3, dynamically constructing a mask vector for each layer of the feedforward neural network according to the importance score of each neuron in each layer of the feedforward neural network for each knowledge template, the mask vector indicating the neurons in the feedforward neural network that need to be updated and the neurons that do not need to be updated; S4, training the large language model using the multiple knowledge templates and the mask vector to obtain a trained large language model, wherein during training, knowledge updates are performed only on the neurons in each layer of the feedforward neural network in the large language model that need to be updated.

[0024] In some embodiments of the present invention, in S2, the quantification method of the importance score predicted by each neuron for each knowledge template includes: inputting the knowledge template into a large language model to obtain a first activation vector output by each layer of the feedforward neural network and a sub-value vector output by the neuron in that layer of the feedforward neural network; perturbing each output sub-value vector based on the first activation vector to obtain a second activation vector; calculating a first probability of outputting target knowledge under the first activation vector and a second probability of outputting target knowledge under the second activation vector through the final output layer of the large language model; and obtaining the importance score of the neuron based on the difference between the second probability and the first probability.

[0025] In some embodiments of the present invention, in S2, the importance score of each neuron in each layer of the feedforward neural network for each knowledge template prediction is quantified as follows;

[0026] ,

[0027] in, This represents the importance score of the i-th neuron in the feedforward neural network for knowledge template prediction. This indicates that the final output layer of the large language model is based on The output target knowledge after calculation The second probability, Representing target knowledge, , This represents the second activation vector. This represents the first activation vector output by the feedforward neural network layer after the knowledge template is input into the large language model. This represents the disturbance amplitude factor. represents a sub-value vector of an i-th neuron output in the layer of the feedforward neural network after inputting the knowledge template into the large language model, represents a first probability of the output target knowledge after the calculation.

[0028] In some embodiments of the present application, in the S3, the mask vector comprises mask values corresponding to all neurons in the feedforward neural network, wherein the mask values corresponding to all neurons are set by using a generality analysis method to obtain the mask vector, and the generality analysis method comprises: calculating a knowledge generality score of each neuron, wherein the knowledge generality score of each neuron is the sum of the importance scores predicted by the neuron for each knowledge template; when the knowledge generality score of the neuron is greater than or equal to a first preset threshold corresponding to the layer of the feedforward neural network, the mask value corresponding to the neuron is set to a first value, otherwise, the mask value corresponding to the neuron is set to a second value, and the first value indicates that the corresponding neuron needs to be updated, and the second value indicates that the corresponding neuron does not need to be updated.

[0029] In some embodiments of the present application, in the S3, the mask vector obtained by setting the mask values corresponding to all neurons by using the generality analysis method is adjusted to obtain an adjusted mask vector, wherein the adjustment method comprises: determining a knowledge specificity score of each neuron, wherein the knowledge specificity score of each neuron is the highest importance score predicted by each neuron for each knowledge template; when the knowledge specificity score of the neuron is greater than or equal to a second preset threshold corresponding to the layer of the feedforward neural network, the mask value corresponding to the neuron is set to the first value; when the knowledge generality score of the neuron is less than the first preset threshold and the knowledge specificity score of the neuron is less than the second preset threshold, the mask value corresponding to the neuron is set to the second value.

[0030] In some embodiments of the present application, the setting method of the first preset threshold comprises: processing the importance scores predicted by each neuron in each layer of the feedforward neural network for each knowledge template by using a Softmax activation function to obtain a first probability distribution corresponding to each neuron under each knowledge template; calculating an average normalized entropy value according to the first probability distribution corresponding to all neurons in the layer of the feedforward neural network under each knowledge template, and calculating a first selection proportion of the neuron according to the average normalized entropy value; and determining the first preset threshold according to the first selection proportion and the knowledge generality score of all neurons in the layer of the feedforward neural network.

[0031] ​In some embodiments of the present application, the second preset threshold is set in the following manner: based on the knowledge-specific scores of all neurons in each layer of the feedforward neural network, the knowledge-specific scores of the neurons are normalized to obtain a second probability distribution corresponding to each neuron; based on the second probability distributions corresponding to all neurons in the layer of the feedforward neural network, a distribution entropy value is calculated, and a second selection ratio of the neurons is calculated based on the distribution entropy value; and based on the second selection ratio and the knowledge-specific scores of all neurons in the layer of the feedforward neural network, the second preset threshold is determined.

[0032] According to a second aspect of the present application, a knowledge question and answer method based on a large language model is provided, which comprises: using the trained large language model obtained by the method of the first aspect of the present application, outputting a target knowledge answer text according to an input knowledge prompt text question related to the target knowledge.

[0033] According to a third aspect of the present application, an electronic device is provided, which comprises: one or more processors; and a memory for storing executable instructions; the one or more processors are configured to implement the steps of the method of the first aspect and the method of the second aspect of the present application by executing the executable instructions.

[0034] Compared with the prior art, the present application has the following advantages:

[0035] On the one hand, the large language model is trained using multiple knowledge templates under the same target knowledge, which increases the generalization ability of the LLM. On the other hand, based on the importance scores of each neuron in each layer of the feedforward neural network of the evaluated large language model for predicting each knowledge template, the neurons that need to be updated and the neurons that do not need to be updated in the feedforward neural network are determined, so that the present application realizes a fine-grained parameter updating method at the neuron level, i.e., only the neurons more related to the target knowledge are updated, which significantly reduces the interference with the original knowledge and general ability of the model, thereby alleviating the model knowledge forgetting and collapse. The present application not only retains the original knowledge but also learns new knowledge, which can effectively improve the effect and editing accuracy of LLM lifelong knowledge editing. BRIEF DESCRIPTION OF DRAWINGS

[0036] The embodiments of the present application will be further described below with reference to the accompanying drawings, in which:

[0037] Figure 1 The figure is a flowchart of the knowledge editing method according to the embodiments of the present application;

[0038] Figure 2 The figure is a schematic diagram of the knowledge editing process of the large language model according to the embodiments of the present application;

[0039] Figure 3 The figure is a schematic diagram of the comparison results of the generalization ability according to the embodiments of the present application. DETAILED DESCRIPTION

[0040] For the purposes of the present application, the technical solutions and advantages will be clearer. The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0041] As mentioned in the background section, with the increasing number of knowledge updates, the existing internal parameter editing method will damage the LLM internal parameter structure and cause the LLM knowledge forgetting due to the coarse-grained updating method of the LLM internal parameters. And in the lifelong editing scenario, the errors of both the existing external parameter editing method and the internal parameter editing method will accumulate exponentially with the continuous editing operation.

[0042] To solve the above problems, the inventors propose a fine-grained knowledge editing method based on neuron attribution and dynamic masking. On the one hand, the present application acquires the target knowledge to be newly learned and edits multiple knowledge templates for the target knowledge. The multiple knowledge templates under the same target knowledge are used to train the large language model, which increases the generalization ability of the LLM. On the other hand, the present application determines the neurons in the feedforward neural network that need to be updated and the neurons that do not need to be updated according to the importance scores of each neuron in each layer of the feedforward neural network of the evaluated large language model for predicting each knowledge template. The present application realizes a fine-grained parameter updating method at the neuron level, i.e., more accurately locates the neurons more relevant (i.e., more important) to the target knowledge, and only updates the neurons more relevant to the target knowledge, which significantly reduces the interference to the original knowledge and general ability of the model, thereby alleviating the model knowledge forgetting and collapse. The present application preserves the original knowledge and learns new knowledge during the editing process, which can effectively alleviate the problem of decline in editing accuracy and generalization ability caused by error accumulation during the editing process, and improve the effect of lifelong knowledge editing of the LLM.

[0043] According to one embodiment of the present application, when facing multiple types of target knowledge to be newly learned by the large language model, multiple knowledge editing of the large language model is needed. For each type of target knowledge to be newly learned, the knowledge editing method of the present application is used to edit the knowledge of the large language model. Referring to Figure 1 which is a flowchart of the knowledge editing method of the present application. The method includes steps S1, S2, S3 and S4. In order to better understand the present application, each step will be described in detail below with reference to specific embodiments.

[0044] Step S1, acquiring a large language model and its target knowledge to be newly learned, and editing multiple knowledge templates for the target knowledge.

[0045] According to one embodiment of the present application, each target knowledge to be newly learned is edited as a sample, knowledge editing is performed based on a sample, and a plurality of knowledge templates corresponding to the sample are obtained. For example, a target knowledge to be newly learned is edited as the date of the next Olympic Games is 2028, and the knowledge templates edited for the target knowledge include: 1. The next Olympic Games is planned to be held in 2028, 2. The next Olympic Games will be held in 2028, 3. The date of the next Olympic Games is 2028, and the like. The technical solution of this embodiment can at least achieve the following beneficial technical effects: for the same target knowledge, a plurality of knowledge templates with different expression forms are constructed to train the large language model, which can effectively increase the generalization ability of the LLM.

[0046] In step S2, neuron-level attribution quantization is performed on the multi-layer feedforward neural network of the large language model based on the plurality of knowledge templates, and an attribution score matrix is obtained, which includes the importance scores of each neuron in each layer of the feedforward neural network for predicting each knowledge template.

[0047] According to one embodiment of the present application, the attribution score matrix is a two-dimensional matrix, and the attribution score matrix refers to: for a plurality of knowledge templates of each target knowledge, the matrix obtained by calculating the importance scores of each neuron for predicting each knowledge template. For example, the multi-layer feedforward neural network of the LLM includes 4096 neurons, and 3 knowledge templates are edited for the target knowledge, and the corresponding attribution score matrix has a dimension of 3x4096.

[0048] According to one embodiment of the present application, in step S2, the quantization manner of the importance scores of each neuron for predicting each knowledge template includes the following steps a1, a2 and a3:

[0049] Step a1: input the knowledge template into the large language model to obtain a first activation vector output by each layer of the feedforward neural network and a sub-value vector output by the neuron in the layer of the feedforward neural network.

[0050] According to one embodiment of the present application, the first activation vector output by the layer of the feedforward neural network after inputting the knowledge template into the large language model is denoted as , , and if the layer of the feedforward neural network has m neurons, then , , and

[0051] Step a2: based on the first activation vector, perturbation processing is performed on each output sub-value vector to obtain a second activation vector.

[0052] According to one embodiment of the present application, the perturbation processing can be performed in the following manner:

[0053] ,

[0054] wherein, represents the second activation vector, represents the perturbation amplitude factor, represents the sub-value vector output by the i-th neuron in the layer of the feedforward neural network after inputting the knowledge template into the large language model. The technical solution of the embodiment can at least achieve the following beneficial technical effects: the influence of the neuron on the output of the entire feedforward neural network is amplified by the perturbation amplitude factor. It can also be understood that the sub-value vector output by the neuron is the degree of contribution to the current knowledge. If the neuron is more sensitive to the current knowledge, the degree of contribution is greater, and the value of the perturbation is greater. That is, the perturbation processing method can better evaluate the importance score in the subsequent step.

[0055] Step a3, through the last output layer of the large language model, the first probability of outputting the target knowledge under the first activation vector and the second probability of outputting the target knowledge under the second activation vector are calculated, and based on the difference between the second probability and the first probability, the importance score of the neuron is obtained.

[0056] According to an embodiment of the present application, the quantification method of the importance score predicted by each neuron in each layer of the feedforward neural network for each knowledge template can adopt the following method:

[0057] ,

[0058] wherein, represents the importance score predicted by the i-th neuron of the layer of the feedforward neural network for the knowledge template, represents the second probability of outputting the target knowledge after calculation based on by the last output layer of the large language model, represents the target knowledge, represents the first probability of outputting the target knowledge after calculation based on by the last output layer of the large language model.

[0059] According to an embodiment of the present application, the quantification method of the importance score predicted by each neuron in each layer of the feedforward neural network for each knowledge template can also adopt the following method:

[0060] .

[0061] ​​​​The technical solutions of the above embodiments can at least achieve the following beneficial technical effects: by weighting and perturbing the activation vector of each neuron, and measuring the values ​​before and after the perturbation... and The difference in the predicted probability of the target knowledge between the two neurons is used to determine the importance of the neuron to the target knowledge. This method of importance analysis can more accurately locate neurons that are more relevant to the target knowledge.

[0062] Step S3: Based on the importance scores of each neuron in each layer of the feedforward neural network for each knowledge template prediction, dynamically construct the mask vector of each layer of the feedforward neural network. The mask vector indicates which neurons in the feedforward neural network need to be updated and which do not need to be updated.

[0063] According to an embodiment of the present invention, in step S3, the mask vector includes mask values ​​corresponding to each neuron in the feedforward neural network. When the mask value is a first value, it indicates that the corresponding neuron needs to perform knowledge updates. When the mask value is a second value, it indicates that the corresponding neuron does not need to perform knowledge updates.

[0064] According to one embodiment of the present invention, the mask vector of each layer of the feedforward neural network is constructed in the following ways:

[0065] Construction Method 1

[0066] According to one embodiment of the present invention, a universal analysis method is used to set the mask values ​​corresponding to each neuron to obtain a mask vector. The universal analysis method includes the following steps b1 and b2:

[0067] Step b1: Calculate the general knowledge score for each neuron, where the general knowledge score for each neuron is the sum of the importance scores of that neuron for each knowledge template prediction.

[0068] According to one embodiment of the present invention, if three knowledge templates are edited for the target knowledge, the importance scores predicted by the neuron for the three knowledge templates are summed to obtain the general knowledge score of the neuron.

[0069] Step b2: When the general knowledge score of a neuron is greater than or equal to the first preset threshold corresponding to that layer of the feedforward neural network, set the mask value corresponding to the neuron to the first value; otherwise, set it to the second value.

[0070] According to an embodiment of the present invention, in step b2, the setting of the first preset threshold includes steps b21, b22, and b23:

[0071] Step b21: processing the importance score of each neuron in each layer of the feedforward neural network for predicting each knowledge template by using a Softmax activation function to obtain a first probability distribution corresponding to each neuron under each knowledge template.

[0072] According to an embodiment of the present application, the processing of the importance score of each neuron under each knowledge template by using a Softmax activation function is as follows:

[0073] ,

[0074] wherein, represents the first probability distribution obtained by Softmax processing of the importance score of the i-th neuron in the j-th layer of the feedforward neural network under the k-th knowledge template, represents the importance score of the i-th neuron in the j-th layer of the feedforward neural network under the k-th knowledge template, represents the importance score of the i-th neuron in the j-th layer of the feedforward neural network under the k-th knowledge template. represents the first probability distribution obtained by Softmax processing of the importance score of the i-th neuron in the j-th layer of the feedforward neural network under the k-th knowledge template, represents the knowledge template number, represents the neuron number, and softmax() represents a Softmax activation function, represents a temperature parameter for adjusting the score difference, represents the importance score of the i-th neuron in the j-th layer of the feedforward neural network under the k-th knowledge template.

[0075] Step b22: calculating an average normalized entropy value according to the first probability distribution corresponding to each neuron under each knowledge template in the layer of the feedforward neural network, and calculating a first selection ratio of the neuron according to the average normalized entropy value.

[0076] According to an embodiment of the present application, the average normalized entropy value can be calculated in the following manner:

[0077] ,

[0078] wherein, represents the average normalized entropy value, represents the total number of knowledge templates for the target knowledge editor, represents the total number of neurons in the layer of the feedforward neural network.

[0079] According to an embodiment of the present application, the first selection ratio can be calculated in the following manner:

[0080] ,

[0081] wherein, represents the first selection ratio, represents a constant scaling factor,​​​​ This indicates the bias term.

[0082] Step b23: Determine the first preset threshold based on the first selection ratio and the knowledge common score of all neurons in the feedforward neural network of this layer.

[0083] According to one embodiment of the present invention, after arranging the general knowledge scores of all neurons in the feedforward neural network in descending order, the neurons in the next layer are selected and arranged in order of their general knowledge scores. The knowledge of quantiles is generally used as the first preset threshold. . No. Quantile: Represents the general knowledge score corresponding to the position corresponding to the product of the first selection ratio and the total number of neurons in that layer of the feedforward neural network. For example... =1 / 4, the total number of neurons in this feedforward neural network is 100, so the knowledge general score ranked 25th is selected.

[0084] According to one embodiment of the present invention, neurons with a general knowledge score greater than or equal to a first preset threshold are considered as target knowledge-general neurons, and the mask value corresponding to the target knowledge-general neurons is set to a first value; otherwise, it is set to a second value. Target knowledge-general neurons show positive contributions under different knowledge templates, exhibiting high predictive generality for target knowledge. The technical solution of this embodiment can at least achieve the following beneficial technical effects: filtering out neurons with high importance scores for each knowledge template prediction, and updating more important neurons.

[0085] Construction Method Two

[0086] While some neurons may not perform well in terms of general knowledge scores, they may exhibit low importance scores for predicting one knowledge template of the target knowledge but high scores for predicting another. These neurons demonstrate a high correlation with the target knowledge, and using only generality analysis for mask vector construction would ignore them. Therefore, according to an embodiment of the present invention, in step S3, the mask vector obtained by setting the mask values ​​corresponding to all neurons using generality analysis is adjusted to obtain an adjusted mask vector. The adjustment method includes steps c1, c2, and c3:

[0087] Step c1: Determine the knowledge-specific score for each neuron, where the knowledge-specific score for each neuron is the highest importance score predicted by each neuron for each knowledge template.

[0088] According to one embodiment of the present invention, if three knowledge templates are edited for the target knowledge, the highest score predicted by the neuron for the importance of the three knowledge templates is selected as the knowledge-specific score of the neuron.

[0089] Step c2: When the knowledge-specific score of a neuron is greater than or equal to the second preset threshold corresponding to the feedforward neural network layer, set the mask value corresponding to the neuron to the first value.

[0090] According to one embodiment of the present invention, the setting method of the second preset threshold includes steps c21, c22 and c23:

[0091] Step c21: Based on the knowledge-specific scores of all neurons in each layer of the feedforward neural network, normalize the knowledge-specific scores of each neuron to obtain the second probability distribution corresponding to each neuron.

[0092] According to one embodiment of the present invention, the normalization process can be performed as follows:

[0093] ,

[0094] in, Indicates the first The second probability distribution corresponding to each neuron Indicates the first Each neuron has its own knowledge-specific score. This represents the sum of the knowledge-specific scores of all neurons in this layer of the feedforward neural network.

[0095] Step c22: Calculate the distribution entropy value based on the second probability distribution corresponding to each neuron in this layer of the feedforward neural network, and calculate the second selection ratio of the neurons based on the distribution entropy value.

[0096] According to one embodiment of the present invention, the distribution entropy value can be calculated in the following manner:

[0097] ,

[0098] in, This represents the distribution entropy value.

[0099] According to one embodiment of the present invention, the second selection ratio can be calculated in the following manner:

[0100] ,

[0101] in, This indicates the proportion of the second choice. Indicates a constant scaling factor. This indicates the bias term.

[0102] Step c23: Determine the second preset threshold based on the second selection ratio and the knowledge-specific scores of all neurons in the feedforward neural network of this layer.

[0103] According to one embodiment of the present invention, after arranging the knowledge-specific scores of all neurons in the feedforward neural network in descending order, the neurons in the next layer are selected and arranged in the order of knowledge-specific scores. The knowledge-specific score of quantiles is used as the second preset threshold. . No. Quantile: Represents the knowledge-specific score corresponding to the position corresponding to the product of the second selection ratio and the total number of neurons in that layer of the feedforward neural network. For example... =1 / 2, the total number of neurons in this layer of the feedforward neural network is 100, so the knowledge-specific score ranked 50th is selected as the second preset threshold.

[0104] Step c3: When the general knowledge score of a neuron is less than the first preset threshold and its knowledge-specific score is less than the second preset threshold, set the mask value corresponding to the neuron to the second value.

[0105] According to one embodiment of the present invention, when the general knowledge score of a neuron is greater than or equal to a first preset threshold corresponding to the feedforward neural network layer, the mask value corresponding to the neuron is set to a first value. Neurons with a knowledge-specific score greater than or equal to a second preset threshold are considered as target knowledge-specific neurons. Target knowledge-specific neurons exhibit high positive contributions under certain specific knowledge templates and have high predictive relevance to the target knowledge. Therefore, when the knowledge-specific score of a neuron is greater than or equal to the second preset threshold corresponding to the feedforward neural network layer, the mask value corresponding to the neuron is set to the first value. When the general knowledge score of a neuron is less than the first preset threshold and its knowledge-specific score is less than the second preset threshold, the mask value corresponding to the neuron is set to the second value. The technical solution of this embodiment can at least achieve the following beneficial technical effects: it can both filter out neurons with high general knowledge scores for the target knowledge and identify target knowledge-specific neurons that selectively activate specific knowledge expressions through knowledge-specific scores. It can more accurately locate neurons that are more relevant (i.e., more important) to the target knowledge, which is beneficial for better subsequent neuron knowledge updates.

[0106] According to one embodiment of the present invention, the mask value in the mask vector corresponding to this feedforward neural network layer can be set in the following manner:

[0107] ,

[0108] in, Indicates the first The first layer of the feedforward neural network The mask values ​​corresponding to each neuron are 1 for the first value and 0 for the second value. Indicates the first The general knowledge score for each neuron Indicates the first a knowledge-specific score of a neuron.

[0109] It should be noted that the mask vector can be obtained by selecting the first construction mode or the second construction mode according to actual needs.

[0110] Step S4, training the large language model by using the plurality of knowledge templates and the mask vector, to obtain a trained large language model, and only the neurons in each layer of the feedforward neural network in the large language model that need to be updated are updated during the training.

[0111] According to an embodiment of the present application, if the current layer of the feedforward neural network includes 10 neurons, the corresponding mask vector is [1, 1, 0, 0, 0, 1, 0, 1, 0, 0], which belongs to a sparse mask vector, and only the 0th, 1st, 5th and 7th neurons of the layer need to be updated. Among them, for a plurality of new target knowledge, the LLM is edited by steps S1-S4 multiple times to obtain the edited LLM.

[0112] According to an embodiment of the present application, referring to Figure 2 which is a schematic diagram of the principle of the knowledge editing process of the large language model. In the figure, the LLM includes a plurality of large language model layer modules, each of which includes an attention network and a feedforward neural network. The settings before knowledge editing: for example, the old knowledge of the LLM is: the date of the next Olympic Games is 2024. The target knowledge to be learned is set as: the date of the next Olympic Games is 2028, and a plurality of knowledge templates are constructed for the new target knowledge, wherein the first knowledge template input to the LLM is: In which year will the next Olympic Games be held? If the 1st the knowledge in the 1st the knowledge in the 1st the knowledge in the 1st the knowledge in the 1st the knowledge in the 1st the knowledge in the 1st

[0113] Among them, the knowledge editing process: according to the input knowledge template, the neurons in the three-layer feedforward neural network are attributed and quantized at the neuron level, to obtain the importance score of each neuron in predicting the knowledge template, and the importance score ranges from [-1, 1]. According to the importance score of each neuron for the plurality of knowledge templates of the target knowledge, the process of step S3 of the above embodiment is adopted, and the target knowledge-general neuron and the target knowledge-specific neuron that need to be updated are selected by the second construction mode, and the 1st the 1st the 1st The mask vector of the respective neuron level of each layer masks the neurons that do not need to be updated according to the mask vector, and the neurons subjected to the mask processing are taken as the masked neurons.

[0114] According to an embodiment of the present application, a knowledge question and answer method based on a large language model is provided, which includes: using the trained large language model obtained by the knowledge editing method in the above embodiment, outputting the target knowledge answer text according to the input knowledge prompt text question related to the target knowledge. Illustratively, the LLM old knowledge is that the date of the next Olympic Games is 2024, and after knowledge editing, it is updated to 2028. At this time, the question is asked: when is the date of the next Olympic Games? The LLM answer is updated to 2028.

[0115] In order to verify the beneficial effects of the present application, the inventors conducted the following experiments:

[0116] 1) ZsRE dataset and CounterFact dataset are used as knowledge editing performance benchmarks. ZsRE dataset source: Zero-Shot Relation Extraction via Reading Comprehension; Counter-Fact dataset source: Locating and Editing Factual Associations in GPT.

[0117] 2) Seven existing methods are selected for comparison experiments with the method of the present application (NMKE), including FT method, KN method, ROME method, MEMIT method, PMET method, WISE method and AlphaEdit method. The comparison experiments include the following comparison mode 1 and comparison mode 2. Among them, the seven existing methods are as follows:

[0118] FT method: Zhu C, Rawat A S, Zaheer M, et al. Modifying memories in transformer models[J]. arXiv preprint arXiv:2012.00363, 2020.

[0119] KN method: Dai D, Dong L, Hao Y, et al. Knowledge neurons in pretrained transformers[J]. arXiv preprint arXiv:2104.08696, 2021.

[0120] ROME method: reference

[10] in the background art;

[0121] MEMIT method: reference

[11] in the background art;

[0122] PMET method: Li X, Li S, Song S, et al. Pmet: Precise model editing in a transformer [C] / / Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(17): 18564-18572.

[0123] WISE method: Wang P, Li Z, Zhang N, et al. Wise: Rethinking the knowledge memory for lifelong model editing of large language models [J]. Advances in Neural Information Processing Systems, 2024, 37: 53764-53797.

[0124] AlphaEdit method: reference

[12] in the background art.

[0125] Comparison mode 1

[0126] Select five cross-domain tasks to compare the generalization ability of the FT method, the KN method, the ROME method, the MEMIT method, the PMET method, the AlphaEdit method and the method of the present application (referred to as NMKE). The five cross-domain tasks are MMLU tasks, GSM8K tasks, CommonsenseQA tasks, BBH-Zeroshot tasks and HumanEval tasks.

[0127] The generalization ability comparison results are shown in Table 1. Figure 3 Figure 3 ​The plurality of coordinate graphs include a horizontal coordinate of knowledge editing times T. The first four coordinate graphs in the first row are the accuracy experimental results of each existing method and the method of the application on the MMLU task, the GSM8K task, the CommonsenseQA task and the BBH-Zeroshot task under the ZsRE data set, and the vertical coordinate of each coordinate graph is the accuracy. The last coordinate graph in the first row is the Pass@k result of the HumanEval task under the ZsRE data set, and the vertical coordinate is Pass@k, which is used to represent the probability that the LLM can answer correctly at least once in k opportunities. The first four coordinate graphs in the second row are the accuracy experimental results of each existing method and the method of the application on the MMLU task, the GSM8K task, the CommonsenseQA task and the BBH-Zeroshot task under the CounterFact data set, and the vertical coordinate of each coordinate graph is the accuracy. The last coordinate graph in the second row is the Pass@k result of the HumanEval task under the CounterFact data set, and the vertical coordinate is Pass@k.

[0128] Figure 3 The experimental results show that the traditional method (such as the FT method and the ROME method) has a significant generalization performance decay when the knowledge editing times T>500, the AlphaEdit method reduces the accuracy of GSM8K and HumanEval to 0 when T=1500; while the application (NMKE) still maintains the accuracy of MMLU=0.59 when T=2000, has strong generalization ability, and is improved by an average of 42% compared with the baseline.

[0129] The MMLU task: Hendrycks D, Burns C, Basart S, et al. Measuring massive multitask language understanding[J]. arXiv preprint arXiv:2009.03300, 2020.

[0130] The GSM8K task: Cobbe K, Kosaraju V, Bavarian M, et al. Training verifiers to solve math word problems[J]. arXiv preprint arXiv:2110.14168, 2021.

[0131] CommonsenseQA task: Talmor A, Herzig J, Lourie N, et al. Commonsenseqa: A question answering challenge targeting commonsense knowledge [J]. arXiv preprint arXiv:1811.00937, 2018.

[0132] BBH-Zeroshot task: Suzgun M, Scales N, Schärli N, et al. Challenging big-bench tasks and whether chain-of-thought can solve them [J]. arXiv preprint arXiv:2210.09261, 2022.

[0133] HumanEval task: Chen M, Tworek J, Jun H, et al. Evaluating large language models trained on code [J]. arXiv preprint arXiv:2107.03374, 2021.

[0134] Comparison method 2

[0135] Experiments are conducted on the LLaMA3-8B-Instruct architecture with cumulative editing times T e {10, 100,..., 2000}. The success rate of editing (Rel.), prompt generalization (Gen.), and knowledge retention (Loc.) are used to evaluate each method. The evaluation results on the ZsRE dataset are shown in Table 1. Similarly, the evaluation results on the Counter-Fact dataset are shown in Table 2. The best results in Tables 1 and 2 are shown in bold, and the second best results are underlined.

[0136] LLaMA3-8B-Instruct architecture: Grattafiori A, Dubey A, Jauhri A, et al. The llama 3 herd of models [J]. arXiv preprint arXiv:2407.21783, 2024.

[0137] Table 1

[0138]

[0139] Table 2

[0140]

[0141] From the evaluation results of the above table 1 and table 2, in 2000 rounds of sequential editing, most of the existing methods (such as the FT method and the ROME method) have a sharp decline in performance after T>100, the editing success rate of the AlphaEdit method on the Counter-Fact dataset decreases by 0.78; while the method of the present application (NMKE) maintains an editing success rate of 0.81 and 0.79 on ZsRE and CounterFact datasets respectively by virtue of neuron-level editing update, verifying that sparse modification of the most important neurons of the target knowledge has a great protective effect on model stability.

[0142] In general, the technical solutions of the above embodiments can at least achieve the following beneficial technical effects: compared with the original coarse-grained parameter update method (i.e. updating the parameters of all neurons of one or more layers of neural network), the method of the present application has the following advantages by updating the neurons through dynamic mask vector:

[0143] 1. Adaptive parameter intervention under different target knowledge editing tasks is achieved. That is, different target knowledge is used to dynamically construct different mask vectors to limit the parameter update range of the neural network.

[0144] 2. Reduces the redundant intervention area significantly and protects the original model capability. Only the most important neurons related to the target knowledge are updated, which significantly reduces the interference with the original knowledge and general ability of the model, such as the damage to the general ability of common sense knowledge question answering and calculation of mathematical problems. In addition, the ability to master and understand new target knowledge is significantly increased.

[0145] 3. The parameter update is strictly constrained in the neurons strongly related to the target knowledge, which is easy to explain and can effectively control and alleviate model forgetting and collapse, improve the effect of LLM lifelong knowledge editing (i.e. improve the lifelong editing success rate of the optimizer), and facilitate integration into existing optimization frameworks.

[0146] It should be noted that although the above describes the steps in a specific order, it does not mean that the steps must be performed in the above specific order, in fact, some of the steps can be performed concurrently, or even change the order, as long as the desired function can be achieved.

[0147] The present application can be a system, a method and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions loaded thereon, which are used to cause a processor to implement various aspects of the present application.

[0148] A computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves.

[0149] Embodiments of the application have been described above, with the understanding that these embodiments are exemplary only, and are not restrictive, in terms of the scope of the embodiments disclosed. Many modifications and variations of the described embodiments are possible, in light of the above teachings, without departing from the scope and spirit of the described embodiments. The choice of words in this document is intended to best explain the principles of the embodiments, practical application, or technical improvement in the art, or to enable others skilled in the art to utilize the embodiments disclosed herein.

Claims

1. A fine-grained knowledge editing method based on neuron attribution and dynamic mask, comprising: S1, obtaining a large language model and target knowledge to be newly learned by the large language model, and editing a plurality of knowledge templates for the target knowledge; S2, based on the plurality of knowledge templates, performing neuron-level attribution quantization on a plurality of layers of a feedforward neural network of the large language model to obtain an attribution score matrix, the attribution score matrix including an importance score of each neuron in each layer of the feedforward neural network for predicting each knowledge template; S3, dynamically constructing a mask vector for each layer of the feedforward neural network according to the importance score of each neuron in each layer of the feedforward neural network for predicting each knowledge template, the mask vector indicating neurons in the feedforward neural network that need to be updated with knowledge and neurons in the feedforward neural network that do not need to be updated with knowledge; S4, training the large language model using the plurality of knowledge templates and the mask vector to obtain a trained large language model, and updating knowledge only for neurons in each layer of the feedforward neural network of the large language model that need to be updated with knowledge during the training.

2. The method of claim 1, wherein, In the S2, the quantization manner of the importance score of each neuron for predicting each knowledge template comprises: inputting the knowledge template into the large language model to obtain a first activation vector output by each layer of the feedforward neural network and a sub-value vector output by the neuron in the layer of the feedforward neural network; performing perturbation processing on each output sub-value vector based on the first activation vector to obtain a second activation vector; calculating a first probability of outputting the target knowledge under the first activation vector and a second probability of outputting the target knowledge under the second activation vector through a last output layer of the large language model, and obtaining the importance score of the neuron based on a difference between the second probability and the first probability.

3. The method of claim 1, wherein, In the S2, the quantization manner of the importance score of each neuron in each layer of the feedforward neural network for predicting each knowledge template is as follows: ， in, This represents the importance score of the i-th neuron in the feedforward neural network for knowledge template prediction. This indicates that the final output layer of the large language model is based on The output target knowledge after calculation The second probability, Representing target knowledge, , This represents the second activation vector. This represents the first activation vector output by the feedforward neural network layer after the knowledge template is input into the large language model. This represents the disturbance amplitude factor. This represents the sub-value vector output by the i-th neuron in the feedforward neural network of this layer after the knowledge template is input into the large language model. This indicates that the final output layer of the large language model is based on The output target knowledge after calculation The first probability.

4. The method of claim 1, wherein, In the S3, the mask vector includes mask values corresponding to all neurons in the feedforward neural network, wherein the mask values corresponding to all neurons are set by a generality analysis manner to obtain the mask vector, and the generality analysis manner comprises: calculating a knowledge generality score of each neuron, wherein the knowledge generality score of each neuron is a sum of the importance scores of the neuron for predicting each knowledge template; when the knowledge generality score of the neuron is greater than or equal to a first preset threshold value corresponding to the layer of the feedforward neural network, setting the mask value corresponding to the neuron as a first numerical value, and otherwise, setting the mask value corresponding to the neuron as a second numerical value, the first numerical value indicating that the corresponding neuron needs to be updated with knowledge, and the second numerical value indicating that the corresponding neuron does not need to be updated with knowledge.

5. The method of claim 4, wherein, In the S3, the mask vector obtained by setting the mask values corresponding to all neurons by the generality analysis manner is adjusted to obtain an adjusted mask vector, wherein the adjustment manner comprises: determining a knowledge specificity score of each neuron, wherein the knowledge specificity score of each neuron is a highest importance score of each neuron for predicting each knowledge template; when the knowledge specificity score of the neuron is greater than or equal to a second preset threshold value corresponding to the layer of the feedforward neural network, setting the mask value corresponding to the neuron as the first numerical value. When the knowledge general score of the neuron is less than a first preset threshold and the knowledge specific score of the neuron is less than a second preset threshold, a mask value corresponding to the neuron is set as a second numerical value.

6. The method of claim 4, wherein, The first preset threshold is set in a manner comprising: The Softmax activation function is used to process the importance score of each neuron in each layer of the feedforward neural network for predicting each knowledge template, to obtain a first probability distribution corresponding to each neuron under each knowledge template. The average normalized entropy value is calculated according to the first probability distribution corresponding to each neuron in the layer of the feedforward neural network under each knowledge template, and the first selection proportion of the neuron is calculated according to the average normalized entropy value. The first preset threshold is determined according to the first selection proportion and the knowledge general score of all neurons in the layer of the feedforward neural network.

7. The method of claim 5, wherein, The second preset threshold is set in a manner comprising: The knowledge specific score of each neuron in each layer of the feedforward neural network is normalized to obtain a second probability distribution corresponding to each neuron. The distribution entropy value is calculated according to the second probability distribution corresponding to each neuron in the layer of the feedforward neural network, and the second selection proportion of the neuron is calculated according to the distribution entropy value. The second preset threshold is determined according to the second selection proportion and the knowledge specific score of all neurons in the layer of the feedforward neural network.

8. A knowledge question and answer method based on a large language model, characterized in that, The question and answer method comprises: The trained large language model obtained by the method of any one of claims 1-7 is used to output a target knowledge answer text according to an input knowledge prompt text question related to the target knowledge.

9. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method of any one of claims 1-8.

10. An electronic device, comprising: Comprise: One or more processors; And A memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1-8 by executing the executable instructions.

Images