A processing method and device for parameter editing of a pre-trained large model

Abstract

Embodiments of the present application relate to a processing method and device for parameter editing of a pre-trained large model, the method comprising: recording a pre-trained large model requiring parameter editing as a target model; configuring a weight parameter editing model for each linear layer of the target model as an editor g i ; receiving input-output transformation data set input by a user as a data set D set ; selecting a set of transformation data from the data set D set as a basic sample, setting a specified number of relevant samples and an irrelevant sample for the basic sample, and forming a training data set D tr from all the samples; and training all the editors g i based on the training data set D tr and the target model; after the training of all the editors is completed, performing parameter editing on all the weight parameters w i of the target model based on the data set D set and all the editors g i . The present application can improve the updating flexibility and efficiency of model parameters and reduce the updating cost.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and apparatus for editing parameters of a pre-trained large model. Background Technology

[0002] Pre-trained large language models, also known as pre-trained large models, refer to large language models that have undergone pre-training on massive corpora. Some model parameters of pre-trained large models, similar to knowledge, are time-sensitive and require timely updates. For example, the correct answer to a question x might be answer y at time t, but be changed to answer y+1 at time t+1. * Theoretically, the model parameters should be updated promptly at the beginning of time period t+1 so that the updated y can be generated based on the input x. * .

[0003] Currently, parameter updates for pre-trained large models primarily rely on fine-tuning techniques. However, when the dataset is small, conventional fine-tuning methods are prone to overfitting. In such cases, even if there is only one input x, the output y needs to be adjusted to yi. * It is also necessary to collect multiple additional input-output pairs for this (x, y) condition. * These elements, together, form a large dataset for fine-tuning. In other words, this conventional update method is effective when dealing with a small number of input-output transformations (x, y → y). * When parameter update tasks are triggered by ), the update efficiency is relatively low and the update cost is relatively high. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a method, apparatus, electronic device, and computer-readable storage medium for editing parameters of a pre-trained large model. This invention denotes the pre-trained large model requiring parameter editing as the target model, and assigns a weight parameter editing model, denoted as editor g, to each linear layer of the target model. i ; and the input-output transformation dataset received from the user Then, from the current dataset D set Choose any set of transformed data As the base sample, and with a small number of related samples (x) set up as the base sample. k ,y k ) and an unrelated sample (x) un ,y un The training dataset D consists of all the samples. tr ; and based on the training dataset D tr and target model for all editors gi Perform training; and after training, based on dataset D set and all editors g i For all weight parameters w of the target model i Perform parameter editing on dataset D. set It can contain at least one set of transformed data. The editor g provided in this invention i This is a simple editing model based on a two-layer nonlinear network (each layer consists of fully connected layers, activation layers, and residual connection modules). It features a small number of parameters, high training efficiency, and low training cost. This invention utilizes the trained editor g... i The technical solution for editing the parameters of the target model is a parameter editing scheme based on backpropagation gradient calculation. This scheme has low computational cost, high editing flexibility, high editing efficiency, and low editing cost. It is particularly suitable for handling parameters derived from a small number of input-output transformations (x,y→y). * When a parameter update task is triggered by a problem, using the solution of this invention can improve update flexibility, increase update efficiency, and reduce update costs.

[0005] To achieve the above objectives, a first aspect of the present invention provides a method for editing parameters of a pre-trained large model, the method comprising:

[0006] The pre-trained large model that requires parameter editing is denoted as the target model; the target model contains N w Linear layer, N w The value is a positive integer greater than 1; the overall model parameters of the target model are denoted as θ, and the weight parameters and bias parameters of each linear layer are denoted as w. i b i 1 ≤ level index i ≤ N w w i b i ∈θ; the input vector of each linear layer in the forward prediction process of the target model is denoted as u. i The gradient of the model loss L with respect to the output vector of each linear layer is denoted as δ. i+1 The model loss L is related to each weight parameter w. i The gradient is denoted as The model loss L is assumed to be the negative log-likelihood loss function L. NLL Let L be the loss function, and L = L NLL (θ, y|x)=-log(P) θ (y|x)), where x is the model input, y is the model output, and P θ (y|x) represents the probability that the target model outputs y when the model parameters are the overall model parameters θ and the model input is x;

[0007] Configure a weight parameter for each linear layer of the target model and edit the model; denoted as editor g. i All editors mentioned g i The model structures are all based on two-layer nonlinear networks, each consisting of a fully connected layer, an activation layer, and a residual connection module; all the editors mentioned above are based on two-layer nonlinear networks. i Both are used to process the vector u input to the current editor through a two-layer nonlinear network. i gradient δ i+1 Perform fine-tuning and output the adjusted vector u new,i gradient δ new,i+1 ;

[0008] The input-output transformation dataset that receives user input is denoted as dataset D. set The dataset D set Including N D Group Transformation Data N D It is a configurable positive integer, with a minimum value of 1; 1 ≤ data index j ≤ N D x j Input to the model, For input x j The corresponding latest output tag;

[0009] From the dataset D set Choose any set of transformed data as the base sample; and set a specified number N for the base sample. K One relevant sample and one irrelevant sample, and consisting of the base sample, N K The equivalent samples and the irrelevant samples constitute the training dataset D. tr ; and based on the training dataset D tr The target model applies to all editors g. i Training is performed; the specified number N K This is a preset positive integer, which is less than 10 by default; the basic sample is denoted as... The equivalent sample is denoted as (x k ,y k ), 1 ≤ sample index k ≤ N K The unrelated samples are denoted as (x un ,y un ); the input x of each of the equivalent samples k All are related to the input x of the basic sample. s Related, output label y k All are related to the output labels of the base samples. Relevant; the input x of the irrelevant sample un With the input x of the base sample sIrrelevant, output label y un Also related to the output labels of the base sample Irrelevant;

[0010] After all editors have been trained, based on the dataset D set and all the editors mentioned g i For all the weight parameters w of the target model i Edit the parameters.

[0011] Preferably, each of the editors g i Each module consists of an input splicing module, a first fully connected layer, a first activation layer, a first residual connection module, a second fully connected layer, a second activation layer, a second residual connection module, and an output splitting module.

[0012] The input concatenation module has its input terminal connected to the input terminal of the current editor, and its output terminal connected to the input terminal of the first fully connected layer and the first input terminal of the first residual connection module, respectively. The output terminal of the first fully connected layer is connected to the input terminal of the first activation layer. The output terminal of the first activation layer is connected to the second input terminal of the first residual connection module. The output terminal of the first residual connection module is connected to the input terminal of the second fully connected layer and the first input terminal of the second residual connection module, respectively. The output terminal of the second fully connected layer is connected to the input terminal of the second activation layer. The output terminal of the second activation layer is connected to the second input terminal of the second residual connection module. The output terminal of the second residual connection module is connected to the input terminal of the output splitting module. The output terminal of the output splitting module is connected to the output terminal of the current editor.

[0013] The input concatenation module is used to receive the vector u currently input by the editor. i and the gradient δ i+1 ; and for the vector u i and the gradient δ i+1 The corresponding initial vector z is obtained by concatenating the vectors. 0,i Send to the first fully connected layer and the first residual connection module;

[0014] The model parameters of the first fully connected layer include the scale vector parameter p. 1,i Offset vector parameter q 1,i Low-rank matrix parameters A1 and B1; the scale vector parameter p 1,i and the offset vector parameter q 1,i For each of the editors mentioned g i Private parameters; the low-rank matrix parameters A1 and B1 are for all the editors g i Shared parameters;

[0015] The first fully connected layer is used to adjust the scale vector parameter p. 1,i The offset vector parameter q 1,i The low-rank matrix parameters A1 and B1 are related to the initial vector z. 0,i Perform a fully connected computation to obtain the corresponding hidden vector h. 1,i Send to the first activation layer;

[0016] Wherein, the hidden vector h 1,i For: h 1,i =p 1,i ⊙(A1B1z 0,i +b)+q 1,i ; b is the preset bias parameter, and all the editors mentioned g i The bias parameter b of the first fully connected layer is the same; ⊙ is the Hadamard product operator;

[0017] The first activation layer is implemented based on a class of nonlinear activation functions σ1; the nonlinear activation function σ1 includes at least the ReLU activation function;

[0018] The first activation layer is used to apply the latent vector h according to the nonlinear activation function σ1. 1,i The activation operation is performed to obtain the corresponding activation vector a. 1,i Send to the first residual connection module;

[0019] Wherein, the activation vector a 1,i For: a 1,i =σ1(h 1,i );

[0020] The first residual connection module is used to process the initial vector z according to the residual connection method. 0,i and the activation vector a 1,i The first vector z is obtained by adding elements one by one. 1,i Send to the second fully connected layer;

[0021] Wherein, the first vector z 1,i z 1,i =z 0,i +a 1,i ;

[0022] The model parameters of the second fully connected layer include the scale vector parameter p. 2,i Offset vector parameter q 2,i Low-rank matrix parameters A2 and B2; the scale vector parameter p 2,i and the offset vector parameter q 2,i For each of the editors mentioned g iPrivate parameters; the low-rank matrix parameters A2 and B2 are for all the editors g i Shared parameters;

[0023] The second fully connected layer is used to adjust the scale vector parameter p. 2,i The offset vector parameter q 2,i The low-rank matrix parameter A2 and the low-rank matrix parameter B2 affect the first vector z. 1,i Perform a fully connected computation to obtain the corresponding hidden vector h. 2,i Send to the second activation layer;

[0024] Wherein, the hidden vector h 2,i For: h 2,i =p 2,i ⊙(A2B2z 1,i )+q 2,i ;

[0025] The second activation layer is implemented based on a class of nonlinear activation functions σ²; the nonlinear activation function σ² includes at least the ReLU activation function;

[0026] The second activation layer is used to apply the latent vector h according to the nonlinear activation function σ2. 2,i The activation operation is performed to obtain the corresponding activation vector a. 2,i Send to the second residual connection module;

[0027] Wherein, the activation vector a 2,i For: a 2,i =σ2(h 2,i );

[0028] The second residual connection module is used to process the first vector z according to the residual connection method. 1,i and the activation vector a 2,i The second vector z is obtained by adding elements one by one. 2,i Send to the output splitting module;

[0029] Wherein, the second vector z 2,i z 2,i =z 1,i +a 2,i ;

[0030] The output splitting module is used to split the initial vector z. 0,i The vector u mentioned in i and the gradient δ i+1 Taking the concatenation position as a reference, from the second vector z 2,i Extract the corresponding vector u new,i and the gradient δ new,i+1 And output it.

[0031] Preferably, the training dataset D is used as the basis for... tr The target model applies to all editors g. i The training includes:

[0032] Step 31, from the training dataset D tr Extract the corresponding base sample N K The equivalent samples (x) k ,y k ) and the unrelated samples (x) un ,y un The overall model parameters θ of the target model are backed up and stored as backup parameters θ0; and the current overall model parameters θ are used as the corresponding current model parameters θ0. now ; and by all the editors mentioned g i The editor parameters form the corresponding editor parameter set φ; and all parameters of the editor parameter set φ are initialized to obtain the current parameter set φ. now And set a first counter initialized to 0;

[0033] Wherein, the editor parameter set φ is composed of all the editors g i Private scale vector parameter p 1,i / p 2,i Offset vector parameter q 1,i / q 2,i It consists of shared low-rank matrix parameters A1 / A2 and B1 / B2;

[0034] Step 32, based on the current parameter set φ now For all the editors mentioned g i Configure the editor parameters;

[0035] Step 33, the basic sample x s The target model is input for forward prediction processing, and the vectors u generated during the processing are analyzed. i Cache the data and calculate the corresponding model loss.

[0036] in, The target model has its model parameters as the current model parameters θ. now And the model input is x s In this case, its model output is The probability of;

[0037] Step 34, based on the current model parameters θnow The weight parameters w in i and its corresponding bias parameter b i The vector u i Calculate the corresponding linear layer output vector u i+1 And calculate the model loss L for each vector u. i+1 The gradient is obtained by the corresponding gradient δ. i+1 ; and each of the gradients δ i+1 and its corresponding vector u i Input the corresponding editor g i Fine-tuning is performed to obtain the corresponding vector u. new,i and the gradient δ new,i+1 ; and based on each of the vectors u new,i and its corresponding gradient δ new,i+1 Calculate the new weight gradient And based on each of the weight parameters w i and the corresponding weight gradient Calculate the corresponding weight parameters and the current model parameter θ now The weight parameters w in i Reset to the corresponding weight parameters

[0038] in,

[0039] The vector u i+1 The calculation method is as follows:

[0040] u i+1 =w i u i +b i ;

[0041] The gradient δ i+1 The calculation method is as follows:

[0042]

[0043] The vector u new,i and the gradient δ new,i+1 The calculation method is as follows:

[0044] g i (u i ,δ i+1 )→(u new,i ,δ new,i+1 ), g i (u i ,δ i+1 ) is the editor g i The expression;

[0045] The weight gradient The calculation method is as follows:

[0046] T is the transpose symbol;

[0047] The weight parameters The calculation method is as follows:

[0048]

[0049] Step 35, based on the latest current model parameters θ now The model parameters of the target model are reset; and each of the equivalent samples (x) is... k ,y k ) of x k The target model is input for positive prediction processing, and the corresponding first model loss L is calculated. 1,k ;

[0050] Wherein, the first model loss L 1,k The default is to use the negative log-likelihood loss function L. NLL The loss function is as follows:

[0051]

[0052] The target model is defined with its model parameters being the latest current model parameters θ. now And the model input is x k In this case, the model output is the corresponding y k The probability of;

[0053] Step 36, the irrelevant samples (x) un ,y un ) of x un The target model is input for positive prediction processing, and the corresponding post-edit probability is calculated. Then, based on the backup parameter θ0, the model parameters of the target model are reset; and the irrelevant samples (x) are... un ,y un ) of x un The target model is input for positive prediction processing, and the corresponding pre-edit probability is calculated. And based on the pre-edit probability r pre The probability r after editing post Sum of divergence loss function L KL Calculate the corresponding second model loss L2;

[0054] in, The target model is defined with its model parameters being the latest current model parameters θ.now And the model input is x un In this case, its model output is y un The probability of;

[0055] The target model has its model parameters as the backup parameter θ0 and its model input is x. un In this case, its model output is y un The probability of;

[0056] The second model loss L2 is assumed to use the divergence loss function L. KL The loss function is as follows:

[0057]

[0058] Step 37, by N K The first model loss L 1,k Together with the second model loss L2, they form the corresponding editor loss L. G ;

[0059] Wherein, the editor loss L G for:

[0060]

[0061] λ is a preset balance parameter;

[0062] Step 38, towards causing the editor to lose L G The direction to reach the minimum value is based on the preset first model optimizer for all editors g. i The corresponding current parameter set φ now Perform one round of parameter modulation; and use the modulated editor parameter set as the new current parameter set φ. now ;

[0063] The first model optimizer includes at least the Adam optimizer;

[0064] Step 39: Increment the first counter by 1; and check whether the incremented first counter exceeds a preset counter threshold; if it does not exceed the threshold, return to step 32 to continue training; if it exceeds the threshold, stop training and adjust the current parameter set φ based on the latest value. now For all the editors mentioned g i The editor parameters are fixed, and the training of all editors is confirmed to be complete.

[0065] Preferably, the dataset D is based on set and all the editors mentioned g i For all the weight parameters w of the target modeli Parameter editing includes:

[0066] The model parameters of the target model are set based on the backup parameter θ0; and the dataset D is... set All of the aforementioned transformation data Perform one round of traversal; and during this round of traversal, transform the data of the currently traversed data. As the corresponding current transformation data (x) c ,y c ); and the current transformation data (x) c ,y c ) of x c The target model is input for forward prediction processing, and the vectors u generated during the processing are analyzed. i Cache the data and calculate the corresponding model loss. And according to each of the weight parameters w in the backup parameter θ0 i and its corresponding bias parameter b i The vector u i Calculate the corresponding linear layer output vector u i+1 And calculate the model loss L for each vector u. i+1 The gradient is obtained by the corresponding gradient δ. i+1 ; and each of the gradients δ i+1 and its corresponding vector u i Input the corresponding editor g i Fine-tuning is performed to obtain the corresponding vector u. new,i and the gradient δ new,i+1 ; and based on each of the vectors u new,i and its corresponding gradient δ new,i+1 Calculate the new weight gradient And based on each of the weight parameters w i and the corresponding weight gradient Calculate the corresponding weight parameters And each of the weight parameters w in the current backup parameter θ0 i Reset to the corresponding weight parameters And based on the latest backup parameter θ0, the model parameters of the target model are reset once;

[0067] Wherein, the vector u i+1 The calculation method is as follows:

[0068] u i+1 =w i u i +b i ;

[0069] The gradient δ i+1 The calculation method is as follows:

[0070]

[0071] The vector u new,i and the gradient δ new,i+1 The calculation method is as follows:

[0072] g i (u i ,δ i+1 )→(u new,i ,δ new,i+1 ), g i (u i ,δ i+1 ) is the editor g i The expression;

[0073] The weight gradient The calculation method is as follows:

[0074] T is the transpose symbol;

[0075] The weight parameters The calculation method is as follows:

[0076]

[0077] A second aspect of the present invention provides an apparatus for implementing the processing method for editing parameters of a pre-trained large model as described in the first aspect above. The apparatus includes: a first preprocessing module, a second preprocessing module, a data receiving module, an editor training module, and a model parameter editing module.

[0078] The first preprocessing module is used to denote the pre-trained large model that requires parameter editing as the target model; the target model contains N w Linear layer, N w The value is a positive integer greater than 1; the overall model parameters of the target model are denoted as θ, and the weight parameters and bias parameters of each linear layer are denoted as w. i b i 1 ≤ level index i ≤ N w w i b i ∈θ; the input vector of each linear layer in the forward prediction process of the target model is denoted as u. i The gradient of the model loss L with respect to the output vector of each linear layer is denoted as δ. i+1 The model loss L is related to each weight parameter w. i The gradient is denoted as The model loss L is assumed to be the negative log-likelihood loss function L.NLL Let L be the loss function, and L = L NLL (θ, y|x)=-log(P) θ (y|x)), where x is the model input, y is the model output, and P θ (y|x) represents the probability that the target model outputs y when the model parameters are the overall model parameters θ and the model input is x;

[0079] The second preprocessing module is used to configure a weight parameter for each linear layer of the target model and edit the model, denoted as editor g. i All editors mentioned g i The model structures are all based on two-layer nonlinear networks, each consisting of a fully connected layer, an activation layer, and a residual connection module; all the editors mentioned above are based on two-layer nonlinear networks. i Both are used to process the vector u input to the current editor through a two-layer nonlinear network. i gradient δ i+1 Perform fine-tuning and output the adjusted vector u new,i gradient δ new,i+1 ;

[0080] The data receiving module is used to receive the user's input-output transformed dataset, denoted as dataset D. set The dataset D set Including N D Group Transformation Data N D It is a configurable positive integer, with a minimum value of 1; 1 ≤ data index j ≤ N D x j Input to the model, For input x j The corresponding latest output tag;

[0081] The editor training module is used to train data from the dataset D. set Choose any set of transformed data as the base sample; and set a specified number N for the base sample. K One relevant sample and one irrelevant sample, and consisting of the base sample, N K The equivalent samples and the irrelevant samples constitute the training dataset D. tr ; and based on the training dataset D tr The target model applies to all editors g. i Training is performed; the specified number N K This is a preset positive integer, which is less than 10 by default; the basic sample is denoted as... The equivalent sample is denoted as (x k ,y k ), 1 ≤ sample index k ≤ NK The unrelated samples are denoted as (x un ,y un ); the input x of each of the equivalent samples k All are related to the input x of the basic sample. s Related, output label y k All are related to the output labels of the base samples. Relevant; the input x of the irrelevant sample un With the input x of the base sample s Irrelevant, output label y un Also related to the output labels of the base sample Irrelevant;

[0082] The model parameter editing module is used to, after all editors have completed training, edit the model parameters based on the dataset D. set and all the editors mentioned g i For all the weight parameters w of the target model i Edit the parameters.

[0083] A third aspect of the present invention provides an electronic device, including: a memory, a processor, and a transceiver;

[0084] The processor is used to couple with the memory, read and execute instructions in the memory to implement the steps of the method described in the first aspect above;

[0085] The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

[0086] A fourth aspect of the present invention provides a computer-readable storage medium storing computer instructions that, when executed by a computer, cause the computer to perform the instructions described in the first aspect.

[0087] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for editing parameters of a pre-trained large model. As described above, this invention designates the pre-trained large model requiring parameter editing as the target model, and assigns a weight parameter editing model, denoted as editor g, to each linear layer of the target model. i ; and the input-output transformation dataset received from the user Then, from the current dataset D set Choose any set of transformed data As the base sample, and with a small number of related samples (x) set up as the base sample. k ,y k ) and an unrelated sample (x) un ,yun The training dataset D consists of all the samples. tr ; and based on the training dataset D tr and target model for all editors g i Perform training; and after training, based on dataset D set and all editors g i For all weight parameters w of the target model i Perform parameter editing on dataset D. set The minimum requirement is that it can include only one set of transformed data. The editor g provided in this embodiment of the invention... i This is a simple editing model based on a two-layer nonlinear network (each layer consists of a fully connected layer, an activation layer, and a residual connection module). It features a small number of parameters, high training efficiency, and low training cost. This invention utilizes the trained editor g... i The technical solution for editing the parameters of the target model is a parameter editing scheme based on backpropagation gradient calculation. This scheme has low computational cost, high editing flexibility, high editing efficiency, and low editing cost. It is particularly suitable for handling parameters derived from a small number of input-output transformations (x,y→y). * When a parameter update task is triggered by a certain event, using the solution of the present invention can achieve the technical effects of improving update flexibility, improving update efficiency, and reducing update cost. Attached Figure Description

[0088] Figure 1 This is a schematic diagram of a method for editing parameters of a pre-trained large model according to Embodiment 1 of the present invention;

[0089] Figure 2 The editor g provided in Embodiment 1 of the present invention i Module structure diagram;

[0090] Figure 3 This is a module structure diagram of a processing device for editing parameters of a pre-trained large model, provided in Embodiment 2 of the present invention.

[0091] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0092] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0093] Embodiment 1 of the present invention provides a method for parameter editing of a pre-trained large model, such as... Figure 1 The diagram illustrates a method for editing parameters of a pre-trained large model according to Embodiment 1 of the present invention. This method mainly includes the following steps:

[0094] Step 1: Denote the pre-trained large model that requires parameter editing as the target model.

[0095] Here, the pre-trained large model in this embodiment of the invention can be any type of large language model implemented based on the Transformer model structure, any type of large language model implemented based on the LSTM / Bi-LSTM model structure, or any type of large language model implemented based on a convolutional neural network. It should be noted that, regardless of the structure upon which the pre-trained large model is implemented, this embodiment of the invention requires that the pre-trained large model have a large number of linear layers, and that each key component within the model (e.g., embedded encoding module, encoder, decoder, etc.) should have a corresponding linear layer within its component or at its output, and that the model output should also have a corresponding linear layer.

[0096] The following describes some key parameters of the pre-trained large model, i.e., the target model, in embodiments of the present invention:

[0097] 1) The target model contains N w Linear layer, N w It is a positive integer greater than 1;

[0098] 2) The overall model parameters of the target model are denoted as θ, and the weight parameters and bias parameters of each linear layer are denoted as w. i b i Where 1 ≤ level index i ≤ N w w i b i ∈θ;

[0099] 3) In the forward prediction process of this target model, the input vector of each linear layer is denoted as u. i The model loss L affects the output vector u of each linear layer. i+1 The gradient is denoted as δ i+1 The model loss L is related to the weight parameters w of each linear layer. i The gradient is denoted as This gradient can be further expressed as: T is the transpose symbol;

[0100] Here, in this embodiment of the invention, the model loss L of the target model is assumed to be the negative log-likelihood loss function L. NLL The loss function is calculated as follows:

[0101] L = LNLL (θ, y|x)=-log(P) θ (y|x)),

[0102] Where x is the model input, y is the model output, and P is the model input. θ (y|x) represents the probability that the target model outputs y given that the model parameters are the global model parameters θ and the model input is x.

[0103] Step 2: Configure a weight parameter for each linear layer of the target model and edit the model (let's call it editor g). i .

[0104] Here, all editors g in this embodiment of the invention i The model structures are all based on two-layer nonlinear networks, each consisting of a fully connected layer, an activation layer, and a residual connection module. All editors g in this embodiment of the invention... i Both are used to process the vector u input to the current editor through a two-layer nonlinear network. i gradient δ i+1 Perform fine-tuning and output the adjusted vector u new,i gradient δ new,i+1 .

[0105] The following describes the editor g in an embodiment of the present invention. i Please provide a detailed explanation.

[0106] like Figure 2 The editor g provided in Embodiment 1 of the present invention i As shown in the module structure diagram, the various editors g in this embodiment of the invention... i Each module consists of an input splicing module, a first fully connected layer, a first activation layer, a first residual connection module, a second fully connected layer, a second activation layer, a second residual connection module, and an output splitting module.

[0107] like Figure 2 As shown, the editor g i The connection relationships of the components are as follows: the input end of the input splicing module is connected to the input end of the current editor, and its output end is connected to the input end of the first fully connected layer and the first input end of the first residual connection module, respectively; the output end of the first fully connected layer is connected to the input end of the first active layer; the output end of the first active layer is connected to the second input end of the first residual connection module; the output end of the first residual connection module is connected to the input end of the second fully connected layer and the first input end of the second residual connection module, respectively; the output end of the second fully connected layer is connected to the input end of the second active layer; the output end of the second active layer is connected to the second input end of the second residual connection module; the output end of the second residual connection module is connected to the input end of the output splitting module; and the output end of the output splitting module is connected to the output end of the current editor.

[0108] editor g i The functions of each component are shown below.

[0109] 1) Input splicing module:

[0110] The input concatenation module in this embodiment of the invention is used to receive the vector u currently input by the editor. i and gradient δ i+1 ; and for vector u i and gradient δ i+1 The corresponding initial vector z is obtained by concatenating the vectors. 0,i Send to the first fully connected layer and the first residual connection module.

[0111] 2) First fully connected layer:

[0112] The model parameters of the first fully connected layer in this embodiment of the invention include the scale vector parameter p. 1,i Offset vector parameter q 1,i Low-rank matrix parameters A1 and B1; where the scale vector parameter p 1,i and offset vector parameter q 1,i For each editor g i Private parameters; low-rank matrix parameters A1 and B1 are for all editors g i Shared parameters.

[0113] The first fully connected layer is used to adjust the scale vector parameter p. 1,i Offset vector parameter q 1,i The low-rank matrix parameters A1 and B1 affect the initial vector z. 0,i Perform a fully connected computation to obtain the corresponding hidden vector h. 1,i Send to the first active layer.

[0114] Here, the latent vector h 1,i The specific calculation formula is as follows:

[0115] h 1,i =p 1,i ⊙(A1B1z 0,i +b)+q 1,i ;

[0116] Where b is a pre-set bias parameter, and all editors g i The bias parameter b is the same for all of them; ⊙ is the Hadamard product operator.

[0117] 3) First activation layer:

[0118] The first activation layer in this embodiment of the invention is implemented based on a class of nonlinear activation functions σ1. Here, the nonlinear activation function σ1 includes at least the ReLU activation function.

[0119] The first activation layer is used to apply the latent vector h according to the nonlinear activation function σ1. 1,i The activation operation is performed to obtain the corresponding activation vector a. 1,i Send to the first residual connection module.

[0120] Here, the activation vector a 1,i The specific calculation formula is as follows:

[0121] a 1,i =σ1(h 1,i ).

[0122] 4) First residual connection module:

[0123] The first residual connection module in this embodiment of the invention is used to process the initial vector z according to the residual connection method. 0,i and activation vector a 1,i The first vector z is obtained by adding elements one by one. 1,i Send to the second fully connected layer.

[0124] Here, the first vector z 1,i The specific calculation formula is as follows:

[0125] z 1,i =z 0,i +a 1,i .

[0126] 5) Second fully connected layer:

[0127] The model parameters of the second fully connected layer include the scale vector parameter p. 2,i Offset vector parameter q 2,i Low-rank matrix parameters A2 and B2. The scale vector parameter p... 2,i and offset vector parameter q 2,i For each editor g i Private parameters; low-rank matrix parameters A2 and B2 are for all editors g i Shared parameters.

[0128] The second fully connected layer is used to adjust the scale vector parameter p. 2,i Offset vector parameter q 2,i The low-rank matrix parameters A2 and B2 affect the first vector z. 1,i Perform a fully connected computation to obtain the corresponding hidden vector h. 2,i Send to the second activation layer.

[0129] Here, the latent vector h2,i The specific calculation formula is as follows:

[0130] h 2,i =p 2,i ⊙(A2B2z 1,i )+q 2,i .

[0131] 6) Second activation layer:

[0132] The second activation layer in this embodiment of the invention is implemented based on a class of nonlinear activation functions σ². Here, the nonlinear activation function σ² includes at least the ReLU activation function.

[0133] The second activation layer is used to apply the latent vector h according to the nonlinear activation function σ². 2,i The activation operation is performed to obtain the corresponding activation vector a. 2,i Send to the second residual connection module.

[0134] Here, the activation vector a 2,i The specific calculation formula is as follows:

[0135] a 2,i =σ2(h 2,i ).

[0136] 7) Second residual connection module:

[0137] The second residual connection module in this embodiment of the invention is used to process the first vector z according to the residual connection method. 1,i and activation vector a 2,i The second vector z is obtained by adding elements one by one. 2,i Send to the output splitting module.

[0138] Here, the second vector z 2,i The specific calculation formula is as follows:

[0139] z 2,i =z 1,i +a 2,i .

[0140] 8) Output split module:

[0141] The output splitting module of this embodiment of the invention is used to split the initial vector z. 0,i mid-vector u i and gradient δ i+1 Taking the concatenation position as a reference, from the second vector z 2,i Extract the corresponding vector u new,i and gradient δ new,i+1 And output it.

[0142] Step 3: Receive user input and output transformation dataset, denoted as dataset D.set .

[0143] Here, the dataset D of this embodiment of the invention set Including N D Group Transformation Data N D It is a configurable positive integer, with a minimum value of 1; 1 ≤ data index j ≤ N D x j Input to the model, For input x j The corresponding latest output tag.

[0144] Step 4, from dataset D set Choose any set of transformed data as the base sample; and set a specified number N for the base sample. K There are one relevant sample and one irrelevant sample, and consists of a base sample and N. K The training dataset D consists of equivalent samples and irrelevant samples. tr ; and based on the training dataset D tr The target model applies to all editors g. i Conduct training;

[0145] Specifically, this includes: Step 41, from dataset D set Choose any set of transformed data as the base sample; and set a specified number N for the base sample. K There are one relevant sample and one irrelevant sample, and consists of a base sample and N. K The training dataset D consists of equivalent samples and irrelevant samples. tr ;

[0146] Here, the specified number N in the embodiments of the present invention K It is a pre-set positive integer, which is less than 10 by default; the basic sample of this embodiment is denoted as Equivalent samples in embodiments of the present invention are denoted as (x k ,y k ), 1 ≤ sample index k ≤ N K In this embodiment of the invention, irrelevant samples are denoted as (x un ,y un );

[0147] It should be noted that the various equivalent samples (x) in the embodiments of the present invention k ,y k Input x k All are related to the input x of the base sample s Related, output label y k All are the same as the output labels of the base samples Relevant; it should also be noted that irrelevant samples (x) in the embodiments of the present inventionun ,y un Input x un Input x of the base sample s Irrelevant, output label y un Also related to the output labels of the base samples Irrelevant;

[0148] How to understand equivalent samples (x) k ,y k ), irrelevant samples (x) un ,y un ) and base sample The correlation and uncorrelation between them are illustrated with a simple example below:

[0149] For example, the target model has learned the following knowledge after the last pre-training or parameter update: Knowledge 1 "Person A's father is Person B, and their mother is Person C", Knowledge 2 "Person B's address is address B, and Person C's address is address C", and Knowledge 3 "Person D, who has no relation to Person A, B, or C, has an address of address D". However, through fact lookup, we find that Person A's father should actually be Person E, and Person E's address is address E. Therefore, the model parameters of the target model need to be updated based on the new knowledge content "Person A's father is Person E, and Person E's address is address E".

[0150] Assume the base sample x s For the question "Who is person A's father?", then, without updating the parameters of the target model, the corresponding output label y... s The answer should be "Character A's father is Character B"; after the parameters of the target model have been updated, the corresponding output label... The answer should be "Character A's father is Character E";

[0151] In x s For the question "Who is person A's father?", Given the answer "Character A's father is Character E", question x k "What is the address of Person A's father?" (Answer: y) k "The address of Person A's father is address E" is a set of data related to the base sample. Related equivalent samples (x) k ,y k ); while problem x un "What is the address of character D?", Answer y un "The address of person D is address D" is a set of data related to the base sample. Unrelated samples (x) un ,y un );

[0152] In summary: 1) For equivalent samples (x) k ,y k For example, its problem x k The answers before and after the model parameters are updated are different, and the changes in the model's output answer are consistent with the actual answers. The changes are related to; 2) while for unrelated samples (x) un ,y un For example, its problem x un The answer is the same before and after the model parameters are updated, i.e., question x. un The corresponding model output answer will not change;

[0153] Step 42, and based on the training dataset D tr The target model applies to all editors g. i Conduct training;

[0154] Specifically, this includes: Step 421, from the training dataset D tr Extract the corresponding base samples N K Equivalent samples (x) k ,y k ) and irrelevant samples (x) un ,y un The overall model parameters θ of the target model are backed up and stored as the corresponding backup parameters θ0; and the current overall model parameters θ are used as the corresponding current model parameters θ0. now ; and by all editors g i The editor parameters form the corresponding editor parameter set φ; and all parameters in the editor parameter set φ are initialized to obtain the current parameter set φ. now And set a first counter initialized to 0;

[0155] Among them, the editor parameter set φ is composed of all editors g i Private scale vector parameter p 1,i Offset vector parameter q 1,i Scale vector parameter p 2,i Offset vector parameter q 2,i It consists of shared low-rank matrix parameters A1, B1, A2, and B2.

[0156] Step 422, based on the current parameter set φ now For all editors g i Configure the editor parameters;

[0157] Step 423, base sample xs The target model is input for forward prediction processing, and the vectors u generated during the processing are analyzed. i Cache the data and calculate the corresponding model loss L;

[0158] Here, the model loss L calculated in the current step is:

[0159]

[0160] in, The target model has its model parameters as the current model parameters θ. now And the model input is x s In this case, its model output is The probability of;

[0161] Step 424, based on the current model parameters θ now The various weight parameters w in i and its corresponding bias parameter b i Vector u i Calculate the corresponding linear layer output vector u i+1 And calculate the model loss L for each vector u. i+1 The gradient is obtained by finding the corresponding gradient δ. i+1 ; and each gradient δ i+1 and its corresponding vector u i Enter the corresponding editor g i Fine-tuning yields the corresponding vector u new,i and gradient δ new,i+1 ; and based on each vector u new,i and its corresponding gradient δ new,i+1 Calculate the new weight gradient And based on each weight parameter w i and its corresponding weight gradient Calculate the corresponding weight parameters and the current model parameters θ now The various weight parameters w in i Reset to the corresponding weight parameters

[0162] here:

[0163] a. Vector u i+1 The calculation method is as follows: u i+1 =w i u i +b i ;

[0164] b. Gradient δ i+1 The calculation method is as follows:

[0165] c. Vector u new,i and gradient δ new,i+1 The calculation method is as follows:

[0166] g i (u i ,δ i+1 )→(u new,i ,δ new,i+1 ), g i (u i ,δ i+1 ) is the editor g i The expression;

[0167] d. Weight gradient The calculation method is as follows:

[0168] e. Weight parameters The calculation method is as follows:

[0169] Step 425, based on the latest current model parameters θ now The model parameters of the target model are reset; and each equivalent sample (x) is... k ,y k ) of x k Input the target model for positive prediction and calculate the corresponding first model loss L. 1,k ;

[0170] Here, the first model loss L in each embodiment of the present invention 1,k By default, the negative log-likelihood loss function L is used. NLL The loss function is as follows:

[0171]

[0172] in, The target model has its model parameters set to the latest current model parameters θ. now And the model input is x k In this case, the model output is the corresponding y k The probability of;

[0173] Step 426, remove irrelevant samples (x) un ,y un ) of x un Input the target model for positive prediction processing and calculate the corresponding post-edited probability. Then, the model parameters of the target model are reset based on the backup parameter θ0; and the irrelevant samples (x) are... un ,y un ) of x un Input the target model for positive prediction processing and calculate the corresponding pre-edit probability. And based on the probability r before editing pre The probability r after editing post Sum of divergence loss function L KL Calculate the corresponding second model loss L2;

[0174] Here, the embodiments of the present invention The target model has its model parameters set to the latest current model parameters θ. now And the model input is x un In this case, its model output is y un The probability of; embodiments of the present invention The target model has backup parameters θ0 and model input x. un In this case, its model output is y un The probability; the second model loss L2 in this embodiment of the invention defaults to the divergence loss function L. KL The loss function is as follows:

[0175]

[0176] Step 427, by N K The first model loss L 1,k The editor loss L is composed of the second model loss L2. G ;

[0177] Here, the editor loss L in this embodiment of the invention G for:

[0178]

[0179] Wherein, λ is a pre-set balance parameter;

[0180] Step 428, towards causing the editor to lose L G The direction to reach the minimum value is based on the preset first model optimizer for all editors g. i The corresponding current parameter set φ now Perform one round of parameter modulation; and use the modulated editor parameter set as the new current parameter set φ. now ;

[0181] Here, the first model optimizer in this embodiment of the invention includes at least the Adam optimizer;

[0182] Step 429: Increment the first counter by 1; and check whether the incremented first counter exceeds a preset counter threshold; if it does not exceed the threshold, return to step 422 to continue training; if it exceeds the threshold, stop training and adjust the current parameter set φ accordingly. now For all editors g iThe editor parameters are fixed, and the training of all editors is confirmed to be complete.

[0183] Here, the counter threshold is a pre-set threshold parameter.

[0184] Step 5: After all editors have been trained, based on dataset D... set and all editors g i For all weight parameters w of the target model i Perform parameter editing;

[0185] Specifically, this includes: setting the model parameters of the target model based on the backup parameter θ0; and setting the dataset D. set All transformation data Perform one iteration; and during this iteration, transform the data of the current iteration. As the corresponding current transformation data (x) c ,y c ); and will the current transformed data (x) c ,y c ) of x c The target model is input for forward prediction processing, and the vectors u generated during the processing are analyzed. i Cache the data and calculate the corresponding model loss. And based on the weight parameters w in the backup parameter θ0 i and its corresponding bias parameter b i Vector u i Calculate the corresponding linear layer output vector u i+1 And calculate the model loss L for each vector u. i+1 The gradient is obtained by finding the corresponding gradient δ. i+1 ; and each gradient δ i+1 and its corresponding vector u i Enter the corresponding editor g i Fine-tuning yields the corresponding vector u new,i and gradient δ new,i+1 ; and based on each vector u new,i and its corresponding gradient δ new,i+1 Calculate the new weight gradient And based on each weight parameter w i and its corresponding weight gradient Calculate the corresponding weight parameters And the weight parameters w in the current backup parameter θ0 i Reset to the corresponding weight parameters And based on the latest backup parameters θ0, the model parameters of the target model are reset once.

[0186] Here, in the current step The target model has backup parameters θ0 and model input x. c In this case, its model output is y c The probability; the calculation method for each parameter in the current step is similar to that in step 424, that is:

[0187] a. Vector u i+1 The calculation method is as follows: u i+1 =w i u i +b i ;

[0188] b. Gradient δ i+1 The calculation method is as follows:

[0189] c. Vector u new,i and gradient δ new,i+1 The calculation method is as follows:

[0190] g i (u i ,δ i+1 )→(u new,i ,δ new,i+1 ), g i (u i ,δ i+1 ) is the editor g i The expression;

[0191] d. Weight gradient The calculation method is as follows:

[0192] e. Weight parameters The calculation method is as follows:

[0193] Figure 3 This is a module structure diagram of a processing device for editing parameters of a pre-trained large model according to Embodiment 2 of the present invention. This device can be a terminal device or server implementing the aforementioned method embodiments, or it can be a device that enables the aforementioned terminal device or server to implement the aforementioned method embodiments. For example, the device can be a device or chip system of the aforementioned terminal device or server. Figure 3 As shown, the device includes: a first preprocessing module 201, a second preprocessing module 202, a data receiving module 203, an editor training module 204, and a model parameter editing module 205.

[0194] The first preprocessing module 201 is used to denote the pre-trained large model that requires parameter editing as the target model; the target model contains N w Linear layer, N wThe integer is a positive integer greater than 1; the overall model parameters of the target model are denoted as θ, and the weight parameters and bias parameters of each linear layer are denoted as w. i b i 1 ≤ level index i ≤ N w w i b i ∈θ; In the forward prediction process of the target model, the input vector of each linear layer is denoted as u. i The gradient of the model loss L with respect to the output vector of each linear layer is denoted as δ. i+1 Model loss L versus each weight parameter w i The gradient is denoted as The model loss L is by default the negative log-likelihood loss function L. NLL Let L be the loss function, and L = L NLL (θ, y|x)=-log(P) θ (y|x)), where x is the model input, y is the model output, and P θ (y|x) represents the probability that the target model outputs y given that the model parameters are the global model parameters θ and the model input is x.

[0195] The second preprocessing module 202 is used to configure a weight parameter for each linear layer of the target model and edit the model, denoted as editor g. i All editors g i The model structures are all based on two-layer nonlinear networks, each consisting of a fully connected layer, an activation layer, and a residual connection module; all editors g i Both are used to process the vector u input to the current editor through a two-layer nonlinear network. i gradient δ i+1 Perform fine-tuning and output the adjusted vector u new,i gradient δ new,i+1 .

[0196] Data receiving module 203 is used to receive user input and output transformation datasets, denoted as dataset D. set Dataset D set Including N D Group Transformation Data N D It is a configurable positive integer, with a minimum value of 1; 1 ≤ data index j ≤ N D x j Input to the model, For input x j The corresponding latest output tag.

[0197] Editor training module 204 is used to train data from dataset D. set Choose any set of transformed data as the base sample; and set a specified number N for the base sample.K There are one relevant sample and one irrelevant sample, and consists of a base sample and N. K The training dataset D consists of equivalent samples and irrelevant samples. tr ; and based on the training dataset D tr The target model applies to all editors g. i Perform training; specify the number N. K It is a preset positive integer, which is less than 10 by default; the base sample is denoted as . Equivalent samples are denoted as (x k ,y k ), 1 ≤ sample index k ≤ N K Unrelated samples are denoted as (x un ,y un ); Input x for each equivalent sample k All are related to the input x of the base sample s Related, output label y k All are the same as the output labels of the base samples The input x of relevant and irrelevant samples un Input x of the base sample s Irrelevant, output label y un Also related to the output labels of the base samples Irrelevant.

[0198] The model parameter editing module 205 is used to edit the model parameters based on dataset D after all editors have finished training. set and all editors g i For all weight parameters w of the target model i Edit the parameters.

[0199] The present invention provides a processing device for editing parameters of a pre-trained large model, which can execute the method steps in the above method embodiments. Its implementation principle and technical effect are similar, and will not be repeated here.

[0200] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, the first preprocessing module can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and called and executed by a processing element of the device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through integrated logic circuits in the hardware of the processor element or through software instructions.

[0201] For example, these modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). As another example, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a System-on-a-Chip (SOC).

[0202] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the foregoing method embodiments are generated. The computer described above can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The aforementioned computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the aforementioned computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, Bluetooth, microwave, etc.) means. The aforementioned computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks (SSDs)).

[0203] Figure 4 This is a schematic diagram of an electronic device provided in Embodiment 3 of the present invention. This electronic device can be a terminal device or server implementing the methods of the aforementioned embodiments, or it can be a terminal device or server connected to the aforementioned terminal device or server implementing the methods of the aforementioned embodiments. Figure 4 As shown, the electronic device may include: a processor 301 (e.g., CPU), a memory 302, and a transceiver 303; the transceiver 303 is coupled to the processor 301, and the processor 301 controls the transmission and reception operations of the transceiver 303. The memory 302 may store various instructions for performing various processing functions and implementing the processing steps described in the foregoing embodiments. Preferably, the electronic device involved in the embodiments of the present invention further includes: a power supply 304, a system bus 305, and a communication port 306. The system bus 305 is used to realize communication connections between components. The communication port 306 is used for communication between the electronic device and other peripherals.

[0204] exist Figure 4The system bus 305 mentioned can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface is used to enable communication between the database access device and other devices (e.g., clients, read-write libraries, and read-only libraries). Memory may include Random Access Memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0205] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), graphics processing units (GPUs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0206] It should be noted that the embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when run on a computer, cause the computer to perform the methods and processes provided in the above embodiments.

[0207] This invention provides a method, apparatus, electronic device, and computer-readable storage medium for editing parameters of a pre-trained large model. As described above, this invention designates the pre-trained large model requiring parameter editing as the target model, and assigns a weight parameter editing model, denoted as editor g, to each linear layer of the target model. i ; and the input-output transformation dataset received from the user Then, from the current dataset D set Choose any set of transformed data As the base sample, and with a small number of related samples (x) set up as the base sample. k ,y k ) and an unrelated sample (x) un ,y un The training dataset D consists of all the samples. tr ; and based on the training dataset D tr and target model for all editors g iPerform training; and after training, based on dataset D set and all editors g i For all weight parameters w of the target model i Perform parameter editing on dataset D. set The minimum requirement is that it can include only one set of transformed data. The editor g provided in this embodiment of the invention... i This is a simple editing model based on a two-layer nonlinear network (each layer consists of a fully connected layer, an activation layer, and a residual connection module). It features a small number of parameters, high training efficiency, and low training cost. This invention utilizes the trained editor g... i The technical solution for editing the parameters of the target model is a parameter editing scheme based on backpropagation gradient calculation. This scheme has low computational cost, high editing flexibility, high editing efficiency, and low editing cost. It is particularly suitable for handling parameters derived from a small number of input-output transformations (x,y→y). * When a parameter update task is triggered by a certain event, using the solution of the present invention can achieve the technical effects of improving update flexibility, improving update efficiency, and reducing update cost.

[0208] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0209] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for parameter editing of a pre-trained large model, characterized in that, The method includes: The pre-trained large model that requires parameter editing is denoted as the target model; the target model contains N w Linear layer, N w The value is a positive integer greater than 1; the overall model parameters of the target model are denoted as θ, and the weight parameters and bias parameters of each linear layer are denoted as w. i b i 1 ≤ level index i ≤ N w w i b i ∈θ; the input vector of each linear layer in the forward prediction process of the target model is denoted as u. i The gradient of the model loss L with respect to the output vector of each linear layer is denoted as δ. i+1 The model loss L is related to each weight parameter w. i The gradient is denoted as The model loss L is assumed to be the negative log-likelihood loss function L. NLL Let L be the loss function, and L = L NLL (θ, y|x)=-log(P) θ (y|x)), where x is the model input, y is the model output, and P θ (y|x) represents the probability that the target model outputs y when the model parameters are the overall model parameters θ and the model input is x; Configure a weight parameter for each linear layer of the target model and edit the model; denoted as editor g. i All editors mentioned g i The model structures are all based on two-layer nonlinear networks, each consisting of a fully connected layer, an activation layer, and a residual connection module; all the editors mentioned above are based on two-layer nonlinear networks. i Both are used to process the vector u input to the current editor through a two-layer nonlinear network. i gradient δ i+1 Perform fine-tuning and output the adjusted vector u new,i gradient δ new,i+1 ; The input-output transformation dataset that receives user input is denoted as dataset D. set The dataset D set Including N D Group Transformation Data N D It is a configurable positive integer, with a minimum value of 1; 1 ≤ data index j ≤ N D x j Input to the model, For input x j The corresponding latest output tag; From the dataset D set Choose any set of transformed data as the base sample; and set a specified number N for the base sample. K There are one equivalent sample and one irrelevant sample, and the base sample, N K The equivalent samples and the irrelevant samples constitute the training dataset D. tr ; and based on the training dataset D tr The target model applies to all editors g. i Training is performed; the specified number N K This is a preset positive integer, which is less than 10 by default; the basic sample is denoted as... The equivalent sample is denoted as (x k ,y k ), 1 ≤ sample index k ≤ N K The unrelated samples are denoted as (x un ,y un ); the input x of each of the equivalent samples k All are related to the input x of the basic sample. s Related, output label y k All are related to the output labels of the base samples. Relevant; the input x of the irrelevant sample un With the input x of the base sample s Irrelevant, output label y un Also related to the output labels of the base sample Irrelevant; After all editors have been trained, based on the dataset D set and all the editors mentioned g i For all the weight parameters w of the target model i Edit the parameters.

2. The method for parameter editing of a pre-trained large model according to claim 1, characterized in that, Each of the editors mentioned g i Each module consists of an input splicing module, a first fully connected layer, a first activation layer, a first residual connection module, a second fully connected layer, a second activation layer, a second residual connection module, and an output splitting module. The input end of the input splicing module is connected to the input end of the current editor, and the output end is connected to the input end of the first fully connected layer and the first input end of the first residual connection module, respectively; the output end of the first fully connected layer is connected to the input end of the first activation layer; the output end of the first activation layer is connected to the second input end of the first residual connection module. The output of the first residual connection module is connected to the input of the second fully connected layer and the first input of the second residual connection module, respectively; the output of the second fully connected layer is connected to the input of the second activation layer; the output of the second activation layer is connected to the second input of the second residual connection module; the output of the second residual connection module is connected to the input of the output splitting module; and the output of the output splitting module is connected to the output of the current editor. The input concatenation module is used to receive the vector u currently input by the editor. i and the gradient δ i+1 ; and for the vector u i and the gradient δ i+1 The corresponding initial vector z is obtained by concatenating the vectors. 0,i Send to the first fully connected layer and the first residual connection module; The model parameters of the first fully connected layer include the scale vector parameter p. 1,i Offset vector parameter q 1,i Low-rank matrix parameters A1 and B1; the scale vector parameter p 1,i and the offset vector parameter q 1,i For each of the editors mentioned g i Private parameters; the low-rank matrix parameters A1 and B1 are for all the editors g i Shared parameters; The first fully connected layer is used to adjust the scale vector parameter p. 1,i The offset vector parameter q 1,i The low-rank matrix parameters A1 and B1 are related to the initial vector z. 0,i Perform a fully connected computation to obtain the corresponding hidden vector h. 1,i Send to the first activation layer; Wherein, the hidden vector h 1,i For: h 1,i =p 1,i ⊙(A1B1z 0,i +b)+q 1,i ; b is the preset bias parameter, and all the editors mentioned g i The bias parameter b of the first fully connected layer is the same; ⊙ is the Hadamard product operator; The first activation layer is implemented based on a class of nonlinear activation functions σ1; the nonlinear activation function σ1 includes at least the ReLU activation function; The first activation layer is used to apply the latent vector h according to the nonlinear activation function σ1. 1,i The activation operation is performed to obtain the corresponding activation vector a. 1,i Send to the first residual connection module; Wherein, the activation vector a 1,i For: a 1,i =σ1(h 1,i ); The first residual connection module is used to process the initial vector z according to the residual connection method. 0,i and the activation vector a 1,i The first vector z is obtained by adding elements one by one. 1,i Send to the second fully connected layer; Wherein, the first vector z 1,i z 1,i =z 0,i +a 1,i ; The model parameters of the second fully connected layer include the scale vector parameter p. 2,i Offset vector parameter q 2,i Low-rank matrix parameters A2 and B2; the scale vector parameter p 2,i and the offset vector parameter q 2,i For each of the editors mentioned g i Private parameters; the low-rank matrix parameters A2 and B2 are for all the editors g i Shared parameters; The second fully connected layer is used to adjust the scale vector parameter p. 2,i The offset vector parameter q 2,i The low-rank matrix parameter A2 and the low-rank matrix parameter B2 affect the first vector z. 1,i Perform a fully connected computation to obtain the corresponding hidden vector h. 2,i Send to the second activation layer; Wherein, the hidden vector h 2,i For: h 2,i =p 2,i ⊙(A2B2z 1,i )+q 2,i ; The second activation layer is implemented based on a class of nonlinear activation functions σ²; the nonlinear activation function σ² includes at least the ReLU activation function; The second activation layer is used to apply the latent vector h according to the nonlinear activation function σ2. 2,i The activation operation is performed to obtain the corresponding activation vector a. 2,i Send to the second residual connection module; Wherein, the activation vector a 2,i For: a 2,i =σ2(h 2,i ); The second residual connection module is used to process the first vector z according to the residual connection method. 1,i and the activation vector a 2,i The second vector z is obtained by adding elements one by one. 2,i Send to the output splitting module; Wherein, the second vector z 2,i z 2,i =z 1,i +a 2,i ; The output splitting module is used to split the initial vector z. 0,i The vector u mentioned in i and the gradient δ i+1 Taking the concatenation position as a reference, from the second vector z 2,i Extract the corresponding vector u new,i and the gradient δ new,i+1 And output it.

3. The method for parameter editing of a pre-trained large model according to claim 1, characterized in that, The training dataset D tr The target model applies to all editors g. i The training includes: Step 31, from the training dataset D tr Extract the corresponding base sample N K The equivalent samples (x) k ,y k ) and the unrelated samples (x) un ,y un The overall model parameters θ of the target model are backed up and stored as backup parameters θ0; and the current overall model parameters θ are used as the corresponding current model parameters θ0. now ; and by all the editors mentioned g i The editor parameters form the corresponding editor parameter set φ; and all parameters of the editor parameter set φ are initialized to obtain the current parameter set φ. now And set a first counter initialized to 0; Wherein, the editor parameter set φ is composed of all the editors g i Private scale vector parameter p 1,i / p 2,i Offset vector parameter q 1,i / q 2,i It consists of shared low-rank matrix parameters A1 / A2 and B1 / B2; Step 32, based on the current parameter set φ now For all the editors mentioned g i Configure the editor parameters; Step 33, the basic sample x s The target model is input for forward prediction processing, and the vectors u generated during the processing are analyzed. i Cache the data and calculate the corresponding model loss. in, The target model has its model parameters as the current model parameters θ. now And the model input is x s In this case, its model output is The probability of; Step 34, based on the current model parameters θ now The weight parameters w in i and its corresponding bias parameter b i The vector u i Calculate the corresponding linear layer output vector u i+1 And calculate the model loss L for each vector u. i+1 The gradient is obtained by the corresponding gradient δ. i+1 ; and each of the gradients δ i+1 and its corresponding vector u i Input the corresponding editor g i Fine-tuning is performed to obtain the corresponding vector u. new,i and the gradient δ new,i+1 ; and based on each of the vectors u new,i and its corresponding gradient δ new,i+1 Calculate the new weight gradient And based on each of the weight parameters w i and the corresponding weight gradient Calculate the corresponding weight parameters and the current model parameter θ now The weight parameters w in i Reset to the corresponding weight parameters in, The vector u i+1 The calculation method is as follows: u i+1 =w i u i +b i ; The gradient δ i+1 The calculation method is as follows: The vector u new,i and the gradient δ new,i+1 The calculation method is as follows: g i (u i ,δ i+1 )→(u new,i ,δ new,i+1 ), g i (u i ,δ i+1 ) is the editor g i The expression; The weight gradient The calculation method is as follows: T is the transpose symbol; The weight parameters The calculation method is as follows: Step 35, based on the latest current model parameters θ now The model parameters of the target model are reset; and each of the equivalent samples (x) is... k ,y k ) of x k The target model is input for positive prediction processing, and the corresponding first model loss L is calculated. 1,k ; Wherein, the first model loss L 1,k The default is to use the negative log-likelihood loss function L. NLL The loss function is as follows: The target model is defined with its model parameters being the latest current model parameters θ. now And the model input is x k In this case, the model output is the corresponding y k The probability of; Step 36, the irrelevant samples (x) un ,y un ) of x un The target model is input for positive prediction processing, and the corresponding post-edit probability is calculated. Then, based on the backup parameter θ0, the model parameters of the target model are reset; and the irrelevant samples (x) are... un ,y un ) of x un The target model is input for positive prediction processing, and the corresponding pre-edit probability is calculated. And based on the pre-edit probability r pre The probability r after editing post Sum of divergence loss function L KL Calculate the corresponding second model loss L2; in, The target model is defined with its model parameters being the latest current model parameters θ. now And the model input is x un In this case, its model output is y un The probability of; The target model has its model parameters as the backup parameter θ0 and its model input is x. un In this case, its model output is y un The probability of; The second model loss L2 is assumed to use the divergence loss function L. KL The loss function is as follows: Step 37, by N K The first model loss L 1,k Together with the second model loss L2, they form the corresponding editor loss L. G ; Wherein, the editor loss L G for: λ is a preset balance parameter; Step 38, towards causing the editor to lose L G The direction to reach the minimum value is based on the preset first model optimizer for all editors g. i The corresponding current parameter set φ now Perform one round of parameter modulation; and use the modulated editor parameter set as the new current parameter set φ. now ; The first model optimizer includes at least the Adam optimizer; Step 39: Increment the first counter by 1; and check whether the incremented first counter exceeds a preset counter threshold; if it does not exceed the threshold, return to step 32 to continue training; if it exceeds the threshold, stop training and adjust the current parameter set φ based on the latest value. now For all the editors mentioned g i The editor parameters are fixed, and the training of all editors is confirmed to be complete.

4. The method for parameter editing of a pre-trained large model according to claim 1, characterized in that, The dataset D set and all the editors mentioned g i For all the weight parameters w of the target model i Parameter editing includes: The model parameters of the target model are set based on the backup parameter θ0; and the dataset D is... set All of the aforementioned transformation data Perform one round of traversal; and during this round of traversal, transform the data of the currently traversed data. As the corresponding current transformation data (x) c ,y c ); and the current transformation data (x) c ,y c ) of x c The target model is input for forward prediction processing, and the vectors u generated during the processing are analyzed. i Cache the data and calculate the corresponding model loss. And according to each of the weight parameters w in the backup parameter θ0 i and its corresponding bias parameter b i The vector u i Calculate the corresponding linear layer output vector u i+1 And calculate the model loss L for each vector u. i+1 The gradient is obtained by the corresponding gradient δ. i+1 ; and each of the gradients δ i+1 and its corresponding vector u i Input the corresponding editor g i Fine-tuning is performed to obtain the corresponding vector u. new,i and the gradient δ new,i+1 ; and based on each of the vectors u new,i and its corresponding gradient δ new,i+1 Calculate the new weight gradient And based on each of the weight parameters w i and the corresponding weight gradient Calculate the corresponding weight parameters And each of the weight parameters w in the current backup parameter θ0 i Reset to the corresponding weight parameters And based on the latest backup parameter θ0, the model parameters of the target model are reset once; Wherein, the vector u i+1 The calculation method is as follows: u i+1 =w i u i +b i ; The gradient δ i+1 The calculation method is as follows: The vector u new,i and the gradient δ new,i+1 The calculation method is as follows: g i (u i ,δ i+1 )→(u new,i ,δ new,i+1 ), g i (u i ,δ i+1 ) is the editor g i The expression; The weight gradient The calculation method is as follows: T is the transpose symbol; The weight parameters The calculation method is as follows:

5. An apparatus for performing the processing method for parameter editing of a pre-trained large model as described in any one of claims 1-4, characterized in that, The device includes: a first preprocessing module, a second preprocessing module, a data receiving module, an editor training module, and a model parameter editing module; The first preprocessing module is used to denote the pre-trained large model that requires parameter editing as the target model; the target model contains N w Linear layer, N w The value is a positive integer greater than 1; the overall model parameters of the target model are denoted as θ, and the weight parameters and bias parameters of each linear layer are denoted as w. i b i 1 ≤ level index i ≤ N w w i b i ∈θ; the input vector of each linear layer in the forward prediction process of the target model is denoted as u. i The gradient of the model loss L with respect to the output vector of each linear layer is denoted as δ. i+1 The model loss L is related to each weight parameter w. i The gradient is denoted as The model loss L is assumed to be the negative log-likelihood loss function L. NLL Let L be the loss function, and L = L NLL (θ, y|x)=-log(P) θ (y|x)), where x is the model input, y is the model output, and P θ (y|x) represents the probability that the target model outputs y when the model parameters are the overall model parameters θ and the model input is x; The second preprocessing module is used to configure a weight parameter for each linear layer of the target model and edit the model, denoted as editor g. i All editors mentioned g i The model structures are all based on two-layer nonlinear networks, each consisting of a fully connected layer, an activation layer, and a residual connection module; all the editors mentioned above are based on two-layer nonlinear networks. i Both are used to process the vector u input to the current editor through a two-layer nonlinear network. i gradient δ i+1 Perform fine-tuning and output the adjusted vector u new,i gradient δ new,i+1 ; The data receiving module is used to receive the user's input-output transformed dataset, denoted as dataset D. set The dataset D set Including N D Group Transformation Data N D It is a configurable positive integer, with a minimum value of 1; 1 ≤ data index j ≤ N D x j Input to the model, For input x j The corresponding latest output tag; The editor training module is used to train data from the dataset D. set Choose any set of transformed data as the base sample; and set a specified number N for the base sample. K There are one equivalent sample and one irrelevant sample, and the base sample, N K The equivalent samples and the irrelevant samples constitute the training dataset D. tr ; and based on the training dataset D tr The target model applies to all editors g. i Training is performed; the specified number N K This is a preset positive integer, which is less than 10 by default; the basic sample is denoted as... The equivalent sample is denoted as (x k ,y k ), 1 ≤ sample index k ≤ N K The unrelated samples are denoted as (x un ,y un ); the input x of each of the equivalent samples k All are related to the input x of the basic sample. s Related, output label y k All are related to the output labels of the base samples. Relevant; the input x of the irrelevant sample un With the input x of the base sample s Irrelevant, output label y un Also related to the output labels of the base sample Irrelevant; The model parameter editing module is used to, after all editors have completed training, edit the model parameters based on the dataset D. set and all the editors mentioned g i For all the weight parameters w of the target model i Edit the parameters.

6. An electronic device, characterized in that, include: Memory, processor, and transceiver; The processor is configured to be coupled to the memory, read and execute instructions in the memory to implement the method according to any one of claims 1-4; The transceiver is coupled to the processor, and the processor controls the transceiver to send and receive messages.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method described in any one of claims 1-4.

Images