Model updating method, electronic device, and storage medium

Abstract

This invention provides a model update method, an electronic device, and a storage medium. The method includes: for any linear layer in a first model, performing singular value decomposition on a first matrix of the linear layer to obtain a second matrix; acquiring output data of the linear layers in the first and second models respectively based on input data; the second model is obtained by updating the linear layers in the first model based on the second matrix; updating the second matrix based on the output data of the linear layers in the first and second models, and updating the linear layers in the second model based on the updated second matrix. The updated model in this solution has high computational efficiency when executed on the device, does not require backpropagation to update model parameters, has low memory consumption, and produces better output results, closer to the original uncompressed model than existing model compression methods.

Description

Technical Field

[0001] This invention relates to the field of model update technology, and more particularly to a model update method, electronic device, and storage medium. Background Technology

[0002] Artificial intelligence (AI) has been widely applied in various fields. In most AI models, especially large language models, the computational cost and memory usage of linear layers are often the largest. The optimization of linear layers has a significant impact on the performance of AI models.

[0003] Currently, electronic devices suffer from low computational efficiency and consume a large amount of device memory when executing AI models due to the large amount of computation required. Therefore, it is urgent for those skilled in the art to implement a model optimization method to improve computational efficiency and reduce the consumption of device memory. Summary of the Invention

[0004] This invention provides a model update method, electronic device, and storage medium to address the shortcomings of existing technologies, such as low computational efficiency and high memory consumption. The invention provides a model update method that has high computational efficiency based on the updated model and consumes less device memory.

[0005] This invention provides a model update method, comprising:

[0006] For any linear layer in the first model, the first matrix of the linear layer is subjected to singular value decomposition to obtain the second matrix;

[0007] Based on the input data, the output data of the linear layers in the first model and the second model are obtained respectively; the second model is obtained by updating the linear layers in the first model according to the second matrix;

[0008] The second matrix is ​​updated based on the output data of the linear layers in the first model and the second model, and the linear layers in the second model are updated based on the updated second matrix.

[0009] According to a model update method provided by the present invention, the step of performing singular value decomposition on the first matrix of the linear layer to obtain a second matrix includes:

[0010] The first matrix is ​​subjected to singular value decomposition, and the r value is obtained using formula (1). The second matrix is ​​obtained based on the r value.

[0011]

[0012] Where P represents the preset percentage, σi Let represent the singular values ​​of the first matrix, whose size is [d1, d2], where d is determined based on the size of d1 and d2.

[0013] According to a model update method provided by the present invention, updating the second matrix based on the output data of the linear layers in the first model and the second model includes:

[0014] Based on the output data of the linear layers in the first model and the second model, at least one intermediate matrix of the second matrix is ​​updated to obtain the updated second matrix;

[0015] Wherein, the second matrix W r =U r Σ r V r T U r and V r Let Σ be the intermediate matrix. r It is a diagonal matrix.

[0016] According to a model update method provided by the present invention, updating at least one intermediate matrix of the second matrix based on the output data of the linear layers in the first model and the second model includes:

[0017] For any of the intermediate matrices, the intermediate matrices are updated using the objective function as shown in formula (2) based on the output data of the linear layers in the first model and the second model.

[0018]

[0019] Where y = xW + b, b is the deviation parameter, x is the input data, and W is the first matrix;

[0020] ⊙ represents element-vector multiplication.

[0021] According to a model update method provided by the present invention, the step of updating the intermediate matrix using an objective function as shown in formula (2) based on the output data of the linear layers in the first model and the second model includes:

[0022] Based on the output data of the linear layers in the first model and the second model, the intermediate matrix of the second matrix is ​​updated using formulas (3)-(5);

[0023]

[0024]

[0025]

[0026] Among them, X r =xU r ,|V s | dim This refers to V S Perform L1-norm accumulation operation on the row dimension. It is the matrix condition number. express The maximum singular value, express The minimum singular value, κ max It is the preset maximum condition number, diag{∑ r} refers to ∑ r A vector consisting of the diagonal elements.

[0027] According to a model update method provided by the present invention, the method further includes:

[0028] Calculate using the following formula (6)

[0029]

[0030] Here, linSolve represents a linear equation solver.

[0031] According to a model update method provided by the present invention, before obtaining the output data of the linear layers in the first model and the second model respectively based on the input data, the method further includes:

[0032] When the input data is a three-dimensional tensor, the input data is transformed to obtain input data represented by a two-dimensional matrix.

[0033] The present invention also provides a model update apparatus, comprising:

[0034] The decomposition module is used to perform singular value decomposition on the first matrix of any linear layer in the first model to obtain the second matrix;

[0035] The processing module is used to obtain the output data of the linear layer in the first model and the second model respectively based on the input data; the second model is obtained by updating the linear layer in the first model according to the second matrix;

[0036] The processing module is further configured to update the second matrix based on the output data of the linear layer in the first model and the second model, and update the linear layer in the second model based on the updated second matrix.

[0037] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the model update method as described above.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the model update method as described above.

[0039] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the model update method as described above.

[0040] The model update method, electronic device, and storage medium provided by this invention involve performing singular value decomposition on the first matrix of any linear layer in a first model to obtain a second matrix; obtaining the output data of the linear layers in the first and second models respectively based on the input data; and updating the linear layers in the first model based on the second matrix. In this scheme, singular value decomposition reduces the size of the first matrix in the model, compresses model parameters, reduces computational load when running the updated model, improves computational efficiency, and reduces memory usage. Furthermore, updating the second matrix based on the output data of the linear layers in the first and second models, and updating the linear layers in the second model based on the updated second matrix, utilizes model output data obtained from the input data during the second matrix update process. This makes the model output effect more suitable for the model application scenario, resulting in better model performance and more accurate output results. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is one of the flowcharts illustrating the model update method provided by the present invention;

[0043] Figure 2 This is the second flowchart of the model update method provided by the present invention;

[0044] Figure 3 This is a schematic diagram of the model update device provided by the present invention;

[0045] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0047] First, the application scenarios involved in the embodiments of the present invention will be introduced:

[0048] The method of this invention can be applied to artificial intelligence (AI) models, such as language models, speech recognition models, visual models (e.g., the Visual Transformer (ViT) model), image generation models, and image detection models. It is particularly applicable to linear layers within these models. For example, when applied to an image detection model, the input data can be image data, and image recognition can be performed, such as identifying license plate numbers of vehicles on a road through surveillance images. Similarly, when applied to a language model, the input data can be text information, such as a question, and language prediction can be performed, such as outputting the answer corresponding to the text information. Likewise, when applied to a speech recognition model, the input data can be speech information, and speech recognition can be performed, outputting the corresponding text information.

[0049] For a linear layer in an AI model: y = xW; where x is the input of the linear layer, W is the model weight matrix of the linear layer, and y is the output of the linear layer. Assume the size of W is [d, d], where d can range from several thousand to tens of thousands depending on the model size, and the number of linear layers in the model is typically in the hundreds. Therefore, in most AI models, especially large language models, the computational and memory usage of linear layers is often very high, and optimizing linear layers has a significant impact on the performance of large language models.

[0050] Singular Value Decomposition (SVD) is a neural network compression method used to decompose and reduce the weight matrix of a neural network model. SVD yields a low-rank matrix that is highly similar to the original weight matrix. Therefore, using SVD to decompose the weight matrix into a low-rank matrix can compress model parameters, saving memory and computational resources during model inference. Accordingly, a compressed neural network that uses a low-rank matrix to approximate the original model weight parameter matrix can be called a low-rank parameter neural network.

[0051] This method involves performing a low-rank decomposition on the weight matrix of each layer in the model:

[0052] W=U∑V T

[0053] Here, the U and V matrices (assuming they are square matrices) have dimensions [d, d]; the Σ matrix is ​​a diagonal matrix of dimension [d, d], whose diagonal elements contain the singular values ​​of the matrix. The explicit expression for the Σ matrix is:

[0054]

[0055] Where σ1>σ2>…>σ d These are the singular values ​​of matrix W (also representing the system's energy). Note: the singular values ​​are arranged in descending order, meaning that the recombination of σ1 and its corresponding u1, v1 (the first column vectors of U and V) is the decomposition closest to the original matrix W, in other words, it possesses the largest energy in the system. Therefore, SVD decomposition is often expressed as a summation expansion:

[0056]

[0057] Typically, the first r terms are sufficient to estimate W well.

[0058]

[0059] Here, r≤d is the rank of the matrix after SVD decomposition.

[0060] If the original matrix W is decomposed into an r-rank matrix W using SVD... r This can save a lot of computation and memory usage, as shown in Table 1 below, which compares the performance of a single linear layer:

[0061] Table 1

[0062]

[0063] diag{Σ r} refers to ∑ r A vector consisting of the diagonal elements (singular value vector);

[0064] In other words, if This method would be beneficial in terms of computational cost and memory usage.

[0065] To reduce computational and memory usage while maintaining model performance, it's crucial to find an appropriate value for r. Too small an r will result in poor model performance, while too large an r offers limited optimization for performance and memory usage. A common approach is to determine the value of r as a percentage of the sum of the singular values. For example, if the sum of r singular values ​​is less than 70% of the sum of the singular values ​​(total system energy), i.e., based on the condition σ1+σ2+…+σ r ≤S 70％ , Choose a value for r.

[0066] Therefore, the generalized expression of this formula is:

[0067]

[0068] Where P represents the percentage of the system's energy.

[0069] However, following the above method will negatively impact the model's performance. This is because the decomposed... The matrix only applies to the weights W and W r Direct differences were minimized (e.g., ||WW) r || 2 For different models and different scenarios, the accuracy of the output results may vary greatly.

[0070] Therefore, simply using SVD to decompose the weight matrix of a neural network model leads to a decline in model performance. The mainstream solution is to retrain the low-rank weight matrix. Compared to pure inference tasks, training significantly increases the implementation difficulty, and when the model size is large, the training cost is often enormous, with high computational load and low efficiency.

[0071] Therefore, the method in this embodiment of the invention introduces additional input data to optimize model updates in order to improve model performance, thereby enhancing the performance of the low-rank model and increasing the accuracy of the output results. By using a small amount of benchmark data, the entire model parameter update process only requires one forward pass operation, eliminating the need for conventional iterative model training and tuning, which significantly improves the accuracy of the low-rank model's output results.

[0072] The following is combined with Figures 1-4The technical solutions of the embodiments of the present invention will be described in detail with reference to specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes may not be described again in some examples.

[0073] Figure 1 This is a flowchart illustrating the model update method provided by the present invention. Figure 1 As shown, the method provided in this embodiment includes:

[0074] Step 101: For any linear layer in the first model, perform singular value decomposition on the first matrix of the linear layer to obtain the second matrix;

[0075] Optionally, singular value decomposition is performed on the first matrix, and the r value is obtained using formula (1), and the second matrix is ​​obtained based on the r value;

[0076]

[0077] Where P represents the preset percentage, σ i Let represent the singular values ​​of the first matrix, whose size is [d1, d2], where d is determined based on the size of d1 and d2.

[0078] For example, the first model can be a model built based on a neural network algorithm. The original first matrix (e.g., the weight matrix) W of the linear layer is decomposed by SVD, and the r value corresponding to the preset energy percentage P is selected according to formula (1) to obtain the intermediate matrix U. r V r diagonal matrix ∑ r ; Second matrix W r =U r Σ r V r T .

[0079] Where d1 can be equal to d2, in which case d = d1 = d2, or d1 ≠ d2, in which case d is the smaller value between d1 and d2.

[0080] Step 102: Based on the input data, obtain the output data of the linear layers in the first model and the second model respectively; the second model is obtained by updating the linear layers in the first model according to the second matrix;

[0081] Specifically, the original first matrix of the linear layer in the first model is replaced with the second matrix to obtain the updated second model.

[0082] For example, for a Large Language Model (LLM), a small amount of baseline data is prepared, such as multiple different cue texts. These cue texts are then input into the LLM. At each linear layer, the input data for that linear layer is saved. For a Generative Pre-Training (GPT) model, the input data may include, for example, a cue part (of length L0) and a text generation part (of length 1 multiplied by L1). Therefore, this means that each linear layer will save an input tensor of dimension [B, L0 + L1, d], where B is the number of cue texts.

[0083] Step 103: Update the second matrix based on the output data of the linear layers in the first and second models, and update the linear layers in the second model based on the updated second matrix.

[0084] Specifically, based on the output data y of the linear layer in the first model and the output data y of the linear layer in the second model... r Update the second matrix, for example, using the objective function. Optimize the second matrix, i.e., find the matrix that makes The smallest second matrix.

[0085] The linear layer is updated based on the updated second matrix to obtain the updated second model. Inference is then performed using the updated second model. The computational cost and memory usage are the same as the traditional SVD decomposition method, but the model's performance is more in line with the current scenario, and the model's output data is more accurate.

[0086] Table 2 below compares the method of this invention with the traditional SVD decomposition method:

[0087] Table 2

[0088]

[0089] As can be seen from Table 2, the computational cost and weight memory usage are the same as those of the traditional SVD decomposition method. If the model size is large, i.e., d is large, the computational cost is less than that of the original model, and the weight memory usage is also smaller.

[0090] Here, ⊙ represents element-vector multiplication. r ⊙diag{∑ r The size of} is

[0091] [d,r]⊙[1,r], that is, in each diag{∑ r The elements of} will interact with U r Multiply all elements in each column.

[0092] In this embodiment, for any linear layer in the first model, the first matrix of the linear layer is decomposed using singular value decomposition to obtain a second matrix. Based on the input data, the output data of the linear layers in the first and second models are obtained respectively. The second model is obtained by updating the linear layers in the first model based on the second matrix. In this scheme, singular value decomposition reduces the size of the first matrix in the model, compresses the model parameters, reduces computational load when running the updated model, resulting in higher computational efficiency and reduced memory usage. Furthermore, the second matrix is ​​updated based on the output data of the linear layers in the first and second models, and the linear layers in the second model are updated based on the updated second matrix. Since the update of the second matrix utilizes model output data obtained from the input data, the model output effect is more adapted to the model application scenario, resulting in better model performance and more accurate output results.

[0093] The updated model described above has high computational efficiency when executed on the device, does not require backpropagation to update model parameters, has low memory consumption, and produces better output results, which are closer to the original uncompressed model than existing model compression methods.

[0094] Optionally, such as Figure 2 As shown, step 103 can be achieved in the following way:

[0095] Step 1031: Based on the output data of the linear layer in the first model and the second model, update at least one intermediate matrix of the second matrix to obtain the updated second matrix;

[0096] Step 1032: Update the linear layers in the second model based on the updated second matrix;

[0097] Wherein, the second matrix W r =U r Σ r V r T U r and V r Let Σ be the intermediate matrix. r It is a diagonal matrix.

[0098] Optionally, for any of the intermediate matrices, the intermediate matrix is ​​updated using the objective function of formula (2) based on the output data of the linear layers in the first model and the second model;

[0099]

[0100] Where y = xW + b, b is the deviation parameter, x is the input data, and W is the first matrix;

[0101] ⊙ represents element-vector multiplication.

[0102] Specifically, updating the second matrix can be achieved by updating at least one of its intermediate matrices.

[0103] To optimize the objective function in formula (2), assume that for the intermediate matrix V R After optimization, the objective function of formula (2) can be written as:

[0104] Because J(V) s If is a convex function, its zero gradient location corresponds to its minimum value, which is the optimal point of the function.

[0105]

[0106] Solving this linear equation yields the following formula:

[0107]

[0108] Optionally, using V s For V r Update the intermediate matrix of the second matrix using formulas (3)-(5);

[0109]

[0110]

[0111]

[0112] Among them, X r =xU r ,|V s | dim＝0 This refers to V s Perform L1-norm accumulation operation on the row dimension. It is the matrix condition number. express The maximum singular value, express The minimum singular value, k max It is the preset maximum condition number, diag{∑ r} refers to ∑ r A vector consisting of the diagonal elements.

[0113] Since a matrix approaches non-invertible status when its condition number is too large, this usually indicates a linear dependency in the data. The calculation results are highly likely to be inaccurate. Therefore, in each linear layer that requires parameter updates, when When the condition number is too large, we will adjust V. S Performing L1-norm regularization, i.e., multiplying by α, can ensure the stability of parameter updates to a certain extent and prevent excessively large values ​​from affecting the final model's performance.

[0114] In the above implementation, updating the intermediate matrix of the second matrix has low complexity and high efficiency.

[0115] Alternatively, the following formula (6) can be used to calculate...

[0116]

[0117] Here, linSolve represents a linear equation solver.

[0118] Specifically, for better numerical stability, it can be obtained by solving a linear equation of the form xA = B.

[0119]

[0120] Wherein, linSolve can be any linear equation solver, and the embodiments of the present invention are not limited to this.

[0121] Optionally, before step 102, the following operations can be performed:

[0122] When the input data is a three-dimensional tensor, the input data is transformed to obtain input data represented by a two-dimensional matrix.

[0123] Specifically, if the input data is a three-dimensional tensor, the input data x of each linear layer is reshaped to transform it from a 3D tensor into a 2D matrix. For example, the input data dimension [B,L0+L1,d] can be transformed into [N,d], where N = B × (L0+L1).

[0124] The following is a simulation experiment based on a large language model:

[0125] SVD decomposition is performed on the four linear layers of each attention module in the Omni-Perception Pre-Trainer (OPT) model. The differences in the output probabilities of the original neural network model are compared between traditional SVD decomposition methods and the method of this invention.

[0126] In this experiment, the parameters of the low-rank neural network model of the present invention were updated using 50 benchmark samples. Then, 10 different samples were used to test whether the method of the present invention could more closely approximate the output of the original neural network model compared to the low-rank neural network model after traditional SVD decomposition.

[0127] Define the following comparison formula:

[0128]

[0129]

[0130] in, These represent the output probabilities of the original model, the traditional SVD low-rank model, and the low-rank model of this invention, respectively; where T = 50265 is the dictionary size of the OPT large model, and N varies depending on the value of each sample. Therefore, positive ∈ k The value represents the relative model performance gain of the low-rank neural network of this invention compared to the traditional SVD low-rank neural network, while the negative ∈ k The value represents a decrease in the relative performance of the model.

[0131] Table 3

[0132]

[0133] As can be seen from Table 3 of the experimental results, in most cases, the low-rank neural network model of the present invention has a more accurate performance than the traditional SVD low-rank neural network model on different samples and different singular values.

[0134] The model update apparatus provided by the present invention is described below. The model update apparatus described below and the model update method described above can be referred to in correspondence.

[0135] Figure 3 This is a schematic diagram of the model update device provided by the present invention. Figure 3 As shown, the model update apparatus provided in this embodiment includes:

[0136] The decomposition module 310 is used to perform singular value decomposition on the first matrix of any linear layer in the first model to obtain a second matrix.

[0137] Processing module 320 is used to obtain the output data of the linear layer in the first model and the second model respectively according to the input data; the second model is obtained by updating the linear layer in the first model according to the second matrix;

[0138] The processing module 320 is further configured to update the second matrix based on the output data of the linear layer in the first model and the second model, and update the linear layer in the second model based on the updated second matrix.

[0139] Optionally, the decomposition module 310 is specifically used for:

[0140] The first matrix is ​​subjected to singular value decomposition, and the r value is obtained using formula (1). The second matrix is ​​obtained based on the r value.

[0141]

[0142] Where P represents the preset percentage, σ i Let represent the singular values ​​of the first matrix, whose size is [d1, d2], where d is determined based on the size of d1 and d2.

[0143] Optionally, the processing module 320 is specifically used for:

[0144] Based on the output data of the linear layers in the first model and the second model, at least one intermediate matrix of the second matrix is ​​updated to obtain the updated second matrix;

[0145] Wherein, the second matrix W r =U r Σ r V r T U r and V r Let Σ be the intermediate matrix. r It is a diagonal matrix.

[0146] Optionally, the processing module 320 is specifically used for:

[0147] For any of the intermediate matrices, the intermediate matrices are updated using the objective function as shown in formula (2) based on the output data of the linear layers in the first model and the second model.

[0148]

[0149] Where y = xW + b, b is the deviation parameter, x is the input data, and W is the first matrix;

[0150] ⊙ represents element-vector multiplication.

[0151] Optionally, the processing module 320 is specifically used for:

[0152] Based on the output data of the linear layers in the first model and the second model, the intermediate matrix of the second matrix is ​​updated using formulas (3)-(5);

[0153]

[0154]

[0155]

[0156] Among them, X r =xU r ,|V s | dim＝0 This refers to V s Perform L1-norm accumulation operation on the row dimension. It is the matrix condition number. express The maximum singular value, express The minimum singular value, k max It is the preset maximum condition number, diag{∑ r} refers to ∑ r A vector consisting of the diagonal elements.

[0157] Optionally, the processing module 320 is further configured to:

[0158] Calculate using the following formula (6)

[0159]

[0160] Here, linSolve represents a linear equation solver.

[0161] Optionally, the processing module 320 is further configured to:

[0162] When the input data is a three-dimensional tensor, the input data is transformed to obtain input data represented by a two-dimensional matrix.

[0163] The apparatus of this invention is used to execute the method in any of the foregoing method embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0164] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4As shown, the electronic device may include a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a model update method, which includes: for any linear layer in the first model, performing singular value decomposition on the first matrix of the linear layer to obtain a second matrix;

[0165] Based on the input data, the output data of the linear layers in the first model and the second model are obtained respectively; the second model is obtained by updating the linear layers in the first model according to the second matrix;

[0166] The second matrix is ​​updated based on the output data of the linear layers in the first model and the second model, and the linear layers in the second model are updated based on the updated second matrix.

[0167] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0168] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, the computer program being executed by a processor, the computer being able to execute the model update method provided by the above methods, the method including: for any linear layer in the first model, performing singular value decomposition on the first matrix of the linear layer to obtain a second matrix;

[0169] Based on the input data, the output data of the linear layers in the first model and the second model are obtained respectively; the second model is obtained by updating the linear layers in the first model according to the second matrix;

[0170] The second matrix is ​​updated based on the output data of the linear layers in the first model and the second model, and the linear layers in the second model are updated based on the updated second matrix.

[0171] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the model update method provided by the above methods, the method comprising: for any linear layer in a first model, performing singular value decomposition on a first matrix of the linear layer to obtain a second matrix;

[0172] Based on the input data, the output data of the linear layers in the first model and the second model are obtained respectively; the second model is obtained by updating the linear layers in the first model according to the second matrix;

[0173] The second matrix is ​​updated based on the output data of the linear layers in the first model and the second model, and the linear layers in the second model are updated based on the updated second matrix.

[0174] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0175] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0176] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A model update method, characterized in that, include: For any linear layer in the first model, the first matrix of the linear layer is subjected to singular value decomposition to obtain the second matrix; Based on the input data, obtain the output data of the linear layers in the first model and the second model respectively; The second model is obtained by updating the linear layer in the first model based on the second matrix; The second matrix is ​​updated based on the output data of the linear layers in the first model and the second model, and the linear layers in the second model are updated based on the updated second matrix; The first model is a large language model; the input data includes a prompting part and a text generation part; The step of updating the second matrix based on the output data of the linear layers in the first and second models includes: Based on the output data of the linear layers in the first model and the second model, at least one intermediate matrix of the second matrix is ​​updated to obtain the updated second matrix; Among them, the second matrix , and The intermediate matrix, It is a diagonal matrix; The step of updating at least one intermediate matrix of the second matrix based on the output data of the linear layers in the first model and the second model includes: For any of the intermediate matrices, the intermediate matrices are updated using the objective function as shown in formula (2) based on the output data of the linear layers in the first model and the second model. ；（2） in, , For input data, This is the first matrix; , This represents element-wise vector multiplication. The step of updating the intermediate matrix using the objective function as shown in formula (2) based on the output data of the linear layers in the first model and the second model includes: Based on the output data of the linear layers in the first model and the second model, the intermediate matrix of the second matrix is ​​updated using formulas (3)-(5); （3） （4） （5） in, , It refers to Perform L1-norm accumulation operation on the row dimension. , It is the matrix condition number. express The maximum singular value, express The minimum singular value, It is the preset maximum condition number. It refers to A vector consisting of the diagonal elements.

2. The model update method according to claim 1, characterized in that, The step of performing singular value decomposition on the first matrix of the linear layer to obtain the second matrix includes: Perform singular value decomposition on the first matrix and obtain the result using formula (1). Value, based on the It is worthwhile to obtain the second matrix; （1） Where P represents the preset percentage, Represents the singular values ​​of the first matrix, the size of the first matrix is ​​[ , ], According to and The size is determined.

3. The model update method according to claim 1, characterized in that, The method further includes: Calculate using the following formula (6) ; （6） Here, linSolve represents a linear equation solver.

4. The model update method according to claim 1 or 2, characterized in that, Before obtaining the output data of the linear layers in the first model and the second model respectively based on the input data, the method further includes: When the input data is a three-dimensional tensor, the input data is transformed to obtain input data represented by a two-dimensional matrix.

5. A model update device, characterized in that, include: The decomposition module is used to perform singular value decomposition on the first matrix of any linear layer in the first model to obtain the second matrix; The processing module is used to obtain the output data of the linear layers in the first model and the second model respectively based on the input data; The second model is obtained by updating the linear layer in the first model based on the second matrix; The processing module is further configured to update the second matrix based on the output data of the linear layer in the first model and the second model, and update the linear layer in the second model based on the updated second matrix; The first model is a large language model; the input data includes a prompting part and a text generation part; The step of updating the second matrix based on the output data of the linear layers in the first and second models includes: Based on the output data of the linear layers in the first model and the second model, at least one intermediate matrix of the second matrix is ​​updated to obtain the updated second matrix; Among them, the second matrix , and The intermediate matrix, It is a diagonal matrix; The step of updating at least one intermediate matrix of the second matrix based on the output data of the linear layers in the first model and the second model includes: For any of the intermediate matrices, the intermediate matrices are updated using the objective function as shown in formula (2) based on the output data of the linear layers in the first model and the second model. ；（2） in, , For input data, This is the first matrix; , This represents element-wise vector multiplication. The step of updating the intermediate matrix using the objective function as shown in formula (2) based on the output data of the linear layers in the first model and the second model includes: Based on the output data of the linear layers in the first model and the second model, the intermediate matrix of the second matrix is ​​updated using formulas (3)-(5); （3） （4） （5） in, , It refers to Perform L1-norm accumulation operation on the row dimension. , It is the matrix condition number. express The maximum singular value, express The minimum singular value, It is the preset maximum condition number. It refers to A vector consisting of the diagonal elements.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the model update method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the model update method as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the model update method as described in any one of claims 1 to 4.

Images