Training method, device and equipment for pre-training large model

Abstract

The invention provides a training method, device and equipment for a pre-trained large model, and relates to the technical field of artificial intelligence, and the method comprises the steps: carrying out the initialization of a first low-rank matrix and a second low-rank matrix based on the type of a preset gradient matrix in a fine tuning process of the pre-trained large model, and obtaining a first low-rank matrix and a second low-rank matrix; and iterative training is performed on the to-be-updated model based on the initialized first low-rank matrix and the initialized second low-rank matrix, so that the problem of gradient disappearance or explosion in the model training process caused by performing all-zero initialization on the low-rank matrixes can be avoided, and the effect of improving the convergence speed can be achieved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to a training method, device and equipment for pre-training a large model. Background Art

[0002] In recent years, large language models with hundreds of billions of parameters have demonstrated excellent performance in a variety of tasks. Many applications in natural language processing rely on adapting these large-scale pre-trained models to multiple downstream applications, and this adaptation is usually achieved through fine-tuning. The traditional fine-tuning process updates all parameters of the pre-trained large model to improve the model's performance on a specific task. This process is now called full fine-tuning. Designing efficient and accurate fine-tuning methods is an important goal at present, which will not only improve the application effect of large language models, but also promote their widespread application in various fields.

[0003] Currently, in the process of fine-tuning a pre-trained large model, the initialization method of the first low-rank matrix and the second low-rank matrix uses a combination of all-zero and Gaussian initialization. This approach may cause convergence difficulties in the optimization process, thereby affecting the model performance. Summary of the invention

[0004] The present application provides a training method, device and equipment for a pre-trained large model, which can eliminate the problem of gradient disappearance or explosion caused by using all-zero initialization or Gaussian initialization during fine-tuning of the pre-trained large model.

[0005] The present application provides a method for training a pre-trained large model, comprising: Get training data and preset gradient matrix; A model to be updated including a low-rank module is obtained based on the pre-trained large model; the low-rank module includes at least a first low-rank matrix and a second low-rank matrix; When the gradient matrix is ​​preset as a diagonal matrix, the unit matrix, the second all-zero matrix, and the third all-zero matrix are stacked vertically to initialize the first low-rank matrix, and the unit matrix and the first all-zero matrix are stacked horizontally to initialize the second low-rank matrix; When the preset gradient matrix is ​​a non-diagonal matrix, performing singular value decomposition on the gradient matrix to initialize the first decomposition matrix and the second decomposition matrix; Based on the initialized first low-rank matrix and the second low-rank matrix, the training data is input into the model to be updated for iterative training, and a trained target model is obtained when an iteration stop condition is met.

[0006] In one embodiment of the present application, a model to be updated including a low-rank module is obtained based on a pre-trained large model, including: Obtaining a target layer in the pre-trained large model; A low-rank module is added to the target layer to obtain the model to be updated; the low-rank module is composed of two low-rank decomposition networks in series, and each layer of the low-rank decomposition network corresponds to a low-rank matrix.

[0007] In one embodiment of the present application, adding a low-rank module to the target layer to obtain the model to be updated includes: Two low-rank modules are added to the target layer to obtain the model to be updated.

[0008] In one embodiment of the present application, two low-rank modules are added to the target layer to obtain the model to be updated, including: Two low-rank modules are connected in parallel to the original weight path of the target layer to obtain the model to be updated.

[0009] In one embodiment of the present application, two low-rank modules are added to the target layer to obtain the model to be updated, including: A fine-tuning branch is connected in parallel to the original weight path of the target layer, and the fine-tuning branch includes two parallel low-rank modules.

[0010] In one embodiment of the present application, one of the low-rank modules is a half-precision floating-point branch, and the other low-rank module is a single-precision floating-point branch.

[0011] In one embodiment of the present application, based on the initialized first low-rank matrix and the second low-rank matrix, the training data is input into the model to be updated for iterative training, including: After initializing the first matrix, the second matrix, the third matrix, and the fourth matrix, the first matrix, the second matrix, the third matrix, and the fourth matrix are quantized into single-precision floating-point data; the first matrix and the second matrix are respectively the first low-rank matrix and the second low-rank matrix of the half-precision floating-point branch, and the third matrix and the fourth matrix are respectively the first low-rank matrix and the second low-rank matrix of the single-precision floating-point branch; For any first target layer: quantize the input data into half-precision floating-point data, freeze the parameters in the pre-trained large model and the single-precision floating-point branch in the model to be updated, and perform half-precision training on the first target layer; the first target layer is an odd-numbered layer in the target layer; For any second target layer: quantize the input data into single-precision floating-point data, freeze the parameters in the pre-trained large model and the half-precision floating-point branch in the model to be updated, and perform single-precision training on the second target layer; the second target layer is an even-numbered layer in the target layer; The weight matrix of the first target layer is updated based on the first matrix and the second matrix obtained through training, and the weight matrix of the second target layer is updated based on the third matrix and the fourth matrix obtained through training.

[0012] To achieve the above objectives and other related objectives, the present application provides a training device for pre-training a large model, comprising: Data acquisition module, used to obtain training data and preset gradient matrix; A model processing module, used to obtain a model to be updated including a low-rank module based on a pre-trained large model; the low-rank module includes at least a first low-rank matrix and a second low-rank matrix; A first initialization module is used for, when the preset gradient matrix is ​​a diagonal matrix, vertically stacking the unit matrix, the second all-zero matrix, and the third all-zero matrix to initialize the first low-rank matrix, and horizontally stacking the unit matrix and the first all-zero matrix to initialize the second low-rank matrix; A second initialization module is used for performing singular value decomposition on the gradient matrix when the preset gradient matrix is ​​a non-diagonal matrix, so as to initialize the first decomposition matrix and the second decomposition matrix; The model updating module is used to input the training data into the model to be updated for iterative training based on the initialized first low-rank matrix and the second low-rank matrix, and obtain a trained target model when the iteration stopping condition is met.

[0013] To achieve the above objectives and other related objectives, the present application also provides an electronic device, the electronic device comprising: one or more processors; a memory for storing program code executable by the processor; Wherein, the processor is configured to execute the program code to implement the above-mentioned training method of the pre-trained large model.

[0014] As described above, the present application provides a method, device and equipment for training a pre-trained large model, which has the following beneficial effects: A training method for a pre-trained large model in the present application can avoid the problem of gradient disappearance or explosion caused by all-zero initialization of the low-rank matrix during model training, by initializing a first low-rank matrix and a second low-rank matrix based on a preset gradient matrix type during fine-tuning of the pre-trained large model, and iteratively training the model to be updated based on the initialized first low-rank matrix and the second low-rank matrix, thereby achieving the effect of improving the convergence speed.

[0015] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings: Figure 1 is a flowchart of a training method for a pre-trained large model shown in an exemplary embodiment of the present application; Figure 2 is a schematic diagram of a FoRA architecture in a model to be updated shown in an exemplary embodiment of the present application; Figure 3 is a schematic diagram of a FoRA architecture in a model to be updated shown in another exemplary embodiment of the present application; Figure 4 is a schematic diagram of a FoRA architecture in a model to be updated shown in another exemplary embodiment of the present application; Figure 5 It is a block diagram of a training device for a pre-trained large model shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0017] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0018] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0019] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application. However, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present application difficult to understand.

[0020] In recent years, large language models (LLMs) with hundreds of billions of parameters have demonstrated excellent performance in various tasks. Many applications in natural language processing rely on adapting these large-scale pre-trained language models to multiple downstream applications, and this adaptation is usually achieved through fine-tuning. Designing efficient and accurate fine-tuning methods is an important goal at present, which will not only improve the application effect of large language models, but also promote their widespread application in various fields. The traditional fine-tuning process updates all parameters of the pre-trained large model to improve the performance of the model on a specific task. This process is now called full fine-tuning.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0022] Low-Rank Adaptation (LoRA): An efficient fine-tuning method that adjusts the parameters of a pre-trained large model by introducing a low-rank matrix, thereby significantly reducing the computing resources and storage space required for training while maintaining model performance.

[0023] Full Fine-tuning (FF): refers to retraining all parameters of a pre-trained large model to adapt to a specific task or dataset to maximize the performance of the model.

[0024] Singular Value Decomposition (SVD): A matrix decomposition technique that decomposes an arbitrary matrix into the product of three matrices, revealing the structural characteristics and important components of the matrix. It is often used in fields such as dimensionality reduction, data compression, and recommendation systems.

[0025] Vanishing Gradient / Exploding Gradient: Gradient vanishing means that during the back propagation process of a deep neural network, the gradient gradually becomes very small, resulting in the inability of the model to effectively update parameters; while gradient explosion means that the gradient increases sharply during back propagation, resulting in excessive parameter updates, making model training unstable or unable to converge.

[0026] Large Language Models (LLMs): Advanced language models with a large number of parameters, such as GPT and LLaMa. These models perform well in understanding and generating natural language and have important application value for a variety of natural language processing tasks.

[0027] Pre-trained models are a technique in the field of natural language processing (NLP) that allows models to perform unsupervised learning on large amounts of text data to learn the common features of language. These models can be designed to understand or generate natural language, and can then be fine-tuned for specific tasks through supervised learning, such as text classification, question-answering systems, machine translation, etc.

[0028] See also Figure 1 , Figure 1 FIG. 1 is a flowchart of a training method for a pre-trained large model shown in an exemplary embodiment of the present application. Figure 1 It can be seen that the training method of the pre-trained large model may include: Step S110, obtaining training data and presetting a gradient matrix.

[0029] In one embodiment of the present application, training data and a preset gradient matrix may be obtained. Before executing the training method of the pre-trained large model provided in the embodiment of the present application, a small batch of full-scale fine-tuning may be performed on the pre-trained large model to obtain a preset gradient matrix, which may be used to initialize the first low-rank matrix and the second low-rank matrix. The training data may be determined according to the specific task of judgment or recognition, for example, the training data may include text sentiment classification samples, question-answer samples, or medical data-disease classification samples.

[0030] It should be noted that the execution subject of the embodiment of the present application may be a server or a server cluster or other equipment. The pre-trained large model may include a large number of parameters and has a large scale.

[0031] Step S120, obtaining a model to be updated including a low-rank module based on the pre-trained large model.

[0032] The low-rank module includes at least a first low-rank matrix and a second low-rank matrix.

[0033] In one embodiment of the present application, a model to be updated including a low-rank module can be obtained based on a pre-trained large model. A model to be updated including a low-rank module can be obtained based on a low-rank adaptive optimization strategy and a pre-trained large model. Low-Rank Adaptation (LoRA) is an efficient fine-tuning method, particularly suitable for large pre-trained large models. Its core idea is to adjust the weights of the pre-trained large model by introducing a low-rank matrix instead of directly modifying the parameters of the original model. This method can significantly reduce the number of additional parameters and computational costs while maintaining the performance of the pre-trained large model.

[0034] Step S130, when the preset gradient matrix is ​​a diagonal matrix, the unit matrix, the second all-zero matrix, and the third all-zero matrix are vertically stacked to initialize the first low-rank matrix, and the unit matrix and the first all-zero matrix are horizontally stacked to initialize the second low-rank matrix.

[0035] In one embodiment of the present application, when the preset gradient matrix is ​​a diagonal matrix, the unit matrix, the second all-zero matrix, and the third all-zero matrix can be vertically stacked to initialize the first low-rank matrix, and the unit matrix and the first all-zero matrix can be horizontally stacked to initialize the second low-rank matrix.

[0036] For example, in order to avoid using all-zero initialization, the output of the low-rank module is consistent with the output of the pre-trained large model. We let the first low-rank matrix after low-rank decomposition be and the second low-rank matrix The initialization parameters are and ,in and ,and , and the fine-tuning strategy only needs to satisfy: ; This can achieve the condition that the initial values ​​of the two models are the same.

[0037] In addition, the matrix and The assignment of is very important for the subsequent fine-tuning optimization strategy. Good parameter initialization and hyperparameters The selection of can make the algorithm achieve the same accuracy as full-scale fine-tuning in the LoRA-based fine-tuning process. We make the full-scale fine-tuning process as follows: ; in, represents the loss function, Represents the learning rate. It represents the weight after full fine-tuning optimization. Represents the weights of the pre-trained large model; Represents the preset gradient matrix. Similarly, the LoRA-based fine-tuning process is expressed as: ; ; ; Since the learning rate The value of is small, so we ignore its second-order term, then the optimized matrix It can be approximately expressed as: ; In order to make the quantization results of low-rank adaptation as close as possible to those of full-scale fine-tuning, the two quantization strategies Matrix distance: ; in, is the Frobenius norm, is the Euclidean distance, and the square of the Frobenius norm is used to measure the difference between the matrices. Here, the comprehensive difference between the three is calculated. and From the definition of and The highest rank is , so the rank of the sum of the first two terms in the above formula is at most In order to make the low-rank adapted network weights close to the full fine-tuned weights, the above process is regarded as a search based on The process of low-rank optimal approximation: , in Indicates the rank of the calculated matrix. In addition, let ,and is a positive definite matrix, and argmin is the variable that makes the minimum value hold, which is also the optimal solution.

[0038] exist When is a diagonal matrix, according to the Eckart–Young–Mirsky theorem, its rank does not exceed The best approximation is to keep only the front diagonal The new diagonal matrix is ​​generated by setting the largest element and the remaining elements to 0. elements on the diagonal, we can get: , ; in, express The unit matrix, , and are all zero matrices, Represents the set of real numbers.

[0039] Step S140: When the preset gradient matrix is ​​a non-diagonal matrix, singular value decomposition is performed on the gradient matrix to initialize the first decomposition matrix and the second decomposition matrix.

[0040] In one embodiment of the present application, when the preset gradient matrix is ​​a non-diagonal matrix, singular value decomposition may be performed on the gradient matrix to initialize the first decomposition matrix and the second decomposition matrix.

[0041] Exemplarily, when the preset gradient matrix is ​​a non-diagonal matrix, the preset gradient matrix is ​​decomposed into ,in , is an orthogonal matrix, and is a diagonal matrix, substitute it into The low-rank optimal approximate solution of can be expressed as: ; Therefore, the object to be approached is Transformed into a diagonal matrix , and it is easy to know and The rank has the same properties. Then it can be processed as a diagonal matrix, , ; in, is an orthogonal matrix forward Slice matrix of columns, is an orthogonal matrix Take the first Go to Slice matrix of rows.

[0042] It should be noted that by executing step S130 and step S140, the output of the model before the fine-tuning process can be kept consistent with the output of the pre-trained large model only by presetting the gradient matrix. The low-rank matrix initialized in this way can achieve an effect close to full fine-tuning in the early stage of training, greatly reducing the number of optimization iterations and accelerating the fine-tuning training process. At the same time, it effectively avoids the convergence problem caused by the disappearance or explosion of the gradient caused by all-zero initialization, thereby providing a more stable foundation for subsequent fine-tuning.

[0043] Step S150, based on the initialized first low-rank matrix and the second low-rank matrix, input the training data into the model to be updated for iterative training, and obtain the trained target model when the iteration stop condition is met.

[0044] In one embodiment of the present application, based on the initialized first low-rank matrix and the second low-rank matrix, the training data can be input into the model to be updated for iterative training, and the trained target model can be obtained when the iteration stop condition is met.

[0045] In one embodiment, the process of obtaining the model to be updated including the low-rank module based on the pre-trained large model in step S120 may include step S121 and step S122.

[0046] Step S121, obtaining the target layer in the pre-trained large model.

[0047] In one embodiment, the target layer in the pre-trained large model pre-set by the designer can be obtained. "Layer" refers to different components or modules in the pre-trained large model, and each layer is responsible for performing a specific type of transformation on the input data. When fine-tuning the pre-trained large model, the target layer can refer to a layer containing a trainable weight matrix, and the target layer can optimize the fine-tuning process through low-rank decomposition. The specific layer type depends on the model architecture used and the task requirements.

[0048] Exemplarily, the target layer may include a fully connected layer (Dense / LinearLayers), a convolutional layer (ConvolutionalLayers), a self-attention mechanism layer (Self-AttentionLayers), an embedding layer (EmbeddingLayers) or an LSTM / GRU layer (LongShort-TermMemory / GatedRecurrentUnitLayers), etc.

[0049] Step S122, adding a low-rank module to the target layer to obtain the model to be updated.

[0050] The low-rank module is composed of two low-rank decomposition networks connected in series, and each layer of the low-rank decomposition network corresponds to a low-rank matrix. The low-rank matrix includes the first low-rank matrix and the second low-rank matrix mentioned above.

[0051] See also Figure 2 , which is a schematic diagram of the FoRA architecture in the model to be updated according to an exemplary embodiment of the present application. represents the input of the model, represents the vector to be activated, is the length of the input, represents the weight of the pre-trained large model, represents the first low-rank matrix, represents the second low-rank matrix, is the rank of the matrix.

[0052] Among all the techniques for efficiently fine-tuning large language models, one of the most popular methods is Low-Rank Adaptation (LoRA), which replaces the newly added parameter model weights by introducing a low-rank matrix, approximating the original parameter model with less parameter cost during the fine-tuning process, thereby achieving more efficient training. In addition, during the inference process, LoRA can merge the newly added parameters into the original model, thereby avoiding increasing the inference time of the model. The pre-trained model weights are set to , the change of the model during fine-tuning optimization is expressed as . Use low-rank decomposition to change the matrix To split: ,in and are all low-rank matrices that can be optimized. is the rank of the variable matrix. During the optimization process, the newly added matrix With the original parameter matrix In parallel, they complete the forward / backward propagation process together: .

[0053] In one embodiment, adding a low-rank module to a target layer to obtain a model to be updated includes: adding two low-rank modules to the target layer to obtain a model to be updated.

[0054] See also Figure 3 , which is a schematic diagram of the FoRA architecture in the model to be updated according to another exemplary embodiment of the present application. Wherein, input represents the input of the model, represents the weight of the pre-trained large model, represents the first low-rank matrix, represents the second low-rank matrix, represents the third low-rank matrix, represents the fourth low-rank matrix, and is the rank of the matrix. In one possible implementation, adding a low-rank module to the target layer to obtain the model to be updated includes: connecting two low-rank modules in parallel in the original weight path of the target layer to obtain the model to be updated. The FoRA architecture can be a LoRA based on residual design (ResLoRA).

[0055] The LoRA strategy based on residual design is an improvement on the original LoRA, which aims to reduce the eigenvector in the algorithm to a low dimension. The information loss problem generated during the mapping process is improved. An additional residual design ResLoRABlock is added next to the new matrix in parallel to reduce The influence of too small size on the optimization process. The structure of ResLoRABlock is the same as that of the original LoRA, and the dimension reduction matrix is ​​expressed as , the reduction matrix is ​​expressed as , the forward propagation process of ResLoRA is expressed as: .

[0056] See also Figure 4 , which is a schematic diagram of the FoRA architecture in the model to be updated according to another exemplary embodiment of the present application. represents the first low-rank matrix, represents the second low-rank matrix, represents the third low-rank matrix, Represents the fourth low-rank matrix, FP16-Branch represents a half-precision floating-point branch, and FP32-Branch represents a single-precision floating-point branch. In another possible implementation, a low-rank module is added to the target layer to obtain a model to be updated, including: a fine-tuning branch is connected in parallel to the original weight path of the target layer, and the fine-tuning branch includes two parallel low-rank modules. In one embodiment, one of the low-rank modules is a half-precision floating-point branch, and the other low-rank module is a single-precision floating-point branch. In another embodiment, both low-rank modules can be single-precision floating-point branches, or both low-rank modules are half-precision floating-point branches. Among them, FP16-Branch has the same structure as FP32-Branch, FP16-Branch is responsible for half-precision floating-point operations, and FP32-Branch is responsible for single-precision floating-point operations.

[0057] It should be noted that Figure 4 The FoRA architecture shown can also be called the In-place Residual Structure, which is a design method of residual structure. It avoids additional network branch structure by replacing parameters. So that it can better adapt to our iterative training strategy. This method not only effectively compensates for the accuracy loss of LoRA, but also significantly reduces the memory usage compared to traditional ResLoRA.

[0058] One of the low-rank adaptive structures focuses on the optimization process of half-precision FP6, aiming to quickly iterate to approach the optimal solution, which is the half-precision floating point branch (FP16-Branch); the other low-rank adaptive structure is optimized for single floating point FP32, ensuring that the model can achieve better accuracy when processing fine-grained data. This patent names it as the single-precision floating point branch (FP32-Branch). This dual optimization module cooperates with each other, and can speed up the optimization process while ensuring the stability of accuracy during alternating iterative training.

[0059] In addition, if the optimization conditions are insufficient (such as insufficient cache space), the two branch structures can remain the same. In different iteration rounds, the branches can be quickly exchanged only by replacing parameters, thus avoiding the introduction of additional branch structures.

[0060] In one embodiment, the process of inputting training data into the model to be updated for iterative training based on the initialized first low-rank matrix and the second low-rank matrix in step S150 may include steps S151 to S154.

[0061] Step S151, after initializing the first matrix, the second matrix, the third matrix, and the fourth matrix, quantize the first matrix, the second matrix, the third matrix, and the fourth matrix into single-precision floating-point data; the first matrix and the second matrix are respectively the first low-rank matrix and the second low-rank matrix of the half-precision floating-point branch, and the third matrix and the fourth matrix are respectively the first low-rank matrix and the second low-rank matrix of the single-precision floating-point branch.

[0062] Step S152, for any first target layer: quantize the input data into half-precision floating-point data, freeze the parameters in the pre-trained large model and the single-precision floating-point branch in the model to be updated, and perform half-precision training on the first target layer; the first target layer is an odd-numbered layer in the target layer.

[0063] Step S153, for any second target layer: quantize the input data into single-precision floating-point data, freeze the parameters in the pre-trained large model and the half-precision floating-point branch in the model to be updated, and perform single-precision training on the second target layer; the second target layer is an even-numbered layer in the target layer.

[0064] Step S154, updating the weight matrix of the first target layer based on the first matrix and the second matrix obtained through training, and updating the weight matrix of the second target layer based on the third matrix and the fourth matrix obtained through training.

[0065] In a possible implementation, if the target layer includes multiple layers connected in sequence, each target layer can be alternately arranged as a first target layer and a second target layer, and the output of the first target layer can be used as the input of the next second target layer, and the output of the second target layer can be used as the input of the next first target layer. In another possible implementation, if the target layer includes a plurality of sequentially connected layers, a first preset number of target layers may be set as the first target layer, a second preset number of target layers after the first target layer may be set as the second target layer, and a third preset number of target layers after the second target layer may be set as the first target layer. The sum of the first preset number, the second preset number, and the third preset number may be equal to the number of target layers. Among the first preset number of consecutive first target layers, the output of the last first target layer may be used as the input of the first second target layer, and the output of the last second target layer may not be used as the input of the first first target layer among the third preset number of consecutive first target layers.

[0066] In one embodiment of the present application, on the basis of the pre-trained large model, the updated weight matrix of the first target layer and the updated weight matrix of the second target layer can be determined as the update parameters of each target layer in the pre-trained large model to obtain the target model.

[0067] In one embodiment, the iteration stopping condition may be that the iteration is repeated three times.

[0068] It should be noted that the alternating iterative training strategy provided by steps S151 to S154 can avoid loading all new parameters into the video memory at the same time, thereby reducing the computational burden. This strategy not only improves the training efficiency, but also ensures effective model fine-tuning in resource-limited environments, making the algorithm more feasible and flexible in practical applications. There are two advantages to using the parameters of the last round of the FP16-Branch branch as the initialization parameters of the third round of the FP32-Branch branch: first, these parameter matrices avoid the convergence problem caused by the vanishing or exploding gradient caused by the all-zero initialization in the traditional LoRA algorithm; in addition, compared with other initialization methods, these initialization parameters are closer to the optimal solution, which can reduce the number of iterations of the model and speed up the optimization training process.

[0069] Using the training method of the pre-trained large model provided in the embodiment of the present application, through the new parameter initialization method, the accuracy of the fine-tuned model can be close to the accuracy of the full model in the strategy based on PEFT training. This strategy can effectively solve the impact of the lower parameter space. The problem of high computational complexity in the ResLoRA algorithm is solved by in-situ parameter replacement and single / half-precision iterative training, so that the time complexity of the algorithm can be kept the same as that of ordinary LoRA, while the cache occupancy remains unchanged. By using the existing gradient parameter matrix to initialize the parameters of the low-rank decomposition matrix, it is possible to maintain the same output results as the original pre-trained large model at the beginning of training while avoiding the problem of gradient vanishing or gradient explosion caused by the eigenvector being zero. The training method of the pre-trained large model provided in the embodiment of the present application uses relatively reasonably distributed initialization parameters in the three iterative optimization processes. These parameters enable the model to quickly approach the optimal solution, thereby accelerating the convergence of the model.

[0070] On the sentence pair matching dataset MRPC, the accuracy of the target model obtained using the training method of the pre-trained large model provided in the embodiment of the present application reached 93.1%, which is higher than the accuracy of the model fine-tuned by the ordinary LoRA algorithm (92.3%).

[0071] Figure 5 FIG. 1 is a block diagram of a training device for a pre-trained large model shown in an exemplary embodiment of the present application. Figure 5 As shown, the exemplary pre-trained large model training device 500 includes: The data acquisition module 510 is used to acquire training data and preset a gradient matrix.

[0072] The model processing module 520 is used to obtain a model to be updated including a low-rank module based on a pre-trained large model; the low-rank module includes at least a first low-rank matrix and a second low-rank matrix.

[0073] The first initialization module 530 is used to vertically stack the unit matrix, the second all-zero matrix, and the third all-zero matrix to initialize the first low-rank matrix, and horizontally stack the unit matrix and the first all-zero matrix to initialize the second low-rank matrix when the preset gradient matrix is ​​a diagonal matrix.

[0074] The second initialization module 540 is used to perform singular value decomposition on the gradient matrix when the preset gradient matrix is ​​a non-diagonal matrix, so as to initialize the first decomposition matrix and the second decomposition matrix.

[0075] The model updating module 550 is used to input the training data into the model to be updated for iterative training based on the initialized first low-rank matrix and the second low-rank matrix, and obtain the trained target model when the iteration stop condition is met.

[0076] It should be noted that the training device for the pre-trained large model provided in the above embodiment and the training method for the pre-trained large model provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment and will not be repeated here. In practical applications, the training device for the pre-trained large model provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the system into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0077] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by one or more processors, the electronic device implements the training method of the pre-trained large model provided in the above-mentioned embodiments.

[0078] In the embodiments of the present application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. "Including" and "comprising" mentioned throughout the specification and claims are open-ended terms and should be interpreted as "including but not limited to".

[0079] The above embodiments are merely illustrative of the principles and effects of the present application, and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A training method for a pre-trained large model, characterized in that: include: Get training data and preset gradient matrix; A model to be updated including a low-rank module is obtained based on the pre-trained large model; The low-rank module includes at least a first low-rank matrix and a second low-rank matrix; When the gradient matrix is ​​preset as a diagonal matrix, the unit matrix, the second all-zero matrix, and the third all-zero matrix are stacked vertically to initialize the first low-rank matrix, and the unit matrix and the first all-zero matrix are stacked horizontally to initialize the second low-rank matrix; When the preset gradient matrix is ​​a non-diagonal matrix, performing singular value decomposition on the gradient matrix to initialize the first decomposition matrix and the second decomposition matrix; Based on the initialized first low-rank matrix and the second low-rank matrix, the training data is input into the model to be updated for iterative training, and a trained target model is obtained when an iteration stop condition is met.

2. The training method of the pre-trained large model according to claim 1, characterized in that: Based on the pre-trained large model, the model to be updated including the low-rank module is obtained, including: Obtaining a target layer in the pre-trained large model; A low-rank module is added to the target layer to obtain the model to be updated; the low-rank module is composed of two low-rank decomposition networks in series, and each layer of the low-rank decomposition network corresponds to a low-rank matrix.

3. The training method of the pre-trained large model according to claim 2, characterized in that: Adding a low-rank module to the target layer to obtain the model to be updated includes: Two low-rank modules are added to the target layer to obtain the model to be updated.

4. The training method of the pre-trained large model according to claim 3, characterized in that: Adding two low-rank modules to the target layer to obtain the model to be updated includes: Two low-rank modules are connected in parallel to the original weight path of the target layer to obtain the model to be updated.

5. The training method of the pre-trained large model according to claim 3, characterized in that: Adding two low-rank modules to the target layer to obtain the model to be updated includes: A fine-tuning branch is connected in parallel to the original weight path of the target layer, and the fine-tuning branch includes two parallel low-rank modules.

6. The training method of the pre-trained large model according to claim 5, characterized in that: One of the low-rank modules is a half-precision floating-point branch, and the other low-rank module is a single-precision floating-point branch.

7. The training method of the pre-trained large model according to claim 6, characterized in that: Based on the initialized first low-rank matrix and the second low-rank matrix, inputting the training data into the to-be-updated model for iterative training, comprising: After initializing the first matrix, the second matrix, the third matrix, and the fourth matrix, the first matrix, the second matrix, the third matrix, and the fourth matrix are quantized into single-precision floating-point data; the first matrix and the second matrix are respectively the first low-rank matrix and the second low-rank matrix of the half-precision floating-point branch, and the third matrix and the fourth matrix are respectively the first low-rank matrix and the second low-rank matrix of the single-precision floating-point branch; For any first target layer: quantize the input data into half-precision floating-point data, freeze the parameters in the pre-trained large model and the single-precision floating-point branch in the model to be updated, and perform half-precision training on the first target layer; the first target layer is an odd-numbered layer in the target layer; For any second target layer: quantize the input data into single-precision floating-point data, freeze the parameters in the pre-trained large model and the half-precision floating-point branch in the model to be updated, and perform single-precision training on the second target layer; the second target layer is an even-numbered layer in the target layer; The weight matrix of the first target layer is updated based on the first matrix and the second matrix obtained through training, and the weight matrix of the second target layer is updated based on the third matrix and the fourth matrix obtained through training.

8. A training device for pre-training a large model, characterized in that: include: Data acquisition module, used to obtain training data and preset gradient matrix; A model processing module, used to obtain a model to be updated including a low-rank module based on a pre-trained large model; The low-rank module includes at least a first low-rank matrix and a second low-rank matrix; A first initialization module is used for, when the preset gradient matrix is ​​a diagonal matrix, vertically stacking the unit matrix, the second all-zero matrix, and the third all-zero matrix to initialize the first low-rank matrix, and horizontally stacking the unit matrix and the first all-zero matrix to initialize the second low-rank matrix; A second initialization module is used for performing singular value decomposition on the gradient matrix when the preset gradient matrix is ​​a non-diagonal matrix, so as to initialize the first decomposition matrix and the second decomposition matrix; The model updating module is used to input the training data into the model to be updated for iterative training based on the initialized first low-rank matrix and the second low-rank matrix, and obtain a trained target model when the iteration stopping condition is met.

9. An electronic device, characterized in that: The electronic device comprises: one or more processors; a memory for storing program code executable by the processor; Wherein, the processor is configured to execute the program code to implement the training method of the pre-trained large model as described in any one of claims 1 to 7.

Images