Model training method and device, equipment and medium

Abstract

The invention discloses a model training method and device, equipment and a medium, and belongs to the technical field of computers. The model training method is applied to the electronic equipment, the electronic equipment comprises a graphics processing unit (GPU), a first memory and a second memory, the GPU is electrically connected with the first memory, the first memory is electrically connected with the second memory, and the model training method comprises the following steps: embedding low-rank adapter LoRA parameters into a pre-trained large model through the GPU to obtain a to-be-trained model; in the forward propagation process, calculating network activation of each network layer in the to-be-trained model based on the training sample through the GPU; storing the network activation in the GPU into a second memory through the first memory; in the back propagation process, the network activation stored in the second memory is sent to the GPU through the first memory; and training LoRA parameters in the to-be-trained model based on network activation through the GPU to obtain a trained large model.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and specifically relates to a model training method, device, equipment and medium. Background Art

[0002] In recent years, with the development of artificial intelligence technology, large-scale pre-trained models (referred to as big models) have made significant progress in natural language processing, computer vision, speech and other fields.

[0003] When training a large model, training sample data is usually transmitted from an electronic device to a cloud or a data center through a network, and the large model is trained in the cloud or the data center.

[0004] However, this can easily lead to the leakage of training sample data during transmission, resulting in poor data security, unable to meet users' privacy needs, and relying on the network. Large models cannot be trained without a network. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a model training method, apparatus, device and medium, which can store data in the training process through a second memory, thereby solving the problem of insufficient storage space when training large models through electronic devices, and realizing training of large models through electronic devices. There is no need to transmit training samples to the cloud or data center through the network, thus avoiding the leakage of training sample data during transmission, improving data security, meeting user needs for privacy, and being independent of the network. Large models can also be trained without a network.

[0006] In a first aspect, an embodiment of the present application provides a model training method, which is applied to an electronic device, wherein the electronic device includes a graphics processor GPU, a first memory, and a second memory, wherein the GPU is electrically connected to the first memory, and the first memory is electrically connected to the second memory, and the method includes:

[0007] The low-rank adapter LoRA parameters are embedded into the pre-trained large model through GPU to obtain the model to be trained;

[0008] During the forward propagation process, the GPU is used to calculate the network activation of each network layer in the model to be trained based on the training samples;

[0009] Storing the network activation in the GPU in the second memory via the first memory;

[0010] During back-propagation, the network activations stored in the second memory are sent to the GPU via the first memory;

[0011] The LoRA parameters in the training model are trained based on network activation through GPU to obtain the trained large model.

[0012] In a second aspect, an embodiment of the present application provides a model training device, which is applied to an electronic device, wherein the electronic device includes a graphics processor GPU, a first memory, and a second memory, wherein the GPU is electrically connected to the first memory, and the first memory is electrically connected to the second memory, and the device includes:

[0013] The embedding module is used to embed the low-rank adapter LoRA parameters into the pre-trained large model through the GPU to obtain the model to be trained;

[0014] The computing module is used to calculate the network activation of each network layer in the model to be trained based on the training samples through the GPU during the forward propagation process;

[0015] A first storage module, configured to store the network activation in the GPU in the second memory via the first memory;

[0016] A sending module, configured to send the network activation stored in the second memory to the GPU via the first memory during the back propagation process;

[0017] The training module is used to train the LoRA parameters in the training model based on network activation through GPU to obtain the trained large model.

[0018] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0019] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0020] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0021] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0022] In an embodiment of the present application, the electronic device includes a GPU, a first memory, and a second memory. The GPU can embed the LoRA parameters into a pre-trained large model to obtain a model to be trained; then in the forward propagation process, the GPU calculates the network activation of each network layer in the model to be trained based on the training sample; the network activation in the GPU is stored in the second memory through the first memory, and in the reverse propagation process, the network activation stored in the second memory is sent to the GPU through the first memory; the LoRA parameters in the model to be trained are trained based on the network activation by the GPU to obtain the trained large model. In this way, the data in the training process is stored in the second memory, which solves the problem of insufficient storage space when training a large model through an electronic device, and realizes the training of the large model through the electronic device, without transmitting the training sample to the cloud or data center through the network, avoiding the leakage of the training sample data during the transmission process, improving data security, and being able to meet the user's privacy needs. It is not dependent on the network, and the large model can be trained without a network. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is one of the structural block diagrams of an electronic device according to an exemplary embodiment;

[0024] Figure 2 is one of the flowchart diagrams of a model training method according to an exemplary embodiment;

[0025] Figure 3 is one of the flowchart diagrams of a gradient updating method according to an exemplary embodiment;

[0026] Figure 4 is a second flow chart of a gradient updating method according to an exemplary embodiment;

[0027] Figure 5 is one of the schematic diagrams showing a fine-tuning training process according to an exemplary embodiment;

[0028] Figure 6 is a second schematic diagram of a fine-tuning training process according to an exemplary embodiment;

[0029] Figure 7 is a second structural block diagram of an electronic device according to an exemplary embodiment;

[0030] Figure 8 is a second flowchart of a model training method according to an exemplary embodiment;

[0031] Fig. 9 is a third flowchart of a model training method according to an exemplary embodiment;

[0032] Fig.10 is a structural block diagram of a model training device according to an exemplary embodiment;

[0033] Fig.11 is a second structural block diagram of an electronic device according to an exemplary embodiment;

[0034] Fig.12 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0036] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0037] As mentioned in the background technology, in recent years, with the development of artificial intelligence technology, large models (i.e., models with billions of parameters or more) have made significant progress in natural language processing, computer vision, speech and other fields. Although large models have powerful functions, fine-tuning all parameters of pre-trained large models requires considerable computing power and a considerable amount of time. The training of large models usually requires a large amount of computing resources and data, relying on high bandwidth and powerful servers, and is usually completed in the cloud or data center.

[0038] However, training large models in the cloud requires transferring training sample data to the cloud, which cannot meet users’ privacy and data security needs. At the same time, training large models in the cloud requires data transmission between the end and the cloud to rely on the network, which cannot be achieved without a network.

[0039] The embodiment of the present application provides a model training method, device, equipment and medium, the electronic device includes a GPU, a first memory and a second memory, and the LoRA parameters can be embedded into a pre-trained large model through the GPU to obtain a model to be trained; then in the forward propagation process, the network activation of each network layer in the model to be trained is calculated by the GPU based on the training sample; the network activation in the GPU is stored in the second memory through the first memory, and in the reverse propagation process, the network activation stored in the second memory is sent to the GPU through the first memory; the LoRA parameters in the model to be trained are trained based on the network activation by the GPU to obtain the trained large model. In this way, the data in the training process is stored in the second memory, which solves the problem of insufficient storage space when training a large model through an electronic device, and realizes the training of the large model through the electronic device, without transmitting the training sample to the cloud or data center through the network, avoiding the leakage of the training sample data during the transmission process, improving data security, and being able to meet the user's privacy needs, and not relying on the network, and can also train the large model without the network.

[0040] The following, in conjunction with the accompanying drawings, describes in detail the model training method, device, equipment and medium provided in the embodiments of the present application through specific embodiments and their application scenarios.

[0041] The model training method provided in the embodiment of the present application can be applied to application scenarios where large models need to be trained.

[0042] The model training method provided in the embodiment of the present application can be executed by an electronic device. Figure 1 As shown, the electronic device 100 may include a graphics processing unit (GPU) 101 , a first memory 102 and a second memory 103 .

[0043] The graphics processor 101 may be electrically connected to the first memory 102 , and the first memory 102 may be electrically connected to the second memory 103 .

[0044] For example, the electronic device 100 may be an electronic device having a terminal processing chip. The first memory 102 may be a double data rate synchronous dynamic random access memory (DDR). The second memory 103 may be an external memory provided by the electronic device, such as Flash.

[0045] It should be noted that the above-mentioned execution entities do not constitute a limitation on this application.

[0046] Combine the following Figure 2 The model training method provided in the embodiments of the present application is described in detail.

[0047] Figure 2 The figure is a flowchart of a model training method according to an exemplary embodiment.

[0048] like Figure 2 As shown, the model training method may include the following steps:

[0049] Step 210, embed the LoRA parameters into the pre-trained large model through the GPU to obtain the model to be trained.

[0050] Here, the large model can be trained with Low-Rank Adaptation (LoRA) by embedding the Low-Rank Adaptation (LoRA) parameters into the pre-trained large model. The model parameters of the large model can be fine-tuned through LoRA training.

[0051] Exemplarily, the LoRA parameters can be trainable matrices, and two trainable small matrices can be added to each weight of the original large model. During training, the weights of the original large model can be fixed and the weights of the small matrices can be trained.

[0052] By fine-tuning a large model through LoRA training, better results can be achieved in specific situations, and the number of model parameters, computing power and time required are much lower than fine-tuning the entire large model.

[0053] Exemplarily, the big model may be a speech big model, an image speech generation big model, a multimodal big model, etc. The role of the big model may include but is not limited to answering questions in a manner preferred by the user, and generating pictures and sounds in a style preferred by the user, etc.

[0054] In some implementations, step 210 may include:

[0055] Construct LoRA parameters based on the model parameters to be updated included in the large model through GPU;

[0056] The LoRA parameters are embedded into the network layer of the large model through the GPU to obtain the model to be trained.

[0057] Here, the model parameters to be updated may be parameters that need to be fine-tuned in the large model.

[0058] Specifically, the LoRA parameters can be embedded into each network layer in the large model that contains the model parameters to be updated.

[0059] Exemplarily, for the original weights (i.e., parameters to be updated) included in each transformer layer of the large model: W Q , WK , W V and W O , and the weights of the two FFN layers: W FFN1 and W FFN2 , construct two LoRA weights (i.e. LoRA parameters) of updateable parameters: matrix A and matrix B. Let the original weight be R d×k , then the matrix B∈R d×r , the matrix A∈R r×k , where k>1000, d>1000, r≤32. Gaussian initialization is performed on each constructed matrix A; zero initialization is performed on each constructed matrix B. Then, the constructed model to be trained can be transferred from the external storage to the DDR.

[0060] In this way, by constructing LoRA parameters based on the model parameters to be updated included in the large model, and then embedding the LoRA parameters into the network layer in the large model, a good foundation can be laid for LoRA training.

[0061] Step 220, during the forward propagation process, the GPU is used to calculate the network activation of each network layer in the model to be trained based on the training samples.

[0062] Here, the training samples may be collected by the electronic device when the user uses the large model. The training samples may include user data. The training samples may be stored in the second memory and retrieved from the second memory to the GPU during training.

[0063] For example, when a user uses an electronic device, the electronic device may collect and save user data. Specifically, the user data may include personal preference data during the user's use, which may include multiple modalities, such as: the user's unique way of speaking, the user's favorite large model text output style, the user's preferred image generation style, etc. Then, a training sample may be constructed based on the user data, and the large model may be fine-tuned periodically based on the constructed training sample.

[0064] In some implementations, the model to be trained may include N network layers, where N may be a positive integer, and step 220 may include:

[0065] During the forward propagation process, the GPU is used to calculate the network activation of the first network layer based on the training samples;

[0066] Calculate the network activation of the i+1th network layer based on the network activation of the i-th network layer through the GPU, i∈{1,N-1}, i can be an integer;

[0067] When i is less than N-1, update i to i+1, and return to the step of calculating the network activation of the i+1th network layer based on the network activation of the i-th network layer through the GPU, and stop when i is equal to N-1, to obtain the network activations corresponding to the N network layers respectively.

[0068] Here, the N network layers may be network layers containing model parameters to be updated. The N network layers may be network layers in which LoRA parameters are embedded.

[0069] Model training can include two processes: forward propagation and back propagation. The forward propagation process can include calculating network activation layer by layer.

[0070] To calculate the network activations corresponding to each of the N network layers, you can start from the first network layer.

[0071] Specifically, the network activation of the first network layer can be calculated based on the training sample, and the network activation of the second network layer and the network layer after the second network layer can be calculated based on the network activation of the previous network layer. The calculation of the network activation of N network layers can be achieved through iteration.

[0072] Exemplarily, the network activation of the first network layer can be calculated based on the training samples, and then the network activation of the second network layer is calculated based on the network activation of the first network layer, and the network activation of the third network layer is calculated based on the network activation of the second network layer, and so on, until the network activations of N network layers are obtained.

[0073] In this way, by calculating the network activation of the first network layer based on the training samples, and then iteratively calculating the network activation of the current network layer based on the network activation of the previous network layer, the network activation of each network layer can be obtained quickly and accurately.

[0074] Step 230: Store the network activation in the GPU in the second memory through the first memory.

[0075] Training a large model on the electronic device side requires a large storage space, but the storage space of the first memory is limited and cannot meet the storage requirements during the large model training process, which will cause the floating-point storage accuracy to decrease during model training, resulting in poor training results. If the electronic device is configured with a first memory with a larger storage space, it will increase costs.

[0076] Here, in the process of training the large model, the calculated network activation can be stored in the second memory. The second memory can have a large storage space to meet the storage requirements during the large model training process. The second memory can be an external storage that comes with the electronic device, and there is no need to add a new memory to the electronic device, thereby saving a lot of costs.

[0077] In some implementations, step 230 may include:

[0078] After calculating the network activation of the i+1th network layer, storing the network activation of the i-th network layer in the first memory through the GPU;

[0079] The network activation of the i-th network layer is read and saved from the first memory through the second memory, and the network activation of the i-th network layer is deleted from the first memory.

[0080] Here, after each calculation to obtain the network activation of the current network layer based on the network activation of the previous network layer, the network activation of the previous network layer may be stored in the second memory.

[0081] Specifically, the GPU may first store the network activation to be stored in the first memory, and then the second memory may read and save the network activation from the first memory, and release the storage space occupied by the network activation in the first memory.

[0082] Exemplarily, the calculation formula for network activation may be: y1=w0y0, y2=w1y1. Wherein, y0 may be the network activation of the first network layer, y1 may be the network activation of the second network layer, y1 may be the network activation of the third network layer, w0 may be the LoRA parameter embedded in the second network layer, and w1 may be the LoRA parameter embedded in the third network layer.

[0083] After the GPU calculates y1 based on y0, the GPU can store y0 in DDR, and then Flash reads y0 from DDR and saves it, and DDR releases the storage space occupied by y0; after the GPU calculates y2 based on y1, the GPU can store y1 in DDR, and then Flash reads y1 from DDR and saves it, and DDR releases the storage space occupied by y1.

[0084] In this way, through the above process, after each network activation is used, the network activation can be stored in the second memory, and the storage space of the first memory can be released, so as to avoid occupying a large amount of storage space of the first memory, resulting in insufficient storage space, causing the floating point storage accuracy to be reduced during model training, resulting in poor training results.

[0085] Step 240 , during the back-propagation process, the network activations stored in the second memory are sent to the GPU via the first memory.

[0086] Here, during the back propagation process, the GPU needs to train the LoRA parameters in the model to be trained based on the network activation, so the GPU needs to obtain the previously stored network activation from the second memory before training the LoRA parameters in the model to be trained based on the network activation.

[0087] Step 250, train the LoRA parameters in the to-be-trained model based on network activation through the GPU to obtain a trained large model.

[0088] Here, the LoRA parameters in the model to be trained can be updated based on the network activation, and the trained large model can be obtained based on the updated LoRA parameters.

[0089] In some implementations, step 250 may include:

[0090] Determine the gradients of each network layer in the model to be trained through GPU-based network activation;

[0091] The LoRA parameters are updated based on the gradient by GPU. If the model to be trained has not converged, the step of calculating the network activation of each network layer in the model to be trained based on the training samples by GPU is returned until the model to be trained converges to obtain the updated LoRA parameters.

[0092] The GPU is used to update the model parameters of the large model based on the updated LoRA parameters to obtain the trained large model.

[0093] Here, the convergence of the model to be trained may be the convergence of the loss function, for example, the loss function value satisfies a preset condition.

[0094] Specifically, for any network layer, the gradient of the network layer can be calculated based on the network activation of the network layer, and then the LoRA parameters of the network layer can be updated based on the gradient. The process can be iterated based on multiple training samples until the model to be trained converges to obtain the updated LoRA parameters.

[0095] Exemplarily, the formula for updating the model parameters of the large model based on the updated LoRA parameters can be: W'=W0+B W0 A W0 Where W' can be the updated model parameters, W0 can be the model parameters before the update, and B W0 and A W0 It can be the updated LoRA parameters.

[0096] Since the LoRA parameters are small in size and occupy a small DDR space, the trained LoRA parameters can be stored in the first memory without being stored in the second memory.

[0097] Fine-tuning training on the electronic device side only requires updating the LoRA parameters instead of all large model parameters, so the required DDR memory and time are greatly reduced.

[0098] In this way, by updating the LoRA parameters and then updating the model parameters of the large model based on the updated LoRA, a large number of parameters in the large model can be fine-tuned by training the LoRA parameters, thereby saving computing power and time.

[0099] In some implementations, determining the gradient of each network layer in the to-be-trained model based on network activation by the GPU may include:

[0100] Determine the gradient corresponding to the Nth network layer through the GPU based on the network activation corresponding to the Nth network layer;

[0101] Determine the gradient corresponding to the j-1th network layer through the GPU based on the gradient corresponding to the j-th network layer and the network activation corresponding to the j-1th network layer, j∈{2,N}, j is an integer;

[0102] When j is greater than 2, update j to j-1, and return to the step of determining the gradient corresponding to the j-1th network layer based on the gradient corresponding to the j-th network layer and the network activation corresponding to the j-1th network layer through the GPU, and stop when j is equal to 2 to obtain the gradients corresponding to the N network layers respectively.

[0103] Here, when calculating the gradients of N network layers, we can start from the last network layer, that is, the Nth network layer.

[0104] Specifically, the loss function value of the model to be trained can be calculated based on the network activation corresponding to the Nth network layer, and then back propagation can be performed to calculate the gradient corresponding to the Nth network layer based on the loss function value and the network activation corresponding to the Nth network layer. Then, based on the gradient corresponding to the Nth network layer and the network activation corresponding to the N-1th network layer, the gradient corresponding to the N-1th network layer is calculated, and so on, until the gradients corresponding to the N network layers are obtained.

[0105] The gradient can be stored in the first memory. After the gradients corresponding to the N network layers are obtained, the LoRA parameters in each network layer are updated uniformly.

[0106] Exemplarily, based on the gradient corresponding to the j-th network layer and the network activation corresponding to the j-1-th network layer, the formula for determining the gradient corresponding to the j-1-th network layer may be: j-1 =y j-1 ▽ j Among them, j-1 can be the gradient corresponding to the j-1th network layer, y j-1The network activation corresponding to the j-1th network layer can be j It can be the gradient corresponding to the j-th network layer.

[0107] In this way, by determining the gradient corresponding to the last network layer based on the network activation corresponding to the last network layer, and then iteratively calculating the gradient corresponding to the current network layer based on the gradient corresponding to the next network layer and the network activation of the current network layer, the gradient corresponding to each network layer can be obtained quickly and accurately.

[0108] In some implementations, step 240 may include:

[0109] In the back propagation process, before determining the gradient corresponding to the j-1th network layer based on the gradient corresponding to the j-th network layer and the network activation corresponding to the j-1th network layer through the GPU, obtaining the network activation corresponding to the j-1th network layer from the second memory through the first memory;

[0110] The network activation corresponding to the j-1th network layer is obtained from the first memory through the GPU.

[0111] Here, the GPU may obtain a network activation from the second memory each time before the network activation is needed.

[0112] For example, DDR can be based on ▽ j and j-1 OK j-1 Before, get y from Flash j-1 , and y j-1 Sent to the GPU so that the GPU can j and j-1 Calculation ▽ j-1 After the calculation is completed, y can be released from DDR j-1 The storage space occupied.

[0113] In this way, through the above process, the GPU can obtain the network activation from the second memory each time before a certain network activation is needed, avoiding obtaining the network activations of multiple network layers at the same time, occupying a large amount of storage space of the first memory, resulting in insufficient storage space, causing the floating-point storage accuracy to decrease during model training, and resulting in poor training results.

[0114] Traditional network gradient update methods are usually batch processing methods, that is, reading multiple pieces of data at one time, such as B pieces (B can usually be 16 or more), and then performing forward propagation, back propagation / gradient calculation, and parameter update operations in sequence. This method is generally used in the cloud, when the computing memory is large enough, or for training or fine-tuning of some small-scale networks. For fine-tuning large models on the end side, due to the limitation of the DDR space on the end side, one forward propagation, back propagation / gradient calculation operation can only support a single piece of data. If a single piece of data is used to update the parameters, the training effect will be reduced and the loss function will not converge.

[0115] Based on this, in some implementations, the above-mentioned updating of LoRA parameters based on gradients by GPU may include:

[0116] For each of the N network layers, perform the following steps:

[0117] Determine the number of gradients corresponding to the network layer;

[0118] When the number of gradients corresponding to the network layer reaches a first threshold, determining an average value of multiple gradients corresponding to the network layer;

[0119] The LoRA parameters included in the network layer are updated based on the average value.

[0120] Here, for each network layer, one gradient can be calculated based on one training sample.

[0121] The first threshold value can be set according to actual needs and is not limited here. For example, the first threshold value can be 16 or greater than 16.

[0122] Specifically, after each gradient is calculated based on a single piece of data, the gradient can be stored in the DDR. For each network layer, when the number of gradients accumulates to a first threshold, the multiple gradients of the network layer stored in the DDR are averaged, and the LoRA parameters are updated based on the average value.

[0123] For example, the traditional network gradient update method can be as follows Figure 3 As shown, step 310, input B training samples; step 320, forward propagation; step 330, back propagation; step 340, parameter update. The network gradient update method in the model training method provided in the embodiment of the present application can be as follows Figure 4 As shown, step 410, input a single training sample; step 420, forward propagation; step 430, back propagation; step 440, determine whether the number of gradients reaches B; if so, execute step 450, if not, return to execute step 410; step 450, update the parameters based on the average value of B gradients.

[0124] In this way, for each network layer, by updating the LoRA parameters included in the network layer based on the average value of multiple gradients when the gradient reaches the first threshold, it is possible to update the parameters based on multiple data without increasing the storage space, thereby improving the training effect, avoiding the occurrence of non-convergence of the loss function, and avoiding the inability to conduct fine-tuning training due to memory limitations.

[0125] In some examples, for the update of LoRA parameters, the gradient conduction matrix can be stored in the DDR according to the chain rule.

[0126] Specifically, the calculation formula for network activation can be: y1 = Ay0, y2 = p0 + By1, y3 = p1y2. Among them, A and B are LoRA parameters, and p0 and p1 can be non-updatable parameters. If L is the loss function value, then:

[0127] remember because In DDR, the gradient conduction matrix MW1 needs to be stored instead of the matrix M, through the formula Update B; and because and Therefore, the gradient conduction matrix MW1B needs to be stored in DDR instead of the matrix MW1, through the formula Update A, and so on, where α is the learning rate. The gradient of each trainable parameter can be stored in DDR first, and then updated uniformly after all gradient calculations are completed.

[0128] After the network training is completed, for any original model weight matrix W0, there are two trained matrices A W0 , B W0 Correspondingly, let W'=W0+B W0 A W0 , replacing the original large model weight matrix W0 with W', the model parameters of the large model can be updated based on the updated LoRA parameters, and the LoRA parameters can be eliminated to complete the model update, without any additional overhead in subsequent model reasoning.

[0129] In some examples, such as Figure 5 As shown in the figure, during the fine-tuning training process, forward propagation can be performed first, then the loss function value is calculated, and then back propagation can be performed.

[0130] During the forward propagation process, forward calculations can be performed on each network layer starting from the first network layer, that is, calculating network activations. Since the network activations of the current network layer need to be calculated based on the network activations of the previous network layer, the network activations of the previous network layer can be called out to the external memory after each calculation to obtain the network activations of the current network layer.

[0131] During the back propagation process, each network layer can be reversely calculated starting from the last network layer, that is, the gradient can be calculated. Since the gradient of the current network layer needs to be calculated based on the network activation of the current network layer, the network activation of the current network layer can be loaded from the external memory before calculating the gradient of the current network layer each time.

[0132] Exemplarily, after the calculation of y1 is completed, y0 can be transferred to the external memory, and the space of y0 can be released in DDR. After the calculation of y2 is completed, y1 can be transferred to the external memory, and the space of y1 can be released in DDR. After the calculation of y3 is completed, y2 can be transferred to the external memory, and the space of y2 can be released in DDR. Finally, the loss function value is calculated. In the back propagation process, the gradient of the loss function value to y3 is first calculated, and then y2 is transferred from the external memory to calculate the gradient of w2, y1 is transferred from the external memory to calculate the gradient of w1, and y0 is transferred from the external memory to calculate the gradient of w0. w1, w2 and w0 can be LoRA parameters of each network layer.

[0133] In some examples, such as Figure 6 As shown in the figure, the fine-tuning training process is shown in the form of blocks. F0 can be the forward propagation process of the first network layer, F1 can be the forward propagation process of the second network layer, F2 can be the forward propagation process of the third network layer, and F3 can be the forward propagation process of the fourth network layer. B0 can be the back propagation process of the first network layer, B1 can be the back propagation process of the second network layer, B2 can be the back propagation process of the third network layer, and B3 can be the back propagation process of the fourth network layer. The image also demonstrates that the gradient update of back propagation needs to be transferred from the external memory to the network activation of the corresponding network layer to complete the calculation. It also demonstrates the update process of the gradient conduction matrix. The matrix is ​​required when calculating the gradient of w2. And update the gradient conduction matrix as After calculating the gradient of w1, the gradient conduction matrix is ​​updated to This matrix is ​​used to complete the gradient calculation of w0. Then, after completing the gradient calculation of all trainable parameters, the parameters can be updated uniformly.

[0134] In some embodiments, during fine-tuning training, DDR and GPU can be used for transmission of network weights at a faster speed; DDR and external memory can be used for transmission of network activations at a slower speed.

[0135] In some implementations, during training, an Adam optimizer or an AdamW optimizer may be used for optimization. The optimizer may be stored in the first memory and read from the DDR when the GPU needs to use it.

[0136] In some implementations, after step 250, the method may further include:

[0137] The trained large model is stored in the first memory through the GPU;

[0138] The trained large model is read and saved from the first memory through the second memory, and the trained large model is deleted from the first memory.

[0139] Here, since the large model requires a larger storage space, the trained large model can be stored in the second memory.

[0140] Specifically, the GPU can first store the trained large model in the first memory, and then the second memory can read and save the trained large model from the first memory, and release the storage space occupied by the trained large model in the first memory.

[0141] When the user subsequently needs to use the large model or train the large model again, the trained large model can be called from the second memory.

[0142] In this way, by storing the trained large model in the second memory of the electronic device, it is possible to avoid the large model occupying a large amount of storage space in the first memory, and it can be called from the second memory when used without being called from the cloud through the network. Therefore, there is no need to rely on the network, and users can still use the large model even without a network.

[0143] In some implementations, step 210 may include:

[0144] When the number of training samples is greater than the second threshold, the LoRA parameters are embedded into the pre-trained large model through the GPU to obtain the model to be trained.

[0145] Here, the second threshold value can be set according to actual needs and is not limited here. For example, the second threshold value can be 300.

[0146] Specifically, an incremental learning method can be used to start fine-tuning the large model when the accumulation of training samples exceeds a second threshold.

[0147] In this way, by starting fine-tuning the large model when the number of training samples is greater than the second threshold, it is possible to periodically fine-tune the large model so that the large model can continuously adapt to the personalized needs of users.

[0148] In some embodiments, when the number of training samples is greater than the second threshold, embedding the LoRA parameters into the pre-trained large model through the GPU to obtain the model to be trained may include:

[0149] When the number of training samples is greater than the second threshold, displaying a prompt message;

[0150] Receiving a first input determining to train a large model;

[0151] In response to the first input, the LoRA parameters are embedded into the pre-trained large model through the GPU to obtain the model to be trained.

[0152] Here, the prompt information can be used to prompt the user to determine whether to train the large model. The first input can be used to trigger the training of the large model.

[0153] Specifically, when the accumulation of training samples exceeds a second threshold, a prompt message may be displayed to prompt the user to determine whether to train the large model. When a first input is received from the user to determine whether to train the large model, training of the large model may be started.

[0154] For example, when the number of training samples is greater than 300, the electronic device may display a prompt message, which may include a question: "Do you agree to train the large model?" and selection controls "Agree" and "Disagree". If the user clicks the "Agree" control, the electronic device may start training the large model. If the user does not want to train the large model or does not want to train the large model at this moment, the user may click the "Disagree" control.

[0155] In this way, through the above process, the user can choose whether to train the large model, so that the training of the large model is more in line with user needs and avoids affecting the user's use of electronic devices due to the training of the large model.

[0156] In some embodiments, embedding LoRA parameters into a pre-trained large model through a GPU to obtain a model to be trained may include:

[0157] When the electronic device is in a charging state, the LoRA parameters are embedded into the pre-trained large model through the GPU to obtain the model to be trained.

[0158] Here, the power consumption is relatively high when training a large model through an electronic device, so the training of the large model needs to be started only when the electronic device is in a charging state.

[0159] Specifically, the electronic device can detect whether it is in a charging state. If it is in a charging state, the large model training can be started; if it is not in a charging state, a charging prompt message can be output to prompt the user to charge the electronic device.

[0160] In this way, by starting the large model training only when the electronic device is in a charging state, it is possible to avoid the situation where the electronic device is shut down due to low battery during the large model training process, thereby avoiding the interruption of the large model training.

[0161] In some embodiments, the electronic device may further include a central processing unit (CPU), and the CPU may be located on a target chip, and the target chip may be different from a main processor chip of the electronic device;

[0162] The CPU may be used to control data transmission between the GPU and the first memory, and data transmission between the first memory and the second memory.

[0163] Here, the target chip may be a non-main processor chip.

[0164] Specifically, the CPU may control data transmission between the GPU and the first memory, and control data transmission between the first memory and the second memory by sending instructions.

[0165] For example, Figure 7 As shown, the electronic device 700 may include a target chip 710 and a second memory 720. The target chip 710 may include a central processing unit 711, a first memory 712, and a graphics processor 713. The central processing unit 711 may be electrically connected to the first memory 712, the first memory 712 may be electrically connected to the graphics processor 713, and the first memory 712 may be electrically connected to the second memory 720.

[0166] In this way, using the CPU on a non-main processor chip to control data transmission during large model training can avoid overloading the main processor.

[0167] Thus, the electronic device includes a GPU, a first memory, and a second memory, and the LoRA parameters can be embedded into a pre-trained large model through the GPU to obtain a model to be trained; then in the forward propagation process, the network activation of each network layer in the model to be trained is calculated by the GPU based on the training sample; the network activation in the GPU is stored in the second memory through the first memory, and in the reverse propagation process, the network activation stored in the second memory is sent to the GPU through the first memory; the LoRA parameters in the model to be trained are trained based on the network activation by the GPU to obtain the trained large model. In this way, the data in the training process is stored in the second memory, which solves the problem of insufficient storage space when training a large model through an electronic device, and realizes the training of the large model through the electronic device, without transmitting the training sample to the cloud or data center through the network, avoiding the leakage of the training sample data during the transmission process, improving data security, and being able to meet the user's privacy needs, and is independent of the network, and can also be trained in the absence of a network.

[0168] In order to better describe the entire solution, based on the above embodiments, a specific example is given. Figure 8 As shown, the model training method may include steps 801 to 808, which are explained in detail below.

[0169] Step 801, embed the LoRA parameters into the pre-trained large model through the GPU to obtain the model to be trained.

[0170] Step 802, during the forward propagation process, the network activation of each network layer in the to-be-trained model is calculated based on the training samples by the GPU.

[0171] Step 803: Store the network activation in the GPU in the second memory through the first memory.

[0172] Step 804 , during the back-propagation process, the network activation stored in the second memory is sent to the GPU via the first memory.

[0173] Step 805, determining the gradient of each network layer in the model to be trained based on network activation through the GPU.

[0174] Step 806, updating the LoRA parameters based on the gradient through the GPU.

[0175] Step 807, determine whether the loss function value meets the convergence condition.

[0176] If not, return to step 802; if so, execute step 808.

[0177] Step 808: Update the model parameters of the large model based on the updated LoRA parameters through the GPU to obtain the trained large model.

[0178] The specific process can be found in the above embodiments and will not be described in detail here.

[0179] In the embodiment of the present application, fine-tuning training of a large model can be implemented on the electronic device side without the help of any cloud device, thereby avoiding the leakage of user privacy and improving data security. And fine-tuning training of a large model can be implemented on the electronic device side, so that the large model can better understand the user and meet the user's personalized needs while maintaining its powerful functions.

[0180] In order to better describe the entire solution, based on the above embodiments, another specific example is given. Fig. 9 As shown, the model training method may include steps 901 to 905, which are explained in detail below.

[0181] Step 901, determining whether the number of training samples is greater than a second threshold.

[0182] If yes, execute step 902; if no, return to execute step 901.

[0183] Step 902: display prompt information.

[0184] Step 903, determine whether the user agrees to conduct large model training.

[0185] If yes, execute step 904; if no, end.

[0186] Step 904, determining whether the electronic device is in a charging state.

[0187] If yes, execute step 905; if no, end.

[0188] Step 905, perform large model training.

[0189] The specific process can be found in the above embodiments and will not be described in detail here.

[0190] Based on the same inventive concept, the present application also provides an electronic device.

[0191] like Figure 1 As shown, the electronic device 100 may include a graphics processor 101 , a first memory 102 , and a second memory 103 .

[0192] The graphics processor 101 may be electrically connected to the first memory 102 , and the first memory 102 may be electrically connected to the second memory 103 .

[0193] in:

[0194] The graphics processor 101 can be used to embed the LoRA parameters into the pre-trained large model to obtain a model to be trained;

[0195] The graphics processor 101 may also be used to calculate the network activation of each network layer in the model to be trained based on the training samples during the forward propagation process;

[0196] The first memory 102 may be used to store the network activation in the graphics processor 101 into the second memory 103;

[0197] The first memory 102 may also be used to send the network activations stored in the second memory 103 to the graphics processor 101 during the back-propagation process;

[0198] The graphics processor 101 may also be used to train the LoRA parameters in the to-be-trained model based on network activation through the graphics processor 101 to obtain a trained large model.

[0199] The specific process can be found in the above embodiment and will not be described again here.

[0200] Thus, the electronic device includes a GPU, a first memory, and a second memory, and the LoRA parameters can be embedded into a pre-trained large model through the GPU to obtain a model to be trained; then in the forward propagation process, the network activation of each network layer in the model to be trained is calculated by the GPU based on the training sample; the network activation in the GPU is stored in the second memory through the first memory, and in the reverse propagation process, the network activation stored in the second memory is sent to the GPU through the first memory; the LoRA parameters in the model to be trained are trained based on the network activation by the GPU to obtain the trained large model. In this way, the data in the training process is stored in the second memory, which solves the problem of insufficient storage space when training a large model through an electronic device, and realizes the training of the large model through the electronic device, without transmitting the training sample to the cloud or data center through the network, avoiding the leakage of the training sample data during the transmission process, improving data security, and being able to meet the user's privacy needs, and is independent of the network, and can also be trained in the absence of a network.

[0201] The model training method provided in the embodiment of the present application can be executed by a model training device. In the embodiment of the present application, the model training device executing the model training method is taken as an example to illustrate the model training device provided in the embodiment of the present application.

[0202] Based on the same inventive concept, the present application also provides a model training device. Fig.11 The model training device provided in the embodiment of the present application is described in detail.

[0203] Fig.10 It is a structural block diagram of a model training device according to an exemplary embodiment.

[0204] like Fig.10 As shown, the model training device 1000 can be applied to an electronic device, which may include a GPU, a first memory, and a second memory. The GPU may be electrically connected to the first memory, and the first memory may be electrically connected to the second memory. The model training device 1000 may include:

[0205] An embedding module 1001 is used to embed LoRA parameters into a pre-trained large model through a GPU to obtain a model to be trained;

[0206] A computing module 1002 is used to calculate the network activation of each network layer in the model to be trained based on the training samples through the GPU during the forward propagation process;

[0207] A first storage module 1003 is used to store the network activation in the GPU into the second memory through the first memory;

[0208] A sending module 1004 is used to send the network activation stored in the second memory to the GPU through the first memory during the back propagation process;

[0209] The training module 1005 is used to train the LoRA parameters in the training model based on network activation through the GPU to obtain a trained large model.

[0210] The above-mentioned model training device 1000 is described in detail below, as shown below:

[0211] In one embodiment, the model to be trained may include N network layers, where N may be a positive integer, and the calculation module 1002 may include:

[0212] The first computing submodule is used to calculate the network activation of the first network layer based on the training samples through the GPU during the forward propagation process;

[0213] The second calculation submodule is used to calculate the network activation of the i+1th network layer based on the network activation of the i-th network layer through the GPU, where i∈{1,N-1}, i is an integer;

[0214] The first updating submodule is used to update i to i+1 when i is less than N-1, and return to execute the step of calculating the network activation of the i+1th network layer based on the network activation of the i-th network layer through the GPU, and stop when i is equal to N-1, to obtain the network activations corresponding to the N network layers respectively.

[0215] In one embodiment, the first storage module 1003 may include:

[0216] A storage submodule, configured to store the network activation of the i-th network layer in the first memory through the GPU after calculating the network activation of the i+1-th network layer;

[0217] The reading submodule is used to read and save the network activation of the i-th network layer from the first memory through the second memory, and delete the network activation of the i-th network layer from the first memory.

[0218] In one embodiment, the training module 1005 may include:

[0219] Determine a submodule for determining the gradients of each network layer in the model to be trained based on network activation through the GPU;

[0220] The second updating submodule is used to update the LoRA parameters based on the gradient through the GPU. When the model to be trained has not converged, it returns to the step of calculating the network activation of each network layer in the model to be trained based on the training sample through the GPU until the model to be trained converges to obtain the updated LoRA parameters;

[0221] The third updating submodule is used to update the model parameters of the large model based on the updated LoRA parameters through the GPU to obtain the trained large model.

[0222] In one embodiment, the determining submodule may include:

[0223] A first determining unit, configured to determine a gradient corresponding to the Nth network layer based on a network activation corresponding to the Nth network layer through a GPU;

[0224] A second determining unit is used to determine the gradient corresponding to the j-1th network layer based on the gradient corresponding to the j-th network layer and the network activation corresponding to the j-1th network layer through the GPU, where j∈{2,N}, j is an integer;

[0225] The updating unit is used to update j to j-1 when j is greater than 2, and return to execute the step of determining the gradient corresponding to the j-1th network layer based on the gradient corresponding to the j-th network layer and the network activation corresponding to the j-1th network layer through the GPU, until j is equal to 2 and stop, and obtain the gradients corresponding to the N network layers respectively.

[0226] In one embodiment, the sending module 1004 may include:

[0227] A first acquisition submodule is used for acquiring the network activation corresponding to the j-1th network layer from the second memory through the first memory during the back propagation process before determining the gradient corresponding to the j-1th network layer through the GPU based on the gradient corresponding to the j-th network layer and the network activation corresponding to the j-1th network layer;

[0228] The second acquisition submodule is used to obtain the network activation corresponding to the j-1th network layer from the first memory through the GPU.

[0229] In one embodiment, the second updating submodule may be specifically used for:

[0230] For each of the N network layers, perform the following steps:

[0231] Determine the number of gradients corresponding to the network layer;

[0232] When the number of gradients corresponding to the network layer reaches a first threshold, determining an average value of multiple gradients corresponding to the network layer;

[0233] The LoRA parameters included in the network layer are updated based on the average value.

[0234] In one embodiment, the model training device 1000 may further include:

[0235] A second storage module is used to train the LoRA parameters in the to-be-trained model based on network activation through the GPU, and after obtaining the trained large model, store the trained large model in the first memory through the GPU;

[0236] The reading module is used to read and save the trained large model from the first memory through the second memory, and delete the trained large model from the first memory.

[0237] In one of the embodiments, the electronic device may further include a central processing unit (CPU), and the CPU may be located on a target chip, and the target chip may be different from a main processor chip of the electronic device;

[0238] The CPU may be used to control data transmission between the GPU and the first memory, and data transmission between the first memory and the second memory.

[0239] Thus, the electronic device includes a GPU, a first memory, and a second memory, and the LoRA parameters can be embedded into a pre-trained large model through the GPU to obtain a model to be trained; then in the forward propagation process, the network activation of each network layer in the model to be trained is calculated by the GPU based on the training sample; the network activation in the GPU is stored in the second memory through the first memory, and in the reverse propagation process, the network activation stored in the second memory is sent to the GPU through the first memory; the LoRA parameters in the model to be trained are trained based on the network activation by the GPU to obtain the trained large model. In this way, the data in the training process is stored in the second memory, which solves the problem of insufficient storage space when training a large model through an electronic device, and realizes the training of the large model through the electronic device, without transmitting the training sample to the cloud or data center through the network, avoiding the leakage of the training sample data during the transmission process, improving data security, and being able to meet the user's privacy needs, and is independent of the network, and can also be trained in the absence of a network.

[0240] The model training device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtualreality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personalcomputer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.

[0241] The model training device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0242] The model training device provided in the embodiment of the present application can achieve Figure 1-9The various processes implemented in the method embodiment achieve the same technical effect and will not be described again here to avoid repetition.

[0243] In some embodiments, Fig.11 As shown, an embodiment of the present application also provides an electronic device 1100, including a processor 1101 and a memory 1102, wherein the memory 1102 stores programs or instructions that can be executed on the processor 1101, and when the program or instructions are executed by the processor 1101, the various steps of the above-mentioned model training method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0244] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0245] Fig.12 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0246] The electronic device 1200 includes but is not limited to: a radio frequency unit 1201, a network module 1202, an audio output unit 1203, an input unit 1204, a sensor 1205, a display unit 1206, a user input unit 1207, an interface unit 1208, a memory 1209, and a processor 1210 and other components.

[0247] Those skilled in the art will appreciate that the electronic device 1200 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 1210 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.12 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0248] The processor 1210 is used to embed the low-rank adapter LoRA parameters into the pre-trained large model through the GPU to obtain a model to be trained;

[0249] During the forward propagation process, the GPU is used to calculate the network activation of each network layer in the model to be trained based on the training samples;

[0250] Memory 1209, used to store the network activation in the GPU in the second memory via the first memory;

[0251] During back-propagation, the network activations stored in the second memory are sent to the GPU via the first memory;

[0252] Processor 1210 is also used to train the LoRA parameters in the model to be trained based on network activation through the GPU to obtain a trained large model.

[0253] Thus, the electronic device includes a GPU, a first memory, and a second memory, and the LoRA parameters can be embedded into a pre-trained large model through the GPU to obtain a model to be trained; then in the forward propagation process, the network activation of each network layer in the model to be trained is calculated by the GPU based on the training sample; the network activation in the GPU is stored in the second memory through the first memory, and in the reverse propagation process, the network activation stored in the second memory is sent to the GPU through the first memory; the LoRA parameters in the model to be trained are trained based on the network activation by the GPU to obtain the trained large model. In this way, the data in the training process is stored in the second memory, which solves the problem of insufficient storage space when training a large model through an electronic device, and realizes the training of the large model through the electronic device, without transmitting the training sample to the cloud or data center through the network, avoiding the leakage of the training sample data during the transmission process, improving data security, and being able to meet the user's privacy needs, and is independent of the network, and can also be trained in the absence of a network.

[0254] In some embodiments, the model to be trained includes N network layers, where N is a positive integer, and the processor 1210 is further configured to calculate the network activation of the first network layer based on the training samples through the GPU during the forward propagation process;

[0255] Calculate the network activation of the i+1th network layer based on the network activation of the i-th network layer through the GPU, i∈{1,N-1}, i is an integer;

[0256] When i is less than N-1, update i to i+1, and return to the step of calculating the network activation of the i+1th network layer based on the network activation of the i-th network layer through the GPU, and stop when i is equal to N-1, to obtain the network activations corresponding to the N network layers respectively.

[0257] In this way, by calculating the network activation of the first network layer based on the training samples, and then iteratively calculating the network activation of the current network layer based on the network activation of the previous network layer, the network activation of each network layer can be obtained quickly and accurately.

[0258] In some embodiments, the processor 1210 is further configured to store the network activation of the i-th network layer in the first memory through the GPU after calculating the network activation of the i+1-th network layer;

[0259] The memory 1209 is further configured to read and save the network activation of the i-th network layer from the first memory through the second memory, and delete the network activation of the i-th network layer from the first memory.

[0260] In this way, through the above process, after each network activation is used, the network activation can be stored in the second memory, and the storage space of the first memory can be released, so as to avoid occupying a large amount of storage space of the first memory, resulting in insufficient storage space, causing the floating point storage accuracy to be reduced during model training, resulting in poor training results.

[0261] In some embodiments, the processor 1210 is further configured to determine the gradient of each network layer in the model to be trained based on network activation through the GPU;

[0262] The LoRA parameters are updated based on the gradient by GPU. If the model to be trained has not converged, the step of calculating the network activation of each network layer in the model to be trained based on the training samples by GPU is returned until the model to be trained converges to obtain the updated LoRA parameters.

[0263] The GPU is used to update the model parameters of the large model based on the updated LoRA parameters to obtain the trained large model.

[0264] In this way, by updating the LoRA parameters and then updating the model parameters of the large model based on the updated LoRA, a large number of parameters in the large model can be fine-tuned by training the LoRA parameters, thereby saving computing power and time.

[0265] In some embodiments, the processor 1210 is further configured to determine, through the GPU, a gradient corresponding to the Nth network layer based on the network activation corresponding to the Nth network layer;

[0266] Determine the gradient corresponding to the j-1th network layer through the GPU based on the gradient corresponding to the j-th network layer and the network activation corresponding to the j-1th network layer, j∈{2,N}, j is an integer;

[0267] When j is greater than 2, update j to j-1, and return to the step of determining the gradient corresponding to the j-1th network layer based on the gradient corresponding to the j-th network layer and the network activation corresponding to the j-1th network layer through the GPU, and stop when j is equal to 2 to obtain the gradients corresponding to the N network layers respectively.

[0268] In this way, by determining the gradient corresponding to the last network layer based on the network activation corresponding to the last network layer, and then iteratively calculating the gradient corresponding to the current network layer based on the gradient corresponding to the next network layer and the network activation of the current network layer, the gradient corresponding to each network layer can be obtained quickly and accurately.

[0269] In some embodiments, the memory 1209 is further used to obtain the network activation corresponding to the j-1th network layer from the second memory through the first memory during the back propagation process before determining the gradient corresponding to the j-1th network layer through the GPU based on the gradient corresponding to the j-1th network layer and the network activation corresponding to the j-1th network layer;

[0270] Processor 1210 is further configured to obtain the network activation corresponding to the j-1th network layer from the first memory through the GPU.

[0271] In this way, through the above process, the GPU can obtain the network activation from the second memory each time before a certain network activation is needed, avoiding obtaining the network activations of multiple network layers at the same time, occupying a large amount of storage space of the first memory, resulting in insufficient storage space, causing the floating-point storage accuracy to decrease during model training, and resulting in poor training results.

[0272] In some embodiments, the processor 1210 is further configured to perform the following steps for each of the N network layers:

[0273] Determine the number of gradients corresponding to the network layer;

[0274] When the number of gradients corresponding to the network layer reaches a first threshold, determining an average value of multiple gradients corresponding to the network layer;

[0275] The LoRA parameters included in the network layer are updated based on the average value.

[0276] In this way, for each network layer, by updating the LoRA parameters included in the network layer based on the average value of multiple gradients when the gradient reaches the first threshold, it is possible to update the parameters based on multiple data without increasing the storage space, thereby improving the training effect, avoiding the occurrence of non-convergence of the loss function, and avoiding the inability to conduct fine-tuning training due to memory limitations.

[0277] In some embodiments, the processor 1210 is further configured to train the LoRA parameters in the to-be-trained model based on network activation through the GPU, and after obtaining the trained large model, store the trained large model in the first memory through the GPU;

[0278] The memory 1209 is also used to read and save the trained large model from the first memory through the second memory, and delete the trained large model from the first memory.

[0279] In this way, by storing the trained large model in the second memory of the electronic device, it is possible to avoid the large model occupying a large amount of storage space in the first memory, and it can be called from the second memory when used without being called from the cloud through the network. Therefore, there is no need to rely on the network, and users can still use the large model even without a network.

[0280] In some embodiments, the electronic device may further include a central processing unit (CPU), the CPU may be located on a target chip, and the target chip may be different from a main processor chip of the electronic device;

[0281] The CPU may be used to control data transmission between the GPU and the first memory, and data transmission between the first memory and the second memory.

[0282] In this way, using the CPU on a non-main processor chip to control data transmission during large model training can avoid overloading the main processor.

[0283] It should be understood that in the embodiment of the present application, the input unit 1204 may include a graphics processor (Graphics Processing Unit, GPU) 12041 and a microphone 12042, and the graphics processor 12041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1206 may include a display panel 12061, and the display panel 12061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1207 includes a touch panel 12071 and at least one of other input devices 12072. The touch panel 12071 is also called a touch screen. The touch panel 12071 may include two parts: a touch detection device and a touch controller. Other input devices 12072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0284] The memory 1209 can be used to store software programs and various data. The memory 1209 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 1209 may include a volatile memory or a non-volatile memory, or the memory 1209 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 1209 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0285] The processor 1210 may include one or more processing units; in some embodiments, the processor 1210 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 1210.

[0286] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned model training method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0287] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory, a random access memory, a magnetic disk or an optical disk.

[0288] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned model training method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0289] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0290] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned model training method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0291] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0292] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0293] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A model training method, characterized in that: Applied to an electronic device, the electronic device comprises a graphics processor (GPU), a first memory and a second memory, the GPU is electrically connected to the first memory, the first memory is electrically connected to the second memory, the method comprises: Embedding the low-rank adapter LoRA parameters into the pre-trained large model through the GPU to obtain a model to be trained; In the forward propagation process, the GPU calculates the network activation of each network layer in the to-be-trained model based on the training samples, and stores the network activation in the GPU into the second memory through the first memory; During the back propagation process, the network activation stored in the second memory is sent to the GPU through the first memory, and the LoRA parameters in the model to be trained are trained by the GPU based on the network activation to obtain a trained large model.

2. The method according to claim 1, characterized in that The model to be trained includes N network layers, where N is a positive integer. In the forward propagation process, the network activation of each network layer in the model to be trained is calculated by the GPU based on the training samples, including: In the forward propagation process, the GPU calculates the network activation of the first network layer based on the training sample; Calculate the network activation of the i+1th network layer based on the network activation of the i-th network layer by the GPU, i∈{1,N-1}, where i is an integer; When i is less than N-1, i is updated to i+1, and the step of calculating the network activation of the i+1th network layer based on the network activation of the i-th network layer by the GPU is returned to execution until i is equal to N-1, and the network activations corresponding to the N network layers are obtained.

3. The method according to claim 2, characterized in that The storing, through the first memory, the network activation in the GPU into the second memory comprises: After calculating the network activation of the (i+1)th network layer, storing the network activation of the (i)th network layer in the first memory through the GPU; The network activation of the i-th network layer is read and saved from the first memory through the second memory, and the network activation of the i-th network layer is deleted from the first memory.

4. The method according to claim 2, characterized in that: The step of training the LoRA parameters in the model to be trained based on the network activation by the GPU to obtain a trained large model includes: Determining, by the GPU based on the network activation, the gradient of each network layer in the model to be trained; The LoRA parameters are updated by the GPU based on the gradient, and when the model to be trained has not converged, the step of calculating the network activation of each network layer in the model to be trained based on the training sample by the GPU is returned until the model to be trained converges, thereby obtaining the updated LoRA parameters; The GPU updates the model parameters of the large model based on the updated LoRA parameters to obtain the trained large model.

5. The method according to claim 4, characterized in that The determining, by the GPU based on the network activation, the gradient of each network layer in the to-be-trained model comprises: Determining, by the GPU based on the network activation corresponding to the Nth network layer, a gradient corresponding to the Nth network layer; Determine, by the GPU, the gradient corresponding to the j-1th network layer based on the gradient corresponding to the j-th network layer and the network activation corresponding to the j-1th network layer, where j∈{2,N}, j is an integer; When j is greater than 2, update j to j-1, and return to the step of determining the gradient corresponding to the j-1th network layer based on the gradient corresponding to the j-th network layer and the network activation corresponding to the j-1th network layer by the GPU, until j is equal to 2 and stop, to obtain the gradients corresponding to the N network layers respectively.

6. The method according to claim 5, characterized in that The method of sending the network activation stored in the second memory to the GPU via the first memory during the back propagation process includes: In the back propagation process, before determining the gradient corresponding to the j-1th network layer based on the gradient corresponding to the j-th network layer and the network activation corresponding to the j-1th network layer by the GPU, obtaining the network activation corresponding to the j-1th network layer from the second memory through the first memory; Obtain, through the GPU, from the first memory, a network activation corresponding to the j-1th network layer.

7. The method according to claim 4, characterized in that The updating of the LoRA parameters based on the gradient by the GPU includes: For each of the N network layers, perform the following steps respectively: Determining the number of gradients corresponding to the network layer; When the number of gradients corresponding to the network layer reaches a first threshold, determining an average value of multiple gradients corresponding to the network layer; The LoRA parameters included in the network layer are updated based on the average value.

8. The method according to claim 1, characterized in that After the LoRA parameters in the model to be trained are trained based on the network activation by the GPU to obtain the trained large model, the method further includes: Storing the trained large model in the first memory through the GPU; The trained large model is read and saved from the first memory through the second memory, and the trained large model is deleted from the first memory.

9. The method according to claim 1, characterized in that: The electronic device further comprises a central processing unit (CPU), wherein the CPU is located on a target chip, and the target chip is different from a main processor chip of the electronic device; The CPU is used to control data transmission between the GPU and the first memory, and data transmission between the first memory and the second memory.

10. A model training device, characterized in that: Applied to an electronic device, the electronic device comprises a graphics processor GPU, a first memory and a second memory, the GPU is electrically connected to the first memory, the first memory is electrically connected to the second memory, and the device comprises: An embedding module, used to embed the low-rank adapter LoRA parameters into the pre-trained large model through the GPU to obtain a model to be trained; A computing module, used for computing the network activation of each network layer in the model to be trained based on the training samples through the GPU during the forward propagation process; A first storage module, configured to store the network activation in the GPU into the second memory through the first memory; a sending module, configured to send the network activation stored in the second memory to the GPU through the first memory during back propagation; A training module is used to train the LoRA parameters in the model to be trained based on the network activation through the GPU to obtain a trained large model.

11. An electronic device, characterized in that: It includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the model training method as described in any one of claims 1 to 9 are implemented.

12. A readable storage medium, characterized in that: The readable storage medium stores programs or instructions, and when the programs or instructions are executed by the processor, the steps of the model training method as described in any one of claims 1-9 are implemented.

Images