A model training method

Abstract

The present application relates to the technical field of deep learning model, specifically to a model training method. The present application firstly pre-trains the original model on the GPU to obtain a pre-training weight matrix, then adds noise to the pre-training weight matrix, and then continues to train the model to update the weight matrix until the model converges to obtain a trained weight matrix, and finally writes it to a non-volatile memory device. Because the present application introduces noise in the process of model training, the model learns to be insensitive to weight changes. When the trained weight matrix is written from the GPU to the non-volatile memory device, although the trained weight matrix has changed due to the characteristics of the non-volatile memory device, the model of the present application has learned to be insensitive to weight changes, so the model in the non-volatile memory device still has high prediction accuracy. The present application uses LoRA fine-tuning technology to significantly improve the efficiency of noise-resistant training.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning model, and particularly relates to a model training method. BACKGROUND

[0002] In training a deep learning model on a GPU, the weights of the model after training are written from the GPU to a non-volatile memory device (such as a resistive random memory). The structure of the non-volatile memory device (including a plurality of resistors) is shown in FIG. 1, and the weights are written to the non-volatile memory device, that is, the conductance values of the resistors are adjusted to values suitable for the weights. When it is necessary to use the trained model for inference application, the non-volatile memory device uses the model weights written in advance to perform inference calculation. Since the non-volatile memory device stores the model weights and performs inference calculation at the same physical location, the transmission time of the model weights between the memory and the processor is avoided, thereby saving the time required for model inference calculation. However, in the process of writing the model weights from the GPU to the non-volatile memory device, due to the inherent properties of the non-volatile memory device, the model weights on the GPU cannot be written to the non-volatile memory device without distinction, thereby reducing the accuracy of the model on the non-volatile memory device. Figure 2

[0003] In summary, the existing model training method reduces the model accuracy.

[0004] Therefore, the prior art still needs to be improved and improved. SUMMARY

[0005] To solve the above technical problems, the present application provides a model training method, which solves the problem that the existing model training method reduces the model accuracy.

[0006] To achieve the above purpose, the present application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a model training method, comprising:

[0008] pre-training the original model by a GPU to obtain a pre-trained model, and obtaining a pre-training weight matrix of the pre-trained model;

[0009] generating a preset noise corresponding to the non-volatile memory device, and adding the preset noise into the pre-training weight matrix to obtain a pre-training noise weight matrix;

[0010] continuing to train the pre-trained model based on the pre-training noise weight matrix by the GPU to obtain a trained weight matrix corresponding to a trained model, and writing the trained weight matrix to the non-volatile memory device.​

[0011] In an implementation, the pre-training, by the GPU, of the original model to obtain a pre-trained model comprises:

[0012] constructing a training data set without noise;

[0013] pre-training, by the GPU, of the original model based on the training data set until the model converges to obtain a pre-trained model.

[0014] In an implementation, the generating of a preset noise corresponding to the non-volatile memory device and adding the preset noise into the pre-trained weight matrix to obtain a pre-trained noise weight matrix comprises:

[0015] obtaining data loss between the GPU and the non-volatile memory device;

[0016] generating a preset noise corresponding to the data loss;

[0017] determining a noise matrix corresponding to the preset noise;

[0018] obtaining a pre-trained noise weight matrix according to the noise matrix and the pre-trained weight matrix.

[0019] In an implementation, the continuing, by the GPU, to train the pre-trained model based on the pre-trained noise weight matrix to obtain a trained weight matrix corresponding to a trained model comprises:

[0020] obtaining the number of rows and the number of columns contained in the pre-trained noise weight matrix, and generating an auxiliary matrix according to the number of rows and the number of columns of the pre-trained noise weight matrix;

[0021] continuing to train the pre-trained model by the GPU, and updating the auxiliary matrix each time of iteration training until the model converges again to obtain a final auxiliary update matrix;

[0022] obtaining a trained weight matrix according to the pre-trained noise weight matrix and the final auxiliary update matrix.

[0023] In an implementation, the generating of the auxiliary matrix according to the number of rows and the number of columns of the pre-trained noise weight matrix comprises:

[0024] generating a first matrix having the same number of columns as the pre-trained noise weight matrix;

[0025] generating a second matrix having the same number of rows as the pre-trained noise weight matrix, the second matrix and the first matrix being applicable to a matrix multiplication rule;

[0026] Randomly initialize element values of the first matrix to obtain a first auxiliary matrix; and initialize element values of the second matrix to zeros to obtain a second auxiliary matrix;

[0027] Alternatively, initialize element values of the first matrix to zeros to obtain a first auxiliary matrix; and randomly initialize element values of the second matrix to obtain a second auxiliary matrix.

[0028] In an implementation manner, the final auxiliary update matrix includes a first auxiliary update matrix corresponding to the first auxiliary matrix and a second auxiliary update matrix corresponding to the second auxiliary matrix, and the trained weight matrix is obtained according to the pre-training noise weight matrix and the final auxiliary update matrix, including:

[0029] Multiplying the second auxiliary update matrix by the first auxiliary update matrix to obtain a multiplication matrix;

[0030] Adding the pre-training noise weight matrix to the multiplication matrix to obtain the trained weight matrix.

[0031] In an implementation manner, the trained model includes a large language model or an image segmentation model.

[0032] In a second aspect, an embodiment of the present application further provides a model training device, wherein the device includes the following components:

[0033] A pre-training module is configured to pre-train an original model by using a GPU to obtain a pre-training model and acquire a pre-training weight matrix of the pre-training model.

[0034] A noise adding module is configured to generate a preset noise corresponding to a non-volatile memory device and add the preset noise into the pre-training weight matrix to obtain a pre-training noise weight matrix.

[0035] A weight matrix updating module is configured to continue training the pre-training model by using the GPU based on the pre-training noise weight matrix to obtain a trained weight matrix corresponding to a trained model, and write the trained weight matrix into the non-volatile memory device.

[0036] In a third aspect, an embodiment of the present application further provides a terminal device, wherein the terminal device includes a memory, a processor, and a model training program stored in the memory and executable on the processor, and the processor implements steps of the model training method when executing the model training program.

[0037] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, which stores a model training program, and the model training program, when executed by a processor, implements the steps of the model training method described above.

[0038] Beneficial effects: The present application firstly pre-trains the original model on the GPU to obtain a pre-trained model and a pre-trained weight matrix of the model, then adds noise to the pre-trained weight matrix, and then continues to train the model to update the weight matrix in the process of iterative training until the model converges, at which time the weight matrix is the final trained weight matrix, and finally writes the trained weight matrix to the non-volatile memory device. Because the present application introduces noise in the process of model training, the model learns to be insensitive to weight changes, that is, even if the weight changes, the model can still maintain high prediction accuracy. When the trained weight matrix is written from the GPU to the non-volatile memory device, although the trained weight matrix changes due to the characteristics of the non-volatile memory device, as can be seen from the above analysis, the model of the present application has learned to be insensitive to weight changes, so the model in the non-volatile memory device still has high prediction accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 The overall flow of the present application.

[0040] Figure 2 The structure diagram of the non-volatile memory device in the embodiments of the present application;

[0041] Figure 3 The model training flowchart in the embodiments of the present application;

[0042] Figure 4 The weight matrix splitting schematic diagram in the embodiments of the present application;

[0043] Figure 5 The schematic diagram of the additional training layer in the embodiments of the present application;

[0044] Figure 6 The model training device structure diagram provided by the present application;

[0045] Figure 7 The internal structure principle block diagram of the terminal device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the present application are described below in conjunction with the embodiments and the accompanying drawings. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of the present application.

[0047] It is found through research that when training a deep learning model on a GPU, the weights of the model after training are written from the GPU to a non-volatile memory device. The structure of the non-volatile memory device (including a plurality of resistors) is as shown in Figure 2 The weights are written to the non-volatile memory device, that is, the conductance values of the resistors are adjusted to values adapted to the weights. When it is necessary to use the trained model for inference application, the non-volatile memory device uses the model weights written in advance to perform inference calculation. Since the non-volatile memory device stores the model weights and the inference calculation at the same physical location, the transmission time of the model weights between the memory and the processor is avoided, thereby saving the time required for model inference calculation. However, in the process of writing the model weights from the GPU to the non-volatile memory device, due to the inherent properties of the non-volatile memory device, the model weights on the GPU cannot be written to the non-volatile memory device without difference, thereby reducing the accuracy of the model on the non-volatile memory device.

[0048] To solve the above technical problems, the present application provides a model training method, which solves the problem of reducing the model precision of the existing model training method. In specific implementation, first, the original model is pre-trained by a GPU to obtain a pre-trained model, and a pre-trained weight matrix of the pre-trained model is obtained; then a preset noise corresponding to a non-volatile memory device is generated, and the preset noise is added to the pre-trained weight matrix to obtain a pre-trained noise weight matrix; finally, based on the pre-trained noise weight matrix, the pre-trained model is further trained by the GPU to obtain a trained weight matrix corresponding to a trained model, and the trained weight matrix is written to the non-volatile memory device. The present application trains the model under the premise of noise, so that the model with the trained weight matrix learns to be insensitive to weight changes, thereby enabling the model on the non-volatile memory device to still maintain high precision based on the trained weight matrix with writing errors.

[0049] For example, first, the original model is trained on the GPU until the precision of the model reaches the set precision, and the training is stopped. At this time, the model is a pre-trained model, and each weight value of the pre-trained model is saved in the form of a pre-trained weight matrix. For example, the pre-trained model has nine weights, and the values of the nine weights are a1, a2, a3, a4, a5, a6, a7, a8, and a9. Therefore, the pre-trained weight matrix is Then, according to the characteristics of the non-volatile memory device (including RRAM or PCM), corresponding noise is generated, and the noise is introduced into the pre-training weight matrix, that is, the nine weight values a1, a2, a3, a4, a5, a6, a7, a8, and a9 in the pre-training weight matrix are respectively added or subtracted by a number, so as to simulate the phenomenon that the pre-training weight matrix cannot be accurately written into the non-volatile memory device, that is, to simulate the change of the pre-training weight matrix caused by writing the pre-training weight matrix into the non-volatile memory device. On the basis of the pre-training weight matrix with the introduced noise (that is, the pre-training noise weight matrix), the model is continuously trained on the GPU until the accuracy of the model reaches the set accuracy again or the model converges, so as to obtain a trained model, and the trained weight matrix of the trained model is written into the non-volatile memory device. Although the trained weight matrix changes due to the characteristics of the non-volatile memory device during the writing process, since the model has learned the insensitivity of weight change, the accuracy of the model on the non-volatile memory device will not decrease although the weight matrix changes.

[0050] The model training method of the embodiment can be applied to a terminal device, which can be a terminal product with a data processing function, such as a computer, etc. In the embodiment, as shown in Figure 1 The model training method specifically includes the following steps:

[0051] S100, pre-training an original model by a GPU to obtain a pre-training model, and obtaining a pre-training weight matrix of the pre-training model;

[0052] S200, generating a preset noise corresponding to a non-volatile memory device, and adding the preset noise into the pre-training weight matrix to obtain a pre-training noise weight matrix;

[0053] S300, based on the pre-training noise weight matrix, continuing to train the pre-training model by the GPU to obtain a trained weight matrix corresponding to a trained model, and writing the trained weight matrix into the non-volatile memory device.

[0054] In the embodiment, step S100 includes the following specific steps: constructing a training data set without noise; based on the training data set, pre-training an original model by the GPU until the model converges to obtain a pre-training model.

[0055] The weight values of the pre-training model in the form of a matrix are a pre-training weight matrix W0∈R d×d .

[0056] The training data set in this embodiment has no noise, that is, the model is trained with actual samples, so that the model has the ability to learn the true samples.

[0057] In this embodiment, the original model is a deep learning model, the training data set includes a plurality of medical diagnosis sample texts and a true classification result corresponding to each medical diagnosis sample text, the true classification result includes whether to be ill and the degree of illness, the medical diagnosis sample text is input into the deep learning model located on the GPU, the prediction classification result output by the deep learning model is obtained, the difference between the prediction classification result and the true classification result (that is, the loss function) is calculated, and the weight values of the deep learning model are adjusted according to the difference, after the adjustment, the medical diagnosis sample text is input into the deep learning model again, until the difference is less than the set value, that is, the model accuracy reaches the set accuracy, or the number of iterations reaches the set number, or the model converges, to obtain the pre-trained large language model (that is, the pre-trained model), and the matrix composed of the weight values of each weight value on the pre-trained large language model is the pre-trained weight matrix.

[0058] In embodiment two, based on embodiment one, in this embodiment, step S200 includes the following specific steps S201, S202, S203 and S204:

[0059] S201, obtaining data loss between the GPU and the non-volatile memory device.

[0060] The data loss is how much a data is reduced when the data is written from the GPU to the non-volatile memory device, for example, the data is 5, after being written to the non-volatile memory device, the data becomes 4.9, and the data loss is 0.1.

[0061] S202, generating a preset noise corresponding to the data loss.

[0062] Different noises are selected according to the size of the data loss, and the noises include Gaussian noise, Bernoulli noise, uniform noise and random inactivation.

[0063] S203, determining a noise matrix W corresponding to the preset noise noise .

[0064] That is, the preset noise is represented in the form of a matrix.

[0065] S204, obtaining a pre-training noise weight matrix W' according to the noise matrix and the pre-training weight matrix.

[0066] As shown in Figure 3 , the expression of the pre-training noise weight matrix W' is as follows:

[0067] W' = W0 + Wnoise

[0068] In the third embodiment based on the first or second embodiment, step S300 comprises the following specific steps S301, S302, S303 and S304:

[0069] S301, obtain the number of rows and the number of columns contained in the pre-trained noise weight matrix, generate a first matrix with the same number of columns as the pre-trained noise weight matrix; generate a second matrix with the same number of rows as the pre-trained noise weight matrix, and the second matrix and the first matrix are applicable to the matrix multiplication rule.

[0070] S302, randomly initialize the element values of the first matrix to obtain a first auxiliary matrix W A ; initialize the element values of the second matrix to zero to obtain a second auxiliary matrix W B .

[0071] Alternatively, initialize the element values of the first matrix to zero to obtain a first auxiliary matrix W A ; randomly initialize the element values of the second matrix to obtain a second auxiliary matrix W B .

[0072] The element values of one of the first auxiliary matrix W A and the second auxiliary matrix W B are all zero, so as to ensure that the value of W B W A is zero, thereby not affecting the value of W' at the beginning of training, and the element values of only one of the first auxiliary matrix W A and the second auxiliary matrix W B are all zero, which can also ensure the training efficiency of the model.

[0073] W0∈R d×d ,W A ∈R r×d ,W B ∈R d×r , that is, W A and W0 have the same number of columns d, and W B and W0 have the same number of rows d. The value of r includes 1 or 2 or 4, and d is 4096, that is, the dimensions of W A and W B are much smaller than the dimension of W0. The number of elements of W B W A is equal to the number of elements of W0.

[0074] S303, continue training the pre-trained model by the GPU, update the auxiliary matrix every time iteration training, until the model converges again, get the final auxiliary update matrix.

[0075] As shown in Figure 4 , W A and W B are located in the additional training layer. By adding a low-dimensional matrix in the model, fine-tuning for specific tasks is realized without large-scale parameter update of the entire model. The required data and computing resources can be effectively reduced while maintaining or improving the performance of the model.

[0076] Every time iteration training, only the first auxiliary matrix W A and the second auxiliary matrix W B are updated, and the pre-trained noise weight matrix W' is not updated. When iteration to model convergence, get the final first auxiliary matrix W A ' and the final second auxiliary matrix W B '.

[0077] When the training is over, the additional training layer does not participate in the prediction of the model.

[0078] As shown in Figure 5 , when the model (which can be GPT or BERT) structure includes an input layer, a convolutional layer 1, a convolutional layer 2, a linear layer and an output layer. Convolutional layer 1 is connected in parallel with additional training layer 1, convolutional layer 1 and additional training layer 1 have the same structure, and the weights on convolutional layer 1 are not updated during training, only the weights on additional training layer 1 are updated. Convolutional layer 2 is connected in parallel with additional training layer 2, convolutional layer 2 and additional training layer 2 have the same structure, and the weights on convolutional layer 2 are not updated during training, only the weights on additional training layer 2 are updated. Linear layer is connected in parallel with additional training layer 3, linear layer and additional training layer 3 have the same structure, and the weights on linear layer are not updated during training, only the weights on additional training layer 3 are updated.

[0079] S304, the final auxiliary update matrix includes the first auxiliary update matrix corresponding to the first auxiliary matrix and the second auxiliary update matrix corresponding to the second auxiliary matrix, multiply the final second auxiliary update matrix by the final first auxiliary update matrix to get the multiplication matrix W B 'W A '; add the pre-trained noise weight matrix W' to the multiplication matrix W B 'W A ', get the trained weight matrix W new .

[0080] W new =W'+W B 'W A '.

[0081] get W new After that, each element value in W new is written to the non-volatile memory device respectively, that is, the conductance value of each resistor of the non-volatile memory device (the structure of the non-volatile memory device is shown in Figure 2 ) is set to the corresponding element value in W new . Then the non-volatile memory device has the function of the trained model.

[0082] In summary, by introducing noise in the training process, the model learns to be insensitive to the change of weights, thereby significantly improving the performance in the presence of hardware noise and other uncertainty factors. By introducing low-rank matrix for training, the required computing resources and storage space are greatly reduced, and the training cost is reduced. Since the parameter amount of W A and W B is much smaller than the parameter amount of W0, the training process is more efficient, and the fine-tuning of the model can be completed in a short time.

[0083] The embodiment also provides a model training device, as shown in Figure 6 , the device comprises the following components:

[0084] The pre-training module 01 is used for pre-training the original model by the GPU to obtain a pre-trained model and obtain a pre-training weight matrix of the pre-trained model;

[0085] The noise adding module 02 is used for generating a preset noise corresponding to the non-volatile memory device and adding the preset noise into the pre-training weight matrix to obtain a pre-training noise weight matrix;

[0086] The weight matrix updating module 03 is used for continuing to train the pre-trained model based on the pre-training noise weight matrix by the GPU to obtain a trained weight matrix corresponding to the trained model, and writing the trained weight matrix to the non-volatile memory device.

[0087] Based on the above embodiment, the application also provides a terminal device, and the principle block diagram can be as shown in Figure 7The terminal device includes a processor, a memory, a network interface, and a display screen connected through a system bus. The processor of the terminal device is configured to provide computing and control capabilities. The memory of the terminal device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a model training method. The display screen of the terminal device can be a liquid crystal display screen or an electronic ink display screen.

[0088] Those skilled in the art can understand that Figure 7 The principle block diagram shown in the figure is only a block diagram of part of the structure related to the present application, and does not constitute a limitation on the terminal device to which the present application is applied. The specific terminal device can include more or fewer components than those shown in the figure, or combine some components, or have a different component arrangement.

[0089] In one embodiment, a terminal device is provided, which includes a memory, a processor, and a model training program stored in the memory and executable on the processor. When the processor executes the model training program, the following operation instructions are implemented:

[0090] The original model is pre-trained by a GPU to obtain a pre-trained model, and a pre-training weight matrix of the pre-trained model is obtained;

[0091] A preset noise corresponding to the non-volatile memory device is generated, and the preset noise is added to the pre-training weight matrix to obtain a pre-training noise weight matrix;

[0092] Based on the pre-training noise weight matrix, the pre-trained model is further trained by the GPU to obtain a trained weight matrix corresponding to a trained model, and the trained weight matrix is written into the non-volatile memory device.

[0093] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, storage, database or other medium used in each embodiment of the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0094] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A model training method, characterized in that, The method comprises the following steps: pre-training an original model by using a GPU to obtain a pre-trained model and obtain a pre-training weight matrix of the pre-trained model; generating preset noise corresponding to a non-volatile memory device and adding the preset noise into the pre-training weight matrix to obtain a pre-training noise weight matrix; continuing to train the pre-trained model by using the GPU based on the pre-training noise weight matrix to obtain a trained weight matrix corresponding to a trained model, and writing the trained weight matrix into the non-volatile memory device; the method of generating preset noise corresponding to a non-volatile memory device and adding the preset noise into the pre-training weight matrix to obtain a pre-training noise weight matrix comprises: obtaining data loss between the GPU and the non-volatile memory device; generating preset noise corresponding to the data loss; determining a noise matrix corresponding to the preset noise; obtaining a pre-training noise weight matrix according to the noise matrix and the pre-training weight matrix; the method of continuing to train the pre-trained model by using the GPU based on the pre-training noise weight matrix to obtain a trained weight matrix corresponding to a trained model comprises: obtaining the number of rows and the number of columns contained in the pre-training noise weight matrix, and generating an auxiliary matrix according to the number of rows and the number of columns of the pre-training noise weight matrix; continuing to train the pre-trained model by using the GPU, and updating the auxiliary matrix at each iteration until the model converges again to obtain a final auxiliary update matrix; obtaining a trained weight matrix according to the pre-training noise weight matrix and the final auxiliary update matrix. 2.The model training method of claim 1, wherein, the method of pre-training an original model by using a GPU to obtain a pre-trained model comprises: constructing a noise-free training data set; pre-training the original model by using the GPU based on the training data set until the model converges to obtain a pre-trained model. 3.The model training method of claim 1, wherein, the method of generating an auxiliary matrix according to the number of rows and the number of columns of the pre-training noise weight matrix comprises: generating a first matrix having the same number of columns as the pre-training noise weight matrix; generating a second matrix having the same number of rows as the pre-training noise weight matrix, the second matrix and the first matrix being applicable to matrix multiplication rules; randomly initializing the element values of the first matrix to obtain a first auxiliary matrix, and initializing the element values of the second matrix to zero to obtain a second auxiliary matrix; or, initializing the element values of the first matrix to zero to obtain a first auxiliary matrix, and randomly initializing the element values of the second matrix to obtain a second auxiliary matrix. 4.The model training method of claim 3, wherein, The final auxiliary update matrix comprises a first auxiliary update matrix corresponding to the first auxiliary matrix and a second auxiliary update matrix corresponding to the second auxiliary matrix, and the method of obtaining a trained weight matrix according to the pre-training noise weight matrix and the final auxiliary update matrix comprises: multiplying the second auxiliary update matrix by the first auxiliary update matrix to obtain a multiplication matrix; adding the pre-training noise weight matrix to the multiplication matrix to obtain a trained weight matrix.

5. The model training method of any one of claims 1-4, wherein, The trained model comprises a large language model or an image segmentation model.

Images