A control method, device and medium for model training

Abstract

The present application discloses a control method, device and medium for model training, which includes: constructing a model to be trained; obtaining a training data set for training the model to be trained; performing parameter block division on the model to be trained to obtain multiple parameter blocks; assigning an initial Hessian approximation information to each parameter block and the unblocked target parameter to replace the original Hessian information; determining the target Hessian approximation information of the current iteration; and iteratively training the model to be trained using the training data set based on the target Hessian approximation information of the current iteration until the iterative training conditions are met to obtain the target model. Therefore, compared to calculating a complete Hessian matrix, the present application divides the model parameters into blocks and assigns a Hessian approximation information to each parameter block and the unblocked target parameter. Each parameter block can be processed separately, reducing the amount of calculation while reducing storage requirements.

Description

Technical Field

[0001] The present application relates to the field of optimization technology, and in particular to a control method, device and medium for model training. Background Art

[0002] In recent years, with the rapid development of deep learning technology, large-scale pre-trained language models such as GPT-3 (Generative Pre-training Transformer 3) have been gradually applied to various fields of natural language processing. As the scale of pre-trained language models increases, the computing and storage resources required for training the models also increase dramatically, becoming a key bottleneck restricting the further expansion of pre-trained language models. Therefore, the design and improvement of model optimization algorithms have become a key factor in improving the training efficiency of pre-trained language models.

[0003] Currently, stochastic gradient descent (SGD) and its variants are commonly used to improve model training speed and effectiveness. However, when training complex neural network models, SGD is prone to getting stuck in local optimal solutions. In another feasible implementation, the Adam (Adaptive Moment Estimation) optimizer can adaptively adjust the learning rate during training, thereby overcoming SGD's slow convergence and local optimal solutions. However, during the specific optimization process of the Adam optimizer, the first-order momentum (m) and second-order momentum (v) must be stored to update each parameter, resulting in enormous storage and computational pressure. This is unbearable for many hardware platforms, especially in cases with limited memory, and can lead to memory overflow during training.

[0004] Therefore, how to ensure the efficiency of model training while solving the computing and storage pressure faced during the training process is an urgent problem to be solved by technical personnel in this field. Summary of the Invention

[0005] In view of this, one aspect of the present application provides a control method for model training, the method comprising:

[0006] Constructing a model to be trained; and obtaining a training data set for training the model to be trained;

[0007] Dividing the parameters of the model to be trained into blocks to obtain multiple parameter blocks;

[0008] Assigning an initial Hessian approximation information to each of the parameter blocks and the unblocked target parameter to replace the original Hessian information;

[0009] Determine the target Hessian approximation information of the current iteration;

[0010] Based on the target Hessian approximation information of the current iteration, the model to be trained is iteratively trained using the training data set until an iterative training condition is met and a target model is obtained.

[0011] Optionally, the model to be trained is a model based on a Transformer architecture; and parameter segmentation of the model to be trained includes:

[0012] For the matrix parameters in the model to be trained, the parameters are divided into blocks in units of rows; wherein the matrix parameters include the embedding layer of the Transformer architecture, the value matrix of the multi-head attention layer in the Transformer architecture, the output layer of the multi-head attention layer, the output layer of the model to be trained, and the fully connected layer of the Transformer architecture;

[0013] For the query matrix and key matrix of the multi-head self-attention layer, parameter blocks are performed based on the attention head;

[0014] For the layer normalization layer of the Transformer architecture, parameter blocks are performed based on the entire layer.

[0015] Optionally, the initial Hessian approximation information includes an initial Hessian approximation scalar and an initial Hessian approximation matrix; each parameter block and unblocked target parameter is assigned an initial Hessian approximation information, including:

[0016] For the parameter block, one initial Hessian approximation scalar is allocated correspondingly based on the parameter block;

[0017] For the target parameter, an initial Hessian approximation matrix having the same shape as the target parameter is assigned; the target parameter includes a bias layer of the model to be trained.

[0018] Optionally, determining target Hessian approximation information of the current iteration includes:

[0019] Determine whether the current number of iterations is an integer multiple of a preset value; the preset value is an integer greater than 1;

[0020] If yes, determine the target Hessian approximation information of the current iteration based on the target Hessian approximation information of the previous iteration;

[0021] If not, the target Hessian approximation information of the previous iteration is used as the target Hessian approximation information of the current iteration.

[0022] Optionally, the target Hessian approximation information includes a target Hessian approximation scalar and a target Hessian approximation matrix; and determining the target Hessian approximation information of the current iteration based on the target Hessian approximation information of the previous iteration includes:

[0023] Determine the target Hessian approximation scalar of the current iteration according to the mean of the target Hessian approximation scalar of the previous iteration and the gradient Hessian approximation value of the current iteration;

[0024] The target Hessian approximation matrix of the current iteration is determined according to the target Hessian approximation matrix of the previous iteration.

[0025] Optionally, if the parameter block is the matrix parameter, the mean is calculated row by row; if the parameter block is the query matrix and the key matrix, the mean is calculated per attention head; if the parameter block is the layer normalization layer, the mean is calculated per layer normalization layer.

[0026] Optionally, based on the target Hessian approximation information of the current iteration, iteratively training the model to be trained using the training dataset includes:

[0027] Before each round of iterative training, determining the gradient of the current model to be trained and the weight decay behavior parameters of the current model;

[0028] Based on the gradient, determining the current momentum exponential moving average;

[0029] Determining an update amount of the parameter block according to the target Hessian approximation information of the current iteration and the current momentum exponential moving average;

[0030] Determining the weight parameters of the model currently being trained based on the update amount, the current number of iterations, the current model weight decay behavior parameter, and the current momentum exponential moving average;

[0031] Under the model weight parameters, the training data set is input into the model to be trained for iterative training.

[0032] Another aspect of the present application provides a control device for model training, the device comprising:

[0033] Model building module, used to build the model to be trained;

[0034] A data acquisition module, used to acquire a training data set for training the model to be trained;

[0035] A block module is used to block the parameters of the model to be trained to obtain multiple parameter blocks;

[0036] an allocating module, configured to allocate an initial Hessian approximation information to each of the parameter blocks and the unblocked target parameter to replace the original Hessian information;

[0037] A Hessian approximation determination module, used to determine the target Hessian approximation information of the current iteration;

[0038] The training module is used to iteratively train the model to be trained based on the target Hessian approximation information of the current iteration through the training data set until the iterative training conditions are met and the target model is obtained.

[0039] Another aspect of the present application provides a control device for model training, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the steps of the control method for model training are implemented.

[0040] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the control method for model training when the program is executed by a processor.

[0041] The present application provides a control method, device, and medium for model training, which have the following beneficial effects: Compared with calculating a complete Hessian matrix, the present application divides the model parameters into blocks and assigns a Hessian approximation information to each parameter block and the target parameter of the block. Each parameter block can be processed separately, reducing the amount of calculation while reducing storage requirements. That is, based on the parameter block, the Hessian information originally required to be stored is replaced by the Hessian approximation information, which ensures the accuracy of model training while also improving training efficiency and reducing storage pressure. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flow chart of a control method for model training provided in an embodiment of the present application;

[0043] Figure 2 A schematic diagram of the structure of a model based on the Transformer architecture provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of the structure of a multi-head self-attention layer provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of the principle of parameter block partitioning of the fully connected layer in a Transformer architecture provided in an embodiment of the present application;

[0046] Figure 5 A flowchart of a control method for model training provided in another embodiment of the present application;

[0047] Figure 6 A schematic diagram of the structure of a control device for model training provided in an embodiment of the present application;

[0048] Figure 7 A schematic structural diagram of a control device for model training provided in another embodiment of the present application.

[0049] The reference numerals are as follows: 70 is a memory, 71 is a processor, 72 is a display screen, 73 is an input and output interface, 74 is a communication interface, 75 is a power supply, 76 is a communication bus, 701 is a computer program, 702 is an operating system, and 703 is data. DETAILED DESCRIPTION

[0050] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0051] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0052] Figure 1 A flow chart of a control method for model training provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the method includes:

[0053] S10: Build a model to be trained; and obtain a training data set for training the model to be trained;

[0054] In a specific embodiment, a training model to be trained and optimized is first constructed, and a training dataset that can be used to train the training model is obtained. For example, for the GPT2-small model, OpenWebTxt can be selected as the training dataset. OpenWebTxt is a large text dataset used for training language models, primarily for generating natural language text.

[0055] In an optional embodiment, the model to be trained can be various large language models, which is not limited in this application. When selecting a training data set, a data set for training the model to be trained can be constructed based on a pre-training task or a fine-tuning task. This application also does not limit the method of obtaining the training data set.

[0056] S11: Divide the parameters of the training model into blocks to obtain multiple parameter blocks;

[0057] It is understandable that for large-scale models, the parameter matrix may be very large. Directly processing these parameters will lead to insufficient memory, thereby affecting the training efficiency of the model. Therefore, to solve this technical problem, in an optional embodiment, the parameters of the above-constructed model to be trained are partitioned. Through partitioning, the parameters can be divided into multiple smaller parameter blocks, each of which can be processed separately, thereby reducing memory usage.

[0058] It should be noted that in order to subsequently assign Hessian approximation information to each parameter block, in a specific embodiment, the parameters are partitioned based on dense sub-blocks of the Hessian matrix. Therefore, by rationally partitioning the parameters, the training efficiency and performance of the model can be improved while reducing memory usage and computational complexity.

[0059] In addition, it should be noted that the Hessian approximate information of the parameter block in the embodiment of the present application is not the second-order derivative in the strict sense, but only considers the second-order derivative of each element in the parameter block with respect to itself, ignoring the cross-second-order derivatives between different elements, ensuring that the resulting Hessian approximate information is consistent with the parameter block in dimension. Therefore, when the parameter block is in vector form, its Hessian approximate information is a vector of equal size, and when the parameter block is in matrix form, its Hessian approximate information is a matrix of equal size. In addition, the present application does not directly calculate the second-order derivative of the parameter, but uses the first-order derivative of the parameter to estimate the approximate Hessian information of each parameter.

[0060] S12: assigning an initial Hessian approximation information to each parameter block and the unblocked target parameter to replace the original Hessian information;

[0061] It's understandable that the Hessian matrix is a symmetric matrix whose size is proportional to the square of the model parameter dimensions. For large-scale models, calculating the complete Hessian matrix is very time-consuming and inefficient. Furthermore, during model optimization, updating the Hessian matrix or its inverse is a complex operation, which also affects the efficiency of model training and optimization.

[0062] Therefore, based on the above embodiment, an initial Hessian approximation is assigned to each parameter block, and an initial Hessian approximation is also assigned to the target parameter that has not been classified. In a specific embodiment, the approximate Hessian information can quickly estimate the update direction and step size of the parameters, thereby accelerating the optimization process. Therefore, the parameters in each parameter block share the same Hessian approximation information, thereby speeding up the module training speed.

[0063] In a specific embodiment, the approximate Hessian information can smooth the impact of noisy data, thereby improving the stability of the model in a noisy environment. In addition, the storage requirement of the Hessian approximation information is smaller than the original Hessian mean of the model, thereby reducing the storage pressure of the model training.

[0064] S13: Determine the target Hessian approximation information of the current iteration;

[0065] S14: Based on the target Hessian approximation information of the current iteration, the to-be-trained model is iteratively trained using the training data set until the iterative training conditions are met and the target model is obtained.

[0066] Furthermore, after obtaining a training dataset and assigning initial Hessian approximation information to each parameter block, the model to be trained can be iteratively trained using the training dataset. Prior to training, the model to be trained is initialized. Initialization includes, but is not limited to, initializing hyperparameters, learning rates, assigning initial Hessian approximation information, and initializing first-order momentum for each parameter in the model to be trained.

[0067] After initialization, the model to be trained is trained for the first time using the training dataset under the initial Hessian approximation matrix. Starting from the second training, steps S13 and S14 are executed in a loop. Specifically, the Hessian approximation information of the parameter block is continuously updated, and the model to be trained is iteratively trained based on the updated Hessian approximation information.

[0068] That is, starting from the second training, each round of training determines the target Hessian approximation information used in the current iteration, and iterative training is performed under the updated target Hessian approximation information. This cycle is repeated until the iteration condition is met. The iteration condition can be reaching a preset number of training times or the convergence of the model loss function curve, which is not limited in this application.

[0069] Therefore, the control method for model training provided in the embodiment of the present application, compared to calculating a complete Hessian matrix, divides the model parameters into blocks and assigns a Hessian approximation to each parameter block and the target parameter of the block. Each parameter block can be processed separately, reducing the amount of calculation while reducing storage requirements. That is, based on the parameter block, the Hessian approximation replaces the original Hessian information that needs to be stored, ensuring the accuracy of model training while also improving training efficiency and reducing storage pressure.

[0070] In an optional embodiment, the model to be trained is a model based on the Transformer architecture, for example, a model based on the Transformer architecture is constructed using a deep learning framework such as PyTorch. On this basis, the parameters of the model to be trained are divided into blocks, including:

[0071] For the matrix parameters in the model to be trained, the parameters are divided into blocks in units of rows. The matrix parameters include the embedding layer of the Transformer architecture, the value matrix of the multi-head attention layer in the Transformer architecture, the output layer of the multi-head attention layer, the output layer of the model to be trained, and the fully connected layer of the Transformer architecture.

[0072] For the query matrix and key matrix of the multi-head self-attention layer, the parameters are divided into blocks based on the attention head;

[0073] For the layer normalization layer of the Transformer architecture, parameters are divided into blocks based on the entire layer.

[0074] Figure 2 A schematic diagram of the structure of a model based on the Transformer architecture provided in an embodiment of the present application is shown in FIG. Figure 2 As shown in the figure, the model includes an input layer, a word embedding layer, a position embedding layer, a Transformer layer, and an output layer. The Transformer layer (i.e., the Transformer architecture) includes a multi-head self-attention layer, a layer normalization layer, and a fully connected layer.

[0075] Figure 3 A schematic diagram of the structure of a multi-head self-attention layer provided in an embodiment of the present application is shown as follows: Figure 3 As shown, in an optional embodiment, the multi-head self-attention layer of the Transformer architecture includes a query matrix Q, a key matrix K, and a value matrix V. Each matrix enters the attention calculation after passing through the linear layer, and then the result after the attention calculation is transmitted to the connection layer and output after passing through a linear layer.

[0076] In a specific embodiment, the embedding layer, the value matrix of the multi-head attention layer, the output layer of the multi-head attention layer, and the fully connected layer in the Transformer architecture are partitioned into rows of matrix parameters. Specifically, each layer of the matrix parameter includes a neuron, and each neuron corresponds to a row of the matrix parameter. Therefore, neurons can also be understood as independent parameter blocks.

[0077] In addition, if Figure 2 As shown in , for the output layer of the model to be trained, the blocks are also divided into rows of matrix parameters. For example, assuming the output layer matrix is ,in, is the output dimension, is the input dimension. Then each row ( ) is an independent parameter block.

[0078] Figure 4A schematic diagram of the principle of parameter block division of the fully connected layer in a Transformer architecture provided in an embodiment of the present application is shown as follows: Figure 4 As shown, in the original fully connected layer, the parameters include 3 rows of parameters, the first row includes parameters (3, 2, 1), the second row includes parameters (6, 4, 2), and the third row includes parameters (4, 2, 1). After dividing into blocks in rows, three parameter blocks are obtained, including the first parameter block, the second parameter block, and the third parameter block.

[0079] It should be noted that the block partitioning of the value matrix V allows for more efficient use of the parallelism of matrix operations during the computation process. By partitioning the value matrix by rows, unnecessary computational overhead can be reduced while maintaining computational flexibility.

[0080] As for the query matrix Q and key matrix K, it's easy to understand that the core idea of the multi-head attention mechanism is to capture the features of the input data from different perspectives through multiple independent attention heads. The features learned by each attention head are independent, so partitioning the query matrix Q and key matrix K by attention head ensures that each head can independently calculate the attention score. In other words, for the query matrix Q and key matrix K of the multi-head self-attention layer, the parameters are partitioned by attention head.

[0081] In addition, in an optional embodiment, the layer normalization layer of the Transformer architecture typically includes a scaling factor and an offset factor. Therefore, when the layer normalization layer is divided into blocks, the parameters of the normalization layer of the entire layer are treated as a separate parameter block. For example, if the layer normalization layer includes a scaling factor and an offset factor, the scaling factor and the offset factor are divided into a separate parameter block.

[0082] As an optional embodiment, the initial Hessian approximation information includes an initial Hessian approximation scalar and an initial Hessian approximation matrix. On this basis, each parameter block and unblocked target parameter is respectively assigned an initial Hessian approximation information, including:

[0083] For the parameter block, an initial Hessian approximation scalar is assigned to the parameter block as a unit;

[0084] For the target parameters, assign an initial Hessian approximation matrix of the same shape as the target parameters; the target parameters include the bias layers of the model to be trained.

[0085] On the basis of the above embodiment, after the parameters of the model to be trained are divided into blocks, an initial Hessian approximation scalar is allocated to each parameter block in units of parameter blocks, so that all parameters in the parameter block share one Hessian approximation scalar.

[0086] As for the target parameter, for example, the bias layer, in a specific embodiment, the bias layer is usually a one-dimensional vector. Therefore, when the parameters are divided into blocks, the bias layer is not divided into blocks, but is processed as a whole. That is, the bias layer is used as the target parameter.

[0087] When assigning the initial Hessian approximation information to the unblocked paranoid layer, it is assigned an initial Hessian approximation matrix of the same shape. The same shape means that the number of rows and columns in the matrix remains the same.

[0088] After allocating initial Hessian approximation information to the parameter block and the target parameter, the Hessian approximation information needs to be initialized. In an optional embodiment, the initial Hessian approximation scalar is initialized to zero, and the initial Hessian approximation matrix is initialized to an all-zero matrix.

[0089] Specifically, for the parameter blocks corresponding to the matrix parameters (including the embedding layer, the value matrix of the multi-head attention layer, the output layer of the multi-head attention layer, the model output layer, and the fully connected layer), the initial Hessian approximation scalar after initialization is For the parameter blocks corresponding to the query matrix and key matrix of the multi-head self-attention layer, the initial Hessian approximation scalar after initialization is For the parameter block corresponding to the layer normalization layer, the initial Hessian approximation scalar after initialization is For the target parameters (e.g., bias layer), the initial Hessian approximation matrix after initialization is In addition, the first-order momentum is initialized for each parameter in the model to be trained as .

[0090] In an optional embodiment, determining target Hessian approximation information of a current iteration includes:

[0091] Determine whether the current number of iterations is an integer multiple of a preset value; the preset value is an integer greater than 1;

[0092] If yes, determine the target Hessian approximation information of the current iteration based on the target Hessian approximation information of the previous iteration;

[0093] If not, the target Hessian approximation information of the previous iteration is used as the target Hessian approximation information of the current iteration.

[0094] In an optional embodiment, the target Hessian approximation information of the parameter block and the target parameter can be updated once before each round of iterative training. However, performing the target Hessian approximation information calculation each time will undoubtedly increase the computational pressure and affect the overall training efficiency of the model.

[0095] Therefore, in another optional embodiment, before updating the target Hessian approximation information of the current iteration, it is first determined whether the number of the current iteration is an integer multiple of a preset value, for example, whether it is a multiple of 10. If so, it indicates that the conditions for updating the Hessian approximation information have been met. At this time, the target Hessian approximation information of the current iteration is calculated and updated based on the target Hessian approximation information of the previous iteration.

[0096] Of course, if the current number of iterations is not an integer multiple of the preset value, it means that the target Hessian approximation information can be temporarily not updated. At this time, when training the model, the target approximation Hessian information of the previous round is still used as the parameter condition, that is, the target Hessian approximation information of the previous round of iteration is used as the target Hessian approximation information of the current iteration.

[0097] It should be noted that in a specific embodiment, the preset value is an integer greater than 1. The preset value can be set based on empirical values or actual business needs. Specifically, when the business needs require fast model training, a larger preset value can be appropriately selected. When the model requires higher accuracy, a smaller preset value can be appropriately selected.

[0098] Based on the above embodiment, as an optional embodiment, the target Hessian approximation information includes a target Hessian approximation scalar and a target Hessian approximation matrix; and the target Hessian approximation information of the current iteration is determined based on the target Hessian approximation information of the previous iteration, including:

[0099] Determine the target Hessian approximation scalar of the current iteration according to the mean of the target Hessian approximation scalar of the previous iteration and the gradient Hessian approximation value of the current iteration;

[0100] The target Hessian approximation matrix of the current iteration is determined according to the target Hessian approximation matrix of the previous iteration.

[0101] It can be understood that the Hessian approximation information includes the Hessian approximation scalar and the Hessian approximation matrix. Therefore, when determining that the target Hessian approximation information needs to be updated, it is necessary to simultaneously calculate the Hessian approximation information corresponding to different parameter blocks and the Hessian approximation matrix corresponding to the target parameters.

[0102] Specifically, when calculating the target Hessian approximation scalar of the current iteration, the target Hessian approximation scalar of the current iteration corresponding to the matrix parameters (including the embedding layer, the value matrix of the multi-head attention layer, the output layer of the multi-head attention layer, the model output layer and the fully connected layer) is calculated according to formula (1):

[0103] (1)

[0104] in, is the target Hessian approximation scalar of the current iteration corresponding to the matrix parameter, is the target Hessian approximation scalar of the previous iteration corresponding to the matrix parameter, is a hyperparameter, It is a row-by-row mean function, which can also be understood as finding the mean function by parameter block or by neuron. That is, if the parameter block is a matrix parameter, the mean is calculated by row. is the Hessian approximation of the matrix parameters.

[0105] For the target Hessian approximation scalar of the current iteration corresponding to the query matrix and key matrix of the multi-head self-attention layer, it is calculated according to formula (2):

[0106] (2)

[0107] in, is the target Hessian approximation scalar of the current iteration corresponding to the query matrix and the key matrix, is the target Hessian approximation scalar of the previous iteration corresponding to the query matrix and the key matrix, To find the mean function by attention head, that is, if the parameter block is the query matrix and the key matrix, the mean is calculated by attention head. is the Hessian approximation of the query matrix and the key matrix.

[0108] For the target Hessian approximation scalar of the current iteration corresponding to the layer normalization layer, it is calculated according to formula (3):

[0109] (3)

[0110] in, is the target Hessian approximation scalar of the current iteration corresponding to the layer normalization layer, is the target Hessian approximation scalar of the previous iteration corresponding to the layer normalization layer, To find the mean function, that is, if the parameter block is a layer normalization layer, the mean is calculated in units of layer normalization layers. is the Hessian approximation of the layer normalization layer.

[0111] For target parameters (e.g., bias layers), the target Hessian approximation matrix of the current iteration is calculated according to formula (4):

[0112] (4)

[0113] in, is the target Hessian approximation matrix of the current iteration corresponding to the paranoid layer, is the target Hessian approximation matrix of the previous iteration corresponding to the paranoid layer, is the Hessian approximation of the paranoid layer, .

[0114] It should be noted that in formulas (1) to (4), is the Hessian approximation, and ,in, , is the gradient after forward propagation and back propagation of the model to be trained. ” is the element-by-element multiplication symbol.

[0115] Figure 5 A flow chart of a control method for model training provided in another embodiment of the present application is provided. In an optional embodiment, as Figure 5 As shown, based on the target Hessian approximation information of the current iteration, the training model is iteratively trained through the training dataset, including:

[0116] S50: Before each round of iterative training, determine the gradient of the current model to be trained and the weight decay behavior parameters of the current model;

[0117] It is understandable that in the process of iterative training of the training model through the training data set, each iteration can complete an update of the model weight parameters. Based on the separation of parameters in this application and the Hessian approximation information of parameter block allocation, when the training model is iteratively trained, before each round of training, the gradient of the current model to be trained is calculated. and the current model weight decay behavior parameters Among them, the current model weight decay behavior parameter can be calculated according to formula (5):

[0118] (5)

[0119] in, is the weight decay behavior parameter of the current model, is the model weight decay behavior parameter of the previous iteration, is the learning rate, is the weight decay coefficient.

[0120] S51: Determine the current momentum exponential moving average based on the gradient;

[0121] Furthermore, according to formula (6) the gradient Calculate the current Momentum Exponential Moving Average:

[0122] (6)

[0123] in, is the current momentum exponential moving average, is the momentum exponential moving average of the previous iteration, is a hyperparameter.

[0124] S52: Determine the update amount of the parameter block according to the target Hessian approximation information of the current iteration and the current momentum exponential moving average;

[0125] Furthermore, the update amount of this round of iterative training is calculated based on different parameter blocks. Specifically, it is calculated according to formula (7):

[0126] (7)

[0127] in, is the update amount, is a hyperparameter, and in an optional embodiment, it can be set to 0.0001. is the total number of sample training times, that is, the current batch size. is a truncation function, which means that each element value of x is limited to [-1,1]. is the target Hessian approximation information of the current iteration.

[0128] It is understandable that different parameter blocks calculate different update amounts, and different parameter blocks use the target Hessian approximation information of the current iteration. The calculation is performed according to one of the formulas (1) to (4) in the above embodiment.

[0129] S53: Determine the weight parameters of the currently trained model based on the update amount, the current number of iterations, the current model weight decay behavior parameter, and the current momentum exponential moving average;

[0130] S54: Under the model weight parameters, the training data set is input into the model to be trained for iterative training.

[0131] Furthermore, the model weight parameter update of the current iteration is completed. Specifically, the update amount calculated according to the above steps is , the learning rate at the current iteration , current model weight decay behavior parameters and the current Momentum Exponential Moving Average For calculation, see formula (8)

[0132] (8)

[0133] in, is the weight parameter of the currently trained model, Is the learning rate. Get the model weight parameters Then, based on the weight parameters, the training data set is input into the model to be trained for iterative training. This cycle is repeated until the iteration condition position is reached.

[0134] Therefore, the model training control method provided by the embodiments of this application can reduce the optimizer's storage requirements while maintaining the training performance of large language models, thereby providing support for the further expansion of large language models. Furthermore, the design can avoid excessive fluctuations in the optimization algorithm's training results, ensuring effective model training.

[0135] To help those skilled in the art better understand the technical solution of this application, the following examples are given. For example, the construction of the GPT2-small model is completed. In the GPT2-small model, the number of transmitter layers is set to 12, the number of multi-head attention head layers is set to 12, and the dimension of the embedding layer is set to 768. When obtaining a dataset, you can choose the OpenWebTxt dataset.

[0136] Furthermore, based on the parameter partitioning principle described in the above embodiment, the parameters of the GPT2-small model are partitioned. In an optional embodiment, the partitioning can also be performed based on the name of each layer. For example, when using the PyTorch framework, the word embedding layer is named transformer.wte.weight.

[0137] After the parameter block method based on the above embodiment, Hessian approximation information is assigned to each parameter block and the unblocked target parameter, and the hyperparameters of the optimization algorithm are 、 and , and the learning rate In an optional embodiment, , , , ,In addition, the maximum number of iterations of the model can be preset, for example, ,set to 10000.

[0138] Furthermore, based on the iterative method described in the above embodiment, the current target Hessian approximation information and model weight parameters are continuously updated, and based on the updated parameters, the OpenWebTxt dataset is input into the GPT2-small model for iterative training.

[0139] Table 1 is a training information table of a GPT2-small model provided in an embodiment of the present application under different optimization methods. As shown in Table 1, the optimization methods include Adam optimizer, Adam-mini optimizer and the optimization method provided in this application.

[0140] Table 1. Training information of GPT2-small model under different optimization methods

[0141]

[0142] As shown in Table 1, the model training control method provided by this application achieves faster convergence speed and minimizes storage space usage compared to the Adam optimizer and Adam-mini optimizer under the same number of iterations. In other words, it ensures model training results while reducing storage idle pressure.

[0143] In the above embodiments, the control method for model training is described in detail. The present application also provides an embodiment corresponding to a control device for model training.

[0144] Figure 6 A schematic diagram of a control device for model training provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the device includes:

[0145] A model building module 60 is used to build a model to be trained;

[0146] A data acquisition module 61 is used to acquire a training data set for training a model to be trained;

[0147] A block module 62 is used to block the parameters of the training model to obtain multiple parameter blocks;

[0148] an allocating module 63 for allocating an initial Hessian approximation information to each parameter block and the unblocked target parameter to replace the original Hessian information;

[0149] A Hessian approximation determination module 64 is used to determine target Hessian approximation information of the current iteration;

[0150] The training module 65 is used to iteratively train the to-be-trained model based on the target Hessian approximation information of the current iteration through the training data set until the iterative training conditions are met and the target model is obtained.

[0151] In addition, the control device for model training provided in the embodiment of the present application also includes:

[0152] The first block submodule is used to block the matrix parameters in the model to be trained in units of rows. The matrix parameters include the embedding layer of the Transformer architecture, the value matrix of the multi-head attention layer in the Transformer architecture, the output layer of the multi-head attention layer, the output layer of the model to be trained, and the fully connected layer of the Transformer architecture.

[0153] The second block submodule is used to block the query matrix and key matrix of the multi-head self-attention layer parameters based on the attention head;

[0154] The third block submodule is used to block the parameters of the layer normalization layer of the Transformer architecture based on the entire layer.

[0155] A first allocation submodule is configured to allocate an initial Hessian approximation scalar to a parameter block in units of the parameter block;

[0156] The second allocation submodule is used to allocate an initial Hessian approximation matrix of the same shape as the target parameter to the target parameter; the target parameter includes the bias layer of the model to be trained.

[0157] The iteration number determination module is used to determine whether the current iteration number is an integer multiple of a preset value; the preset value is an integer greater than 1; if so, call the first confirmation module; if not, call the second confirmation module;

[0158] A first confirmation module is used to determine the target Hessian approximation information of the current iteration based on the target Hessian approximation information of the previous iteration;

[0159] The second confirmation module is configured to use the target Hessian approximation information of the previous iteration as the target Hessian approximation information of the current iteration.

[0160] The first confirmation module is further used to determine the target Hessian approximation scalar of the current iteration based on the target Hessian approximation scalar of the previous iteration and the average of the gradient Hessian approximation value of the current iteration; and determine the target Hessian approximation matrix of the current iteration based on the target Hessian approximation matrix of the previous iteration.

[0161] The first parameter determination module is used to determine the gradient of the current model to be trained and the weight decay behavior parameters of the current model before each round of iterative training;

[0162] The second parameter determination module is used to determine the current momentum exponential moving average based on the gradient;

[0163] An update amount determination module is used to determine the update amount of the parameter block based on the target Hessian approximation information of the current iteration and the current momentum exponential moving average;

[0164] The model weight parameter determination module is used to determine the weight parameters of the currently trained model based on the update amount, the current number of iterations, the current model weight decay behavior parameters, and the current momentum exponential moving average;

[0165] The training submodule is used to input the training data set into the model to be trained for iterative training under the model weight parameters.

[0166] Figure 7 A schematic diagram of a control device for model training provided in another embodiment of the present application is shown in FIG. Figure 7 As shown, the control device for model training includes: a memory 70 for storing computer programs;

[0167] The processor 71 is configured to implement the steps of the control method for model training as described in the above embodiment when executing a computer program.

[0168] The control device for model training provided in this embodiment may include but is not limited to a tablet computer, a laptop computer, or a desktop computer.

[0169] Among them, the processor 71 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 71 can be implemented in at least one hardware form of a digital signal processor (DSP), a field programmable gate array (FPGA), and a programmable logic array (PLA). The processor 71 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 71 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 71 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0170] The memory 70 may include one or more computer-readable storage media, which may be non-transitory. The memory 70 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 70 is at least used to store the following computer program 701, wherein, after the computer program is loaded and executed by the processor 71, it can implement the relevant steps of the control method of model training disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 70 may also include an operating system 702 and data 703, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 702 may include Windows, Unix, Linux, etc. The data 703 may include but is not limited to the relevant data involved in the control method of model training, etc.

[0171] In some embodiments, the control device for model training may also include a display screen 77 , an input / output interface 72 , a communication interface 74 , a power supply 75 , and a communication bus 76 .

[0172] Those skilled in the art will understand that Figure 7 The structure shown in does not constitute a limitation on the control device for model training and may include more or fewer components than shown in the figure.

[0173] The control device for model training provided in an embodiment of the present application includes a memory and a processor. When the processor executes the program stored in the memory, it can implement the control method for model training in the above embodiment.

[0174] It should be noted that although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

Claims

1. A control method for model training, characterized in that: The method comprises: Constructing a model to be trained; and obtaining a training data set for training the model to be trained; wherein the training data set is a text data set; Dividing the parameters of the model to be trained into blocks to obtain multiple parameter blocks; Assigning an initial Hessian approximation information to each of the parameter blocks and the unblocked target parameter to replace the original Hessian information; Determine the target Hessian approximation information of the current iteration; Based on the target Hessian approximation information of the current iteration, iteratively training the model to be trained using the training data set until an iterative training condition is met and a target model is obtained; The model to be trained is a model based on the Transformer architecture; parameter segmentation of the model to be trained includes: For the matrix parameters in the model to be trained, the parameters are divided into blocks in units of rows; wherein the matrix parameters include the embedding layer of the Transformer architecture, the value matrix of the multi-head attention layer in the Transformer architecture, the output layer of the multi-head attention layer, the output layer of the model to be trained, and the fully connected layer of the Transformer architecture; For the query matrix and key matrix of the multi-head self-attention layer, parameter blocks are performed based on the attention head; For the layer normalization layer of the Transformer architecture, parameters are divided into blocks based on the entire layer; The target Hessian approximation information includes a target Hessian approximation scalar and a target Hessian approximation matrix; determining the target Hessian approximation information of the current iteration includes: determining the target Hessian approximation information of the current iteration based on the target Hessian approximation information of the previous iteration; Determining the target Hessian approximation information of the current iteration based on the target Hessian approximation information of the previous iteration includes: Determine the target Hessian approximation scalar of the current iteration according to the mean of the target Hessian approximation scalar of the previous iteration and the gradient Hessian approximation value of the current iteration; The target Hessian approximation matrix of the current iteration is determined according to the target Hessian approximation matrix of the previous iteration.

2. The control method for model training according to claim 1, characterized in that: The initial Hessian approximation information includes an initial Hessian approximation scalar and an initial Hessian approximation matrix; each parameter block and unblocked target parameter is assigned an initial Hessian approximation information, including: For the parameter block, one initial Hessian approximation scalar is allocated correspondingly based on the parameter block; For the target parameter, an initial Hessian approximation matrix having the same shape as the target parameter is assigned; the target parameter includes a bias layer of the model to be trained.

3. The control method for model training according to claim 1, characterized in that: Before determining the target Hessian approximation information of the current iteration based on the target Hessian approximation information of the previous iteration, the method includes: Determine whether the current number of iterations is an integer multiple of a preset value; the preset value is an integer greater than 1; If yes, proceed to the step of determining the target Hessian approximation information of the current iteration based on the target Hessian approximation information of the previous iteration; If not, the target Hessian approximation information of the previous iteration is used as the target Hessian approximation information of the current iteration.

4. The control method for model training according to claim 1, characterized in that: If the parameter block is the matrix parameter, the mean is calculated row by row; if the parameter block is the query matrix and the key matrix, the mean is calculated per attention head; if the parameter block is the layer normalization layer, the mean is calculated per layer normalization layer.

5. The control method for model training according to claim 1, characterized in that: Iteratively training the model to be trained using the training data set based on the target Hessian approximation information of the current iteration, including: Before each round of iterative training, determining the gradient of the current model to be trained and the weight decay behavior parameters of the current model; Based on the gradient, determining the current momentum exponential moving average; Determining an update amount of the parameter block according to the target Hessian approximation information of the current iteration and the current momentum exponential moving average; Determining the weight parameters of the model currently being trained based on the update amount, the current number of iterations, the current model weight decay behavior parameter, and the current momentum exponential moving average; Under the model weight parameters, the training data set is input into the model to be trained for iterative training.

6. A control device for model training, characterized in that: The device comprises: Model building module, used to build the model to be trained; A data acquisition module, configured to acquire a training data set for training the model to be trained; wherein the training data set is a text data set; A block module is used to block the parameters of the model to be trained to obtain multiple parameter blocks; an allocating module, configured to allocate an initial Hessian approximation information to each of the parameter blocks and the unblocked target parameter to replace the original Hessian information; A Hessian approximation determination module is used to determine the target Hessian approximation information of the current iteration; the target Hessian approximation information includes a target Hessian approximation scalar and a target Hessian approximation matrix; A training module, configured to iteratively train the model to be trained using the training data set based on the target Hessian approximation information of the current iteration until an iterative training condition is met and a target model is obtained; The first confirmation module is used to determine the target Hessian approximation scalar of the current iteration based on the target Hessian approximation scalar of the previous iteration and the average of the gradient Hessian approximation value of the current iteration; and determine the target Hessian approximation matrix of the current iteration based on the target Hessian approximation matrix of the previous iteration.

7. A control device for model training, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the program, the steps of the control method for model training described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the control method for model training described in any one of claims 1 to 5 are implemented.

Images