Model training method and device, equipment and storage medium

Abstract

The invention provides a model training method and device, equipment and a storage medium. The method comprises the following steps: determining a first optimizer state parameter at a current time step according to a loss function value obtained by forward propagation calculation of a neural network model; the first optimizer state parameter, the second optimizer state parameter and the loss function value have a first corresponding relation; substituting the first optimizer state parameter and the loss function value into a first corresponding relation in response to the detection that the training of the current time step meets a preset training abnormal condition, so as to obtain a second optimizer state parameter; substituting the second optimizer state parameter into the loss function, and updating a loss function value; substituting the updated loss function value and the second optimizer state parameter into the first corresponding relation, and determining a third optimizer state parameter; and using the third optimizer state parameter to execute the training of the next time step on the neural network model.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a model training method, apparatus, device, and storage medium. Background Technology

[0002] Deep learning is an important branch of artificial intelligence, widely used in fields such as image recognition and natural language processing. Its core principle is to simulate the workings of the human brain through neural networks, thereby enabling the learning and analysis of complex data. Model training and optimization techniques are a crucial aspect of deep learning, involving how to effectively adjust model parameters to improve training efficiency and model performance.

[0003] In model training involving image processing, text processing, speech processing, and video processing, the amount of data is often huge, which can easily lead to sudden errors and involves a large number of adjustments and updates to optimizer state parameters. Therefore, how to reduce sudden errors and improve model performance in this scenario is one of the urgent problems to be solved in this field. Summary of the Invention

[0004] In view of the above, this disclosure provides a model training method, apparatus, device, and storage medium to solve at least one problem existing in the prior art.

[0005] To achieve the above objectives, the technical solution of this disclosure embodiment is implemented as follows:

[0006] On one hand, embodiments of this disclosure provide a model training method, including:

[0007] During training at the current time step, the first optimizer state parameters are determined based on the loss function value calculated by forward propagation of the neural network model; wherein, the loss function value is the function value corresponding to the second optimizer state parameters in the previous time step; the first optimizer state parameters, the second optimizer state parameters, and the loss function value have a first correspondence relationship;

[0008] In response to the detection that the training at the current time step meets the preset training anomaly conditions, the first optimizer state parameters and the loss function value are substituted into the first correspondence to obtain the second optimizer state parameters;

[0009] Substitute the second optimizer state parameters into the loss function and update the loss function value;

[0010] Substitute the updated loss function value and the second optimizer state parameter into the first correspondence to determine the third optimizer state parameter;

[0011] The neural network model is trained for the next time step using the state parameters of the third optimizer.

[0012] On the other hand, embodiments of this disclosure also provide a model training apparatus, including:

[0013] The first determining module is configured to determine the first optimizer state parameters in the training at the current time step based on the loss function value calculated by forward propagation of the neural network model; wherein the loss function value is the function value corresponding to the second optimizer state parameters in the previous time step; the first optimizer state parameters, the second optimizer state parameters, and the loss function value have a first correspondence relationship;

[0014] The rollback module is configured to, in response to detecting that the training at the current time step meets a preset training anomaly condition, substitute the first optimizer state parameters and the loss function value into the first correspondence to obtain the second optimizer state parameters.

[0015] The update module is configured to substitute the second optimizer state parameters into the loss function and update the loss function value;

[0016] The second determining module is configured to substitute the updated loss function value and the second optimizer state parameter into the first correspondence to determine the third optimizer state parameter.

[0017] The training module is configured to train the neural network model for the next time step using the state parameters of the third optimizer.

[0018] In another aspect, embodiments of this disclosure also provide a model training device, including a memory, a processor, and a computer program stored in the memory; when the processor executes the computer program, it implements any of the above-described model training methods.

[0019] In another aspect, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the above-described model training methods.

[0020] In the technical solution provided in this disclosure, during training at the current time step, the first optimizer state parameters are determined based on the loss function value. When a training anomaly is detected, the model is backtracked to the second optimizer state parameters through this correspondence, thereby preventing erroneous parameters from continuing to participate in training. Then, the loss function value is recalculated based on the second optimizer state parameters, and a third optimizer state parameter is further generated, allowing the model to continue training based on the correct state. This approach effectively reduces resource waste and model performance degradation caused by errors by promptly identifying problems during training through an anomaly detection mechanism and quickly restoring the model state through an inverse operation rollback strategy. Furthermore, it eliminates the need for extensive storage of historical state information, reducing GPU memory usage. Attached Figure Description

[0021] Figure 1 The flow of the model training method provided in the embodiments of this disclosure Figure 1 ;

[0022] Figure 2 The flow of the model training method provided in the embodiments of this disclosure Figure 2 ;

[0023] Figure 3 The flow of the model training method provided in the embodiments of this disclosure Figure 3 ;

[0024] Figure 4 This is a structural block diagram of the model training device provided in the embodiments of this disclosure;

[0025] Figure 5 This is a structural block diagram of the model training device provided in an embodiment of the present disclosure. Detailed Implementation

[0026] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the specific embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0027] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of this disclosure. However, it will be apparent to those skilled in the art that this disclosure may be practiced without one or more of these details. In other instances, to avoid confusion with this disclosure, certain technical features well-known in the art have not been described; that is, not all features of actual embodiments are described herein, nor are well-known functions and structures described in detail.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. When used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprise” and / or “comprising,” when used in this specification, identify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups. When used herein, the term “and / or” includes any and all combinations of the associated listed items.

[0029] To address potential errors during the training of large-scale deep learning models, relevant technologies primarily employ the following solutions:

[0030] First, directly loading the pre-saved optimizer state has the advantage of being simple and direct, and can quickly restore the previously saved optimizer state, thus achieving error rollback. However, since the optimizer state of a large model usually requires hundreds of gigabytes of storage space, directly saving and loading such a large state will consume a lot of GPU memory resources, thus causing a significant drain on training resources in practical applications.

[0031] Second, loading pre-saved checkpoints: When an error occurs during training, the most recent checkpoint can be loaded and training can continue from that point. However, this method may require rolling back to a very old time step, thus wasting training resources and potentially causing a loss of model performance. This is especially problematic when the model is close to convergence during training, as this method results in a significant waste of training resources.

[0032] This disclosure provides a model training method to improve the efficiency of optimizer state rollback, enabling rapid and effective rollback of the optimizer state when errors occur during training, thereby reducing training interruptions and resource waste. In this disclosure, the neural network model can be used to perform any one of image processing, speech processing, text processing, and video processing tasks.

[0033] This disclosure illustrates an embodiment using a model training method applied to an image processing task as an example. Image processing includes image classification, object detection, image segmentation, and image generation. Taking image generation as an example, the model is trained using a large image set, enabling it to generate new images that are different from any image in the set. The trained model aims to generate images that are visually indistinguishable from real images. To achieve this, a large amount of data, a complex model training process, and updates to the optimizer's state parameters are required, thus necessitating substantial training resources and higher model accuracy requirements. Therefore, a more efficient error rollback mechanism is needed during training, while minimizing the consumption of training resources.

[0034] like Figure 1 As shown, this disclosure provides a model training method, including:

[0035] Step S101: In the training at the current time step, the first optimizer state parameters are determined based on the loss function value calculated by forward propagation of the neural network model; wherein, the loss function value is the function value corresponding to the second optimizer state parameters in the previous time step; the first optimizer state parameters, the second optimizer state parameters, and the loss function value have a first correspondence relationship;

[0036] Step S102: In response to detecting that the training at the current time step meets the preset training anomaly conditions, substitute the first optimizer state parameters and the loss function value into the first correspondence to obtain the second optimizer state parameters.

[0037] Step S103: Substitute the state parameters of the second optimizer into the loss function and update the loss function value;

[0038] Step S104: Substitute the updated loss function value and the second optimizer state parameters into the first correspondence to determine the third optimizer state parameters;

[0039] Step S105: Using the state parameters of the third optimizer, train the neural network model for the next time step.

[0040] Here, forward propagation refers to a computational process in a neural network that involves the flow of data from the input layer to the output layer. During the training phase, forward propagation is used to compute the model's predicted outputs, which are then compared to the true values ​​to calculate the loss. During the inference (or prediction) phase, forward propagation is used to generate predictions for new input data.

[0041] The aforementioned loss function, also known as the cost function or objective function, is used in machine learning and deep learning to measure the difference between the model's predicted values ​​and the true values. The purpose of the loss function is to provide a quantifiable metric to evaluate the model's performance and guide parameter optimization during model training. In this embodiment, the loss function is a function of the second optimizer state parameters in the previous time step; that is, the corresponding loss function can be obtained for each time step through the optimizer state parameters.

[0042] The choice of loss function typically depends on the specific problem type, such as classification, regression, or clustering. Common loss functions include Mean Squared Error (MSE), Cross-Entropy Loss, Mean Absolute Error (MAE), Logarithmic Loss, Hinge Loss, Mean Absolute Percentage Error (MAPE), and custom loss functions. In the embodiments of this disclosure, any of the loss functions described above may be used, without specific limitation.

[0043] In deep learning, an optimizer is an algorithm used to adjust the parameters (such as weights and biases) of a neural network during training to minimize the loss function. The optimizer is one of the key components in training a deep learning model, determining the speed and efficiency of the model's learning.

[0044] Common optimizers include:

[0045] 1. SGD (Stochastic Gradient Descent): This is the most basic optimizer. It calculates the gradient of the loss function with respect to each parameter and updates the parameters to reduce the loss.

[0046] 2. Momentum: An extension of SGD, it adds a momentum term to gradient updates, which can accelerate convergence and reduce oscillations.

[0047] 3. NAG (“Nesterov” Accelerated Gradient): An improved version of “Momentum” that takes future position into account when calculating momentum.

[0048] 4. "AdaGrad": Adaptive gradient algorithm, which adjusts the learning rate individually for each parameter, allowing sparse parameters to learn faster.

[0049] 5. "RMSprop": This solves the problem of the learning rate of "AdaGrad" decreasing too quickly by calculating the moving average of the squared gradient through exponential decay.

[0050] 6. "Adam" (Adaptive Moment Estimation): It combines the advantages of Momentum and RMSprop, simultaneously calculating the first moment (mean) and second moment (variance) of the gradient, and adaptively adjusting the learning rate of each parameter.

[0051] 7. "dadelta": Similar to "AdaGrad", but uses a moving average to calculate the square of the gradient, thus avoiding the problem of the learning rate decreasing too quickly.

[0052] 8. "Adamax": A variant of Adam that uses a different update rule to compute the infinite norm of the gradient.

[0053] 9. Nadam (“Nesterov-accelerated Adaptive Moment Estimation”): Combines NAG and Adam, using the momentum term of NAG and the adaptive learning rate of Adam.

[0054] This disclosure uses the "Adam" optimizer as an example for illustration. In other embodiments, any of the above-mentioned optimizers can be used to achieve error rollback based on the inventive concept provided in this disclosure. The use of any of the above-mentioned optimizers is within the scope of protection of this disclosure, and this disclosure does not impose any limitations.

[0055] The optimizer state parameters mentioned above refer to a set of variables maintained by the optimizer during training. These variables are used to control and record the optimization process. Different optimizers may maintain different state information, which may include gradient values, momentum, learning rate decay, parameter history values, etc.

[0056] Since the optimizer state needs to be updated at each time step, the forward propagation process is performed, and the loss function value of the model training method in the current time step is calculated.

[0057] The loss function value at the current time step is calculated by the difference between the neural network model's prediction of the input data and the true label, and is used to measure the prediction accuracy of the neural network model. The choice of loss function can be adjusted according to the specific task, such as mean squared error (MSE), cross-entropy loss, etc.

[0058] Forward propagation refers to the process of calculating the output of each layer in a neural network model, sequentially from the input layer to the output layer. The results of forward propagation are used to calculate the loss function value and serve as the basis for backpropagation. The results of forward propagation not only affect the loss function value at the current time step but also influence the state parameter updates of the subsequent first optimizer.

[0059] The loss function value is a quantitative metric used to evaluate the performance of a neural network model. A smaller loss function value indicates better predictive performance from the neural network model. In this embodiment, the loss function value is determined by the state parameters of the second optimizer in the previous time step. This fact means that the loss function value at the current time step is calculated based on the state parameters of the second optimizer from the previous time step.

[0060] The aforementioned first correspondence refers to the mathematical relationship between the first optimizer state parameters, the second optimizer state parameters, and the loss function value. This mathematical relationship is typically determined by the optimizer's algorithm, such as the parameter update formula in the Adam optimizer. Under this mathematical relationship, when the model calculates the loss function value at the current time step, it uses the second optimizer state parameters from the previous time step as input for calculation.

[0061] Error rollback refers to restoring the model to a previously known good state when an anomaly or error is encountered during training. This helps reduce the waste of time and resources caused by training failures. The model training method continuously monitors for anomalies during training. When a training anomaly is detected, such as a computational error, memory overflow, gradient explosion, or a sudden increase in the loss function value, the model training method triggers a rollback mechanism.

[0062] Preset training anomaly conditions refer to the judgment criteria set in the model training method to identify abnormal situations that occur during training, which may lead to failure in updating neural network model parameters or a decrease in model performance. Examples include computational errors, memory overflow, gradient explosion, or a sudden increase in the loss function value. To promptly detect and handle training anomalies, the model training method sets preset training anomaly conditions, such as a loss function value greater than a preset loss function value threshold, a gradient greater than a preset gradient threshold, or an accuracy less than a preset accuracy. The error rollback method used in this embodiment is based on the inverse operation of the first optimizer state parameters with errors, thereby restoring the second optimizer state parameters to the previous time step.

[0063] Here, inverse operation refers to the process of deriving and restoring the state parameters of the second optimizer from the previous time step based on the first optimizer state parameters of the current time step and the first correspondence relationship. The core of inverse operation lies in using the parameters of the current time step and the known first correspondence relationship to derive the correct parameters of the previous time step, thereby achieving state rollback.

[0064] Next, the recovered second optimizer state parameters are substituted back into the function of the first correspondence to update the loss function value at the current time step. It's understandable that the loss function value corresponding to an anomaly is erroneous, therefore the obtained first optimizer state parameters are also incorrect. In the above inverse operation, the erroneous first optimizer state parameters and the erroneous loss function value are substituted into the functions of the first correspondence, and the process is rolled back to the previous time step, restoring the correct second optimizer state parameters. It's understandable that in this way, regardless of whether an error actually occurred, or the specific parameters where the error occurred, it's unnecessary to confirm before rolling back to the previous time step, thus simplifying the training process.

[0065] After obtaining the optimizer state parameters from the previous time step, the system returns to the training node of the previous time step, allowing training to continue. These parameters are then substituted into the loss function to recalculate the loss function value, aiming to obtain the most accurate possible value. Next, the training process continues at the current time step. Based on the loss function value and the second optimizer state parameters, the first correspondence is used to obtain the third optimizer state parameters.

[0066] It's understandable that the third optimizer state parameters are essentially corrected versions of the first optimizer state parameters. Ideally, the third optimizer state parameters are correct, rectifying the errors in the original first optimizer state parameters. However, this process still requires detecting training anomalies, meaning the third optimizer state parameters might still be incorrect. If errors persist during recalculation, a rollback can be performed again to recalculate the new third optimizer parameters. Of course, if multiple rollbacks still result in errors, other methods can be used, or model training can be terminated; no specific limitations are specified here.

[0067] It is understandable that the first, second, and third optimizer state parameters mentioned above are used to distinguish different states of optimizer state parameters for ease of description of the above process. In fact, these names represent the states of each optimizer state parameter at different time steps. Therefore, at different time points during the training process, the names of the first, second, and third optimizer state parameters mentioned above can refer to the same set of optimizer state parameters. In other words, at different time steps, the same set of optimizer state parameters may be described using the above different names. Therefore, for ease of description, the general description of the first, second, or third optimizer state parameters mentioned above will be collectively referred to as optimizer state parameters in the following text.

[0068] In summary, by using the above method, when a training anomaly is detected at a certain time step, the inverse operation can be immediately performed by substituting the current optimizer state parameters into the first correspondence, rolling back to the state of the previous time step, thus facilitating recalculation. This effectively reduces model performance loss caused by training anomalies and eliminates the need to store large amounts of optimizer state for fixed-point rollback, avoiding the need to roll back to a state from a long time ago, thereby saving significant training resources and improving training efficiency.

[0069] Furthermore, it should be noted that the neural network model in the above embodiments can be used to perform any one of image processing tasks, speech processing tasks, text processing tasks, and video processing tasks. Taking image processing as an example, the method may further include:

[0070] Image data is input into the input layer of the neural network model and processed by polar convolution in the convolutional layer to obtain a feature map. The obtained feature map is then processed by the pooling layer and activation function before entering the fully connected layer to obtain the feature vector corresponding to the feature map. Finally, it is passed to the output layer to obtain the corresponding loss function.

[0071] Backpropagation is performed based on the loss function to calculate the gradient and other state parameters of the first optimizer and update the parameters of the neural network model.

[0072] Based on the first optimizer state parameters obtained from the above processing, it is detected whether there is a training anomaly. If there is a training anomaly, it is processed according to the above steps S102 to S104 to obtain the third optimizer state parameters before continuing the model training for the next time step.

[0073] In some embodiments, such as Figure 2 As shown, the method further includes:

[0074] Step S201: Detect the specified training parameters of the neural network model at the current time step;

[0075] Step S202: If the value of the specified training parameter is within the preset parameter value range, then it is determined that the training at the current time step meets the training anomaly condition.

[0076] Specifying training parameters refers to a class of parameters used during the training of a neural network model to determine whether anomalies have occurred during training. In some embodiments, specified parameters include one or more of the following: loss function, gradient value, and accuracy, wherein the loss function includes mean squared error, cross-entropy loss, etc.

[0077] Specifying training parameters reflects the training status and performance of a neural network model at the current time step and is an important indicator for evaluating the health of the neural network model. By monitoring the changing trends or values ​​of specified training parameters, the system can promptly identify potential problems during the training process, such as gradient explosion, convergence stagnation, or model performance degradation. For example, in image classification tasks, if the loss function suddenly increases significantly, it may indicate that the neural network model has exhibited unstable update behavior, requiring further investigation or intervention by the system.

[0078] The smaller the loss function value of a neural network model, the closer its prediction is to the actual target. By monitoring changes in the loss function value of a neural network model, we can determine whether the model has converged normally or whether there are abnormal fluctuations during the training process.

[0079] The gradient value represents the derivative of the model parameters calculated using the loss function of the neural network model during backpropagation. The gradient value of a neural network model reflects the direction and magnitude of parameter adjustments. When the gradient value of a neural network model is too large, it may cause the model to become unstable or even diverge; conversely, when the gradient value is too small, it may indicate that the model has fallen into a local optimum or that the learning process has stalled. Therefore, monitoring the gradient value of a neural network model helps to promptly detect any anomalies that may occur during training.

[0080] Accuracy is a crucial metric for evaluating the classification performance of a neural network model, typically used in supervised learning tasks. Higher accuracy indicates a stronger predictive ability on the test data. A sudden drop in accuracy during training may indicate training anomalies, such as overfitting or data contamination.

[0081] In some embodiments, the specified training parameters include one or more of the following: loss function value, gradient value, and accuracy of the neural network model; training anomaly conditions include: loss function value greater than a preset loss function threshold, gradient value greater than a preset gradient threshold, or accuracy less than a preset accuracy; the method further includes: adjusting the training anomaly conditions according to the training process of the neural network model.

[0082] Since the requirements for anomaly detection may differ during training, the above training anomaly conditions can be adjusted according to the needs of the training process. For example, the size of the loss function threshold, the size of the gradient threshold, or the accuracy can be adjusted.

[0083] For example, considering that model training is an iterative process of forward and backward propagation of a neural network, in the early stages of training, the model has high discreteness and low convergence, but the convergence speed may be faster, i.e., the gradient is high. Therefore, at this time, a higher loss function threshold and gradient threshold can be set, without needing excessively high accuracy. In the later stages of training, the model is close to convergence, so the loss function threshold and gradient value can be set lower, and the accuracy can be set higher, and so on.

[0084] In practical applications, training anomaly conditions can also be set or adjusted according to the type of model or requirements.

[0085] In some embodiments, the first optimizer state parameters include: a first gradient value, a first first moment estimate, a first second moment estimate, and a first weight parameter; the second optimizer state parameters include: a second gradient value, a second first moment estimate, a second second moment estimate, and a second weight parameter.

[0086] The first correspondence includes: the first sub-relation of the first gradient value, the first first-order moment estimate, and the second first-order moment estimate; the second sub-relation of the first gradient value, the first second-order moment estimate, and the second second-order moment estimate; and the third sub-relation of the first first-order moment estimate, the first second-order moment estimate, the first weight parameter, and the second weight parameter.

[0087] For example, the process of determining the state parameters of the first optimizer is the parameter update process of the Adam algorithm, and the process of calculating the gradient is as follows: Formula (1):

[0088]

[0089] Among them, g t Let L be the first gradient value and L be the loss function. This represents the estimated value of the loss function, and ▽ represents the amount of change, i.e. θ represents the change in the estimated value of the loss function. t The weight parameter is the weight parameter for the current time step, where t is an integer greater than 1, representing the t-th time step.

[0090] Based on the gradient g above t Determine the first first-order moment estimate m t and the first second-order moment estimate v t The following formula (2) represents the relationship between the first subrelation and the second subrelation:

[0091]

[0092] Wherein, β1 and β2 are hyperparameters controlling the exponential decay rate. For example, the value of β1 can be between 0.8 and 0.999, and for example, it can be 0.9. This hyperparameter can stably optimize the direction and suppress gradient noise. The larger the value, the stronger the momentum effect and the smoother the direction. The value of β2 can be between 0.98 and 0.9999, and for example, it can be 0.999. This hyperparameter is used to adaptively adjust the learning rate of the parameters. The larger the value, the more stable the learning rate scaling factor. t-1 This represents the first first-order moment estimate, i.e., the second first-order moment estimate, at the previous time step (t-1), v t-1 This represents the first second-order moment estimate, or second second-order moment estimate, of the previous time step (t-1). The first-order moment estimate represents the mean of the gradient, reflecting the average level of the gradient, while the second-order moment estimate represents the squared mean of the gradient, reflecting the dispersion of the data. t The first gradient value mentioned above, It is the square of the first gradient value.

[0093] In some embodiments, the third sub-relationship includes:

[0094] The first first-order moment estimate is the corrected first-order moment estimate obtained after bias correction based on the first attenuation rate; the first second-order moment estimate is the corrected second-order moment estimate obtained after bias correction based on the second attenuation rate, as shown in the following formula (3);

[0095] The second correspondence between the corrected first-order moment estimate, the corrected second-order moment estimate, the first weight parameter, and the second weight parameter is as follows (4).

[0096] Here, we can estimate m for the first first moment mentioned above. t and the second-order moment estimate v t After bias correction, the corrected first-order moment estimate is obtained. and first and second moment estimates As shown in formula (3) below:

[0097]

[0098] Next, the corrected first-order moment estimation is used. and first and second moment estimates Perform parameter updates to obtain the updated first weight parameter θ. t+1 That is, the weight parameters for the next time step (t+1), as shown in the following formula (4):

[0099]

[0100] Where η represents the learning rate, which determines the size of the parameter update step, and ε represents the minimum value, which is essentially a very small constant, such as 10. -8 Or 10 -7 Its function is to prevent the denominator from being zero, ensuring the stability of the value. Through the above process, we obtain the first gradient value g. t First-order moment estimate m t and the second-order moment estimate v t Corrected first-order moment estimate and first and second moment estimates and the first weight parameter θ t+1 The state parameters of each of the first optimizers, including those mentioned above. If an anomaly occurs during the above processing, one or more of these parameters may become incorrect, thus requiring a rollback.

[0101] In some embodiments, step S102 above, which substitutes the first optimizer state parameters and the loss function value into the first correspondence to obtain the second optimizer state parameters, includes:

[0102] Substitute the first gradient value and the first first-order moment estimate into the first sub-relation to determine the second first-order moment estimate;

[0103] Substitute the first gradient value and the first second-order moment estimate into the second sub-relation to determine the second second-order moment estimate;

[0104] Substitute the first weight parameter, the first first-order moment estimate, and the first second-order moment estimate into the third sub-relation to determine the second weight parameter.

[0105] For example, when a training anomaly is detected, the parameters obtained in the previous step can be inversely calculated, that is, the first gradient value g can be obtained by performing an inverse operation. t and the first-order moment estimate m t Substitute the first sub-relation and the first gradient value g t And the first and second moment estimates v t Substituting the second sub-relation into the following formula (6), we obtain the second first-order moment estimate m. t-1 Second-order moment estimate v t-2 ; and estimate the first first moment m t And the first and second moment estimates v t Substitute into formula (3) and then combine with the first weight parameter θ t+1Substitute them together into the following formula (5):

[0106]

[0107]

[0108] Thus, we revert to the state of the previous time step. Here, θ t m is the second weighting parameter. t-1 For the second first-order moment estimate, v t-1 This is the second second-order moment estimate, i.e., the state parameters of the second optimizer mentioned above.

[0109] It is understandable that errors may have occurred in the gradient calculation during the previous processing, resulting in the first optimizer state parameters being incorrect, while the second optimizer state parameters from the previous time step are correct. Therefore, by using the incorrect parameters for inverse operation, we can roll back to the previous step without needing to determine the extent of the data error, and we can achieve rollback simply and efficiently.

[0110] In some embodiments, step S103 above, which substitutes the second optimizer state parameters into the loss function and updates the loss function value, includes:

[0111] Substitute the second weight parameter into the loss function to determine the updated loss function value.

[0112] In other words, after rolling back and obtaining the second optimizer state parameters from the previous time step, the second weight parameter θ, which is related to the loss function, can be used. t Substituting the loss function back into the formula (1) above, the resulting loss function value is the updated third gradient value g′. t It may overwrite the first gradient value g from the previous calculation. t The error is corrected, and the loss function value is updated to the correct value. In some embodiments, in step S104 above, the third optimizer state parameters include: the third gradient value g′ t Third first moment estimate m′ t Third second-order moment estimate v′ t and the third weight parameter θ′ t Substitute the updated loss function value and the second optimizer state parameters into the first correspondence to determine the third optimizer state parameters, including:

[0113] First, determine the third gradient value based on the updated loss function value, that is, substitute the updated loss function value into the above formula (1) to obtain the third gradient value g′. t It is understandable that the third gradient value g′ t It equals the loss function value.

[0114] Second, the third gradient value g′ t Second first-order moment estimate m t-1 Substituting into the first sub-relation, i.e., the above formula (2), we determine the third first-order moment estimate m′. t ;

[0115] Third, the third gradient value g′ t Second-order moment estimate v t-1 Substituting the second subrelation into the above formula (2), we determine the third second-order moment estimate v′. t ;

[0116] Fourth, estimate the third first moment m′ t Third second-order moment estimate v′ t The second weighting parameter θ t Substituting into the third sub-relation, i.e., formulas (3) and (4) above, we determine the third weight parameter θ′. t+1 .

[0117] It is understandable that this process is similar to the process in step S101 above, that is, the parameters of the previous time step are processed again to obtain the parameters that need to be updated in the next time step. That is, the corresponding optimizer state parameters are obtained by recalculating using the above formulas (1) to (4).

[0118] Here, the obtained third optimizer state parameters are essentially the same parameters as the first optimizer state parameters from the same process. If there are still anomalies at this time, the current third optimizer state parameters can be used as the aforementioned first optimizer state parameters, and the inverse operation can be performed again to revert to the state of the previous time step until no training anomalies occur.

[0119] In some embodiments, such as Figure 3 As shown, the above method also includes:

[0120] Step S301: In response to the training not meeting the training exception conditions at the current time step, the neural network model is trained for the next time step using the state parameters of the first optimizer.

[0121] After obtaining the first optimizer state parameters through the processing in step S101 of the above embodiments, if no training anomaly conditions are detected, it indicates that the current first optimizer state parameters are correct parameters that meet the conditions and can continue to be used in the subsequent training process. Specifically, in each time step, the first optimizer state parameters need to be obtained through backpropagation based on the loss function obtained by forward propagation, and then the optimizer state is updated. After one round of iterative updates to the optimizer states of the neural network, the next round of iteration can be performed, thus realizing the iterative training of the neural network model.

[0122] Of course, if a training anomaly is detected, the third optimizer state parameters are obtained after processing steps S102 and S104 above. At this point, we return to the situation in step S101. If no training anomaly is detected, the third optimizer state parameters can be used as the first optimizer state parameters to continue subsequent training.

[0123] For ease of understanding, the embodiments of this disclosure provide the following examples:

[0124] This disclosure addresses potential errors during the training of large-scale deep learning models in tasks such as image processing, speech processing, text processing, and video processing. Specifically, it addresses the issue of abnormal situations occurring during model training by enabling rapid and effective rollback of the optimizer state. This avoids training interruptions, reduces resource waste, and improves the efficiency and stability of model training. Based on this, this disclosure provides the following model training apparatus, including:

[0125] Rollback Decision Module: This module monitors key metrics such as loss function or accuracy during model training to detect any anomalies. When an anomaly occurs, it triggers a rollback signal. Anomalies may include computational errors, insufficient GPU memory, etc. The rollback decision module needs to flexibly adjust its decision criteria based on the real-time situation during training to ensure timely response to various anomalies.

[0126] The decision criteria can be dynamically adjusted based on the real-time status of model training to ensure timely response to various anomalies. For example, a loss function threshold can be set, and a rollback signal can be triggered when the loss function value exceeds the threshold.

[0127] Weight rollback module: Once the rollback signal is triggered, the weight rollback module will be responsible for restoring the model's weight parameters to the previously saved state, that is, the state of the previous time step during training. The advantage is that it does not require additional storage of other information and has no additional GPU memory overhead. It should be noted that the gradient of the previous time step is required here. Specifically, the rollback can be achieved by using the above formula (5), that is, the inverse operation of formula (4).

[0128] Momentum rollback module: Similarly, when the rollback signal is triggered, the momentum rollback module is responsible for restoring the first-order momentum parameters of the optimizer to the previously saved state. This process needs to ensure the correct restoration of the momentum parameters to guarantee the continuity and stability of the training process. Specifically, the rollback can be achieved by using the above formula (6), which is the inverse operation of formula (2).

[0129] Continue Training Module: If the rollback determination module does not trigger a rollback signal, it indicates that the training process is proceeding normally. In this case, the Continue Training module will be responsible for continuing the model training and updating the optimizer state as needed. Specifically, after each new optimizer state parameter is obtained, it can be determined whether there is an anomaly. If there is no anomaly, the optimizer state parameters of the current model are received and the optimizer state is updated. This may involve parameter adjustments and optimizer updates to adapt to dynamic changes during the training process.

[0130] Specifically, during the forward propagation process of model training, the model performs forward computation to obtain the true labels (Logits), which are then compared with the loss labels (Labels) to calculate the loss function (Loss). Backpropagation is then performed to obtain the gradients. After backpropagation, the optimizer state updates the parameters. After calculating the loss function, it can be determined whether the update in this step is reasonable. Furthermore, the gradients from the previous implementation step still exist before the backpropagation is performed. The state of the parameters in the above process forms the basis for the implementation of the scheme in this embodiment.

[0131] Referring to formulas (1) to (4) above, the first moment estimate m t and second-order moment estimate v t , calculated at time step t, represent the moving average of the gradient at the current time step and the moving average of the squared gradient. The weight parameter θ to be updated. t+1 This is the corrected estimate at time step t. Therefore, although the above process is performed at time step t, its purpose is to update the weight parameter θ for the next time step. t+1 .

[0132] The core of the above scheme lies in the expectation of using all information at the current time step (including gradient values ​​and their historical values) to update the model parameters, ensuring a more stable and efficient optimization process. However, training anomalies often occur during backpropagation, not during the optimizer's parameter updates. Therefore, in general, the above first-order moment estimate m... t and second-order moment estimate v t The parameters are correct, while the source of error is often the gradient value g. t But this does not prevent the use of g t Restore the optimizer state to the previous time step. That is, utilize the incorrect g... t , and by the error of g t The calculated first-order moment estimate m t and second-order moment estimate v t Perform the inverse operation to obtain the correct optimizer state from the previous time step.

[0133] The above embodiments can achieve the following beneficial technical effects:

[0134] Improve training stability: The rollback judgment module monitors abnormal situations in real time during the training process and triggers a rollback operation in a timely manner when an abnormality is detected, which can effectively avoid interruptions in the training process and ensure the continuity and stability of the training process.

[0135] Reducing resource waste: Traditional training methods often require restarting training when errors occur, wasting significant computational resources. This embodiment of the present disclosure, by rolling back the optimizer state, can quickly restore the system to its previous normal state after an error occurs, reducing the waste of repeated computation.

[0136] Improve training efficiency: By detecting anomalies in real time during training and rolling back to the previous step, it is possible to quickly recover when errors occur, avoiding long training rollbacks and significantly improving overall training efficiency.

[0137] Reduced GPU memory usage: Compared to directly saving the entire optimizer state, the embodiments of this disclosure reduce GPU memory usage by rolling back the optimizer state, enabling efficient training of large models even in environments with limited GPU memory resources.

[0138] Optimize model performance: By quickly responding to and handling anomalies during training, frequent training interruptions are avoided, allowing the model to be trained under more stable conditions, ultimately improving the model's performance and accuracy.

[0139] Flexibility and scalability: The rollback technology proposed in this disclosure can be flexibly applied to different deep learning optimizers and model training processes, and has strong adaptability and scalability, which can adapt to the training needs of models of different scales and complexities.

[0140] In summary, the optimizer state rollback technique of this invention significantly improves the training effect of large-scale deep learning models by enhancing the stability and efficiency of the training process and reducing resource waste. It has important application value and broad prospects for promotion.

[0141] Based on the same inventive concept, such as Figure 4 As shown, this disclosure provides a model training apparatus 400, comprising:

[0142] The first determining module 401 is configured to determine the first optimizer state parameters in the training at the current time step based on the loss function value calculated by forward propagation of the neural network model; wherein the loss function value is the function value corresponding to the second optimizer state parameters in the previous time step; the first optimizer state parameters, the second optimizer state parameters, and the loss function value have a first correspondence relationship;

[0143] The rollback module 402 is configured to, in response to detecting that the training at the current time step meets a preset training anomaly condition, substitute the first optimizer state parameters and the loss function value into the first correspondence to obtain the second optimizer state parameters.

[0144] The update module 403 is configured to substitute the second optimizer state parameters into the loss function and update the loss function value;

[0145] The second determining module 404 is configured to substitute the updated loss function value and the second optimizer state parameter into the first correspondence to determine the third optimizer state parameter.

[0146] Training module 405 is configured to perform training on the neural network model for the next time step using the state parameters of the third optimizer.

[0147] In some embodiments, the apparatus further includes:

[0148] The detection module is configured to detect specified training parameters of the neural network model at the current time step;

[0149] The second determining module is configured to determine that the training at the current time step meets the training anomaly condition if the value of the specified training parameter is within a preset parameter value range.

[0150] In some embodiments, the specified training parameters include one or more of the loss function value, gradient value, and accuracy of the neural network model; the training anomaly conditions include: the loss function value being greater than a preset loss function threshold, the gradient value being greater than a preset gradient threshold, or the accuracy being less than a preset accuracy.

[0151] The device further includes an adjustment module configured to adjust the training anomaly conditions according to the training process of the neural network model.

[0152] In some embodiments, the first optimizer state parameters include: a first gradient value, a first first moment estimate, a first second moment estimate, and a first weight parameter; the second optimizer state parameters include: a second gradient value, a second first moment estimate, a second second moment estimate, and a second weight parameter.

[0153] The first correspondence includes: a first sub-relationship of the first gradient value, the first first moment estimate, and the second first moment estimate; a second sub-relationship of the first gradient value, the first second moment estimate, and the second second moment estimate; and a third sub-relationship of the first first moment estimate, the first second moment estimate, the first weight parameter, and the second weight parameter.

[0154] In some embodiments, the rollback module 402 includes:

[0155] The first determining submodule is configured to substitute the first gradient value and the first first moment estimate into the first sub-relation to determine the second first moment estimate;

[0156] The second determining submodule is configured to substitute the first gradient value and the first second-order moment estimate into the second sub-relation to determine the second second-order moment estimate;

[0157] The third determining submodule is configured to substitute the first weight parameter, the first first moment estimate, and the first second moment estimate into the third sub-relation to determine the second weight parameter.

[0158] In some embodiments, the update module 403 is specifically configured to substitute the second weight parameter into the loss function to determine the updated loss function value.

[0159] In some embodiments, the third optimizer state parameters include: a third gradient value, a third first-order moment estimate, a third second-order moment estimate, and a third weight parameter.

[0160] In some embodiments, the second determining module 404 includes:

[0161] The fourth determining submodule is configured to determine the third gradient value based on the updated loss function value;

[0162] The fifth determining submodule is configured to substitute the third gradient value and the second first-order moment estimate into the first sub-relation to determine the third first-order moment estimate;

[0163] The sixth determining submodule is configured to substitute the third gradient value and the second second moment estimate into the second sub-relation to determine the third second moment estimate;

[0164] The seventh determination submodule is configured to substitute the third first-order moment estimate, the third second-order moment estimate, and the second weight parameter into the third sub-relation to determine the third weight parameter.

[0165] In some embodiments, the third sub-relationship includes:

[0166] The first first-order moment estimate is a corrected first-order moment estimate obtained after bias correction based on the first attenuation rate;

[0167] The first second-order moment estimate is a corrected second-order moment estimate obtained after bias correction based on the second attenuation rate;

[0168] There is a second correspondence between the corrected first-order moment estimate, the corrected second-order moment estimate, the first weight parameter, and the second weight parameter.

[0169] In some embodiments, the training module is further configured to: in response to the training at the current time step not satisfying the training anomaly condition, perform training on the neural network model at the next time step using the first optimizer state parameters.

[0170] In some embodiments, the neural network model is used to perform any one of image processing tasks, speech processing tasks, text processing tasks, and video processing tasks.

[0171] Based on the same inventive concept, this disclosure also discloses a model training device, such as... Figure 5 As shown, the model training device 500 includes a processor 501 and a memory 502, wherein the memory 502 stores a computer program 5021 that can run on the processor 501, and the processor 501 can implement the method in any of the above embodiments when executing the computer program 5021.

[0172] In some embodiments, memory 502 stores a computer program 5021 that can run on a processor. Memory 502 is configured to store instructions and applications executable by processor 501. It can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in processor 501 and various modules in model training device 500. This can be implemented using flash memory or random access memory (RAM).

[0173] In some embodiments, the processor 501 executes the steps of any of the methods described above when executing the program. The processor 501 typically controls the overall operation of the model training device 500.

[0174] Based on the same inventive concept, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the methods described above.

[0175] Based on the same inventive concept, this disclosure also provides a computer program product, which includes a computer program that, when executed by a processor, implements any of the above methods.

[0176] In some specific examples, the processor described above can be at least one of the following: Application-Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that the electronic device implementing the above processor function can also be other types, and this disclosure does not specifically limit its application.

[0177] In some specific examples, the aforementioned computer storage medium / memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it can also be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0178] The methods disclosed in the several method embodiments provided in this disclosure can be arbitrarily combined without conflict to obtain new method embodiments.

[0179] The features disclosed in the several device embodiments provided in this disclosure can be arbitrarily combined without conflict to obtain new device embodiments.

[0180] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A model training method, characterized in that, The method includes: During training at the current time step, the first optimizer state parameters are determined based on the loss function value calculated by forward propagation of the neural network model; wherein, the loss function value is the function value corresponding to the second optimizer state parameters in the previous time step; the first optimizer state parameters, the second optimizer state parameters, and the loss function value have a first correspondence relationship; In response to the detection that the training at the current time step meets the preset training anomaly conditions, the first optimizer state parameters and the loss function value are substituted into the first correspondence to obtain the second optimizer state parameters; Substitute the second optimizer state parameters into the loss function and update the loss function value; Substitute the updated loss function value and the second optimizer state parameter into the first correspondence to determine the third optimizer state parameter; The neural network model is trained for the next time step using the state parameters of the third optimizer.

2. The model training method according to claim 1, characterized in that, The method further includes: Detect the specified training parameters of the neural network model at the current time step; If the value of the specified training parameter is within the preset parameter value range, then the training at the current time step is determined to meet the training anomaly condition.

3. The model training method according to claim 2, characterized in that, The specified training parameters include one or more of the loss function value, gradient value, and accuracy of the neural network model; the abnormal training conditions include: the loss function value is greater than a preset loss function threshold, the gradient value is greater than a preset gradient threshold, or the accuracy is less than a preset accuracy. The method further includes: The abnormal training conditions are adjusted according to the training process of the neural network model.

4. The model training method according to claim 1, characterized in that, The first optimizer state parameters include: a first gradient value, a first first moment estimate, a first second moment estimate, and a first weight parameter; the second optimizer state parameters include: a second gradient value, a second first moment estimate, a second second moment estimate, and a second weight parameter. The first correspondence includes: a first sub-relationship of the first gradient value, the first first moment estimate, and the second first moment estimate; a second sub-relationship of the first gradient value, the first second moment estimate, and the second second moment estimate; and a third sub-relationship of the first first moment estimate, the first second moment estimate, the first weight parameter, and the second weight parameter. The step of substituting the first optimizer state parameters and the loss function value into the first correspondence to obtain the second optimizer state parameters includes: Substitute the first gradient value and the first first-order moment estimate into the first sub-relation to determine the second first-order moment estimate; Substitute the first gradient value and the first second-order moment estimate into the second sub-relation to determine the second second-order moment estimate; Substitute the first weight parameter, the first first moment estimate, and the first second moment estimate into the third sub-relation to determine the second weight parameter.

5. The model training method according to claim 4, characterized in that, The step of substituting the second optimizer state parameters into the loss function and updating the loss function value includes: Substitute the second weight parameter into the loss function to determine the updated loss function value.

6. The model training method according to claim 5, characterized in that, The third optimizer state parameters include: a third gradient value, a third first-order moment estimate, a third second-order moment estimate, and a third weight parameter; the step of substituting the updated loss function value and the second optimizer state parameters into the first correspondence to determine the third optimizer state parameters includes: Based on the updated loss function value, determine the third gradient value; Substitute the third gradient value and the second first-order moment estimate into the first sub-relation to determine the third first-order moment estimate; Substitute the third gradient value and the second second moment estimate into the second sub-relation to determine the third second moment estimate; The third first-order moment estimate, the third second-order moment estimate, and the second weight parameter are substituted into the third sub-relation to determine the third weight parameter.

7. The model training method according to any one of claims 4 to 6, characterized in that, The third sub-relation includes: The first first-order moment estimate is a corrected first-order moment estimate obtained after bias correction based on the first attenuation rate; The first second-order moment estimate is a corrected second-order moment estimate obtained after bias correction based on the second attenuation rate; There is a second correspondence between the corrected first-order moment estimate, the corrected second-order moment estimate, the first weight parameter, and the second weight parameter.

8. The model training method according to any one of claims 1 to 6, characterized in that, The method further includes: In response to the training failure at the current time step, the neural network model is trained for the next time step using the first optimizer state parameters.

9. The model training method according to any one of claims 1 to 6, characterized in that, The neural network model is used to perform any one of the following tasks: image processing, speech processing, text processing, and video processing.

10. A model training device, characterized in that, The device includes: The first determining module is configured to determine the first optimizer state parameters in the training at the current time step based on the loss function value calculated by forward propagation of the neural network model; wherein the loss function value is the function value corresponding to the second optimizer state parameters in the previous time step; the first optimizer state parameters, the second optimizer state parameters, and the loss function value have a first correspondence relationship; The rollback module is configured to, in response to detecting that the training at the current time step meets a preset training anomaly condition, substitute the first optimizer state parameters and the loss function value into the first correspondence to obtain the second optimizer state parameters. The update module is configured to substitute the second optimizer state parameters into the loss function and update the loss function value; The second determining module is configured to substitute the updated loss function value and the second optimizer state parameter into the first correspondence to determine the third optimizer state parameter. The training module is configured to train the neural network model for the next time step using the state parameters of the third optimizer.

11. A model training device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory; when the processor executes the computer program, it implements the model training method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the model training method according to any one of claims 1 to 9.

Images