Model training method based on gradient variance reduction and data rearrangement

Abstract

The present application discloses a model training method based on gradient variance reduction and data rearrangement, and relates to the field of computer technology. The method includes: obtaining the initial model parameters and sample sequence corresponding to the model to be trained; rearranging the sample sequence at the beginning of the iteration cycle; determining the sample average gradient of the samples in the sample sequence at the beginning of the current iteration cycle in the current iteration cycle; selecting target samples in turn, determining the sample current gradient of the target samples under the current model parameters, adjusting the sample current gradient according to the sample average gradient to obtain the sample adjustment gradient, and updating the model parameters according to the sample adjustment gradient; if the sample traversal is completed, entering the next iteration cycle and returning to execute the step of rearranging the sample sequence until the preset training termination condition is met and the model parameters when the training is completed are obtained. In this way, the convergence speed, training rate and training effect of the model training can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model training method based on gradient variance reduction and data rearrangement. Background Art

[0002] With the development of science and technology, various models are being used more and more widely and can be applied in different fields and scenarios to solve different problems. Before using the model, it is necessary to train the model first.

[0003] In the model training process in the related art, the gradient corresponding to the sample is usually calculated directly according to the model parameters corresponding to the sample, so as to update the model parameters directly based on the gradient. However, for some models, such as models whose training process involves non-convex saddle point problems, the variance of the gradient estimate corresponding to the sample is large, which affects the convergence speed of the model training, and is not conducive to improving the training rate and training effect of the model.

[0004] Therefore, relevant technologies still need to be improved and developed. Summary of the invention

[0005] The main purpose of this application is to provide a model training method based on gradient variance reduction and data rearrangement, aiming to solve the technical problem that when the gradient corresponding to the sample is calculated directly according to the model parameters corresponding to the sample in the related technology, and the model parameters are updated directly based on the gradient, the variance of the gradient estimate corresponding to the sample is large, which affects the convergence speed of the model training, and is not conducive to improving the training rate and training effect of the model.

[0006] In order to achieve the above-mentioned object, the first aspect of the present application provides a model training method based on gradient variance reduction and data rearrangement, wherein the model training method based on gradient variance reduction and data rearrangement includes:

[0007] Obtain the initial model parameters corresponding to the model to be trained and a sample sequence consisting of multiple samples;

[0008] At the beginning of each iteration cycle, the samples in the sample sequence are reordered to update the sample sequence;

[0009] For the current iteration cycle of model training, determine the sample average gradient of all samples in the sample sequence under the model parameters at the beginning of the current iteration cycle;

[0010] Selecting samples in the sample sequence as target samples in sequence, determining, for each target sample, a sample current gradient of the target sample under the current corresponding model parameters, adjusting the sample current gradient according to the sample average gradient to obtain a sample adjusted gradient corresponding to the target sample, and updating the model parameters according to the sample adjusted gradient, wherein a gradient variance calculated based on the sample adjusted gradient is less than a gradient variance calculated based on the sample current gradient;

[0011] If the samples in the above sample sequence are traversed, enter the next iteration cycle and return to execute the above step of reordering the samples in the above sample sequence to update the above sample sequence at the beginning of each iteration cycle until the preset training termination condition is met and the model parameters when the training is completed are obtained.

[0012] Optionally, the reordering of samples in the sample sequence to update the sample sequence includes:

[0013] Randomly rearrange the sample index corresponding to each sample in the sample sequence to obtain a randomly rearranged index sequence;

[0014] According to the above sample sequence and the randomly rearranged index sequence, a sample sequence after the sample positions are reordered is generated.

[0015] Optionally, the reordering of samples in the sample sequence to update the sample sequence includes:

[0016] Obtaining an initial sequence, wherein the initial sequence is a sample sequence that has not undergone any reordering processing corresponding to any iteration cycle;

[0017] Randomly rearrange the sample index corresponding to each sample in the initial sequence to obtain an initial index sequence after random rearrangement;

[0018] According to the above initial sequence and the randomly rearranged initial index sequence, a sample sequence after the sample positions are reordered is generated.

[0019] Optionally, the sample arrangement orders corresponding to the sample sequences in any two iteration cycles are different.

[0020] Optionally, adjusting the current gradient of the sample according to the sample average gradient to obtain a sample adjustment gradient corresponding to the target sample includes:

[0021] Determine the sample initial gradient corresponding to the target sample under the model parameters at the beginning of the current iteration cycle;

[0022] According to the sample initial gradient and the sample average gradient, the sample current gradient is adjusted to obtain the sample adjustment gradient corresponding to the target sample.

[0023] Optionally, adjusting the current gradient of the sample according to the initial gradient of the sample and the average gradient of the sample to obtain the sample adjustment gradient corresponding to the target sample includes:

[0024] Determine the target adjustment value according to the difference between the sample average gradient and the sample initial gradient;

[0025] The sum of the current gradient of the sample and the target adjustment value is used as the sample adjustment gradient corresponding to the target sample.

[0026] Optionally, the above method further includes:

[0027] For each iteration cycle, at the end of the iteration cycle, a single projection operation is performed on the model parameters based on a preset constraint set.

[0028] As can be seen from the above, in the scheme of the present application, the initial model parameters corresponding to the model to be trained and the sample sequence consisting of multiple samples are obtained; at the beginning of each iteration cycle, the samples in the above sample sequence are reordered to update the above sample sequence; for the current iteration cycle of model training, the sample average gradient of all samples in the above sample sequence under the model parameters at the beginning of the above current iteration cycle is determined; the samples in the above sample sequence are selected as target samples in turn, and for each of the above target samples, the sample current gradient of the above target sample under the current corresponding model parameters is determined, and the above sample current gradient is adjusted according to the above sample average gradient to obtain the sample adjustment gradient corresponding to the above target sample, and the above model parameters are updated according to the above sample adjustment gradient, wherein the gradient variance calculated based on the above sample adjustment gradient is less than the gradient variance calculated based on the above sample current gradient; if the samples in the above sample sequence are traversed, the next iteration cycle is entered and the step of reordering the samples in the above sample sequence to update the above sample sequence at the beginning of each iteration cycle is returned to execute until the preset training termination condition is met and the model parameters when the training is completed are obtained.

[0029] Compared with the prior art, the present application does not directly update the model parameters based on the current gradient of the sample under the current corresponding model parameters, but first calculates the sample average gradient, and then adjusts the sample gradient based on the global sample average gradient to obtain the sample adjustment gradient, and then updates the model parameters according to the sample adjustment gradient. It is beneficial to reduce the variance of the gradient used for model parameter adjustment, thereby facilitating the convergence speed of model training, and further facilitating the training rate and training effect of the model. In addition, at the beginning of each iteration cycle, the samples in the sample sequence are reordered, which can avoid the dependence on the fixed order of the sample sequence during the model training process, and is beneficial to improving the training effect of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0031] Figure 1 It is a flowchart of a model training method based on gradient variance reduction and data rearrangement provided in an embodiment of the present application;

[0032] Figure 2 It is a specific flow chart of a model training method based on gradient variance reduction and data rearrangement provided in an embodiment of the present application;

[0033] Figure 3 It is a test result schematic diagram provided in an embodiment of the present application;

[0034] Figure 4 It is a test result schematic diagram provided in an embodiment of the present application;

[0035] Figure 5 It is a test result schematic diagram provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0037] It should be understood that when used in this specification, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0038] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0039] It should be further understood that the term “and / or” used in the specification of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0040] As used in this specification, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to being classified into", depending on the context. Similarly, the phrase "if it is determined" or "if classified into [described condition or event]" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce classified into [described condition or event]" or "in response to being classified into [described condition or event]", depending on the context.

[0041] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0043] At present, saddle point problems have attracted widespread attention. They have various applications in adversarial learning, reinforcement learning, robust controller design, resource allocation and model predictive control. In the process of model training for models involving saddle point problems, if the gradient corresponding to the sample is calculated directly according to the model parameters corresponding to the sample, and the model parameters are updated directly based on the gradient, the variance of the gradient estimate corresponding to the sample may be large, which may affect the convergence speed of the model training, thereby not being conducive to improving the training rate and training effect of the model. Specifically, the form of the saddle point problem is: .in and is a convex and compact set, exist The top is convex, However, many saddle point problems in practical applications do not have good convexity, and it is difficult to obtain stable points in non-convex settings. Non-convex saddle point problem is an important problem, which refers to the problem that the objective function is non-convex with respect to some variables in the optimization process.

[0044] Non-convex saddle point problems are very common in machine learning training and constrained network optimization. For smooth non-convex saddle point problems with finite sum structures, random algorithms have been widely studied due to their scalability and efficiency. The most commonly used random algorithm is the stochastic gradient descent ascent algorithm, which is a natural extension of the stochastic gradient descent algorithm on the saddle point problem. As a natural way to deal with the saddle point problem, the stochastic gradient descent ascent algorithm randomly selects a sample in each step to calculate the gradient, and performs gradient descent and ascent operations on different variables respectively to try to find the saddle point. In an application scenario, the saddle point problem can also be solved based on some other random algorithms, but these random algorithms usually rely on the assumption of bounded gradient variance, and their convergence results are affected by the gradient variance. High gradient variance may lead to unstable optimization process and slow convergence speed. Common unequal step size dual time scale stochastic gradient methods and other random algorithms usually involve multiple hyperparameters (such as learning rate, step size), and their performance is extremely sensitive to these hyperparameters. The appropriate hyperparameter configuration often depends on the specific problem and data set, and requires a lot of experiments to adjust.

[0045] The non-convex saddle point problem is an important problem in game theory, robust optimization, and machine learning. Existing randomized algorithms rely on independent and identically distributed sampling replacement schemes, which cannot guarantee that each sample contributes to the model update and have low sample utilization. However, in practice, the sampling-without-replacement shuffle strategy is easier and faster to implement, such as the TensorFlow and PyTorch packages.

[0046] In order to solve at least one of the above-mentioned technical problems, in the solution of the present application, the initial model parameters corresponding to the model to be trained and a sample sequence consisting of multiple samples are obtained; at the beginning of each iteration cycle, the samples in the sample sequence are reordered to update the sample sequence; for the current iteration cycle of model training, the sample average gradient of all samples in the sample sequence under the model parameters at the beginning of the current iteration cycle is determined; the samples in the sample sequence are selected as target samples in turn, and for each of the target samples, the sample current gradient of the target sample under the current corresponding model parameters is determined, the sample current gradient is adjusted according to the sample average gradient to obtain the sample adjustment gradient corresponding to the target sample, and the model parameters are updated according to the sample adjustment gradient, wherein the gradient variance calculated based on the sample adjustment gradient is less than the gradient variance calculated based on the sample current gradient; if the samples in the sample sequence are traversed, the next iteration cycle is entered and the step of reordering the samples in the sample sequence to update the sample sequence at the beginning of each iteration cycle is returned to execute until the preset training termination condition is met and the model parameters when the training is completed are obtained.

[0047] Compared with the prior art, the present application does not directly update the model parameters based on the current gradient of the sample under the current corresponding model parameters, but first calculates the sample average gradient, and then adjusts the sample gradient based on the global sample average gradient to obtain the sample adjustment gradient, and then updates the model parameters according to the sample adjustment gradient. It is beneficial to reduce the variance of the gradient used for model parameter adjustment, thereby facilitating the convergence speed of model training, and further facilitating the training rate and training effect of the model. In addition, at the beginning of each iteration cycle, the samples in the sample sequence are reordered, which can avoid the dependence on the fixed order of the sample sequence during the model training process, and is beneficial to improving the training effect of the model.

[0048] Specifically, the model training method based on gradient variance reduction and data rearrangement provided in the embodiment of the present application can be regarded as a model training method of a variance-reduced stochastic gradient algorithm based on a random recombination scheme. The algorithm accurately converges to a stable point through a variance reduction (variance adjustment) step without assuming that the gradient variance is bounded, significantly reduces the variance of the gradient estimate, and accelerates the convergence of the algorithm.

[0049] Furthermore, a random shuffling step can be introduced. During the model training process, a sampling strategy without replacement is adopted. Data is shuffled at the beginning of each iteration cycle, and all samples are used without replacement in one iteration cycle. This can further reduce variance and optimize performance.

[0050] like Figure 1As shown, the embodiment of the present application provides a model training method based on gradient variance reduction and data rearrangement. Specifically, the above method includes the following steps:

[0051] Step S100, obtaining initial model parameters corresponding to the model to be trained and a sample sequence consisting of multiple samples.

[0052] The above-mentioned model to be trained is a model that needs to be trained, and the initial model parameters are the model parameters of the model before the training begins. Specifically, the specific form and function of the above-mentioned model to be trained are not specifically limited here, for example, it can be an image recognition model, a data prediction model, a text processing model, etc.

[0053] It should be noted that in the embodiment of the present application, the model training process of the above-mentioned model to be trained involves a non-convex saddle point problem. For example: .in, , is The optimization variables to be searched in the n-dimensional real vector space usually represent decision variables or model parameters. , In the constraint set Another optimization variable to be sought in The relative antagonistic variable or another parameter that affects the objective function. In the saddle point problem, we need to find a function that makes the function The largest . , is the objective function, which is about and function. Indicate to take Sub-function The average value of is the total number of data samples or the number of subproblems. is the sum of all sub-functions, each sub-function It may represent the The loss or cost of a data sample or sub-problem. , specific to each , which is a function defined for each individual data sample or subproblem.

[0054] Specifically, in the embodiment of the present application, the initial model parameters of the model to be trained are the initial point and the initial point ,in, is a set of predefined constraints. and These are the initial model parameters that need to be optimized through the model training process.

[0055] It should be noted that for the model training process corresponding to the model to be trained, other parameters required to be used in the training process are also pre-set, such as the step size parameter (or )and (or ), which is the learning rate parameter, is used to control and The update step size in gradient descent (or ascent). The total number of iterations (cycles) can also be pre-set , used to control the training iteration process.

[0056] The above sample sequence is composed of a plurality of pre-set samples arranged in order. It should be noted that the arrangement order of the samples in the initial sample sequence can be randomly generated, or set and adjusted according to the actual situation of sample collection, which is not specifically limited here. In an application scenario, the samples in the sample sequence can be divided into training samples, test samples and verification samples, or only the samples in the sample sequence can be used as training samples, and test samples and verification samples can be additionally set, which is not specifically limited here.

[0057] Step S200: at the beginning of each iteration cycle, samples in the sample sequence are reordered to update the sample sequence.

[0058] Among them, one iteration cycle is an epoch of model training. In one iteration cycle, all samples in the sample sequence corresponding to the iteration cycle are used in sequence for training, and then the next iteration cycle is entered. At the beginning of the next iteration cycle, a new sample sequence is rearranged.

[0059] In an application scenario, the reordering of samples in the sample sequence to update the sample sequence includes:

[0060] Randomly rearrange the sample index corresponding to each sample in the sample sequence to obtain a randomly rearranged index sequence;

[0061] According to the above sample sequence and the randomly rearranged index sequence, a sample sequence after the sample positions are reordered is generated.

[0062] In another application scenario, the reordering of samples in the sample sequence to update the sample sequence includes:

[0063] Obtaining an initial sequence, wherein the initial sequence is a sample sequence that has not undergone any reordering processing corresponding to any iteration cycle;

[0064] Randomly rearrange the sample index corresponding to each sample in the initial sequence to obtain an initial index sequence after random rearrangement;

[0065] According to the above initial sequence and the randomly rearranged initial index sequence, a sample sequence after the sample positions are reordered is generated.

[0066] It should be noted that the sample arrangement order corresponding to the sample sequences in any two iteration cycles is different.

[0067] As can be seen from the above, when reordering the samples in the sample sequence to obtain a new sample sequence, the reordering can be based on the sample sequence of the previous iteration cycle, or based on the initial sequence without any changes. In the embodiment of the present application, reordering based on the initial sequence can better avoid the occurrence of sample sequences with the same sample arrangement order during the reordering process, but it is not a specific limitation.

[0068] It should be further explained that, during the update of the sample sequence, the order of the samples can be adjusted directly, or the order of the sample calls can be adjusted by adjusting the order of the indexes in the index sequence corresponding to the samples. In the embodiment of the present application, the samples are reordered by adjusting the order of the indexes in the index sequence corresponding to the samples. Because each index in the index sequence is different, it is possible to better determine whether the order of the samples in the two generated sample sequences is the same, and the processing efficiency can be improved.

[0069] Specifically, in the embodiment of the present application, at the beginning of each iteration cycle, the samples are randomly reshuffled, that is, the sample indexes are randomly rearranged to generate a new index sequence For example, for each iteration cycle , first randomly generate a non-repeating index sequence It should be noted that the random rearrangement method is adopted in the embodiment of the present application. In actual use, other methods of updating the arrangement order can also be adopted, which are not specifically limited here.

[0070] Step S300, for the current iteration cycle of model training, determine the sample average gradient of all samples in the sample sequence under the model parameters at the beginning of the current iteration cycle.

[0071] Specifically, at the beginning of each iteration cycle, the sample average gradient of all samples under the model parameters at the beginning of the current iteration cycle is calculated. It should be noted that the calculation of the sample average gradient can be performed before the sample sequence is updated or after the sample sequence is updated, without specific limitation.

[0072] Specifically, the sample average gradient includes The average gradient and for The average gradient , calculated according to the following formulas (1) and (2):

[0073] (1);

[0074] (2);

[0075] in, represents the number of samples, Represents the sample index (sample , or the index of the index sequence corresponding to the sample sequence), and Represent the objective function about and These gradients are at the current iteration point Calculated, and for each sample calculate.

[0076] Step S400, sequentially selecting samples in the sample sequence as target samples, determining for each target sample a current sample gradient of the target sample under the currently corresponding model parameters, adjusting the current sample gradient according to the sample average gradient to obtain a sample adjustment gradient corresponding to the target sample, and updating the model parameters according to the sample adjustment gradient, wherein a gradient variance calculated based on the sample adjustment gradient is smaller than a gradient variance calculated based on the sample current gradient.

[0077] Specifically, according to the arrangement order of samples in the sample sequence, each sample is selected as the current target sample and processed accordingly. After the processing of one target sample is completed, the next sample is selected as the target sample and processed accordingly, and the process is repeated in sequence.

[0078] In the embodiment of the present application, the current gradient of the sample is adjusted according to the average gradient of the sample to obtain the sample adjustment gradient corresponding to the target sample, including:

[0079] Determine the sample initial gradient corresponding to the target sample under the model parameters at the beginning of the current iteration cycle;

[0080] According to the sample initial gradient and the sample average gradient, the sample current gradient is adjusted to obtain the sample adjustment gradient corresponding to the target sample.

[0081] Furthermore, adjusting the current gradient of the sample according to the initial gradient of the sample and the average gradient of the sample to obtain the sample adjustment gradient corresponding to the target sample includes:

[0082] Determine the target adjustment value according to the difference between the sample average gradient and the sample initial gradient;

[0083] The sum of the current gradient of the sample and the target adjustment value is used as the sample adjustment gradient corresponding to the target sample.

[0084] Specifically, update the sample order one by one according to the random re-ordering and (That is, update the corresponding model parameters). For each sample From 0 to , based on the following formula (3) and formula (4), the sample is calculated for Sample adjustment gradient and for Sample adjustment gradient :

[0085] (3);

[0086] (4);

[0087] in, Representative After random shuffling in the iteration cycle The index of and Represents at a given index The intermediate variables of the model parameters (i.e. the model parameters corresponding to the current target sample) are updated according to the order after shuffling on the corresponding samples. and Represents the current gradient of the sample (the gradient of the target sample under the current model parameters), and represents the initial gradient of the sample (the gradient corresponding to the target sample at the beginning of the current iteration cycle). It should be noted that The value of The target adjustment value will be The value of The target adjustment value.

[0088] In this way, based on the global average sample gradient in this iteration cycle and the gradient corresponding to the target sample at the beginning of the current iteration cycle, the sample current degree corresponding to the target sample is adjusted. These adjustments are based on the difference between the global gradient and the current sample gradient, which can reduce the gradient variance caused by a single sample.

[0089] After calculating the sample adjustment gradient corresponding to the target sample, the model parameters are updated based on the following formulas (5) and (6):

[0090] (5);

[0091] (6);

[0092] in, and It is the updated model parameter and also the model parameter corresponding to the next target sample. and is the model parameter corresponding to the current target sample, and is the preset step size parameter used to control the update step size.

[0093] Step S500, if the samples in the above sample sequence are traversed, enter the next iteration cycle and return to execute the above step of reordering the samples in the above sample sequence at the beginning of each iteration cycle to update the above sample sequence until the preset training termination condition is met and the model parameters when the training is completed are obtained.

[0094] Specifically, when the samples in the sample sequence complete one traversal, the current iteration cycle ends to enter the next iteration cycle.

[0095] Furthermore, the method further includes: for each iteration cycle, at the end of the iteration cycle, performing a single projection operation on the model parameters based on a preset constraint set.

[0096] Specifically, at the end of each iteration cycle, a single projection operation is performed based on the following formula (7) to ensure that the model parameters Still in the constraint set Inside:

[0097] (7);

[0098] in, Represents the model parameters after projection, and also serves as the model parameters at the beginning of the next iteration cycle. Represents the function corresponding to the preset projection operation, which is used to handle optimization problems with constraints. The specific function can be set and adjusted according to actual needs. Represents the end of the current iteration cycle. At the same time, As the next iteration cycle The model parameters.

[0099] The training termination condition can be that the actual number of iteration cycles reaches a preset value. , may also include other preset conditions, such as the loss value reaching a preset loss threshold, etc., which are not specifically limited here.

[0100] It should be noted that after training is completed, the model parameters at the time of training can be used as the optimized model parameters. and Output and store.

[0101] As can be seen from the above, in the model training method based on gradient variance reduction and data rearrangement provided by the embodiment of the present application, the model parameters are not updated directly based on the current gradient of the sample under the current corresponding model parameters, but the sample average gradient is first calculated, and the sample gradient is adjusted based on the global sample average gradient to obtain the sample adjustment gradient, and then the model parameters are updated according to the sample adjustment gradient. It is beneficial to reduce the variance of the gradient used for model parameter adjustment, so as to improve the convergence speed of model training, and then help improve the training rate and training effect of the model. In addition, at the beginning of each iteration cycle, the samples in the sample sequence are reordered, which can avoid the dependence on the fixed order of the sample sequence during the model training process, which is beneficial to improve the training effect of the model.

[0102] In the embodiment of the present application, the above-mentioned model training method based on gradient variance reduction and data rearrangement is also described based on a specific application scenario. Figure 2 is a specific flow chart of a model training method based on gradient variance reduction and data rearrangement provided in an embodiment of the present application, such as Figure 2 As shown in the figure, the parameters are initialized at the beginning. At the beginning of an iteration cycle, the sample sequence is randomly rearranged, and then the sample average gradient is calculated. The target samples are selected from the sample sequence in turn to calculate the sample current gradient, and the sample adjustment gradient is determined based on the sample current gradient and the sample average gradient. The model parameters are updated based on the sample adjustment gradient. After traversing all samples, the final model parameters are updated and a single projection operation is performed. If the current If the training is completed in one iteration cycle, the model will enter the next iteration cycle and continue training. Otherwise, the training is completed and the final model parameters are obtained.

[0103] In the embodiments of the present application, the model training method based on gradient variance reduction and data rearrangement is also verified and explained. In an application scenario, a data contamination countermeasure logistic regression experiment is conducted. Specifically, a data set containing 1000 samples is generated, where each sample is a vector of length 100, drawn from a standard normal distribution N(0, I), where N(0, I) represents a multidimensional standard normal distribution with a mean of 0 and a covariance matrix of the identity matrix I. At the same time, a small random disturbance term is added, which is a vector drawn from a normal distribution N(0, ) to simulate the data in the real world. The generated data set is randomly divided into a training set Dtrain and a test set Dtest. In the training set Dtrain, data contamination is artificially introduced, that is, a perturbation vector is added to the feature vectors of a small number of samples. The perturbation vector is considered to be added to the added samples to change the characteristics of these samples, making the data of these samples unrealistic or misleading.

[0104] The optimization goal is to minimize the loss of the model in the presence of data contamination, expressed as ,in , represents the perturbation vector added to the training data to simulate data pollution attacks. , represents the model variables to be optimized, Is the model contaminated by the given data The training loss in the case of . Represents the upper bound of the maximum perturbation value.

[0105] Specifically, the contamination ratio was set to 10%, which means that only a small part of the samples were contaminated. = 2 is used to control the regularization term in the loss function. 200 iterations were performed to optimize the loss function. For each algorithm, the learning rate and A grid search was performed in the range of {0.1, 0.01, 0.001}. The most suitable learning rate can be obtained through grid search.

[0106] Figure 3 and Figure 4 is a test result schematic diagram provided in an embodiment of the present application, wherein: Figure 3 represents the change of loss function during optimization. Figure 4 Represents the prediction accuracy on the test data set. Figure 3 It can be seen that the gradient value of the loss function generated by the method provided in the embodiment of the present application decreases rapidly, indicating the effectiveness of the data pollution attack training data model. Figure 4 It can be seen that the accuracy of the model after the data pollution attack on the test set data is very low, which verifies the superior attack performance of the method of this application.

[0107] In another application scenario, a distributed robust optimization experiment is also conducted to consider the non-convex regularized DRO problem in distributed robust learning. A real dataset a9a, which contains 123 features and 32561 samples, is used to simulate the actual scenario. The optimization objective function Evaluations are performed with varying numbers of gradient oracles to verify the convergence and efficiency of the algorithm. It usually represents the worst case of model performance under all possible distribution changes. This expression usually involves a maximization sub-problem to express the model performance under the most unfavorable conditions. The loss function It is usually used to evaluate the performance of a model on a single data point or a data set. It may be an aggregation or average of this worst-case loss over the entire dataset, which reflects the sensitivity and robustness of the model to data changes.

[0108] Figure 5 is a test result diagram provided in an embodiment of the present application. Specifically, Figure 5 As shown, in the distributed robust optimization problem, the gradient value of the objective function generated by the method of the present application converges quickly, ensuring the robustness of the optimization. This shows the superior performance of the solution of the present application in dealing with distributed robust optimization problems.

[0109] It can be seen that the solution of the present application has advantages in terms of convergence speed, effectiveness and optimization quality. Specifically, based on the model training method based on gradient variance reduction and data rearrangement provided in the embodiment of the present application, a new random variance reduction algorithm is adopted, which significantly improves the efficiency and accuracy of processing non-convex saddle point problems by integrating random reshuffling and random variance reduction gradient technology, effectively reduces the variance of gradient estimation, improves the convergence speed during model training, and reduces computational costs.

[0110] Specifically, unlike the commonly used uniform sampling with replacement, the embodiment of the present application ensures that each sample is used exactly once in each iteration cycle. Such a strategy helps to reduce cache misses, thereby speeding up the actual execution speed of the algorithm.

[0111] The variance of the gradient estimate in each iteration is reduced through the stochastic variance reduction gradient method, which allows the algorithm to converge efficiently even without assuming that the gradient is bounded.

[0112] At the same time, compared with other algorithms that require multiple projection operations, the embodiment of the present application only requires one projection operation in each iteration cycle, which simplifies the complexity of implementation. In addition, the algorithm uses a fixed dual time scale step, which is simpler to operate.

[0113] In this way, by randomly reshuffling the data index in each iteration cycle, all samples are ensured to be used fairly during the training process, reducing the overfitting of the model to a specific data order. At the same time, random processing of sample order helps the model to perform stably under different data arrangements, thereby enhancing the generalization ability of the model.

[0114] By calculating the global gradient at the beginning of each cycle and using this global gradient to adjust the gradient of each sample in subsequent updates, the variance of the gradient estimation caused by the randomness of the samples is significantly reduced, thereby accelerating the convergence of the algorithm. Although the global gradient of all samples needs to be calculated once at the beginning of each iteration cycle, the algorithm reduces the computational burden by reducing the computational requirements for updating a single sample as a whole, and is particularly suitable for processing large-scale data sets. It can solve the problems of large gradient variance, slow convergence speed and high computational cost in the model training process in the prior art, effectively reduce the variance and computational cost, and increase the convergence speed, so that the model approaches the ideal training results faster, improving the performance and robustness of the model. In addition, the convergence speed is not affected by the gradient variance, and can accurately converge to the stable point of the optimization problem, thereby improving the model training effect.

[0115] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0116] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A model training method based on gradient variance reduction and data rearrangement, characterized in that: The method comprises: Obtaining initial model parameters corresponding to a model to be trained and a sample sequence consisting of multiple samples; the model to be trained is an image recognition model or a text processing model; At the beginning of each iteration cycle, reordering the samples in the sample sequence to update the sample sequence; For a current iteration cycle of model training, determining a sample average gradient of all samples in the sample sequence under the model parameters at the beginning of the current iteration cycle; Selecting samples in the sample sequence as target samples in sequence, determining, for each target sample, a sample current gradient of the target sample under the current corresponding model parameters, adjusting the sample current gradient according to the sample average gradient to obtain a sample adjusted gradient corresponding to the target sample, and updating the model parameters according to the sample adjusted gradient, wherein a gradient variance calculated based on the sample adjusted gradient is less than a gradient variance calculated based on the sample current gradient; If the samples in the sample sequence are traversed, the next iteration cycle is entered and the step of reordering the samples in the sample sequence to update the sample sequence at the beginning of each iteration cycle is returned to be executed until the preset training termination condition is met and the model parameters when the training is completed are obtained; The reordering of samples in the sample sequence to update the sample sequence comprises: Acquire an initial sequence, wherein the initial sequence is a sample sequence that has not undergone any reordering process corresponding to any iteration cycle; Randomly rearrange the sample index corresponding to each sample in the initial sequence to obtain an initial index sequence after random rearrangement; Generating a sample sequence after reordering sample positions according to the initial sequence and the randomly rearranged initial index sequence; The step of adjusting the current gradient of the sample according to the sample average gradient to obtain the sample adjustment gradient corresponding to the target sample includes: Determine the sample initial gradient corresponding to the target sample under the model parameters at the beginning of the current iteration cycle; According to the sample initial gradient and the sample average gradient, the sample current gradient is adjusted to obtain a sample adjustment gradient corresponding to the target sample; The step of adjusting the current gradient of the sample according to the initial gradient of the sample and the average gradient of the sample to obtain the sample adjustment gradient corresponding to the target sample includes: Determining a target adjustment value according to a difference between the sample average gradient and the sample initial gradient; The sum of the current gradient of the sample and the target adjustment value is used as the sample adjustment gradient corresponding to the target sample.

2. The model training method based on gradient variance reduction and data rearrangement according to claim 1, characterized in that: The reordering of samples in the sample sequence to update the sample sequence comprises: Randomly rearrange the sample index corresponding to each sample in the sample sequence to obtain a randomly rearranged index sequence; A sample sequence after the sample positions are reordered is generated according to the sample sequence and the randomly rearranged index sequence.

3. The model training method based on gradient variance reduction and data rearrangement according to claim 1, characterized in that: The sample arrangement order corresponding to the sample sequence in any two iteration cycles is different.

4. The model training method based on gradient variance reduction and data rearrangement according to any one of claims 1 to 3, characterized in that: The method further comprises: For each iteration cycle, at the end of the iteration cycle, a single projection operation is performed on the model parameters based on a preset constraint set.

Images