Adaptive gradient optimization method and system based on statistical safety boundary

Abstract

This invention discloses an adaptive gradient optimization method and system based on statistical safety boundaries, belonging to the field of machine learning optimization technology. The method includes: acquiring the gradient norm during the warm-up phase, initializing an online quantile estimator and system boundary after tail reduction processing; dynamically updating the statistical reference upper bound at each step through the quantile estimator, fusing gradient direction consistency and amplitude anomaly detection to distinguish signal from noise; calculating the risk probability based on the relative increase of a dual-scale exponential moving average; calculating the dynamic cruise coefficient based on the risk probability and signal quality; and outputting the step size coefficient through a hysteresis-constrained three-modal state machine. This invention replaces a fixed threshold with statistical quantiles, achieves signal acceleration and noise deceleration through signal-noise separation, and eliminates overshoot through a damping ramp. In high-risk scenarios such as large language model training and reinforcement learning, it effectively prevents training collapse caused by gradient spikes while maintaining convergence efficiency.

Description

Technical Field

[0001] This invention relates to the fields of machine learning and artificial intelligence, specifically to an adaptive step-size optimization method, apparatus, and computer-readable storage medium based on gradient statistical quantiles. It can be widely applied to scenarios such as deep learning model training, large language model pre-training and fine-tuning, reinforcement learning strategy optimization, and financial risk model training. This method is also referred to as the "Morley bilateral cruise method" in some literature. Background Technology

[0002] Deep neural network training typically employs optimization methods based on stochastic gradient descent and its adaptive variants (such as Adam and AdamW). Existing adaptive optimizers primarily adjust the step size of each parameter based on the estimation of the first and second moments of the gradient. However, these methods have the following shortcomings: First, it is extremely sensitive to gradient spikes and noise. In scenarios such as large language model training and reinforcement learning, the gradient norm may exhibit sudden spikes, leading to excessive parameter updates, causing training crashes or irreversible increases in loss. Existing techniques, such as gradient clipping, use fixed thresholds or simple adaptive thresholds, lacking dynamic awareness of the statistical characteristics of gradients.

[0003] Second, it cannot distinguish between "deterministic steep slopes" and "random noise". Existing methods treat changes in gradient magnitude uniformly, resulting in the use of the same step size strategy in both steep regions of the objective function (requiring large step sizes) and regions with bursts of noise (requiring small step sizes), sacrificing convergence efficiency or safety.

[0004] Third, it relies on manually preset hyperparameters. Key parameters such as learning rate and pruning threshold require extensive debugging experience and lack zero-configuration startup capability, increasing the barrier to entry and the cost of parameter tuning.

[0005] Fourth, there is a lack of systematic measurement of historical risk. Existing methods have not established statistical references for normal gradient ranges, and cannot quantify the degree of anomalousness of current gradient behavior relative to historical statistics. Summary of the Invention

[0006] The present invention aims to solve the above-mentioned technical problems and provide an adaptive gradient optimization method and system based on statistical safety boundaries.

[0007] The core concept of this invention is as follows: maintaining a statistical reference upper bound for the gradient norm through an online quantile estimation algorithm, serving as a dynamic benchmark for the normal range; distinguishing between deterministic signals and random noise through a two-factor fusion of gradient direction consistency and amplitude anomalies; quantifying historical risk probability through the relative increase of a dual-scale exponential moving average; dynamically adjusting the step size coefficient through a risk-driven three-modal state machine; and achieving zero-configuration startup through automatic acquisition and tail reduction processing during the warm-up period.

[0008] In a first aspect, the present invention provides an adaptive gradient optimization method based on statistical safety boundaries, comprising the following steps: S1: Warm-up Phase. Collect the gradient norm of the preset number of steps, perform tailing processing, and calculate the preset quantiles as the initial values ​​of the system reference boundary; initialize the online quantile estimator with multiple quantiles of the tailed data; statistically calculate the amplitude anomaly threshold based on the warm-up period; initialize the risk index moving average variable.

[0009] S2: Online Update Phase. In each iteration, the system reference boundary is updated using an online quantile estimator; the cosine similarity between the current gradient and the smooth gradient direction is calculated, and the directional signal strength is obtained through nonlinear mapping; the amplitude safety factor is calculated based on the relationship between the current gradient norm and the amplitude anomaly threshold; and the minimum of the two is taken as the comprehensive signal strength factor.

[0010] S3: Risk Probability Calculation. Taking whether the current gradient exceeds the reference boundary as the event, maintain the exponential moving average of the exceedance event using both fast and slow decay factors; calculate the risk probability as the ratio of the difference between the two to the slow value, and then truncate it for protection.

[0011] S4: Dynamic cruise coefficient calculation. Based on the risk probability and the comprehensive signal strength factor, the historical risk suppression term and the prevention suppression term are calculated respectively. After smoothing, the maximum value is taken to obtain the total suppression strength. The cruise coefficient is calculated by dividing the baseline coefficient by one and adding the total suppression strength. The baseline coefficient is dynamically determined based on the current system reference boundary and trust domain parameters. Specifically, the baseline coefficient is equal to the trust domain parameter divided by the system reference boundary.

[0012] S5: Three-mode switching. Based on the ratio of the current gradient norm to the system reference boundary and the continuous over-threshold count, it switches between cruise mode, sprint mode and braking mode, and outputs the step size coefficient of the current step.

[0013] S6: Parameter Update. Update the model parameters by dividing the step size coefficient by the current iteration number.

[0014] Furthermore, in step S5: The cruise mode is in the default state, and the coefficient is the cruise coefficient mentioned above; The sprint mode is entered when the gradient norm exceeds the first threshold and braking is not triggered. The coefficient is increased to the preset maximum value and a safety clamp based on displacement constraints is built in. It exits when the number of steps in the sliding window exceeds a preset proportion and falls below the second threshold. The braking mode is activated when the gradient norm exceeds the third threshold for a consecutive preset number of steps and exceeds a significant multiple for each step. A damping ramp is used to reduce the coefficient to a preset minimum value. After the danger is cleared, the damping is restored to the recovery target locked when braking is activated. When the gradient is lower than the fourth threshold, the mode is switched to the recovery phase.

[0015] Furthermore, the online quantile estimator is a P² quantile estimator that maintains five marker points to achieve online quantile updates with constant storage and constant time.

[0016] Furthermore, the first threshold, the second threshold, the third threshold, and the fourth threshold satisfy hysteresis constraints: the third threshold is greater than the first threshold, the first threshold is greater than the fourth threshold, and the fourth threshold is greater than the second threshold.

[0017] Secondly, the present invention provides an adaptive gradient optimization device based on statistical safety boundaries, comprising: a preheating module, an online quantile estimation module, a signal-noise separation module, a risk probability calculation module, a cruise coefficient calculation module, a mode switching module, and a parameter update module.

[0018] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0019] The beneficial effects of this invention are as follows: By defining safety boundaries using statistical quantiles rather than fixed thresholds, gradient scales can be automatically adapted to different tasks and data distributions.

[0020] For the first time, gradient direction consistency is incorporated into risk measurement, effectively distinguishing between deterministic steep slopes and random noise, thereby accelerating signals and decelerating noise.

[0021] The dual-scale exponential moving average relative increase eliminates the prior dependence on the absolute overshoot rate, enabling true zero-configuration startup.

[0022] Three-mode hysteresis switching achieves a dynamic balance between safety and performance, while the damping ramp eliminates overshoot oscillations.

[0023] The computational cost per step is comparable to that of mainstream optimizers, it does not rely on second-order information, and it can be seamlessly embedded into existing training frameworks. Attached Figure Description

[0024] Figure 1 This is the overall flowchart of the method of the present invention.

[0025] Figure 2 This is a detailed flowchart of the preheating phase.

[0026] Figure 3 This is a schematic diagram of online quantile estimation and signal-noise separation.

[0027] Figure 4 This is a schematic diagram for calculating risk probability and generating dynamic cruise coefficients.

[0028] Figure 5 This is the state transition diagram for a three-modal state machine.

[0029] Figure 6 This is a schematic diagram of the damping ramp curve in braking mode. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0031] Reference Figure 1 The method in this embodiment includes the following steps: Step 101: Warm-up Phase. Before training, perform parameter updates using a conservative step size and collect the gradient norm at each step.

[0032] Step 102: After the warm-up period, the collected gradient norm sequence is shortened: take the first percentile and the second percentile (1% and 99% are recommended), replace the values ​​below the first percentile with the first percentile, and replace the values ​​above the second percentile with the second percentile.

[0033] Take the preset high quantile (recommended 99th percentile) of the shrunken data as the initial value of the system reference boundary; take multiple quantiles of the shrunken data (recommended 1st, 25th, 50th, 75th, and 99th percentiles) as the initial markers of the online quantile estimator; set the amplitude outlier threshold to a preset multiple (recommended 3.0 times) multiplied by the initial value of the system reference boundary; calculate the overshoot rate during the warm-up period, and initialize the fast exponential moving average variable and the slow exponential moving average variable.

[0034] Step 103: Enter the online update phase. Each iteration executes the following sub-steps: Update the system reference boundary using an online quantile estimator; The directional signal strength is obtained by calculating the gradient direction consistency. Calculate the amplitude safety factor; The minimum value of the directional signal strength and the amplitude safety factor is taken as the comprehensive signal strength factor; Update the dual-scale exponential moving average and calculate the risk probability; Calculate the dynamic cruise coefficient; Execute the three-modal state machine and output the step size coefficient; Update the parameters by dividing the step size coefficient by the current iteration number.

[0035] This embodiment details the method for calculating the directional signal strength in step 103.

[0036] Maintaining an exponential moving average in the gradient direction: The smoothed gradient equals the first smoothing coefficient multiplied by the previous smoothed gradient plus one minus the first smoothing coefficient multiplied by the current gradient.

[0037] Calculate the cosine similarity between the current gradient and the smooth gradient: the cosine value is equal to the inner product of the two gradients divided by the product of their norms plus a very small constant.

[0038] The directional signal strength is obtained through nonlinear mapping: the directional signal strength is equal to 1 divided by 1 plus an exponential function, where the parameter of the exponential function is the difference between the negative steepness coefficient multiplied by the cosine value and the preset threshold.

[0039] The recommended values ​​for the first smoothness coefficient are 0.9, the recommended values ​​for the preset threshold are 0.8, and the recommended values ​​for the steepness coefficient are 10.

[0040] The amplitude safety factor is calculated as follows: the amplitude safety factor is equal to the minimum value between 1 and the ratio of the current gradient norm to the amplitude anomaly threshold.

[0041] When the gradient norm exceeds the amplitude anomaly threshold, the amplitude safety factor becomes zero, triggering maximum suppression.

[0042] The event is defined as whether the value exceeds the preset reference coefficient (recommended value is 0.8) multiplied by the system reference boundary.

[0043] The fast exponential moving average equals the fast decay factor multiplied by the fast exponential moving average plus one minus the fast decay factor multiplied by the event indicator value; the slow exponential moving average equals the slow decay factor multiplied by the slow exponential moving average plus one minus the slow decay factor multiplied by the event indicator value.

[0044] The risk probability equals the difference between the fast value and the slow value, divided by the maximum value of the slow value and the lower limit of the denominator, and truncated to between zero and ten.

[0045] The recommended values ​​for the fast attenuation factor are 0.9, the slow attenuation factor are 0.99, and the lower limit of the denominator protection is 0.01.

[0046] Cruise mode (default): The step size factor is equal to the dynamic cruise factor.

[0047] Sprint mode trigger conditions: The gradient norm exceeds the first threshold multiplied by the system reference boundary (the first threshold is recommended to be 1.5) and braking is not triggered. After entering, the step size coefficient is increased to the preset maximum value; when the gradient norm is detected to exceed the safety clamping threshold multiplied by the system reference boundary (recommended to be 2.0), safety clamping is triggered: the step size coefficient is the minimum value of the preset maximum value, the cruise coefficient, the safety clamping threshold multiplied by the system reference boundary divided by the gradient norm. Exit conditions: The number of steps in the sliding window (recommended window size is 5 steps) exceeding the preset proportion (recommended to be 80%) satisfies the gradient norm being less than the second threshold multiplied by the system reference boundary (the second threshold is recommended to be 0.8), or the maximum number of sprint steps is reached (recommended to be 200).

[0048] Braking mode trigger conditions: A preset number of consecutive steps (3 steps recommended) must satisfy a gradient norm greater than a significant multiple multiplied by a third threshold multiplied by the system reference boundary (the third threshold is recommended to be 2.5, and the significant multiple is recommended to be 1.1). Upon entry, a damped ramp is used: the velocity term equals the damping coefficient multiplied by the velocity term plus or minus the damping coefficient multiplied by the difference between the target value and the current value, divided by the time constant; the step size coefficient is taken as the maximum value of the preset minimum value and the current value acceleration term (downward phase) or the minimum value of the recovery target value and the current value acceleration term (upward phase).

[0049] Exit condition: After the gradient falls below the fourth threshold multiplied by the system reference boundary (the fourth threshold is recommended to be 1.0), the uplink recovery phase begins, and cruise resumes after the target is locked.

[0050] As a further preferred implementation, in order to solve the problem of false suppression in the flat region during the later stage of training, a gradient vanishing protection sub-step can be added in step S3.

[0051] The specific steps are as follows: Step S801: Determine whether the current gradient norm is less than the gradient vanishing threshold multiplied by the system reference boundary, where the gradient vanishing threshold is recommended to be 0.01.

[0052] Step S802: If step S801 is satisfied, further determine whether the direction signal strength is greater than the direction safety threshold, where the recommended direction safety threshold is 0.7.

[0053] Step S803: If steps S801 and S802 are satisfied at the same time, and the amplitude anomaly is not triggered at present (i.e. the amplitude safety factor is greater than zero), then the comprehensive signal strength factor is forcibly set to 1.0.

[0054] Step S804: If any condition is not met, the original comprehensive signal strength factor remains unchanged.

[0055] Technical Effects: When the gradient is indeed extremely small and its direction is well consistent, the system cancels the suppression, allowing progress at a normal speed. When the gradient is small but its direction is scattered, the system maintains the suppression to avoid invalid updates. This mechanism effectively solves the problem of false positives and false suppression in flat regions, improving convergence stability.

[0056] Safety Note: Gradient vanishing protection has a lower priority than amplitude anomaly detection. Even if steps S801 and S802 are met, if the gradient norm is greater than or equal to the amplitude anomaly threshold (triggering amplitude anomaly), the system still maintains the amplitude safety factor at zero, thus the overall signal strength factor is zero, triggering suppression. This ensures that the safety priority principle is not violated.

[0057] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. An adaptive gradient optimization method based on statistical safety boundaries, characterized in that, Includes the following steps: S1: During the warm-up phase, the gradient norm of the preset number of steps is collected, and the collected values ​​are shrunken before the preset quantiles are calculated as the initial values ​​of the system reference boundary; the online quantile estimator is initialized with multiple quantiles of the shrunken data; the amplitude anomaly threshold is statistically calculated based on the warm-up period; and the risk index moving average variable is initialized. S2: In each iteration, the system reference boundary is updated by the online quantile estimator; the cosine similarity between the current gradient and the smooth gradient direction is calculated, and the direction signal strength is obtained by nonlinear mapping; the amplitude safety factor is calculated based on the relationship between the current gradient norm and the amplitude anomaly threshold. The minimum of the two values ​​is taken as the comprehensive signal strength factor; S3: The event is whether the current gradient exceeds the preset reference coefficient multiplied by the system reference boundary. The exponential moving average of the overshoot event is maintained by the fast decay factor and the slow decay factor respectively. The risk probability is calculated by dividing the difference between the fast and slow values ​​by the maximum value of the slow value and the lower limit of the protection in the denominator, and then truncating the result. S4: Calculate the historical risk suppression term and the prevention suppression term based on the risk probability and the comprehensive signal strength factor, respectively, and take the maximum value after exponential smoothing to obtain the total suppression strength; The cruise coefficient is calculated by dividing the baseline coefficient by one and adding the total suppression intensity. The baseline coefficient is dynamically determined based on the current system reference boundary and trust domain parameters. Specifically, the baseline coefficient is equal to the trust domain parameters divided by the system reference boundary. S5: Based on the ratio of the current gradient norm to the system reference boundary and the continuous over-threshold count, switch between cruise mode, sprint mode and braking mode, and output the step size coefficient of the current step. S6: Update the model parameters by dividing the step size coefficient by the current iteration number.

2. The method according to claim 1, characterized in that, In step S5: The cruise mode is the default state, and the coefficient is the cruise coefficient. The sprint mode is entered when the gradient norm exceeds the first threshold multiplied by the system reference boundary and braking is not triggered, and the coefficient is increased to the preset maximum value. When the number of steps in the sliding window exceeds the preset proportion and is lower than the second threshold multiplied by the system reference boundary, it exits. In the sprint mode, if the gradient exceeds the safety clamping threshold multiplied by the system reference boundary, a safety clamp based on displacement constraints is triggered. The braking mode is entered when the gradient norm of consecutive preset steps exceeds the third threshold multiplied by the system reference boundary and each step exceeds the preset significant multiple. A damping ramp is used to reduce the coefficient to a preset minimum value. After the danger is cleared, the damping is restored to the recovery target locked when braking is entered. When the gradient is lower than the fourth threshold multiplied by the system reference boundary, the mode enters the recovery phase.

3. The method according to claim 2, characterized in that, The first threshold, the second threshold, the third threshold, and the fourth threshold satisfy the hysteresis constraint: the third threshold is greater than the first threshold, the first threshold is greater than the fourth threshold, and the fourth threshold is greater than the second threshold.

4. The method according to claim 1, characterized in that, The online quantile estimator is a P² quantile estimator, which uses the 1st, 25th, 50th, 75th, and 99th percentiles of the shortened data as initial markers.

5. The method according to claim 1, characterized in that, The tail reduction process in step S1 is as follows: take the first percentile and the second percentile of the gradient norm during the preheating period, replace the values ​​below the first percentile with the first percentile, and replace the values ​​above the second percentile with the second percentile.

6. The method according to claim 1, characterized in that, The calculation of the directional signal strength in step S2 includes: The smoothed gradient is equal to the first smoothing coefficient multiplied by the previous smoothed gradient plus one minus the first smoothing coefficient multiplied by the current gradient; The cosine value is equal to the inner product of the current gradient and the smooth gradient divided by the product of their norms plus a very small constant. The directional signal strength is equal to 1 divided by 1 plus an exponential function, where the parameter of the exponential function is the difference between the negative steepness coefficient multiplied by the cosine value and the preset threshold.

7. The method according to claim 1, characterized in that, The amplitude safety factor in step S2 is calculated as follows: The amplitude safety factor is equal to the minimum value between 1 and the ratio of the current gradient norm to the amplitude anomaly threshold.

8. The method according to claim 1, characterized in that, Step S3 further includes a gradient vanishing protection sub-step: when the current gradient norm is less than the gradient vanishing judgment threshold multiplied by the system reference boundary, and the directional signal strength is greater than the directional safety threshold, and no amplitude anomaly is triggered, the comprehensive signal strength factor is forcibly set to 1.

0.

9. An adaptive gradient optimization device based on statistical safety boundaries, characterized in that, include: The preheating module is used to acquire the gradient norm and initialize system parameters during the preheating phase; An online quantile estimation module is used to maintain the statistical quantiles of the gradient norm; The signal-noise separation module is used to calculate and fuse the directional signal strength and amplitude safety factor. The risk probability calculation module is used to calculate the risk probability based on a dual-scale exponential moving average. The cruise coefficient calculation module is used to calculate the dynamic cruise coefficient based on the risk suppression item and the prevention suppression item; The mode switching module is used to execute the three-modal state machine as described in claim 2; The parameter update module is used to update the model parameters by dividing the step size coefficient by the current iteration number.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

Images