Gradient-free optimization enhancement plug-in algorithm based on bandwidth adaptive kernel density estimation

Abstract

The invention discloses a non-gradient optimization enhancement plug-in algorithm based on bandwidth self-adaption, relates to the field of computer science and technology, and is used for enhancing the optimization performance of a traditional optimization method based on gradient information in a complex non-convex task, multi-modal data and high-dimensional parameter space scene. According to the plug-in, a gradient-free optimization strategy is introduced, a heuristic bandwidth adjustment mechanism is designed, the sampling direction and amplitude of kernel density estimation are accurately guided by utilizing a memory mechanism and based on a prior gradient norm and a parameter variance in a memory space, self-adaptive sampling of a traditional optimizer in the parameter space is achieved, and the global search capability of a model is greatly improved. Meanwhile, a loop learning rate scheduling method is adopted, learning rates of a traditional optimizer in different optimization stages are adjusted in a self-adaptive mode, and optimization efficiency and stability are further improved. Reference performance tests and ablation tests of various typical application scenarios prove that the method is suitable for enhancing the optimization performance of a traditional optimizer based on gradient information in non-weak gradient optimization scenarios, the dependency on parameters is low, and an innovative and effective solution is provided for complex optimization problems.

Description

Technical Field

[0001] The present invention relates to the field of computer science and technology, and in particular to a gradient-free optimization enhancement plug-in algorithm based on bandwidth-adaptive kernel density estimation. Background Art

[0002] In the booming development of modern machine learning and deep learning, optimization algorithms, as the core driving factor, play a vital role in model training and performance improvement. The core goal of the optimization algorithm is to search for a parameter combination in the parameter space that can achieve the optimal value of the objective function, so that the model can show excellent performance in a given task. Traditional gradient optimization methods, such as stochastic gradient descent, stochastic gradient descent with momentum, and adaptive moment estimation, have achieved remarkable results in early machine learning and deep learning tasks due to their simplicity and efficiency. In simple convex optimization problems, these methods can accurately guide the direction of parameter updates with gradient information, quickly and robustly converge to the global optimal solution, and perform well in the training of basic models such as linear regression and logistic regression.

[0003] However, as artificial intelligence application scenarios become increasingly complex and diversified, traditional gradient optimization methods have gradually exposed a series of insurmountable performance bottlenecks when dealing with complex non-convex tasks, multimodal data, and high-dimensional parameter spaces.

[0004] In complex non-convex tasks, the objective function has an extremely complex shape, replete with numerous local optima, saddle points, and flat regions. For example, in deep neural network training, the highly nonlinear structure of deep neural networks leads to highly non-convex loss functions. Traditional gradient optimization methods are prone to becoming trapped in local optima within this complex functional landscape. Because their update strategies rely excessively on the gradient information at the current position, once they enter a local optimum, the gradient approaches zero, causing the algorithm to mistakenly believe it has found the global optimum and cease effective parameter updates, making it difficult to further improve model performance. Furthermore, the presence of saddle points poses significant challenges to traditional gradient optimization methods. While the gradient at a saddle point is also zero, it is not a true optimal solution. Algorithms tend to become stagnant near saddle points, making it difficult to explore more optimal parameter space.

[0005] The processing of multimodal data also poses a severe test to traditional gradient optimization methods. Multimodal data usually contains data from different sources with different feature distributions. For example, in multimedia information processing, data from multiple modalities such as images, audio, and text may be integrated at the same time. The distribution of these data is multimodal and highly complex. Traditional gradient optimization methods, due to their update strategy based on a single gradient information, find it difficult to fully capture the complex distribution characteristics of multimodal data. When processing such data, the algorithm may over-focus on the data characteristics of a certain modality and ignore important information from other modalities, resulting in the model being unable to fully utilize the advantages of multimodal data, which in turn affects the model's generalization ability and performance.

[0006] High-dimensional parameter space presents another major challenge for traditional gradient optimization methods. As model size continues to expand and the number of parameters increases dramatically, the dimensionality of the parameter space grows exponentially. In high-dimensional parameter spaces, traditional gradient optimization methods encounter the curse of dimensionality. On the one hand, the computational cost of the gradient increases exponentially with the increase in dimensionality, significantly extending the algorithm's training time and consuming enormous amounts of computing resources. On the other hand, the direction of the gradient in high-dimensional space becomes increasingly difficult to accurately estimate. Even small noise or perturbations can cause deviations in the gradient direction, affecting the accuracy of parameter updates. Furthermore, the vanishing and exploding gradient problems are more common in high-dimensional parameter spaces. When a neural network has many layers, the gradient undergoes multiplication operations across multiple layers during backpropagation, making it easy for the gradient value to gradually decrease (vanishing gradient) or increase dramatically (exploding gradient). This can lead to unstable model training and difficulty converging to a desired result.

[0007] In critical application areas such as autonomous driving, medical diagnosis, and financial risk prediction, which require extremely high model performance and stability, these limitations of traditional gradient optimization methods severely restrict performance improvements and application expansion. Therefore, there is an urgent need for a new optimization strategy or method that can effectively overcome the performance bottlenecks of traditional gradient optimization methods in complex non-convex tasks, multimodal data, and high-dimensional parameter spaces, and provide solid support for the efficient training and performance improvement of modern machine learning and deep learning models. Summary of the Invention

[0008] The present invention aims to, for example, provide a gradient-free optimization enhancement plug-in algorithm based on bandwidth-adaptive kernel density estimation, which is adaptable to parameter optimization tasks in high-dimensional, multimodal scenarios, and scenarios with no or weak gradients. A kernel density estimation algorithm and a cyclic learning rate adjustment algorithm based on a heuristic adaptive bandwidth adjustment mechanism, in a plug-in-assisted manner, help traditional basic optimizers overcome the difficulty of achieving the desired results in extreme application scenarios such as no or weak gradients.

[0009] The embodiments of the present invention can be implemented as follows:

[0010] An embodiment of the present invention provides a gradient-free optimization plug-in algorithm based on bandwidth-adaptive kernel density estimation, comprising the following steps:

[0011] Step S1: Use the kernel density estimation method (KDE, KernelDensityEstimation) to estimate the current parameters θ = [θ1, θ2, ..., θ n ] distribution, and on this basis, sample new parameters θ′. The calculation formula of the kernel density estimation method is as follows:

[0012]

[0013] Among them, K(·) is the kernel function, represents the bandwidth on dimension d at time step t, n represents the number of parameters distributed in the current sampling space, θ t represents the historical parameter vector at the t-th time step in the sampling space, represents θ t The i-th parameter sample in . In the kernel density estimation method, θ t The resolution of the density function is affected by the fixed bandwidth parameter value. When the bandwidth is large, the kernel density estimation method will extend the influence of the sample points in the adjacent area to a larger range. The deviation can be approximately expressed as:

[0014]

[0015] Taking Gaussian kernel function driving as an example, let It can be expanded to:

[0016]

[0017] The deviation is proportional to the square of the bandwidth, that is, the larger the bandwidth, the greater the deviation, and the more blurred the estimated density function. Conversely, the clearer the density function, which means it is closer to the true density function. Considering the impact of the variance of the kernel density estimation method, it can be approximated as:

[0018]

[0019] The adjustment direction and amplitude of the bandwidth are determined by the parameter variance and the change of the prior gradient norm in the current data space calculated through the above steps.

[0020] Step S2: Using the information changes of the parameter space before and after the intervention of the kernel density estimation method, a heuristic bandwidth adjustment mechanism is designed, and the input parameters are defined as bandwidth λ t , parameter variance The prior gradient norm is The basic formula of the heuristic bandwidth adjustment mechanism is obtained as follows:

[0021]

[0022] Where ψ(·) is the bandwidth adjustment function, which is in the form of:

[0023]

[0024] Where γ is the sensitivity threshold for bandwidth adjustment. The numbers in the formula may cause considerable confusion to the reader, namely 0.9, 0.001, 1.1 and 1.0. In fact, 0.9 and 1.1 are constant factors used to control λ t When the scaling When λ t will be reduced, otherwise it will be increased, and 0.01 and 1.0 are used to control λ t The new parameter θ′ is obtained by sampling from the parameter distribution estimated by the kernel density estimation method using the heuristic bandwidth adjustment mechanism, and its calculation formula is as follows:

[0025]

[0026] Among them, τ is the introduced random noise, is the noise intensity adjusted according to the heuristic strategy to avoid θ′ being too close to θ.

[0027] Step S3: Introduce the memory mechanism and store the historical sampling parameters θ mem Stored in memory space, by calculating θ′ and θ mem The Euclidean distance of is used to characterize the degree of closeness, and a distance threshold δ is set in advance to control the search range of the algorithm in the parameter space. The specific formula is as follows:

[0028]

[0029] Step S4: Independent of the adaptive bandwidth kernel density estimation plug-in of the gradient-free optimization enhancement plug-in, an adaptive cyclic learning rate adjustment (CLR) module is designed to dynamically adjust the learning rate so that the traditional optimizer produces different computational behaviors at different stages of parameter space exploration. The gradient-free optimization enhancement plug-in calculates the adjusted learning rate α according to the selected cycle mode. t , and update the parameter θ t , the learning rate update pattern is as follows.

[0030]

[0031] in represents the periodic pattern, αbase represents the basic learning rate, α max Indicates the maximum learning rate. We initially introduced two modes in the algorithm, namely trigonometric function mode and cosine function mode. The trigonometric function mode is a simple linear learning rate update mode, and the learning rate α base and α max Oscillates back and forth between The numerical change trend of the learning rate in each cycle is divided into two stages: a linear increase stage and a linear decrease stage, where x is the relative position calculated based on the current iteration number t and step size T. The specific formula is as follows.

[0032]

[0033] When x=0, the learning rate is the maximum value α max ; When x = 1, the learning rate is the minimum value α base , that is, in the interval x∈[0,1], the learning rate is α max and α base There is a linear change between .

[0034]

[0035] in A variable indicating the current cycle number. c In this cycle, the amplitude of the learning rate is reduced to half of the original value, thereby preventing overfitting in the post-optimization stage and achieving more stable learning. Another mode we provide is the cosine function mode, in which the learning rate gradually decreases from the maximum value to the minimum value within a cosine cycle. This mode is usually used for "annealing" of the learning rate. In this mode, the cycle pattern expression is

[0036] Step S5: Using the classic algorithm Adam as the enhanced optimizer, perform online learning framework analysis on the regret performance of the gradient-free optimization enhancement plug-in in minimizing a series of non-convex, discontinuous, and non-differentiable functions, that is, online prediction f t (θ t ) and the best fixed parameter f from the feasible set χ t (θ * ) in all previous time steps. At each time step t, our goal is to update the parameter θ t And apply it to the objective function f t Evaluate. Given the non-convexity, possible discontinuity and non-differentiability of the function, the minimum regret is defined as:

[0037]

[0038] in, denotes the optimal fixed parameter at all time steps. We directly derive and prove the regret upper bound of the gradient-free optimization enhancement plugin for non-convex, discontinuous, and non-differentiable functions.

[0039] At each time step t, the parameters θ are updated as follows:

[0040]

[0041] Among them, α t is the learning rate at time step t. Under the adaptive periodic learning rate adjustment mechanism, α t It is dynamically adjusted. The specific adjustment mechanism has been fully described in Section 3; is the bias-corrected moving average of the first-order momentum; It is second order

[0042] Bias-corrected moving average of momentum; Z t It is the error term introduced by the kernel density estimation method sampling, and the regret R(T) at time step t can be recursively deduced as:

[0043] R(t+1)=R(t)+f t+1 (θ t+1 )-f t+1 (θ * )

[0044] f t+1 (θ t+1 ) performs Taylor expansion:

[0045]

[0046] where ξ is a value between θ t and θ t+1 After substituting it into the parameter update formula, we have:

[0047]

[0048] in, represents the gradient descent term, represents the sampling error term of the kernel density estimation method, represents the second-order error term.

[0049] The online regret derivation for the objective function above demonstrates that accumulated regret is essentially the lateral superposition of the gradient descent term, the kernel density estimation sampling error term, and the second-order error term. The gradient descent term is the accumulated regret generated by the gradient-free optimization enhancement plug-in algorithm when rapidly optimizing based on gradient information, leveraging its Adam properties, when faced with a locally convex optimization problem. The kernel density estimation sampling error term is the accumulated regret generated when smoothing the probability density estimation of the parameter space, non-differentiable points, and discontinuities, when faced with globally non-convex, non-differentiable, and discontinuous objective functions.

[0050] The reason for focusing on the second-order error is that in the case of non-convex functions, the Hessian matrix may have negative eigenvalues, which will lead to instability in the introduction of the second-order term. The gradient-free optimization enhancement plug-in algorithm adaptively adjusts the learning rate and bandwidth selection. When faced with such second-order terms, it can better control the update step size, reduce the oscillation caused by negative eigenvalues, and enhance stability in non-convex optimization.

[0051] Step S6: By adding the first-order momentum term and the second-order momentum term Expanding it and bringing it into the gradient descent term to deduce its boundedness, we get:

[0052]

[0053]

[0054] Where β1 and β2 are the decay coefficients of the first-order momentum and the second-order momentum, respectively, which are usually very close to 1; ε is a very small constant to prevent division by zero errors. Considering that the gradient is bounded in the local convex function part, that is,

[0055] Then we have:

[0056]

[0057] Without considering the divergent changes in the learning rate, the gradient descent term can be bounded by:

[0058]

[0059] Considering that β1 is usually close to 1, and Gradually tends to a finite value, so the upper bound of this term is stable and will eventually tend to a finite value. Further derivation The boundedness of the gradient descent term under the influence of different functions on the learning rate parameter. Whether it is the trigonometric function mode or the cosine function mode, It is bounded, and combined with the necessity of reducing the learning rate due to parameter memory, the learning rate will gradually decay in the final stage of optimization. Taking the cosine function mode as an example, the cumulative upper bound of the gradient descent term is:

[0060]

[0061] Based on the AM-GM inequality: for positive numbers a1, a2, …, α n ,have Therefore, the weighted sum of the momentum terms in the gradient descent term can be calculated using This approximation is based on the assumption that β1 is close to 1, so The learning rate term is divided into a constant part and a cosine function part that changes with the number of steps t. base Considered as the main contribution term, the final gradient descent term can be upper bounded as:

[0062]

[0063] It is proved that the gradient descent term of regret accumulation in the gradient-free optimization enhancement plug-in algorithm for optimizing non-convex, discontinuous and non-differentiable functions is stable and will eventually tend to a finite value.

[0064] Step S7: Based on the kernel density estimation method defined in step S3. For any new sample point θ t+1 , the error term of the kernel density estimation method can be calculated as follows:

[0065]

[0066] Among them, f KDE (θ t ) is the sampling distribution of the kernel density estimation method at time t, f real (θ t ) is the true distribution. This part of the error is controlled by the combination of bias and variance. The bias term and error term are respectively expressed as the first and second terms in the following equation:

[0067]

[0068] Based on Jensen’s inequality: for a convex function φ and a random variable X, E[φ(X)]≥φ[E(X)]. Considering that the gradient of the kernel density estimation method in any sampling space (local convex function) is bounded, that is, You can get:

[0069]

[0070] in, Represents the gradient component in the i-th dimension, Z t,iis the sampling error term of the kernel density estimation method in the corresponding dimension. Substituting into the expression of the kernel density estimation method error, we get:

[0071]

[0072] Therefore, it can be proved that the regret accumulation of the sampling error term of the kernel density estimation method is bounded and tends to a finite value with t.

[0073] Step S8: Based on the formulas in steps S6 and S7, the second-order error term is substituted and Taylor expanded to obtain:

[0074]

[0075] The contribution of the second-order error term can be further upper bounded as:

[0076]

[0077] Among them, L is the maximum eigenvalue of the Hessian matrix of the objective function, which represents the upper bound of the complexity of the objective function and also represents the local curvature of the objective function. Further sorting can be obtained as the upper bound is Has boundedness.

[0078] Ignoring λ t In the special case of , under the conditions we described, the final upper bound of the regret accumulation is:

[0079]

[0080] If the special case of dynamic bandwidth adjustment is considered, the dynamic bandwidth parameters are substituted and the deviation and variance of the dynamic bandwidth parameters are introduced. These two items are constrained and bounded.

[0081] It is proved that under non-convex, discontinuous and non-differentiable objective functions, an effective regret upper bound R(T) can be achieved by introducing kernel density estimation method and dynamic learning rate strategy through detailed error analysis. The convergence of the gradient-free optimization enhancement plug-in algorithm is demonstrated, and the influence of various parameters on the final convergence is revealed.

[0082] The beneficial effects of the embodiments of the present application include, for example:

[0083] In summary, the gradient-free optimization enhancement plug-in algorithm based on adaptive bandwidth kernel density estimation provided in this embodiment is used to enhance the optimization performance of traditional gradient information-based optimization methods in complex non-convex tasks, multimodal data and high-dimensional parameter space scenarios. The plug-in algorithm introduces a gradient-free optimization strategy, designs a heuristic bandwidth adjustment mechanism, and uses a memory mechanism based on the prior gradient norm and parameter variance in the memory space to accurately guide the sampling direction and amplitude of the kernel density estimation, thereby realizing adaptive sampling of the traditional optimizer in the parameter space and greatly increasing the global search capability of the model. At the same time, a cyclic learning rate scheduling method is adopted to adaptively adjust the learning rate of the traditional optimizer in different optimization stages, further improving the optimization efficiency and stability. Benchmark performance tests and ablation tests in a variety of typical application scenarios have proved that the invention is suitable for enhancing the optimization performance of traditional optimizers based on gradient information in no or weak gradient optimization scenarios, and has low dependence on parameters, providing an innovative and effective solution for complex optimization problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0085] Figure 1a Graph showing the Rosenbrock benchmark function in an embodiment of the present invention;

[0086] Figure 1b Ackley benchmark test function diagram in an embodiment of the present invention;

[0087] Figure 2 This is a flowchart of an embodiment of the present invention;

[0088] Figure 3 This is a conceptual flow chart of parameter kernel density estimation based on adaptive bandwidth adjustment according to an embodiment of the present invention;

[0089] Figure 4 A conceptual diagram of a cyclic learning rate adjustment mechanism according to an embodiment of the present invention;

[0090] Figure 5 This is the pseudo code of the algorithm for the gradient-free optimization enhancement plug-in based on adaptive bandwidth kernel density estimation in an embodiment of the present invention;

[0091] Figure 6 The optimization result of the Rosenbrock benchmark function in the embodiment of the present invention is shown in FIG.

[0092] Figure 7Optimization results of the Ackley benchmark test function in the embodiments of the present application. DETAILED DESCRIPTION

[0093] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0094] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0095] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0096] In the description of the present application, it should be noted that if the terms such as “upper”, “lower”, “inner”, “outer” and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship when the product of the present application is usually placed, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, therefore, it cannot be understood as a limitation on the present application.

[0097] In addition, if the terms “first”, “second” and the like appear, they are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0098] It should be noted that the features in the embodiments of the present application can be combined with each other without conflict.

[0099] Please refer to Figure 1a-Figure 7 , the present embodiment provides a non-gradient optimization enhancement plug-in algorithm based on adaptive bandwidth kernel density estimation, the present embodiment is a Rosenbrock and Ackley benchmark test function, Figure 1a and Figure 1b respectively shown are the Rosenbrock and Ackley benchmark test function graphs and the best parameter position in the function space is labeled.

[0100] Figure 2The flowchart of the present invention is a gradient-free optimization enhancement plug-in based on bandwidth adaptive kernel density estimation, wherein the parameter kernel density estimation concept flow of adaptive bandwidth adjustment and the cyclic learning rate adjustment mechanism are more vividly shown as follows: Figure 3 and Figure 4 The algorithm specifically includes the following steps:

[0101] Step S1: Use the kernel density estimation (KDE) method to estimate the current parameter θ = [θ1, θ2, ..., θ n ] distribution, and on this basis, sample new parameters θ′. The calculation formula of the kernel density estimation method is as follows:

[0102]

[0103] Among them, K(·) is the kernel function, represents the bandwidth on dimension d at time step t, d is a positive integer, n represents the number of parameters distributed in the current sampling space, θ t represents the historical parameter vector at the t-th time step in the sampling space, represents θ t The i-th parameter sample in . In the kernel density estimation method, θ t The resolution of the density function is affected by the fixed bandwidth parameter value. When the bandwidth is large, the kernel density estimation method will extend the influence of the sample points in the adjacent area to a larger range. The deviation can be approximately expressed as:

[0104]

[0105] Taking Gaussian kernel function driving as an example, let It can be expanded to:

[0106]

[0107] The deviation is proportional to the square of the bandwidth, that is, the larger the bandwidth, the greater the deviation, and the more blurred the estimated density function. Conversely, the clearer the density function, which means it is closer to the true density function. Considering the impact of the variance of the kernel density estimation method, it can be approximated as:

[0108]

[0109] The adjustment direction and amplitude of the bandwidth are determined by the parameter variance and the change of the prior gradient norm in the current data space calculated through the above steps.

[0110] Step S2: Using the information changes of the parameter space before and after the intervention of the kernel density estimation method, a heuristic bandwidth adjustment mechanism is designed, and the input parameters are defined as bandwidth λ t , parameter variance The prior gradient norm is The basic formula of the heuristic bandwidth adjustment mechanism is obtained as follows:

[0111]

[0112] Where ψ(·) is the bandwidth adjustment function, which is in the form of:

[0113]

[0114] Where γ is the sensitivity threshold for bandwidth adjustment. The numbers in the formula may cause considerable confusion to the reader, namely 0.9, 0.001, 1.1 and 1.0. In fact, 0.9 and 1.1 are constant factors used to control λ t When the scaling When λ t will be reduced, otherwise it will be increased, and 0.01 and 1.0 are used to control λ t The new parameter θ′ is obtained by sampling from the parameter distribution estimated by the kernel density estimation method using the heuristic bandwidth adjustment mechanism, and its calculation formula is as follows:

[0115]

[0116] Among them, τ is the introduced random noise, is the noise intensity adjusted according to the heuristic strategy to avoid θ′ being too close to θ.

[0117] Step S3: Introduce the memory mechanism and store the historical sampling parameters θ mem Stored in memory space, by calculating θ′ and θ mem The Euclidean distance of is used to characterize the degree of closeness, and a distance threshold δ is set in advance to control the search range of the algorithm in the parameter space. The specific formula is as follows:

[0118]

[0119] Step S4: Independent of the adaptive bandwidth kernel density estimation plug-in of the gradient-free optimization enhancement plug-in, an adaptive cyclic learning rate adjustment (CLR) module is designed to dynamically adjust the learning rate so that the traditional optimizer produces different computational behaviors at different stages of parameter space exploration. The gradient-free optimization enhancement plug-in calculates the adjusted learning rate α according to the selected cycle mode. t , and update the parameter θ t, the learning rate update pattern is as follows.

[0120]

[0121] in represents the periodic pattern, α base represents the basic learning rate, α max Indicates the maximum learning rate. We initially introduced two modes in the algorithm, namely trigonometric function mode and cosine function mode. The trigonometric function mode is a simple linear learning rate update mode.

[0122] Formula, learning rate α base and α max Oscillates back and forth between The numerical change trend of the learning rate in each cycle is divided into two stages: a linear increase stage and a linear decrease stage, where x is the relative position calculated based on the current iteration number t and step size T. The specific formula is as follows.

[0123]

[0124] When x=0, the learning rate is the maximum value α max ; When x = 1, the learning rate is the minimum value α base , that is, in the interval x∈[0,1], the learning rate is α max and α base There is a linear change between .

[0125]

[0126] in A variable indicating the current cycle number. c In this cycle, the amplitude of the learning rate is reduced to half of the original value, thereby preventing overfitting in the post-optimization stage and achieving more stable learning. Another mode we provide is the cosine function mode, in which the learning rate gradually decreases from the maximum value to the minimum value within a cosine cycle. This mode is usually used for "annealing" of the learning rate. In this mode, the cycle pattern expression is

[0127] In summary, the gradient-free optimization enhancement plug-in algorithm can be expressed in algorithm pseudo code as follows: Figure 5 shown.

[0128] Step S5: Using the classic algorithm Adam as the enhanced optimizer, perform online learning framework analysis on the regret performance of the gradient-free optimization enhancement plug-in in minimizing a series of non-convex, discontinuous, and non-differentiable functions, that is, online prediction f t (θ t ) and the best fixed parameter f from the feasible set χ t (θ* ) in all previous time steps. At each time step t, our goal is to update the parameter θ t And apply it to the objective function f t Evaluate. Given the non-convexity, possible discontinuity and non-differentiability of the function, the minimum regret is defined as:

[0129]

[0130] in, denotes the optimal fixed parameter at all time steps. We directly derive and prove the regret upper bound of the gradient-free optimization enhancement plugin for non-convex, discontinuous, and non-differentiable functions.

[0131] At each time step t, the parameters θ are updated as follows:

[0132]

[0133] Among them, α t is the learning rate at time step t. Under the adaptive periodic learning rate adjustment mechanism, α t It is dynamically adjusted. The specific adjustment mechanism has been fully described in Section 3; is the bias-corrected moving average of the first-order momentum; It is second order

[0134] Bias-corrected moving average of momentum; Z t It is the error term introduced by the kernel density estimation method sampling, and the regret R(T) at time step t can be recursively deduced as:

[0135] R(t+1)=R(t)+f t+1 (θ t+1 )-f t+1 (θ * )

[0136] f t+1 (θ t+1 ) performs Taylor expansion:

[0137]

[0138] where ξ is a value between θ t and θ t+1 After substituting it into the parameter update formula, we get:

[0139]

[0140]

[0141] in, represents the gradient descent term, represents the sampling error term of the kernel density estimation method, represents the second-order error term

[0142] Through the above online regret derivation on the objective function, it is shown that the cumulative regret is actually the transverse superposition of the gradient descent term, the sampling error term of the kernel density estimation method, and the second-order error term. Among them, the gradient descent term is the regret accumulation produced by the Adam-based self-optimization of the gradient-free optimization enhancement plug-in algorithm when it faces local convex optimization problems and quickly optimizes according to the gradient information. The sampling error term of the kernel density estimation method is the regret accumulation produced by the smoothing probability density estimation of the parameter space, non-differentiable points and discontinuous points when the objective function globally exists non-convex, non-differentiable and discontinuous.

[0143] The second-order term error is mainly considered because in the case of non-convex function, the Hessian matrix may have negative eigenvalues, which will cause the introduction process of the second-order term to be unstable. The gradient-free optimization enhancement plug-in algorithm can better control the update step, reduce the shock caused by negative eigenvalues, and enhance the stability in non-convex optimization by adaptively adjusting the learning rate and bandwidth selection when it faces such second-order terms.

[0144] Step S6: Derive the boundedness of the gradient descent term by expanding the first-order momentum term and the second-order momentum term and bringing it into the gradient descent term:

[0145]

[0146] where β1 and β2 are the decay coefficients of the first-order momentum and the second-order momentum, respectively, and are usually very close to 1; ε represents a very small constant to prevent division by zero error. Considering that the gradient has boundedness in the local convex function part, i.e.

[0147] then:

[0148]

[0149] Without considering the divergent change of the learning rate, the gradient descent term can be upper bounded as:

[0150]

[0151] Considering that β1 is usually close to 1, and tends to a finite value gradually, the upper bound of this term is stable and will eventually tend to a finite value. Further derivation The boundedness of the gradient descent term under the influence of the learning rate parameter of different functions. Whether it is a trigonometric function mode or a cosine function mode, It is bounded, and combined with the necessity of reducing the learning rate due to parameter memory, the learning rate will gradually decay in the final stage of optimization. Taking the cosine function mode as an example, the cumulative upper bound of the gradient descent term is:

[0152]

[0153] Based on the AM-GM inequality: for positive numbers a1, a2, ..., a n ,have Therefore, the weighted sum of the momentum terms in the gradient descent term can be calculated using This approximation is based on the assumption that β1 is close to 1, so The learning rate term is divided into a constant part and a cosine function part that changes with the number of steps t. base Considered as the main contribution term, the final gradient descent term can be upper bounded as:

[0154]

[0155] It is proved that the gradient descent term of regret accumulation in the gradient-free optimization enhancement plug-in algorithm for optimizing non-convex, discontinuous and non-differentiable functions is stable and will eventually tend to a finite value.

[0156] Step S7: Based on the kernel density estimation method in step S3, for any new sample point θ t+1 , the error term of the kernel density estimation method can be calculated as follows:

[0157]

[0158] Among them, f KDE (θ t ) is the sampling distribution of the kernel density estimation method at time t, f real (θ t ) is the true distribution. This part of the error is controlled by the combination of bias and variance. The bias term and error term are respectively expressed as the first and second terms in the following equation:

[0159]

[0160] Based on Jensen’s inequality: for a convex function φ and a random variable X, E[φ(X)]≥φ[E(X)]. Considering that the gradient of the kernel density estimation method in any sampling space (local convex function) is bounded, that is, You can get:

[0161]

[0162] in, Represents the gradient component in the i-th dimension, Z t,iis the sampling error term of the kernel density estimation method in the corresponding dimension. Substituting the expression of the error of the kernel density estimation method, we get:

[0163]

[0164] Therefore, it can be proved that the regret accumulation of the sampling error term of the kernel density estimation method has boundedness, which will tend to a finite value as t.

[0165] Step S8: Based on the update formula in steps S6 and S7, the second-order error term is substituted into and Taylor expanded to obtain:

[0166]

[0167] The contribution of the second-order error term can be further upper bounded as:

[0168]

[0169] where L is the maximum eigenvalue of the Hessian matrix of the objective function, representing an upper bound of the complexity of the objective function, and representing the local curvature of the objective function. Further arrangement can obtain an upper bound as which has boundedness.

[0170] Without considering the special case of λ t Under the conditions described by us, the final upper bound of the regret accumulation is obtained as:

[0171]

[0172]

[0173] If the special case of dynamic bandwidth adjustment is considered, i.e., substituting the dynamic bandwidth parameter, introducing the bias and variance of the dynamic bandwidth parameter, and these two items are constrained, which have boundedness. It should be understood that the special case of λt includes the case of overlarge or over-small parameter initialization caused by human error.

[0174] It is proved that under the non-convex, discontinuous, and non-differentiable objective function, by introducing the kernel density estimation method and the dynamic learning rate strategy through detailed error analysis, an effective regret upper bound R(T) can be obtained, which illustrates the convergence of the no-gradient optimization enhancement plug-in algorithm, and also reveals the influence of each parameter on the final convergence. The benchmark test results on the Rosenbrock function and the Ackley function are shown in Figure 6 and Figure 7

[0175] ​In summary, the present invention provides a basic optimizer optimization enhancement plug-in suitable for parameter optimization tasks in high-dimensional, multimodal, and non- or weak-gradient scenarios. A kernel density estimation algorithm and a cyclic learning rate adjustment algorithm based on a heuristic adaptive bandwidth adjustment mechanism, in a plug-in-assisted manner, help traditional basic optimizers overcome the difficulty of achieving the desired effect in extreme application scenarios such as non- or weak-gradient scenarios. This invention can provide technical support for the application of traditional basic optimizers in multimodal and non- or weak-gradient scenarios.

Claims

1. A gradient-free optimization enhancement plug-in algorithm based on adaptive bandwidth kernel density estimation, characterized in that: The method comprises the following steps: Step S1: Use kernel density estimation method to estimate the current parameter θ=[θ1,θ2,…,θ n ] distribution, and on this basis, sample new parameters θ′. The calculation formula of the kernel density estimation method is as follows: Among them, K(·) is the kernel function, represents the bandwidth on dimension d at time step t, n represents the number of parameters distributed in the current sampling space, θ t represents the historical parameter vector at the t-th time step in the sampling space, represents θ t The i-th parameter sample in ; when the bandwidth is large, the kernel density estimation method estimates that the influence of the sample points in the adjacent area is extended to a larger range, The deviation can be approximately expressed as: Taking Gaussian kernel function driving as an example, let It can be expanded to: Based on the influence of the variance of the kernel density estimation method, the formula is approximately expressed as: The parameter variance and the change of the prior gradient norm in the current data space calculated by the above steps are used to determine the adjustment direction and amplitude of the bandwidth; Step S2: Using the information changes of the parameter space before and after the intervention of the kernel density estimation method, a heuristic bandwidth adjustment mechanism is designed, and the input parameters are defined as bandwidth λ t , the parameter variance is The prior gradient norm is The basic formula of the heuristic bandwidth adjustment mechanism is obtained as follows: Where ψ(·) is the bandwidth adjustment function, which is in the form of: Where γ is the sensitivity threshold for bandwidth adjustment; 0.9 and 1.1 are constant factors used to control λ t When the scaling When λ t will be reduced, otherwise it will be increased; 0.01 and 1.0 are the control λ t The new parameter θ′ is obtained by sampling from the parameter distribution estimated by the kernel density estimation method using the heuristic bandwidth adjustment mechanism, and its calculation formula is as follows: Among them, τ is the introduced random noise, is the noise intensity adjusted according to the heuristic strategy to avoid θ′ being too close to θ.

2. The gradient-free optimization enhancement plug-in algorithm according to claim 1, characterized in that: It also includes step S3: introducing a memory mechanism to store the historical sampling parameters θ nem Stored in memory space, by calculating θ′ and θ mem The Euclidean distance is used to characterize the degree of closeness, and a distance threshold δ is set in advance to control the search range of the algorithm in the parameter space. The calculation formula of the Euclidean distance is as follows:

3. The gradient-free optimization enhanced plug-in algorithm according to claim 2, characterized in that: The method further includes step S4: independently of the adaptive bandwidth kernel density estimation plug-in of the gradient-free optimization enhancement plug-in, designing an adaptive cyclic learning rate adjustment module to dynamically adjust the learning rate so that the traditional optimizer produces different computational behaviors at different stages of parameter space exploration; The gradient-free optimization enhancement plugin calculates the adjusted learning rate α according to the selected cycle mode t , and update the parameter θ t .

4. The gradient-free optimization enhancement plug-in algorithm according to claim 3, characterized in that: Introducing trigonometric function mode, learning rate α t The update pattern is as follows: in represents the periodic pattern, α base represents the basic learning rate, α max represents the maximum learning rate; The learning rate is α base and α max Oscillates back and forth between The numerical change trend of the learning rate in each cycle is divided into a linear increase phase and a linear decrease phase, where x is the relative position calculated based on the current number of iterations t and the step size T. The specific formula is as follows: When x=0, the learning rate is the maximum value α max ; When x = 1, the learning rate is the minimum value α base , that is, in the interval x∈[0,1], the learning rate is α max and α base There is a linear change between .

5. The gradient-free optimization enhancement plug-in algorithm according to claim 3, characterized in that: Introducing the cosine function mode, learning rate α t The update pattern is as follows: in, represents the periodic pattern, α base represents the basic learning rate, α max represents the maximum learning rate, A variable representing the current cycle number; the learning rate gradually decreases from the maximum value to the minimum value within a cosine cycle, and its periodic pattern expression is 6. The gradient-free optimization enhancement plug-in algorithm according to claim 4 or 5, characterized in that: The gradient-free optimization enhancement plug-in algorithm involves the convergence process of the traditional optimizer after being enhanced, specifically including the gradient descent term, the kernel density estimation method sampling term and the generated second-order error term; The non-gradient optimization enhancement plug-in algorithm further includes step S5: using the classic algorithm Adam as the enhanced optimizer, performing an online learning framework analysis on the regret performance of the non-gradient optimization enhancement plug-in in minimizing a series of non-convex, discontinuous, and non-differentiable functions, and online predicting f t (θ t ) and the best fixed parameter f from the feasible set χ t (θ * ) in all previous time steps; at each time step t, update the parameter θ t And apply it to the objective function f t Evaluate; given the non-convexity, possible discontinuity and non-differentiability of the function, define the minimum regret as: in, represents the optimal fixed parameters in all time steps; At each time step t, the parameters θ are updated as follows: Among them, α t is the learning rate at time step t. Under the adaptive periodic learning rate adjustment mechanism, α t It is dynamically adjusted; is the bias-corrected moving average of the first-order momentum; is the bias-corrected moving average of the second-order momentum; Z t It is the error term introduced by the kernel density estimation method sampling, and the regret R(T) at time step t can be recursively deduced as: R(t+1)=R(t)+f t+1 (i t+1 )-f t+1 (i * ) f t+1 (θ t+1 ) performs Taylor expansion: where ξ is a value between θ t and θ t+1 At a point between , after bringing it into the parameter update formula, we get: in, represents the gradient descent term, represents the sampling error term of the kernel density estimation method, represents the second-order error term.

7. The gradient-free optimization enhancement plug-in algorithm according to claim 6, characterized in that: The convergence process of the gradient-free optimization enhancement plug-in algorithm involves the horizontal superposition of various error terms, specifically including the boundedness of the gradient descent term, the boundedness of the sampling error term of the kernel density estimation method, and the boundedness of the second-order error term; The gradient-free optimization enhancement plug-in algorithm further includes step S6: by adding the first-order momentum term and the second-order momentum term Expanding it and bringing it into the gradient descent term to deduce its boundedness, we get: Among them, β1 and β2 are the attenuation coefficients of the first-order momentum and the second-order momentum, respectively, which are usually very close to 1; ε represents a very small constant to prevent division by zero errors; considering that the gradient is bounded in the local convex function part, that is, Then we have: Without considering the divergent changes in the learning rate, the gradient descent term can be bounded by: Considering that β1 is usually close to 1, and gradually tends to a finite value, so the upper bound of this term is stable and will eventually tend to a finite value; further deduction The boundedness of the gradient descent term under the influence of different functions on the learning rate parameter; in trigonometric function mode or cosine function mode, are all bounded. Taking the cosine function mode as an example, the cumulative upper bound of the gradient descent term is: Based on the AM-GM inequality: for positive numbers a1, a2, ..., a n ,get The weighted sum of the momentum term in the gradient descent term can be calculated using This approximation is based on the assumption that β1 is close to 1, so The change is not big, and the value of the sum does not change much; the learning rate term is divided into a constant part and a cosine function part that changes with the number of steps t, α base Considered as the main contribution term, the final gradient descent term can be upper bounded as: It is proved that the gradient descent term of regret accumulation in the gradient-free optimization enhancement plug-in algorithm in optimizing non-convex, discontinuous and non-differentiable functions is stable and will eventually tend to a finite value.

8. The gradient-free optimization enhancement plug-in algorithm according to claim 7, characterized in that: The gradient-free optimization enhancement plug-in algorithm further includes step S7: based on the kernel density estimation method definition in step S3, for any new sample point θ t+1 , the error term of the kernel density estimation method can be calculated as follows: Among them, f KDE (θ t ) is the sampling distribution of the kernel density estimation method at time t, f real (θ t ) is the true distribution; this part of the error is controlled by the combination of bias and variance, and the bias term and error term are shown as the first and second terms of the following equation respectively: Based on Jensen's inequality: for convex function φ and random variable X, E[φ(X)]≥φ[E(X)]; considering that the gradient of kernel density estimation method in any sampling space (local convex function) is bounded, that is, get: in, Represents the gradient component in the i-th dimension, Z t,i is the sampling error term of the kernel density estimation method in the corresponding dimension; substituting it into the expression of the kernel density estimation method error, we get: It is proved that the regret accumulation of the sampling error term of the kernel density estimation method is bounded and tends to a finite value with t.

9. The gradient-free optimization enhancement plug-in algorithm according to claim 8, characterized in that: The gradient-free optimization enhancement plug-in algorithm further includes step S8: Based on the formulas in steps S6 and S7, the second-order error term is substituted and Taylor expanded to obtain: The contribution of the second-order error term can be further upper bounded as: Among them, L is the maximum eigenvalue of the Hessian matrix of the objective function, which represents the upper bound of the complexity of the objective function and also represents the local curvature of the objective function. The upper bound can be obtained by sorting out It is bounded; Without considering λ t Under the special condition of , the final upper bound of the regret accumulation is: Considering λ t Under the special condition of , the dynamic bandwidth parameter is substituted and the deviation and variance of the dynamic bandwidth parameter are introduced, and these two items are constrained and bounded.

Images