Machine learning model parameter optimization method based on boa optimization algorithm

Abstract

The invention relates to a machine learning model parameter optimization method based on a boa optimization algorithm, and belongs to the technical field of model optimization. Comprising the steps that an objective function of a target machine learning model to be subjected to parameter optimization, an evaluation index and an initial search area of each snake group are acquired, vectors of each snake in the snake groups are acquired, different snake groups are distributed in different search areas, and the vectors of each snake comprise parameters to be optimized of the target machine learning model; each snake group is controlled to start from the initial search area, and search of the optimal solution of the to-be-optimized parameters is carried out according to the swimming approaching stage, the coiling attack stage and the stability control stage in sequence; and when the variation amplitude of the target function meets a convergence condition or reaches a preset maximum number of iterations, obtaining an optimal solution of a to-be-optimized parameter of the target machine learning model. The method has the advantages of high optimization precision, high convergence speed, high adaptability, good robustness and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of model optimization, and in particular to a method for optimizing machine learning model parameters based on a python optimization algorithm. Background Art

[0002] With the widespread application of optimization problems in science, engineering and economics, the development and improvement of machine learning model parameter optimization algorithms has become an important research topic. Although traditional machine learning model parameter optimization algorithms, such as gradient descent and genetic algorithms, are applicable to many problems, they often encounter problems such as slow convergence and easy to fall into local optimality in high-dimensional, nonlinear and multi-peak function optimization. To solve these problems, biologically inspired swarm intelligence optimization algorithms have gradually emerged in recent years. These algorithms simulate biological behaviors in nature and achieve approximation to the optimal solution through cooperation and competition between groups.

[0003] Based on the complexity of searching for optimal solutions in current machine learning models such as neural networks, particle swarm optimization algorithms (PSO), genetic algorithms (GA), ant colony algorithms (ACO) and other algorithms have been derived and applied to the current state. They have certain advantages in solving the parameter tuning of machine learning models at this stage, but there are also some problems. For example, PSO is prone to fall into local extreme values ​​in multi-peak problems, especially in complex spaces with higher latitudes. The global search ability is weak, and it may face too fast convergence speed in the early iteration, resulting in premature concentration in a certain area, reducing the spatial exploration ability. In addition, PSO is difficult to balance between global search and local search, and specific parameters need to be adjusted manually. The GA algorithm lacks a sophisticated local search strategy, and it may be difficult to converge to the global optimal solution in complex optimization problems. At the same time, GA has a strong randomness in the retention of individuals, and it is difficult to ensure convergence accuracy and stability. ACO uses pheromones to guide the path, but it is easy to have insufficient information evaporation or excessive accumulation in multi-peak problems, resulting in a slow convergence speed. It is also not flexible enough for optimization and local search in continuous space. In complex or dynamic environments, it is easy to converge to a suboptimal path in the early stage. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a method for optimizing machine learning model parameters based on the python optimization algorithm. The technical solution of the present invention is as follows:

[0005] A method for optimizing machine learning model parameters based on a python optimization algorithm, comprising:

[0006] S1, obtaining the objective function, evaluation index and initial search area of ​​each snake group of the target machine learning model to be optimized, and obtaining the vector of each snake in the snake group, wherein different snake groups are distributed in different search areas, and the vector of each snake includes the parameters to be optimized of the target machine learning model;

[0007] S2, control each snake group to start from the initial search area, and search for the optimal solution of the parameters to be optimized in the following stages: swimming approach stage, coiling attack stage and stable control stage;

[0008] S3. When the change amplitude of the objective function meets the convergence condition or reaches the preset maximum number of iterations, the optimal solution of the parameters to be optimized of the target machine learning model is obtained, and the target machine learning model is evaluated by the evaluation index. When the evaluation index meets the preset requirements, the optimal solution of the parameters to be optimized of the target machine learning model is obtained.

[0009] Optionally, the S2 includes:

[0010] Each snake group is controlled to start from the initial search area. In the swimming approach stage, random Gaussian function and sine function are used to search for the optimal solution of the parameters to be optimized. In the coiling and attacking stage, Log attenuation function and linear attenuation function are used to search for the optimal solution of the parameters to be optimized. In the stable control stage, adaptive attenuation function is used to search for the optimal solution of the parameters to be optimized.

[0011] Optionally, the searching for the optimal solution of the parameter to be optimized by using Gaussian function and sine function in the swimming approach stage includes:

[0012] In the swimming approach phase, each snake in the snake group uses the Gaussian distribution function and the sine function to search for the optimal solution of the parameters to be optimized through the position update formulas shown in formulas (1) and (2):

[0013] P new =P current +ω t α 1 N(0,1)·(P best -P current )+(1-ω t )·β·sin(π,γ)·(P bet -P current )(1);

[0014]

[0015] In formula (1) and formula (2), t represents the tth iteration of the optimization process, P new is the updated position, i.e. the position of the snake after the tth iteration, P current is the current position, i.e. the position of the snake before the tth iteration, ω t is the dynamic weight, α 1 and β are step length coefficients, N(0,1) represents the Gaussian function, P best is the current local optimal position of the snake group, T 1is the maximum number of iterations in the swimming approach phase, π represents pi, and γ is a control parameter used to adjust the frequency of the sine function.

[0016] Optionally, the use of a Log decay function and a linear decay function to search for an optimal solution of the parameters to be optimized during the coiling attack phase includes:

[0017] In the coiling attack phase, each snake in the snake group uses the Log decay function and the linear decay function to search for the optimal solution of the parameters to be optimized through the position update formulas shown in formulas (3) to (6):

[0018] P new =P current +Step log ·(P best -P current )+Per 1 (3);

[0019]

[0020] P new =p current +Step linear ·(P best -P current )+Per 2 (5);

[0021]

[0022] In formula (3) to formula (6), t represents the tth iteration of the optimization process, P new is the updated position, i.e. the position of the snake after the tth iteration, P current is the current position, i.e. the position of the snake before the tth iteration, Step log is the logarithmic decay step size, P best is the current local optimal position of the snake group, β 1 is the initial step coefficient of the Log decay function, ε 1 To prevent the denominator from being a small constant of 0, Step linear is the linear attenuation step, Per 1 and Per 2 is the random disturbance term added, β 2 is the initial step coefficient of the linear decay function, T max is the maximum number of iterations of the optimization process;

[0023] In addition, before each iteration, it is determined whether the current iteration number t is less than tp. If t<tp, the Log decay function is used to update the position; if t≥tp, the linear decay function is used to update the position; where tp=0.5×Tmax .

[0024] Optionally, the use of an adaptive attenuation function to search for an optimal solution of the parameter to be optimized in the stable control stage includes:

[0025] In the stable control stage, each snake in the swarm uses an adaptive attenuation function to search for the optimal solution of the parameters to be optimized through the position update formulas shown in formulas (7) to (8):

[0026] P new =P current +Step size ·(P best -P current ) (7);

[0027]

[0028] d current =||P current -P best || (9);

[0029] d initial =||P initial -P best || (10);

[0030] In formula (7) and formula (8), t represents the tth iteration of the optimization process, P new is the updated position, i.e. the position of the snake after the tth iteration, P current is the current position, i.e. the position of the snake before the tth iteration, P best is the current local optimal position of the snake group, Step size is the adaptive attenuation step size, α 2 is the step size coefficient, P initial To stabilize the initial position of the snake during the control phase; d current is the vector difference between the current position and the current local optimal position of the snake group; d initial The vector difference between the initial position of the snake in the stable control phase and the current local optimal position of the snake group; ε 2 To prevent the denominator from being a small constant of 0.

[0031] Optionally, during the search for the optimal solution of the parameters to be optimized in the coiling attack phase and the stable control phase, when the number of iterations reaches the exchange period T swap When , each snake group exchanges information to obtain the local optimal position of other snake groups;

[0032] Among them, if the current iteration number t is T swapWhen the value is an integer multiple of , each snake group is triggered to exchange information. During the information exchange, one snake group obtains the local optimal position shared by other snake groups, and updates its position based on the current local optimal position of the snake group with the best performance among other snake groups through formula (11);

[0033]

[0034] In formula (11), P k new is the updated position after the information exchange between snake group k and snake group m, P k is the current position of snake group k, that is, the current position set of each snake in snake group k; η is the learning rate coefficient, P best-m is the current local optimal position of the snake group with the best performance among other snake groups; the current local optimal position of snake group n is calculated by formula (12):

[0035] P best-n =argmin j f(P nj ) (12);

[0036] In formula (12), argmin j To find the minimum f(P nj ), f is the objective function, P nj is the current position of the jth snake in the snake group n;

[0037] T swap Dynamic adjustment is performed according to the convergence speed of the objective function, and the dynamic adjustment is described by formula (13) and formula (14):

[0038]

[0039] In formula (11), T swap-initial is the initial exchange period, δ is the adjustment factor, f is the objective function, Δf is the difference in the objective function change, ε 3 To prevent small constants with denominators equal to 0, is the global optimal position of the snake group in the previous iteration, is the global optimal position of the snake group after the current iteration.

[0040] Optionally, after updating the position based on the local optimal position of the snake group with the best performance among other snake groups, the method further includes:

[0041] Each snake group uses the updated position to continue to move in the direction of the step length through formula (15), and when it is determined that the positions of multiple snake groups tend to be consistent, each snake group is controlled to narrow the search area;

[0042]

[0043] In formula (15), P k n is the updated position of the snake group after k moves, Step size-k is the decay step length of the current stage of the snake group k, (P best-m -P k ) is the vector difference between the best snake group m and the current local optimal position of the kth snake group, ‖P best-m -P k ‖ is the distance between the best snake group m among other snake groups and the current local optimal position of the kth snake group, and ε is a small constant to prevent the denominator from being zero.

[0044] Optionally, after updating the position based on the current local optimal position of the snake group with the best performance among the other snake groups, it also includes: controlling the search step size of each snake group to be fine-tuned according to the change amplitude of the local optimal position through formula (16):

[0045]

[0046] In formula (16), Step size-new The updated search step size for each snake group, Step size-old Update the previous search step for each snake group, To adjust the parameters, is the variation range of the local optimal position of the kth snake group.

[0047] Optionally, the snake group is controlled to switch between different stages through formula (17) to formula (19):

[0048]

[0049] In formula (17) to formula (19), t represents the tth iteration of the optimization process, and Δf is the change amplitude of the objective function; is the objective function value corresponding to the global optimal position of the snake group in the previous iteration; is the objective function value corresponding to the global optimal position of the snake group after the current iteration; is the threshold at the current iteration t, is the initial threshold.

[0050] Optionally, the initial search area of ​​each snake group is randomly assigned, and the initial position of each snake in the snake group is determined by formula (20):

[0051]

[0052] In formula (20), P ij (0)is the initial position of the jth snake in the i-th snake group, [low d , high d ] are the upper and lower bounds of the search area of ​​snake group i, r~U(0,1) are uniformly distributed random numbers; s i It is the scale parameter for the i-th snake group, which is used to define the position offset of snake group i in the search area.

[0053] All the above optional technical solutions can be combined arbitrarily, and the present invention does not provide detailed descriptions of the structures after the combinations.

[0054] By means of the above scheme, the beneficial effects of the present invention are as follows:

[0055] By setting different snake groups to be distributed in different search areas, the vector of each snake includes the parameters to be optimized of the target machine learning model, and then controlling each snake group to start from the initial search area, and search for the optimal solution of the parameters to be optimized in sequence according to the swimming approach stage, coiling attack stage and stable control stage. This provides a method for simulating the hunting behavior of pythons in nature and achieving efficient and stable parameter optimization. This method avoids the disadvantage of traditional algorithms that they are prone to fall into local optimality through a multi-stage progressive convergence process and a multi-snake group information exchange mechanism; through the initial global search and the later local fine search, it can converge to the global optimal solution within a smaller number of iterations; it is applicable to various types of machine learning models; it shows strong robustness to problems with large parameter space or uneven distribution, and is suitable for processing high-dimensional parameter problems.

[0056] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a flow chart of the present invention.

[0058] Figure 2 It is a schematic diagram of the simulation of the search process in the swimming approach phase in the present invention.

[0059] Figure 3 It is a schematic diagram of simulating the search process in the coiling attack phase of the present invention.

[0060] Figure 4 It is a schematic diagram of the simulation of the search process in the stable control stage of the present invention. DETAILED DESCRIPTION

[0061] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0062] The machine learning model parameter optimization method based on the python optimization algorithm provided in the embodiment of the present invention can be implemented by any electronic device with computing function, such as a PC, a mobile terminal or a server. The method provided in the embodiment of the present invention achieves efficient and stable parameter optimization by simulating the hunting behavior of pythons in nature, thereby improving the adaptability and accuracy of the machine learning model under different data and scenarios.

[0063] Some concepts involved in the embodiments of the present invention are first explained below.

[0064] "Snake group" and "snake" in the embodiment of the present invention are two core concepts, representing different optimization levels. A snake group is a group concept, which is a collection of multiple snakes, including a group of snakes that jointly perform optimization search. Each snake group is responsible for global exploration in a specific search area, and obtains its local optimal solution (local optimal position) through the collaborative search of snakes in the snake group. Different snake groups exchange local optimal solutions with each other through an information sharing mechanism to improve the global search capability and avoid falling into the local optimal solution. At the same time, the snake group covers different search areas in a distributed manner to achieve efficient exploration of the entire search area. The snake is the basic unit in the snake group and is responsible for performing specific optimization tasks in the search area. Each snake explores solutions in the search area through a movement strategy (such as Gaussian perturbation, etc.), and its current position represents a potential solution. The movement trajectory of the snake is dynamically adjusted based on the rules of the optimization algorithm to perform a more refined search locally. In addition, the current position of each snake will be used to calculate the value of the objective function as a basis for evaluating the quality of the solution. Through the above mechanism, the comparison and sharing of the optimal solutions between multiple snake groups greatly improves the global optimization capability of the method. The snake swarm expands the search scope in global search, while the snake refines the search results in local exploration. The two complement each other and together achieve efficient solutions to complex problems.

[0065] Before the specific implementation of the embodiment of the present invention, it is necessary to initialize first, specifically to set the initial snake group, and define each snake as a vector containing the parameters to be optimized, and randomly distribute the snake groups in different areas of the search area to increase the diversity of exploration, avoid falling into local optimization in the initial state, and the search areas of all snake groups cover the entire search area. Set the objective function and evaluation index of the target machine learning model to evaluate the pros and cons of the current position of each snake. Simulate the three stages of python hunting: swimming approach stage, attack and coiling stage, and stable control stage. Through the design of the above three stages, while maintaining global exploration, it converges to the global optimal solution faster, improving the global search and local convergence capabilities. Each snake group is responsible for exploring an area of ​​the search area. The snakes in each snake group are randomly distributed to their corresponding search areas to increase diversity and exploration range. The current position of each snake is represented by a vector, which contains all reference values ​​of the parameters to be optimized. For example: Assume that there are three parameters to be optimized, namely x 1 、x 2 , and x 3 , then the current position of each snake can be expressed as a three-dimensional vector: P = (x 1 , x 2 , x 3 ), x 1 、x 2 , and x 3 Represents the position parameters of the snake.

[0066] Specifically, Figure 1 As shown, the machine learning model parameter optimization method based on the python optimization algorithm provided by the embodiment of the present invention includes the following steps S1 to S3:

[0067] S1, obtain the objective function, evaluation index and initial search area of ​​each snake group of the target machine learning model to be optimized, and obtain the vector of each snake in the snake group, where different snake groups are distributed in different search areas, and the vector of each snake includes the parameters to be optimized of the target machine learning model.

[0068] In specific implementation, based on the above initialization, this step directly obtains the initialization result, and can obtain the objective function, evaluation index and the initial search area of ​​each snake group.

[0069] The initial search area of ​​each snake group is randomly assigned. In the embodiment of the present invention, the snake group is divided into several independent subgroups (snakes), and each subgroup performs independent search in different initial search areas. Suppose there are M snake groups, each with N snakes, and the dimension of the optimization problem is D. The initial position of each snake group can be evenly distributed in different search areas. Specifically, the initial position of each snake in the snake group can be determined by formula (20):

[0070]

[0071] In formula (20), P ij (0) is the initial position of the jth snake in the i-th snake group, [low d , high d ] are the upper and lower bounds of the search area of ​​snake group i, r~U(0,1) are uniformly distributed random numbers; s i It is the scale parameter for the i-th snake group, which is used to define the position offset of snake group i in the search area.

[0072] S2, control each snake group to start from the initial search area, and search for the optimal solution of the parameters to be optimized in sequence according to the swimming approach stage, coiling attack stage and stable control stage.

[0073] The swimming approach stage simulates the swimming approach of a python and conducts extensive spatial exploration of solutions in the early stage of the search to avoid falling into the local optimum. The swimming approach stage enhances the global search capability of the method provided by the embodiment of the present invention. The attack and coiling stage gradually narrows the search range to make the solution approach the optimal solution. It gradually reduces the step size, reduces randomness, balances global exploration and local convergence, and enhances the search accuracy. The stable control stage approaches the position of the solution and makes precise fine-tuning to achieve the result of the global optimal solution.

[0074] In specific implementation, S2 includes: controlling each snake group to start from the initial search area, using random Gaussian function and sine function to search for the optimal solution of the parameters to be optimized in the swimming approach stage, using Log attenuation function and linear attenuation function to search for the optimal solution of the parameters to be optimized in the coiling and attacking stage, and using adaptive attenuation function to search for the optimal solution of the parameters to be optimized in the stable control stage.

[0075] In a specific embodiment, in the early search process, a Gaussian function is used to simulate the randomness of a python in the process of searching for food. The location of the food is updated by generating random numbers that obey the Gaussian distribution. The solution jumps in space, which has uncertainty and diversity. The randomness generated by the Gaussian distribution can enhance the spatial range of the solution. At the same time, the update strategy of the random Gaussian function also brings about the random movement of the solution, making it easier for the method provided by the embodiment of the present invention to jump out of the local optimum and have more opportunities to explore the global optimal area. In a complex space, the Gaussian distribution can ensure the random jump of the solution and avoid getting stuck in a local area. The global search capability and the diversity of the solution are enhanced, and the adaptability to complex spaces is stronger. However, since the Gaussian random jump lacks a fixed direction and a fixed order, the location of the solution may not be close to the optimum, which is not conducive to local convergence. At the same time, the randomness of the Gaussian distribution makes it difficult to estimate the step size, and the jump amplitude may be uncontrollable, thereby affecting the stability of the convergence. In this case, the sine function introduced in the embodiment of the present invention can effectively solve the above problems. That is to say, in the early stage of the search, a Gaussian function is used to implement global random search. After the search area converges, the periodicity and directionality of the sine function are used for precise tracking to improve the global exploration ability and convergence accuracy of the method.

[0076] Among them, the Gaussian function is expressed by the formula: P new =P current +α 1 ·N(0,1)·(P best -P current ), the sine function is expressed by the formula: P new =P current +β·sin(π,γ)·(P best -P current ). In addition, in order to make the transition between the two functions smoother, the embodiment of the present invention introduces a dynamic weight ω t To achieve a smooth transition between Gaussian function and sine function.

[0077] Based on the above content, in the swimming approaching stage, the embodiment of the present invention controls each snake in the snake group to use Gaussian function and sine function to search for the optimal solution of the parameters to be optimized through the position update formulas shown in formula (1) and formula (2):

[0078] P new =P current +ω t α 1 N(0,1)·(P best -P current )+(1-ω t )·β·sin(π,γ)·(P best -P current )(1);

[0079]

[0080] In formula (1) and formula (2), t represents the tth iteration of the optimization process, P new is the updated position, i.e. the position of the snake after the tth iteration, P current is the current position, i.e. the position of the snake before the tth iteration, ω t is the dynamic weight, α 1 and β are step length coefficients, N(0,1) represents the Gaussian distribution function; P best is the current local optimal position of the snake group. In one iteration, its value is calculated by formula (12); T 1 is the maximum number of iterations in the approaching stage; π represents the circumference of a circle, which is used to introduce periodic fluctuations to simulate the swimming trajectory of a python; γ is a control parameter and is used to adjust the frequency of the sine function.

[0081] like Figure 2 As shown in Figure 1, it is a schematic diagram of the simulation of the search process in the swimming approach phase. Figure 2 It can be seen that in the swimming approach stage, the Gaussian function perturbation is used in the first half, which simulates the behavior of snakes searching for targets in a larger range; the sine function perturbation is used in the second half, and the snake group gradually reduces irregular movement and concentrates on the possible optimal area. Figure 2 The “□X” in represents SnakeX, and “□□□□X” represents the initial position of SnakeX.

[0082] In a specific embodiment, the method of using the Log decay function and the linear decay function to search for the optimal solution of the parameter to be optimized in the coiling and attacking stage includes: in the coiling and attacking stage, controlling each snake in the snake group to use the Log decay function and the linear decay function to search for the optimal solution of the parameter to be optimized by using the position update formula shown in formula (3) to formula (6):

[0083] P new =P current +Step log ·(P best -P current )+Per 1 (3);

[0084]

[0085] P new =P current +Step linear ·(P best -P current )+Per 2 (5);

[0086]

[0087] In formula (3) to formula (6), t represents the tth iteration of the optimization process, P new is the updated position, i.e. the position of the snake after the tth iteration, P current is the current position, i.e. the position of the snake before the tth iteration, Step log is the logarithmic decay step size, P best is the current local optimal position of the snake group, β 1 The initial step length coefficient of the Log decay function is used to control the initial value of the step length, ε 1 To prevent the denominator from being a small constant of 0, Step linear is the linear attenuation step; Per 1 and Per 2 is the random disturbance term added to increase the randomness of the search and prevent falling into the local optimum; β 2 is the initial step coefficient of the linear decay function, T max is the maximum number of iterations in the optimization process; (P best -P current ) is the vector difference between the current local optimal position of the snake group and the current position, indicating the moving direction and distance.

[0088] The Log attenuation function controls the step size logarithmically, has strong early jumps and late convergence, and is suitable for the global convergence requirements of high-dimensional complex spaces. However, due to the slow attenuation in the late stage, the convergence time may be increased. Therefore, the embodiment of the present invention adopts a linear attenuation function in the late stage of the coiling and attacking stage. The linear attenuation function is applied to tasks that require precise fine-tuning in the late stage of the coiling and attacking stage because of its smooth transition and easy control.

[0089] Furthermore, in order to make the coiling attack phase transition smoothly, the embodiment of the present invention sets a transition point tp, which is based on the maximum number of iterations T max For example, tp = 0.5 × T max , the specific meaning of this formula is: at the maximum number of iterations T max The attenuation strategy is switched when the value reaches 50%. Of course, this data can be adjusted as needed. Therefore, the embodiment of the present invention determines which attenuation method to use by checking whether the current number of iterations t is less than tp before each iteration. Specifically, the embodiment of the present invention needs to determine whether the current number of iterations t is less than tp before each iteration. If t<tp, the Log attenuation function is used to update the position; if t≥tp, the linear attenuation function is used to update the position; preferably, tp=0.5×T max .

[0090] like Figure 3 As shown in Figure 1, it is a schematic diagram of the search process simulation during the coiling attack phase. Figure 3 It can be seen that the snake group shows a trend of rapid convergence to the "optimal position" (red five-pointed star) in this stage. The step length is calculated using the Log decay function in the first half, which makes the snake gradually approach the optimal position. The step length is calculated using the linear decay function in the second half, which makes the snake converge further.

[0091] In a specific embodiment, the method of using an adaptive attenuation function to search for an optimal solution of the parameter to be optimized in the stable control stage includes: in the stable control stage, controlling each snake in the snake group to use an adaptive attenuation function to search for an optimal solution of the parameter to be optimized by using a position update formula as shown in formula (7) to formula (8):

[0092] P new =P current +Step size ·(P best -P current ) (7);

[0093]

[0094] d current =||P current -P best || (9);

[0095] d initial =||P initial -P best || (10);

[0096] In formula (7) and formula (8), t represents the tth iteration of the optimization process, P new is the updated position, i.e. the position of the snake after the tth iteration, P current is the current position, i.e. the position of the snake before the tth iteration, P best is the current local optimal position of the snake group, Step size is the adaptive attenuation step size, α 2 is the step size coefficient, P initial is the initial position of the snake; d current is the vector difference between the current position and the current local optimal position of the snake group; d initial is the vector difference between the initial position of the snake and the current local optimal position of the snake group, indicating the moving direction and distance; ε 2 To prevent the denominator from being a small constant of 0.

[0097] It can be obtained from formulas (7) to (10) that in the stable control stage, each snake in each snake group in the embodiment of the present invention uses an adaptive attenuation function to adaptively attenuate the step size. The step size of each snake may be different because they are in different search positions, and as the iteration proceeds, the step size will also be updated according to the adaptive attenuation function, so as to automatically adjust the attenuation step size according to the position of the solution during the optimization process, and ensure that the attenuation step size decreases as the search range gradually converges to the optimal solution, thereby improving the convergence accuracy, and at the same time, there is no need to rely on manual parameter determination.

[0098] like Figure 4 As shown in Figure 1, it is a simulation diagram of the search process in the stable control stage. Figure 4 It can be seen that, in this stage, the method provided by the embodiment of the present invention well demonstrates how to achieve accurate approximation to the optimal solution through adaptive step size.

[0099] In addition, the embodiment of the present invention triggers an information exchange mechanism after a certain iteration cycle. The information exchange mechanism exists in the attack and coiling stage and the stable control stage. It exchanges the current local optimal positions of different snake groups, so that each snake group can maintain a certain degree of independence while sharing global information, thereby effectively avoiding falling into the local optimum. Specifically, every certain exchange period T swap An information exchange is carried out. The core idea of ​​the information exchange mechanism is to learn from each other the optimal positions of different snake groups.

[0100] In the specific implementation, during the search for the optimal solution of the parameters to be optimized in the coiling attack stage and the stable control stage, when the number of iterations reaches the exchange period T swap When , each snake group exchanges information to obtain the current local optimal position of other snake groups; if the current iteration number t is T swap When it is an integer multiple of , each snake group is triggered to exchange information. During the information exchange, one snake group obtains the local optimal position shared by other snake groups.

[0101] After exchanging information, the snake group will update its position based on the current local optimal position of the best snake group among other snake groups using formula (11);

[0102]

[0103] In formula (11), P k new is the updated position after the information exchange between snake group k and snake group m. Each snake in snake group k will update its position based on the current local optimal position of the snake group with the best performance among other snake groups through formula (11); P k is the current position of snake group k, that is, the set of positions of each snake in snake group k; η is the learning rate coefficient, which controls the degree of position adjustment; Pbest-m is the current local optimal position of the snake group with the best performance among other snake groups; the current local optimal position of snake group n is calculated by formula (12): P best-n =argmin j f(P nj )(12); in formula (12), argmin j To find the minimum f(P nj ), f is the objective function, P nj is the current position of the jth snake in the snake group n.

[0104] Among them, T swap It can be dynamically adjusted according to the convergence speed of the objective function. The dynamic adjustment is described by formula (13) and formula (14), which is specifically manifested as follows: as the objective function gradually converges, the exchange period will increase and the exchange frequency will be reduced to reduce the interference of excessive information exchange on local optimization.

[0105]

[0106] In formula (11), T swap-initial is the initial exchange period, δ is the adjustment factor, Δf is the difference in the objective function change, ε 3 To prevent small constants with denominators equal to 0, is the global optimal position of the snake group in the previous iteration, is the global optimal position of the snake group after the current iteration. and , the local optimal position of each snake group before and after iteration is substituted into the objective function, and the current local optimal position of the snake group with the smallest difference in the objective function is and

[0107] After exchanging information, each snake group will compare its current local optimal position with the received external local position and select the one with the best performance as the new local optimal position. When selecting the local optimal position with the best performance, the snake group will select the local optimal position with the shortest convergence time, that is, the shortest time to obtain the local optimal position, or the local optimal position with a smaller or more stable step size as the local optimal position of the snake group with the best performance among other snake groups.

[0108] After exchanging information and updating the position, each snake group will use the updated position to continue to move in the direction of the step length. Specifically, each snake group uses the updated position to continue to move in the direction of the step length through formula (15), and when it is determined that the positions of multiple snake groups are tending to be consistent, each snake group is controlled to narrow the search area. Specifically, in the iterative process of the coiling attack phase, before the information is exchanged, the snake group updates the position through formula (3) or (5), and after the information is exchanged, the position is updated through formula (15); in the iterative process of the stable control phase, before the information is exchanged, the snake group updates the position through formula (7), and after the information is exchanged, the position is updated through formula (15).

[0109]

[0110] In formula (15), P k n is the updated position of the snake group after k moves; Step size-k is the attenuation step length of the current stage (coiling attack stage or stable control stage) of snake group k (of course, the calculation formula of the attenuation step length of each snake group is the same, and only snake group k is used as an example here to illustrate), which determines the amplitude of each iterative movement of the snakes in the snake group; (P best-m -P k ) is the vector difference between the best snake group m and the current local optimal position of the kth snake group, ‖P best-m -P k ‖ is the distance between the best snake group m among other snake groups and the current local optimal position of the kth snake group, and ε is a small constant to prevent the denominator from being zero.

[0111] Furthermore, if the positions of multiple snake groups tend to be consistent, that is, if multiple snake groups come to a certain search area and obtain the same local optimal position, then this same position is the best position, indicating that the search area has converged. At this time, the snake group will gradually reduce the step size to conduct local search; if the local optimal positions of the snake groups are quite different, it means that they have not yet converged. The snake group can further expand the search range to improve the global search capability.

[0112] Furthermore, after updating the position based on the current local optimal position of the snake group with the best performance among the other snake groups, the embodiment of the present invention further includes: controlling the search step size of each snake group to be fine-tuned according to the change amplitude of the local optimal position through formula (16):

[0113]

[0114] In formula (16), Step size-new The updated search step size for each snake group, Step size-oldUpdate the previous search step for each snake group, To adjust the parameters; is the change amplitude of the local optimal position of the kth snake group; when the change amplitude is large, the step size will be reduced to perform a more intensive local search; when the change amplitude is small or remains unchanged, the search is stopped.

[0115] It should be noted here that in the stable control stage, before the snake group fine-tunes the step length through formula (16), it can also adjust the step length through formula (21):

[0116]

[0117] The difference between formula (21) and formula (16) is that formula (21) is used to search for solutions in a large range at the beginning of the stable control stage, while formula (16) is used when the optimal solution tends to be stable in the later stage of the stable control stage. The switching between formula (21) and formula (16) is defined by the value of Δf. When Δf is less than the preset threshold, formula (16) is used, and when Δf is greater than the preset threshold, formula (21) is used. The preset threshold is set according to the target machine learning model.

[0118] The above content introduces the position update method of the snake group in three stages. However, the method provided by the embodiment of the present invention also needs to consider how the snake group switches between the three stages. In specific implementation, the embodiment of the present invention controls the snake group to switch between different stages through formulas (17) to (19):

[0119]

[0120] In formula (17) to formula (19), t represents the tth iteration of the optimization process, and Δf is the change amplitude of the objective function; is the objective function value corresponding to the global optimal position of the snake group in the previous iteration; is the objective function value corresponding to the global optimal position of the snake group after the current iteration; is the threshold at the current iteration t, is the initial threshold.

[0121] S3. When the change amplitude of the objective function meets the convergence condition or reaches the preset maximum number of iterations, the optimal solution of the parameters to be optimized of the target machine learning model is obtained, and the target machine learning model is evaluated by the evaluation index. When the evaluation index meets the preset requirements, the optimal solution of the parameters to be optimized of the target machine learning model is obtained.

[0122] Specifically, in each iteration, the embodiment of the present invention can also use evaluation indicators to score the current optimal parameters and record the loss change curve during the optimization process. For example, mae and the like can be selected as evaluation indicators, and this part is not elaborated in detail here. In addition, when the loss function of the target machine learning model no longer decreases significantly in several iterations, or when the preset maximum number of iterations is reached, the iteration ends, and the parameters obtained at this time are the optimal solutions for the parameters to be optimized.

[0123] The termination condition for the convergence of the loss function can be: when the loss function continues to decrease significantly in several iterations, a tolerance threshold σ can be set as the stopping criterion. If the loss change ▽L is less than the preset tolerance threshold σ in T consecutive iterations, the model is considered to have converged and the iteration is stopped. The mathematical expression is |▽L|<σ.

[0124] In addition, based on the final model convergence evaluation, in addition to the above-mentioned dynamic convergence trigger mechanism, an evaluation method based on the Euclidean distance between snake positions can also be introduced. Specifically, by calculating the Euclidean distance distribution of all snakes in the swarm, its degree of convergence is analyzed. When the positions of the snakes gradually tend to be concentrated, that is, when the mean or standard deviation of the Euclidean distance drops to a preset threshold, it can be determined that the search process is close to convergence. This method can more intuitively reflect the aggregation of the snake swarm in the search space, provide additional basis for the termination of the method, and help improve the flexibility and accuracy of model optimization.

[0125] In summary, the method provided by the embodiment of the present invention has the following characteristics:

[0126] 1. A multi-stage hierarchical optimization strategy is proposed.

[0127] The embodiment of the present invention divides the optimization process into a swimming approach stage, an attack and coiling stage, and a stable control stage. Each stage adopts a different update strategy, including Gaussian perturbation, sinusoidal function, logarithmic and linear decay, adaptive step size, etc., to achieve gradual optimization from global search to local convergence. The overall structural design of this multi-stage hierarchical optimization strategy, as well as the specific update function independently adopted for each stage, enables the method provided by the embodiment of the present invention to flexibly strike a balance between global search and local convergence.

[0128] 2. Multi-snake group collaborative optimization and information exchange mechanism.

[0129] The embodiment of the present invention uses parallel search of multiple snake groups, and each snake group starts searching from a different initial search area. By setting an exchange cycle, the local optimal positions are exchanged between the snake groups, which effectively avoids the problem of the optimization process falling into the local optimum. The mechanism of multiple snake groups performing independent optimization in different initial search areas and regularly exchanging local optimal positions ensures that the method provided by the embodiment of the present invention can perform fine optimization in local areas while performing extensive search. The collaborative search and information exchange system is an important innovation with uniqueness and practical application effects.

[0130] 3. Dynamic adaptive step length based on bionics.

[0131] In the stable control stage, the present invention adopts an adaptive step-size attenuation strategy, so that the method provided by the embodiment of the present invention can automatically reduce the step-size when approaching the optimal solution, thereby enhancing the stability of convergence. The design of this dynamic adaptive step-size simulates the behavior of organisms making small adjustments near prey. The design of the dynamic adaptive step-size enables the optimization process to stabilize near the optimal solution in the later stage, and will not jump out of the optimal area due to excessive step-size, thereby improving the accuracy and stability of the method provided by the embodiment of the present invention. The adaptive adjustment method of step-size attenuation is a key update of bionic optimization.

[0132] 4. Random perturbation mechanism for parameter updating.

[0133] The method provided in the embodiment of the present invention is specifically used for parameter optimization of machine learning models. It has the characteristics of universality and strong adaptability, and can be widely used in various machine learning models and complex high-dimensional parameter spaces.

[0134] 5. Protection of the overall framework of the algorithm.

[0135] The method provided in the embodiment of the present invention can be called the snake optimization algorithm (SOA), which includes the above-mentioned multi-stage update strategy, multi-snake group information exchange, adaptive step size adjustment and random perturbation, etc., forming a systematic and well-structured optimization method. This overall architecture has significant innovative value in improving the efficiency of machine learning model parameter optimization.

[0136] The present invention has the following beneficial effects:

[0137] 1) High optimization accuracy: Through a multi-stage progressive convergence process and a multi-snake group information exchange mechanism, the disadvantage of traditional algorithms that they are prone to fall into local optimality is avoided.

[0138] 2) Fast convergence speed: Thanks to the initial global search and the later local fine search, it can converge to the global optimal solution in a smaller number of iterations.

[0139] 3) Strong adaptability: Applicable to various types of machine learning models.

[0140] 4) Good robustness: For problems with large parameter space or uneven distribution, the method provided by the embodiment of the present invention shows strong robustness and is suitable for processing high-dimensional parameter problems.

[0141] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the technical principles of the present invention, and these improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for optimizing machine learning model parameters based on Python optimization algorithm, characterized in that: include: S1, obtaining the objective function, evaluation index and initial search area of ​​each snake group of the target machine learning model to be optimized, and obtaining the vector of each snake in the snake group, wherein different snake groups are distributed in different search areas, and the vector of each snake includes the parameters to be optimized of the target machine learning model; S2, control each snake group to start from the initial search area, and search for the optimal solution of the parameters to be optimized in the following stages: swimming approach stage, coiling attack stage and stable control stage; S3. When the change amplitude of the objective function meets the convergence condition or reaches the preset maximum number of iterations, the optimal solution of the parameters to be optimized of the target machine learning model is obtained, and the target machine learning model is evaluated by the evaluation index. When the evaluation index meets the preset requirements, the optimal solution of the parameters to be optimized of the target machine learning model is obtained.

2. The method for optimizing machine learning model parameters based on the python optimization algorithm according to claim 1, characterized in that: The S2 includes: Each snake group is controlled to start from the initial search area. In the swimming approach stage, random Gaussian function and sine function are used to search for the optimal solution of the parameters to be optimized. In the coiling and attacking stage, Log attenuation function and linear attenuation function are used to search for the optimal solution of the parameters to be optimized. In the stable control stage, adaptive attenuation function is used to search for the optimal solution of the parameters to be optimized.

3. The method for optimizing machine learning model parameters based on the python optimization algorithm according to claim 2, characterized in that: The method of searching for the optimal solution of the parameters to be optimized by using Gaussian function and sine function in the swimming approach stage includes: In the swimming approach phase, each snake in the snake group uses the Gaussian distribution function and the sine function to search for the optimal solution of the parameters to be optimized through the position update formulas shown in formulas (1) and (2): P new =P current +oh t ·α1N(0,1)·(P best -P current )+(1-ω t )·β·sin(π,γ)·(P best -P current (1); In formula (1) and formula (2), t represents the tth iteration of the optimization process, P new is the updated position, i.e. the position of the snake after the tth iteration, P current is the current position, i.e. the position of the snake before the tth iteration, ω t is the dynamic weight, α1 and β are step coefficients, N(0,1) represents the Gaussian function, P best is the current local optimal position of the snake group, T1 is the maximum number of iterations in the swimming approach phase, π represents pi, and γ is a control parameter used to adjust the frequency of the sine function.

4. The method for optimizing machine learning model parameters based on the python optimization algorithm according to claim 2, characterized in that: The method of using the Log decay function and the linear decay function to search for the optimal solution of the parameters to be optimized in the coiling attack stage includes: In the coiling attack phase, each snake in the snake group uses the Log decay function and the linear decay function to search for the optimal solution of the parameters to be optimized through the position update formulas shown in formulas (3) to (6): P new =P current +Step log ·(P best -P current )+Per1 (3); P new =P current +Step linear ·(P best -P current )+Per2 (5); In formula (3) to formula (6), t represents the tth iteration of the optimization process, P new is the updated position, i.e. the position of the snake after the tth iteration, P current is the current position, i.e. the position of the snake before the tth iteration, Step log is the logarithmic decay step size, P best is the current local optimal position of the snake group, β1 is the initial step coefficient of the Log decay function, ε1 is a small constant to prevent the denominator from being 0, Step linear is the linear attenuation step, Per1 and Per2 are the added random disturbance terms, β2 is the initial step coefficient of the linear attenuation function, T max is the maximum number of iterations of the optimization process; In addition, before each iteration, it is determined whether the current iteration number t is less than tp. If t<tp, the Log decay function is used to update the position; if t≥tp, the linear decay function is used to update the position; where tp=0.5×T max .

5. The method for optimizing machine learning model parameters based on the python optimization algorithm according to claim 2, characterized in that: The method of using an adaptive attenuation function to search for an optimal solution of the parameters to be optimized in the stable control stage includes: In the stable control stage, each snake in the swarm uses an adaptive attenuation function to search for the optimal solution of the parameters to be optimized through the position update formulas shown in formulas (7) to (8): P new =P current +Step size ·(P best -P current ) (7); d current =||P current -P best || (9); d initial =||P initial -P best || (10); In formula (7) and formula (8), t represents the tth iteration of the optimization process, P new is the updated position, i.e. the position of the snake after the tth iteration, P current is the current position, i.e. the position of the snake before the tth iteration, P best is the current local optimal position of the snake group, Step size is the adaptive attenuation step size, α2 is the step size coefficient, P initial To stabilize the initial position of the snake during the control phase; d current is the vector difference between the current position and the current local optimal position of the snake group; d initial It is the vector difference between the initial position of the snake in the stable control phase and the current local optimal position of the snake group; ε2 is a small constant to prevent the denominator from being zero.

6. The method for optimizing machine learning model parameters based on the python optimization algorithm according to claim 2, 4 or 5, characterized in that: In the process of searching for the optimal solution of the parameters to be optimized during the coiling attack phase and the stable control phase, when the number of iterations reaches the exchange period T swap When , each snake group exchanges information to obtain the local optimal position of other snake groups; Among them, if the current iteration number t is T swap When the value is an integer multiple of , each snake group is triggered to exchange information. During the information exchange, one snake group obtains the local optimal position shared by other snake groups, and updates its position based on the current local optimal position of the snake group with the best performance among other snake groups through formula (11); In formula (11), P k new is the updated position after the information exchange between snake group k and snake group m, P k is the current position of snake group k, that is, the current position set of each snake in snake group k; η is the learning rate coefficient, P best-m is the current local optimal position of the snake group with the best performance among other snake groups; the current local optimal position of snake group n is calculated by formula (12): P best-n =argmin j f(P nj ) (12); In formula (12), argmin j To find the minimum f(P nj ), f is the objective function, P nj is the current position of the jth snake in the snake group n; T swap Dynamic adjustment is performed according to the convergence speed of the objective function, and the dynamic adjustment is described by formula (13) and formula (14): In formula (11), T swap-initial is the initial exchange period, δ is the adjustment factor, f is the objective function, Δf is the difference in the objective function change, ε3 is a small constant to prevent the denominator from being zero, is the global optimal position of the snake group in the previous iteration, is the global optimal position of the snake group after the current iteration.

7. The method for optimizing machine learning model parameters based on the python optimization algorithm according to claim 6, characterized in that: After the position update based on the local optimal position of the best snake group among other snake groups, it also includes: Each snake group uses the updated position to continue to move in the direction of the step length through formula (15), and when it is determined that the positions of multiple snake groups tend to be consistent, each snake group is controlled to narrow the search area; In formula (15), P k n is the updated position of the snake group after k moves, Step size-k is the decay step length of the current stage of the snake group k, (P best-m -P k ) is the vector difference between the best snake group m and the current local optimal position of the kth snake group, ‖P best-m -P k ‖ is the distance between the best snake group m among other snake groups and the current local optimal position of the kth snake group, and ε is a small constant to prevent the denominator from being zero.

8. The method for optimizing machine learning model parameters based on the python optimization algorithm according to claim 6 or 7, characterized in that: After updating the position based on the current local optimal position of the snake group with the best performance among other snake groups, it also includes: controlling the search step size of each snake group to be fine-tuned according to the change range of the local optimal position through formula (16): In formula (16), Step size-new The updated search step size for each snake group, Step size-old Update the previous search step for each snake group, To adjust the parameters, is the variation range of the local optimal position of the kth snake group.

9. The method for optimizing machine learning model parameters based on the python optimization algorithm according to claim 2 or 6, characterized in that: The control of the snake group is performed by switching between different stages through formula (17) to formula (19): In formula (17) to formula (19), t represents the tth iteration of the optimization process, and Δf is the change amplitude of the objective function; is the objective function value corresponding to the global optimal position of the snake group in the previous iteration; is the objective function value corresponding to the global optimal position of the snake group after the current iteration; is the threshold at the current iteration t, is the initial threshold.

10. The method for optimizing machine learning model parameters based on the python optimization algorithm according to claim 1, characterized in that: The initial search area of ​​each snake group is randomly assigned, and the initial position of each snake in the snake group is determined by formula (20): In formula (20), P ij (0) is the initial position of the jth snake in the i-th snake group, [low d , high d ] are the upper and lower bounds of the search area of ​​snake group i, r~U(0,1) are uniformly distributed random numbers; s i It is the scale parameter for the i-th snake group, which is used to define the position offset of snake group i in the search area.

Images