Hybrid swarm intelligence deep learning model hyper-parameter optimization method

Abstract

The invention discloses a hybrid swarm intelligence deep learning model hyper-parameter optimization method. The method comprises the following steps: selecting a deep neural network to build a software defect prediction model; initializing mixed wolf pack algorithm parameters; updating an individual extreme value and a group extreme value; judging whether an algorithm stopping condition is met or not; updating the speed and the position of each particle; solving an adaptive value on the updated position; solving the optimal adaptive value and the optimal position of the updated first wolf; updating the first wolf after updating the wolf detection position; updating the position of the fierce wolf; updating the adaptive value of the fierce wolf at the current position; calculating an updated position of entering the purse wolf; the worst R wolves are abandoned; calculating the adaptive value of the R piwolf; and obtaining an evaluation index of the model. The hybrid algorithm is used in deep learning model hyper-parameter optimization, can quickly and efficiently approach to the optimal solution, can well solve the problems of a plurality of local solutions and global solutions, and has higher global optimization ability, thereby ensuring the precision of model prediction.

Description

technical field [0001] The invention relates to a deep learning model hyperparameter optimization technology, in particular to a method for optimizing a deep learning model hyperparameter with mixed group intelligence. Background technique [0002] Hyperparameter optimization is also known as hyperparameter tuning. The deep learning algorithm contains thousands of parameters. Some of these parameters can be optimized through training, such as the weights in the neural network, which we call parameters, and some parameters cannot be optimized through training, such as the learning rate. etc., we call hyperparameters. [0003] In deep learning algorithms, hyperparameters are used to adjust the entire network training process, such as the number of hidden layers of the neural network, the size and number of kernel functions, and so on. Different deep learning models and even the same model with different parameters have different application ranges and algorithm effects. Ther...

AI Technical Summary

Problems solved by technology

[0007] 1) Hyperparameter optimization is a dynamic process. Grid and random search can only combine different parameters, and cannot dynamically select and adjust parameters according to the training state of the model, so that the parameters can actively approach the optimal value. Therefore, it is necessary to find a suitable The optimization algorithm of dynamic optimization, such as swarm intelligence algorithm, realizes the automatic and efficient processing of model selection and hyperparameter optimization, and reduces the time and computing power cost caused by violent exhaustive search;

[0008] 2) The hyperparameter optimization problem is a nonlinear problem, so there will be many local optimal solutions, so it is necessary to find an algorithm with strong global optimization capabilities to solve the problem that hyperparameter optimization is easy to fall into local optimal solutions

[0009] 3) A single swarm intelligence algorithm has its own advantages and disadvantages, and the algorithm with fast convergence speed will fall into the trouble of local optimization, so it is necessary to improve the algorithm to improve the performance of the algorithm;

Images