Optimizer learning method and device, electronic equipment and readable storage medium

Abstract

The invention discloses an optimizer learning method and device, electronic equipment and a readable storage medium, and relates to the technical field of deep learning. When optimizer learning is carried out, the implementation scheme adopted comprises the steps of acquiring training data which comprise multiple data sets, and each data set comprises attribute information of a neural network, optimizer information of the neural network and parameter information of an optimizer; and taking the attribute information of the neural network and the optimizer information of the neural network in each data set as input, taking the parameter information of the optimizer in each data set as output, and training a meta-learning model until the meta-learning model converges. According to the invention, the self-adaption of the optimizer can be realized such that the generalization ability of the optimizer is improved.

Description

technical field [0001] The present application relates to the technical field of artificial intelligence, and in particular to an optimizer learning method, device, electronic device and readable storage medium in the technical field of deep learning. Background technique [0002] Deep neural network is a commonly used method in machine learning and has been widely used in various fields in recent years. The training process of the deep neural network needs to use the optimizer (optimizer) to make the network converge, that is, to use the optimizer to update the network parameters to find the optimal point of the network. Therefore, the optimizer directly affects the convergence speed and training effect of the network, and a slower convergence speed will increase the consumption of computing resources. [0003] Existing technologies usually contain human experience-based optimizers and model-based optimizers. Among them, the optimizer based on artificial experience cannot...

AI Technical Summary

Problems solved by technology

Among them, the optimizer based on artificial experience cannot be adapted according to different tasks and different network structures, and it needs to spend manpower and material resources to adjust the parameters in the optimizer at different stages of training; although the model-based optimizer can do to a certain extent Adaptation, but it can only adapt to a certain fixed or similar network structure and the same type of tasks, but does not have the generalization ability for different network structures and different types of tasks

Images