Efficient neural network structure searching method based on probability distribution

Abstract

The invention relates to an efficient neural network structure searching method based on probability distribution. The neural network structure obtained by searching the neural network structure has a very competitive effect in various current computer image tasks and language tasks. How to improve the efficiency of the search strategy and reduce the evaluation cost of the neural structure so as to find a better network structure in a shorter time is still an effort direction. The invention provides a probability distributed algorithm, the number of training sub-networks is greatly reduced, the neural network architecture search process is accelerated, a parameter sharing mode of searching while training is used, the sub-network evaluation cost is reduced, operation with better performance is ensured to obtain more training, and the search process of the neural network architecture is further accelerated. On the CIFAR-10, an optimal neural network structure can be searched out by using GTX1080Ti in only two GPU hours, and 2.69% of test errors are realized under the condition that the network parameter quantity is only 2.8 M. On an ImageNet data set, the precision of the network can reach 76% of top1 precision.

Description

technical field [0001] The invention relates to a method for designing a deep neural network structure in the field of artificial intelligence, in particular to an efficient neural network structure search method. Background technique [0002] Automated neural network search in the space of specified neural network architectures has attracted considerable attention over the past few years. For this reason, many people have proposed many excellent search algorithms and evaluation strategies to find the best neural architecture search (NAS). In general, the NAS framework is divided into three parts, which are search space, search strategy, and evaluation strategy. like figure 1 . [0003] The search space defines the variables of the optimization problem, the variable definitions of the neural network structure and hyperparameters are different, and the different variable scales are not the same for the difficulty of the algorithm. If we find a set of network architecture ...

AI Technical Summary

Problems solved by technology

However, this hypothesis has been verified, and the correct rate is only about 70%. The neural network structure that performed well in the early stage of training may not be the best performance when it is trained to converge.

So although it speeds up the evaluation process, it also affects the final web search results

Images