Neural network architecture searching method and system based on evolutionary computing

Abstract

The invention provides a neural network architecture searching method and system based on evolutionary computing, and the method comprises the steps: setting a target requirement through a target function according to a target requirement and a platform requirement; according to the set search space size, N undirected loop graphs are randomly generated based on the sub-network module set to serve as an evolutionary optimization network search space; under the guidance of heuristic information, in combination with pheromone dynamic volatilization and a probability path selection mechanism, searching N directed acyclic graph optimization paths in randomly generated N undirected acyclic graphs through an ant colony algorithm to form a candidate set; obtaining the accuracy and reasoning time delay of N optimization paths in the candidate set through training and testing, and selecting an optimal result as a current optimal network structure; and evaluating whether the current network architecture meets the target requirement. The method has certain application flexibility and expandability, and a neural network model with good balance between precision and speed is obtained under the condition that resources are limited.

Description

technical field [0001] The present invention relates to the technical field of architecture design and optimization of deep neural networks, in particular, to a neural network architecture search method and system based on evolutionary computation. Background technique [0002] Deep learning has a powerful automatic feature extraction function for unstructured data, and has a strong automatic representation ability, so it has made major breakthroughs and progress in many fields such as machine translation, image recognition, speech recognition, target detection, natural language processing, etc. . Based on the importance of the design of neural network architecture to the feature representation of data and the final performance, researchers focus on designing various complex neural network architectures to obtain good data feature representation. However, the design of neural network architectures relies heavily on researchers' prior knowledge and experience, requiring a lo...

AI Technical Summary

Problems solved by technology

Early NAS trained each candidate network architecture from scratch during the architecture search phase, resulting in a surge in computation; although the parameter sharing strategy was used to speed up the architecture search process, it was likely to lead to inaccurate ranking of candidate architectures, which would make NAS It is difficult to select the optimal network architecture from a large number of candidate architectures, which further reduces the performance of the final searched network architecture

The One-Shot-based differentiable neural network architecture search method relaxes the search space from discrete to continuous, so that gradient descent can be used to simultaneously search for architecture and learning weights, shortening the search time, but when the number of search rounds is too large, it will lead to The search architecture contains many skip connections, making the network shallower

Shallow networks can learn fewer parameters and have weaker expressive capabilities, resulting in a sharp decline in network performance

Although the improved differentiable neural network architecture search method uses the early stopping mechanism to directly control the number of skip connections, the timing of early stopping is an important issue, and premature stopping will lead to incomplete architecture search.

Images