Method and device for training super network

Abstract

The invention relates to the field of artificial intelligence, and particularly discloses a method and device for training a super network. The method comprises the following steps: initializing a to-be-trained super-network and copying the initialized super-network to obtain a first super-network and a second super-network; sequentially executing multiple iterative operations, wherein the iterative operation comprises the following steps: sampling a first super network to obtain a first sub-network sequence, and out-of-order sequencing the first sub-network sequence to obtain a second sub-network sequence; training a first sub-network sequence based on the first super-network, and performing quasi-updating on the first super-network and the second super-network based on training results of the first sub-network sequence and the second sub-network sequence; and in response to determining that the difference between the performance of the first super network to be updated and the performance of the second super network to be updated does not exceed a preset range, taking the first super network to be updated as a new first super network, taking the second super network to be updatedas a new second super network, and executing the next iterative operation. The method improves the precision of the super network.

Description

technical field [0001] Embodiments of the present disclosure relate to the field of computer technology, specifically to the field of artificial intelligence technology, and in particular to a method and device for training a hypernetwork. Background technique [0002] Deep neural networks have achieved important results in many fields. The structure of a deep neural network model has a direct impact on its performance. The structure of the traditional neural network model is designed by experts based on experience, which requires rich expert knowledge, and the design cost of the network structure is relatively high. [0003] NAS (Neural Architecture Search, automatic search for network structure) uses algorithms to replace tedious manual operations to automatically search for the best neural network architecture. In one current approach, hypernetworks are trained by pre-constructing hypernetworks containing all possible model structures. Then in the actual deep learning ...

AI Technical Summary

Problems solved by technology

[0004] However, since all network structures in the supernetwork coexist, there is a mutual exclusion problem in the process of training the supernetwork

In order to take into account the superior performance of all network structures during the training process of the super network, the performance of the network structure will have a large gap with the performance of the independently trained network.

Images