Searching method for high-energy-efficiency neural network architecture

Abstract

The invention discloses a searching method for a high-energy-efficiency neural network architecture. The searching method comprises the following steps: step 1, constructing a polynomial regression model of hierarchical operation time and power; step 2, calculating the overall energy consumption of the neural network architecture based on the operation time and power of the hierarchy; step 3, carrying out serialization on the discrete search space; and step 4, adding the energy consumption as one of search targets into a neural network architecture search process. According to the method, the high-energy-efficiency network architecture is accurately found in a machine search mode, and unnecessary search overhead is reduced. In the measurement of the energy consumption of the network architecture, the energy consumption of a specific architecture is predicted by using a polynomial regression model; in the architecture design process, the architecture meeting the requirements is automatically searched by using a machine instead of manual work, and the design process is more scientific; and a continuous search space and a search method based on gradient descent are utilized, a high-energy-efficiency target is newly added on the basis that only a high-precision neural network architecture is searched originally, and search results are optimized while search efficiency is improved.

Description

technical field [0001] The invention relates to the field of machine learning, in particular to a search method for a high-energy-efficiency neural network architecture. Background technique [0002] With the development of the Internet era, deep neural networks have shown excellent performance in solving various problems, but due to their huge scale, traditional large-scale deep neural networks (VGG, AlexNet, GoogleNet, etc.) Computational speed is often slow and energy cost is high, which poses great challenges to improve its usability in real industrial fields, especially mobile devices and environments with limited energy budgets. Therefore, there is an urgent need for a method that can automatically design a small-scale, high-energy-efficiency, and high-accuracy network architecture for specific problems. [0003] At present, Neural Architecture Search (Neural Architecture Search, hereinafter referred to as NAS) has been widely used. Early NAS mainly used reinforcement...

AI Technical Summary

Problems solved by technology

But the problem that still exists is that the search efficiency is still not improved qualitatively, and the search process requires a huge price

In addition, the searched architecture only pays attention to the prediction accuracy and ignores the energy consumption generated by its reasoning process. Often, the network with higher accuracy is more robust, and the more robust network will generate more energy consumption.

Images