A search method for energy-efficient neural network architectures

Abstract

A search method for a high-energy-efficiency neural network architecture, including: Step 1: Construct a polynomial regression model of hierarchical running time and power; Step 2: Calculate the overall energy consumption of the neural network architecture based on the hierarchical running time and power; Step 3: Discrete Continuous search space; Step 4: Add energy consumption as one of the search objectives to the neural network architecture search process. The present invention accurately discovers high-energy-efficiency network architectures by means of machine search and reduces unnecessary search overhead. In the measurement of network architecture energy consumption, the polynomial regression model is used to predict the energy consumption of a specific architecture; in the architecture design process, machines are used instead of manual methods to automatically search for architectures that meet the requirements, making the design process more scientific; using continuous search space And the search method based on gradient descent, and on the basis of only searching for high-precision neural network architecture, a high-energy-efficiency target is added to improve search efficiency and optimize search results.

Description

technical field [0001] The invention relates to the field of machine learning, in particular to a search method for an energy-efficient neural network architecture. Background technique [0002] With the development of the Internet era, deep neural networks have shown excellent performance in solving various problems, but due to their huge scale, traditional large-scale deep neural networks (VGG, AlexNet, GoogleNet, etc.) Often computationally slow and energy-intensive, it poses a huge challenge to improve its usability in real-world industrial applications, especially in mobile devices and in environments with limited energy budgets. Therefore, there is an urgent need for a method that can automatically design a small-scale, energy-efficient, and high-accuracy network architecture for specific problems. [0003] At present, the application of neural network architecture search (Neural Architecture Search, hereinafter referred to as NAS) has been quite extensive. The early ...

AI Technical Summary

Problems solved by technology

But the problem that still exists is that the search efficiency is still not improved qualitatively, and the search process requires a huge price

In addition, the searched architecture only pays attention to the prediction accuracy and ignores the energy consumption generated by its reasoning process. Often, the network with higher accuracy is more robust, and the more robust network will generate more energy consumption.

Images