Neural Architecture Search with Hardware Cost Function
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The complexity of neural architecture search in deep neural networks is exacerbated by the need to consider hardware-specific factors, making it inefficient and difficult to optimize for specific target systems.
Innovation Solution
A hardware-conscious cost function is introduced to guide an automated architecture search, accounting for properties like latency, energy consumption, and memory bandwidth by mapping parameters to execution costs, allowing for efficient and scalable optimization even when using hardware-dependent techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If hardware-specific factors are considered in neural architecture search, then optimization for specific target systems is improved, but device complexity increases
Solution Approach 1:
The patent segments the architecture search process by separating hardware-specific cost evaluation from the general architecture search. A cost function is introduced that specifically evaluates hardware-dependent properties (latency, energy, memory bandwidth) independently, allowing the search to consider target system constraints without fundamentally complicating the overall search framework.
Solution Approach 2:
The patent introduces hardware-specific parameters (latency, energy consumption, memory bandwidth) into the cost function. By changing the evaluation parameters to include these hardware-dependent metrics, the search can be optimized for specific target systems without requiring a complete restructuring of the search process.
2Reliability
If hardware-dependent optimization techniques are used, then target system performance is improved, but loss of time in architecture search increases
Solution Approach 1:
The patent performs preliminary evaluation of hardware-specific costs by introducing a cost function that estimates latency, energy, and memory bandwidth requirements before final architecture selection. This preliminary assessment allows the search to prune unpromising architectures early, reducing the time spent on detailed hardware-dependent optimization.
Solution Approach 2:
The patent uses lightweight, approximate cost estimation models that provide quick hardware-specific evaluations without requiring exhaustive simulation. These simplified cost functions act as disposable proxies for more accurate but time-consuming hardware performance measurements, enabling fast filtering of architecture candidates.
3Adaptability or versatility
If multiple target systems are considered, then adaptability is improved, but device complexity increases
Solution Approach 1:
The patent creates a universal cost function framework that can evaluate multiple target systems using the same structural approach. The cost function is designed to accommodate different hardware platforms (mobile, embedded, cloud) by adjusting hardware-specific parameters while maintaining the same overall evaluation methodology, enabling multi-target optimization without proportionally increasing complexity.
Data Source
AI summary
A device and a computer-implemented method for a neural architecture search. A first set of values is provided for parameters that define at least one part of an architecture for an artificial neural network, wherein the part of the architecture encompasses a plurality of layers of the artificial neural network and/or a plurality of operations of the artificial neural network, wherein a first value of a function is determined for the first set of values for the parameters, said first value characterizing a property of a target system when the target system executes a task for the part of the artificial neural network that is defined by the first set of values for the parameters.