Coarse-to-Fine Neural Architecture Search for Hardware Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network and hardware architecture design methods are inefficient and not optimized for specific users or use cases, as they rely on manual design approaches that are time-consuming and do not consider varying performance and power consumption metrics.
Innovation Solution
A coarse-to-fine neural architecture search method using a controller-based search algorithm for selecting coarse dimensions and a differential search algorithm for fine dimensions, combined with a two-phase block distillation method and neural hardware predictor techniques to optimize both neural network and hardware architectures based on predicted PPA parameters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If manual design approaches are used for hardware architectures, then working solutions can be provided, but the designs are not optimized for different users or use cases and do not consider varying performance and power consumption metrics
Solution Approach 1:
The patent employs neural architecture search (NAS) to automatically explore and optimize neural network parameters (architecture, operations, coefficients) and hardware parameters (memory size, topology, configuration) to find optimal designs tailored to specific performance and power consumption requirements for different use cases
Solution Approach 2:
The system uses automated algorithms (controller-based search, differential evolution) to perform design optimization without manual intervention, allowing the design process to self-adjust and converge to optimal solutions based on specified constraints and objectives
2Productivity
If manual trial-and-error design approaches are used, then hardware architectures can be designed, but the process is time-consuming and not efficient
Solution Approach 1:
The patent replaces manual trial-and-error design processes with automated computational algorithms including controller-based search algorithms and differential evolution, which systematically explore the design space and converge to optimal solutions much faster than manual methods
Solution Approach 2:
The system performs preliminary optimization by pre-defining design spaces, constraints, and objective functions before the actual design process, enabling the automated algorithms to efficiently navigate toward optimal solutions without exhaustive manual trial-and-error
3Speed
If coarse-to-fine search with two-phased block distillation is used, then convergence speed improves, but the algorithm complexity increases
Solution Approach 1:
The patent divides the neural architecture search process into distinct phases: controller-based search for coarse-grained architecture exploration and differential evolution for fine-grained parameter optimization. This segmentation allows each phase to focus on specific optimization tasks, improving overall convergence efficiency
Solution Approach 2:
The patent introduces a neural hardware predictor as an intermediary component that estimates hardware performance metrics during the search process, allowing the optimization algorithms to make informed decisions without requiring exhaustive actual hardware evaluations, thus accelerating convergence
Data Source
AI summary
Methods, systems, and apparatus for combined or separate implementation of coarse-to-fine neural architecture search (NAS), two-phase block NAS, variable hardware prediction, and differential hardware design are provided and described. A variable predictor is trained, as described herein. Then, a controller or policy may be used to iteratively modify a neural network architecture along dimensions formed by neural network architecture parameters. The modification is applied to blocks (e.g., subnetworks) within the neural network architecture. In each iteration, the remainder of the neural network architecture parameters are modified and learned with a differential NAS method. The training process is performed with two-phase block NAS and incorporates a variable hardware predictor to predict power, performance, and area (PPA) parameters. The hardware parameters may be learned as well using the variable hardware predictor.