Lightweight Iterative Neural Architecture Search Protocol
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural architecture search (NAS) protocols are computationally intensive and time-consuming due to the need to validate each architecture individually, making them impractical for efficient hardware-aware model optimization.
Innovation Solution
The Lightweight Iterative Neural Architecture Search (LINAS) protocol, which iteratively samples and validates a small number of sub-network architectures, uses weakly trained predictors to perform an evolutionary algorithm, reducing the number of validations needed by selecting architecture configurations that align with objectives through an iterative process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional NAS protocols validate each architecture individually, then architecture search accuracy is improved, but computational resources and time consumption increase significantly
Solution Approach 1:
The patent applies partial action by validating only a small subset of architectures (e.g., 10-50 out of thousands) rather than exhaustively validating all candidate architectures. This partial validation approach, combined with weakly-trained predictors, achieves sufficient architecture selection accuracy while dramatically reducing computational resource consumption and validation time.
Solution Approach 2:
The patent uses weakly-trained predictors as simplified copies or surrogates for full architecture validation. These predictors are trained on a small subset of validated architectures and then used to estimate performance of unvalidated architectures, replacing the need for expensive individual validation of each architecture while maintaining reasonable accuracy.
2Measurement precision
If conventional NAS protocols validate each architecture individually, then architecture search accuracy is improved, but search time increases significantly
Solution Approach 1:
The patent applies partial action by validating only a small subset of architectures (e.g., 10-50 out of thousands) rather than exhaustively validating all candidate architectures. This partial validation approach, combined with weakly-trained predictors, achieves sufficient architecture selection accuracy while dramatically reducing computational resource consumption and validation time.
Solution Approach 2:
The patent performs preliminary action by training weak predictors on a small subset of validated architectures before using them to evaluate the full search space. This preliminary training enables rapid estimation of architecture performance without requiring individual validation of each candidate, significantly reducing total search time.
3Use of energy by moving object
If LINAS uses weakly-trained predictors with small subgroup validation, then computational resources are reduced, but validation accuracy may decrease
Solution Approach 1:
The patent introduces weakly-trained predictors as intermediary elements between the small set of validated architectures and the large search space. These predictors serve as mediators that translate limited validation data into performance estimates for unvalidated architectures, enabling resource-efficient search while maintaining acceptable accuracy through iterative refinement.
Solution Approach 2:
The patent implements feedback by iteratively refining the weak predictors using performance data from validated architectures. The predictors are continuously improved based on feedback from actual validation results, allowing the system to maintain or improve accuracy while using fewer computational resources across multiple search iterations.
Data Source
AI summary
Methods, apparatus, systems, and articles of manufacture to iteratively search for an artificial intelligence-based architecture are disclosed. An example apparatus includes an interface to access a first subgroup of architecture configurations from a search space; instructions; and processor circuitry to execute the instructions to: train first predictors based on the first subgroup; generate a first plurality of candidate architecture configurations using the trained first predictors; and generate a second subgroup of architecture configurations by selecting a number of the plurality of candidate architecture configurations; train second predictors based on the first subgroup and the second subgroup; and generate a second plurality of candidate architecture configurations using the trained second predictors.