Hardware Neural Architecture Co-Search Using Pre-Trained Model Zoo
Solution Overview
Problem
Current hardware and neural architecture co-search frameworks are inefficient, requiring hundreds of GPU hours and resulting in high carbon footprints, due to starting from scratch in the design process, which hampers fast turn-around and time-to-market in emerging applications.
Innovation Solution
A computer program that performs hardware and neural architecture co-search by obtaining specifications and design parameters, determining latency of neural architectures within given hardware constraints, and selecting architectures based on latency and accuracy, utilizing a model zoo of pre-trained neural networks to reduce search time and improve efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If co-search frameworks start exploration from scratch (cold start), then they can achieve comprehensive search coverage, but search time increases to hundreds of GPU hours
Solution Approach 1:
The patent applies preliminary action by pre-training multiple neural architectures with different configurations before the actual co-search process. These pre-trained architectures serve as starting points that already have optimized weights and structures, eliminating the need to start from random initialization. This preliminary preparation significantly reduces the search time required during the actual co-search phase while maintaining comprehensive search coverage.
Solution Approach 2:
The patent creates copies of pre-trained neural architectures with different configurations and uses these copies as initial points for the co-search process. Instead of training one architecture from scratch, multiple copied variants are explored in parallel, which accelerates the search process while maintaining diversity in the search space exploration.
2Measurement precision
If co-search frameworks perform exhaustive search, then they achieve high accuracy systems, but carbon footprint increases significantly
Solution Approach 1:
By performing preliminary pre-training of multiple neural architectures before the co-search process, the patent reduces the amount of training required during the actual search phase. This preliminary action ensures that high accuracy is achieved with significantly reduced computational resources during co-search, thereby lowering the carbon footprint while maintaining the ability to discover high-accuracy systems.
Solution Approach 2:
The patent changes key parameters of the search process by using pre-trained architectures as starting points and exploring their neighborhoods through controlled modifications. This parameter-based exploration approach achieves high accuracy systems with fewer iterations compared to exhaustive search, thereby reducing energy consumption and carbon footprint while maintaining measurement precision.
3Adaptability or versatility
If co-search frameworks use default cold start settings, then they ensure thorough exploration, but time-to-market increases for emerging applications
Solution Approach 1:
The patent performs preliminary pre-training of multiple neural architectures with diverse configurations before the co-search process. This preliminary action creates a ready pool of architectures that can be quickly adapted and fine-tuned during the actual search, enabling thorough exploration of the design space while significantly reducing the time-to-market for emerging applications that require fast deployment.
Data Source
AI summary
Hardware and neural architecture co-search may be performed by operations including obtaining a specification of a function and a plurality of hardware design parameters. The hardware design parameters include a memory capacity, a number of computational resources, a communication bandwidth, and a template configuration for performing neural architecture inference. The operations further include determining, for each neural architecture among a plurality of neural architectures, an overall latency of performance of inference of the neural architecture by an accelerator within the hardware design parameters. Each neural architecture having been trained to perform the function with an accuracy. The operations further include selecting, from among the plurality of neural architectures, a neural architecture based on the overall latency and the accuracy.