Automated Proxy Task Design for Neural Architecture Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Designing proxy tasks for neural architecture search is time-consuming and costly, with existing methods requiring significant engineering effort and resources, and lacking efficient tools to compare and select optimal proxy tasks.
Innovation Solution
The development of automated proxy task design tools that determine optimal proxy tasks by evaluating correlation candidate models, generating correlation scores, and ranking proxy task choices based on training time, thereby reducing the neural architecture search cost and computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If manual proxy task design and validation is performed, then engineering control and customization are maintained, but time consumption and engineering effort increase significantly
Solution Approach 1:
The system enables automated self-service for proxy task design through the Proxy Task Designer tool, which automatically generates, validates, and compares proxy task choices without requiring manual engineering intervention. The tool autonomously performs full training runs, computes correlation scores, and ranks proxy tasks based on their effectiveness.
Solution Approach 2:
The patent replaces the manual mechanical process of proxy task design with an automated computational system. The Proxy Task Designer tool substitutes human engineers' manual validation and comparison work with automated machine learning processes that compute correlation scores and evaluate proxy task performance.
2Measurement precision
If comprehensive validation and comparison of proxy task choices is performed, then selection accuracy is improved, but computing resources and processing power are consumed
Solution Approach 1:
The system creates simplified copies of the full training process through proxy tasks. Instead of performing exhaustive validation of all possible proxy tasks with full training runs, the system uses correlation candidate models that are trained on full datasets to create reference scores, then uses these copies to efficiently evaluate multiple proxy task choices through reduced training runs.
Solution Approach 2:
The patent implements partial action by performing reduced training runs for evaluating proxy task choices. The system trains correlation candidate models for a fraction of the full training time (e.g., 1-10% of full training steps) to compute correlation scores, which provides sufficient evaluation accuracy without consuming resources required for complete training of all candidate models.
3Loss of energy
If reduced training runs are used for proxy tasks, then computing cost is reduced, but correlation accuracy with full training may deteriorate
Solution Approach 1:
The system implements feedback mechanisms by computing correlation scores between proxy task results and full training results. The Proxy Task Designer tool performs full training runs on a subset of correlation candidate models to establish reference scores, then uses these feedback references to evaluate and rank reduced training runs of other proxy tasks, ensuring correlation accuracy is maintained.
Solution Approach 2:
The patent applies preliminary action by first training correlation candidate models for full training duration to establish accurate reference scores before using these references to evaluate proxy tasks with reduced training. This preliminary full training creates a baseline that enables accurate correlation computation for subsequent reduced training runs.
4Reliability
If multiple correlation candidate models are trained for full training, then evaluation reliability is improved, but training time and computational resources increase
Solution Approach 1:
The system segments the model evaluation process into two distinct phases: (1) Training a limited number of correlation candidate models for full training duration to establish reference scores, and (2) Using these references to evaluate multiple proxy task choices with reduced training runs. This segmentation allows reliable correlation scoring without the computational burden of fully training all candidate models.
Data Source
AI summary
Aspects of the disclosure are directed to proxy task design tools that automatically find proxy tasks, such as optimal proxy tasks, for neural architecture searches. The proxy task design tools can include one or more tools to search for an optimal proxy task having the lowest neural architecture search cost while meeting a minimum correlation requirement threshold after being provided with a proxy task search space definition. The proxy task design tools can further include one or more tools to select candidate models for computing correlation scores of proxy tasks as well as one or more tools to measure variance of a model. The proxy task design tools can minimize time and effort involved in designing the proxy task.