Neural Network Structure Search with Constraint Predictors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network design methods require manual expertise and numerous experiments, limiting the efficiency of neural architecture search (NAS) due to the inability to search under constraints.
Innovation Solution
A method and apparatus for generating neural networks that involve training multiple neural networks for various performance parameters, training predictors based on these networks, and determining a target neural network using the trained predictors, allowing for automatic search within a constrained search space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual design of neural networks is used, then expertise can be applied to optimize performance, but the process requires large numbers of experiments and extensive time
Solution Approach 1:
The system enables automatic neural network design through self-service mechanisms where the automated design apparatus independently performs network structure search, training, and optimization without requiring manual expert intervention for each design iteration, thereby reducing time loss while maintaining performance quality
Solution Approach 2:
The system performs preliminary actions by pre-training multiple neural networks with different structures and configurations before final selection, allowing the system to evaluate multiple candidates in advance and select the optimal network without extensive experimentation during the design phase
2Extent of automation
If traditional NAS methods are used, then automation is improved, but the ability to search under constraints is lost
Solution Approach 1:
The system segments the neural network design process into multiple independent predictors, each responsible for evaluating specific performance parameters or constraints. This segmentation allows the automated system to handle multiple constraints simultaneously while maintaining high automation levels
Solution Approach 2:
The system changes parameters by adjusting the weights and priorities of different performance parameters and constraints dynamically during the search process, allowing flexible adaptation to different constraint scenarios while maintaining automated operation
3Adaptability or versatility
If multiple performance parameters are evaluated, then comprehensive performance assessment is achieved, but the complexity of the search process increases
Solution Approach 1:
The system segments the evaluation of multiple performance parameters into separate specialized predictors, where each predictor focuses on specific parameters. This reduces the complexity of the overall search process by dividing it into manageable, independent evaluation tasks that can be executed in parallel
Solution Approach 2:
The system creates universal predictors that can evaluate multiple performance parameters through a unified framework, allowing the same predictor structure to handle different parameters (accuracy, latency, model size, etc.) without requiring separate complex evaluation processes for each parameter
Data Source
AI summary
The present document relates to a method and an apparatus for generating a neural network. The method for generating a neural network according to the present document includes: training a plurality of neural networks for a plurality of performance parameters to obtain a plurality of parameter values for each performance parameter; training a plurality of neural network predictors based on the parameter values and the neural networks; and determining a target neural network using trained neural network predictors. According to the technique for generating a neural network herein, automatic searching for a network structure satisfying a preset constraint is enabled in a search space of network structures.