Neural Network Edge Candidate Selection for Processing Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network architecture search techniques, such as gradient-based Neural Architecture Search (NAS), achieve sufficient inference accuracy but often result in increased processing amounts, which can lead to higher costs and longer computation times.
Innovation Solution
An information processing apparatus that generates multiple candidates for neural network edges, calculates losses based on specified candidate numbers, updates weight coefficients, and selects candidates to maintain inference accuracy while reducing processing amounts by ensuring a significant difference in weights between selected and unselected edges.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If gradient-based Neural Architecture Search is used to achieve sufficient inference accuracy, then the inference accuracy is improved, but the processing amount and computation time increase
Solution Approach 1:
The patent extracts and eliminates redundant computation in gradient-based NAS by identifying that only a subset of architecture parameters actually influence the final network structure. By extracting and removing the redundant gradient calculations, the method maintains inference accuracy while significantly reducing processing requirements.
Solution Approach 2:
Instead of directly optimizing all architecture parameters through gradient descent, the patent inverts the approach by first identifying which parameters are redundant and then optimizing only the necessary subset. This inversion transforms a computationally intensive full-parameter optimization into an efficient partial-parameter optimization.
2Measurement precision
If more candidates for neural network edges are evaluated, then the inference accuracy is improved, but the processing amount and memory usage increase
Solution Approach 1:
The patent segments the set of edge candidates into multiple groups or batches rather than evaluating all candidates simultaneously. This segmentation allows the system to process candidates in manageable portions, reducing peak memory usage while still evaluating sufficient candidates to achieve high inference accuracy.
Solution Approach 2:
The patent applies partial action by evaluating a strategically selected subset of edge candidates rather than all possible candidates. By using the weight difference criterion to identify the most promising candidates, the method achieves sufficient accuracy with fewer evaluations, thereby reducing memory requirements.
3Measurement precision
If the number of selected candidates is increased, then the inference accuracy is improved, but the processing speed decreases
Solution Approach 1:
The patent performs preliminary filtering of edge candidates based on weight differences before the final selection stage. By pre-identifying and ranking candidates according to their weight significance, the system can quickly select the top K candidates without performing exhaustive evaluations, thus maintaining speed while ensuring accuracy.
Solution Approach 2:
The patent changes the selection criterion from considering all candidates equally to using weight difference as a discriminating parameter. This parameter change enables the system to prioritize candidates with larger weight differences, achieving a better balance between the number of selected candidates and processing speed.
Data Source
AI summary
The present disclosure makes it possible to learn a neural network architecture for achieving a sufficient inference accuracy while preventing an increase in the amount of processing. An information processing apparatus configured to learn an architecture for optimizing a structure of a neural network generates a plurality of candidates for an edge of the neural network, inputs learning data to the neural network with weight coefficients set to these candidates for the edge, and obtains an inference result. The information processing apparatus calculates a loss of the neural network based on a specified candidate number which is the number of candidates to be selected from the plurality of candidates and on the inference result, and then updates the weight coefficients for the plurality of candidates based on the loss. The information processing apparatus then selects candidates from the plurality of candidates based on the updated weight coefficients.