Target-Data Neural Architecture Search with Dynamic Network Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks with fixed trained structures require the same calculation time and energy consumption regardless of the characteristics of the target data, limiting optimization for performance and energy efficiency.
Innovation Solution
Dynamically analyze the distribution of target data to optimize the neural network structure, sampling a trained neural network into multiple second neural networks, training them using unlabeled target data, and selecting the most suitable network based on performance and energy consumption conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a fixed trained neural network structure is used, then the network can perform inference operations, but the calculation time and energy consumption cannot be optimized for different target data characteristics
Solution Approach 1:
The patent applies dynamics by transitioning from a fixed neural network structure to a dynamic selection mechanism. Multiple pre-trained neural networks with different structures are sampled from a super-network, and the most appropriate network is selected based on the characteristics of the target data distribution, enabling the system to adapt its computational resources to match the specific requirements of different datasets.
Solution Approach 2:
The patent segments the single fixed neural network into multiple candidate networks sampled from a super-network. Each sampled network represents a different architectural configuration, allowing the system to divide the search space into discrete candidate solutions and select the optimal one for the given target data.
2Productivity
If multiple neural networks are sampled and trained, then the calculation time and energy efficiency can be optimized, but the complexity of the system increases
Solution Approach 1:
The patent applies preliminary action by pre-training a super-network and sampling multiple candidate neural networks in advance. These candidate networks are prepared and stored before actual inference operations, so that during runtime, the system only needs to select from pre-prepared options rather than training networks on-the-fly, thus reducing operational complexity.
Solution Approach 2:
The patent uses copying by creating multiple copies of neural network architectures from the super-network through sampling. Each sampled network is a copy with slightly varied parameters and structures, allowing the system to evaluate multiple candidates without having to design and train each from scratch, thereby managing complexity.
3Ease of manufacture
If neural networks are trained without labeled target data, then the training process can be simplified, but the training effectiveness may be reduced
Solution Approach 1:
The patent applies self-service through self-supervised learning where the neural networks train themselves using the target data without requiring external labeled annotations. The networks learn to perform tasks such as data reconstruction, completion, or transformation using only the unlabelled target data, thereby simplifying the data preparation process while maintaining training effectiveness.
Data Source
AI summary
A method and apparatus with neural architecture search are provided. A processor-implemented method includes obtaining target data, sampling a trained first neural network into a plurality of second neural networks, training each of the second neural networks based on a portion of the target data, and selecting a second neural network satisfying a predetermined condition among the trained second neural networks for performing an inference operation.