Hierarchical Neural Architecture Search With Semantic Meta-Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Neural Architecture Search (NAS) methods are inefficient and inflexible when applied to multiple tasks or large-scale datasets, requiring significant computational resources and often result in suboptimal architectures due to the lack of scalability and transfer performance.
Innovation Solution
A hierarchical hybrid neural architecture search method that utilizes semantic clustering and top-k meta-blocks search to identify optimal architectures by combining meta-blocks and scaling neural network models based on meta-features, using a proxy set of a large-scale dataset and various search strategies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing NAS methods are applied to large-scale datasets, then architecture search capability is provided, but computational requirements become enormous and efficiency deteriorates
Solution Approach 1:
The patent segments the architecture search process into multiple hierarchical levels: (1) searching for meta-blocks at the block level, (2) assembling meta-architectures from meta-blocks, and (3) fine-tuning at the full architecture level. This segmentation allows the search to be performed on smaller, more manageable subsets of the architecture space, significantly reducing computational requirements while maintaining search effectiveness.
Solution Approach 2:
The patent performs preliminary actions by pre-defining a set of meta-blocks with specific functions (e.g., convolutional blocks, attention blocks, normalization blocks) before the actual architecture search. These pre-defined meta-blocks serve as building blocks that constrain the search space, allowing the NAS method to efficiently explore architecture combinations without evaluating every possible architecture from scratch, thus improving search efficiency on large-scale datasets.
2Measurement precision
If existing NAS methods are applied to multiple tasks, then architecture optimization is attempted, but the method becomes inflexible and requires searching from scratch for each new task
Solution Approach 1:
The patent creates a universal meta-architecture framework that can be adapted to multiple tasks through the use of task-configurable parameters. The meta-architecture defines a standardized structure with placeholder components that can be instantiated differently for each task. This allows a single meta-architecture design to serve multiple purposes across different datasets and tasks, eliminating the need to search from scratch for each new task while maintaining task-specific optimization capability.
Solution Approach 2:
The patent enables adaptation to multiple tasks by changing parameters within the meta-architecture framework rather than changing the entire architecture. Task-specific parameters such as depth, width, resolution, and data augmentation settings can be modified to suit different tasks while retaining the core meta-architecture structure. This parameter-based adaptation provides flexibility across tasks without requiring complete re-searching of the architecture space.
3Measurement precision
If a sophisticated architecture is used for difficult data samples, then accuracy is improved, but architecture-related costs increase
Solution Approach 1:
The patent introduces dynamic architecture selection where the model complexity is adjusted based on the difficulty of the input sample. For difficult samples that require higher accuracy, the system dynamically activates more sophisticated architecture components (deeper networks, attention mechanisms). For easier samples, a simpler version of the architecture is used, reducing inference costs. This dynamic adaptation allows the system to optimize the trade-off between accuracy and computational cost on a per-sample basis.
Data Source
AI summary
Disclosed herein is an apparatus and method for a hierarchical hybrid neural architecture search. The apparatus assigns a data subset per cluster by retrieving semantic clustering information with meta-feature information similar to target data for a neural architecture search from a large-scale dataset, determines optimal meta-blocks by performing top-k meta-blocks search in the data subset assigned per cluster, scales the size of a neural network model, and determines a neural network architecture based on semantic information of meta-features obtained by combining the optimal meta-blocks.