Multi-Resolution NAS for Dense Prediction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing neural architecture search (NAS) methods for dense prediction tasks, such as semantic segmentation and pose estimation, fail to effectively incorporate high-resolution representations and global context, leading to performance limitations compared to hand-crafted models, due to their focus on image classification and lack of consideration for the specific requirements of dense prediction tasks.

Innovation Solution

The implementation of a high-resolution Neural Architecture Search (HR-NAS) system that utilizes a lightweight transformer with dynamic computational complexity and a multi-branch architecture to encode multiscale contextual information, maintaining high-resolution representations and efficiently exploring the search space for optimal architectures across various tasks and computation resources.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If existing NAS methods directly extend the search space designed for image classification or only search for a feature aggregation head, then the search process is simplified, but the performance of dense prediction tasks is hindered due to lack of consideration for task specificity

Engineering Contradiction:

Improvesearch space complexityVSAvoidperformance accuracy

Core Design Contradiction:

Device complexityVSReliability

Solution Approach 1:

The search space is segmented into three distinct modules: a backbone module for extracting multi-scale features, a fusion module for integrating features across scales, and a head module for task-specific predictions. This segmentation allows each module to be optimized independently for its specific function while maintaining overall system performance.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different parts of the network are designed with specialized functionalities: the backbone uses convolutional layers for local feature extraction, the fusion module employs attention mechanisms for global context integration, and the head module is customized for specific dense prediction tasks. This local optimization ensures each component performs its specific function at the highest quality level.

Inventive Principle:

Principle #3Local quality

2Device complexity

If multi-scale features are combined at the end of the network, then the network structure is simpler, but the integrity of global context and high-resolution representations is compromised

Engineering Contradiction:

Improvenetwork structure complexityVSAvoidglobal context integrity

Core Design Contradiction:

Device complexityVSLoss of information

Solution Approach 1:

The fusion module performs preliminary integration of multi-scale features before the final prediction stage. By fusing features at an intermediate stage rather than at the end, the network preserves high-resolution representations and global context information for subsequent processing, ensuring information integrity while maintaining manageable structural complexity.

Inventive Principle:

Principle #10Preliminary action

3Loss of information

If global attention strategies such as SENet or non-local networks are used to enrich image convolutional features, then the global context is improved, but the computational complexity increases

Engineering Contradiction:

Improveglobal context enrichmentVSAvoidcomputational complexity

Core Design Contradiction:

Loss of informationVSDevice complexity

Solution Approach 1:

The fusion module applies attention mechanisms selectively to key features and regions rather than processing all features uniformly. This partial application of global attention strategies provides sufficient global context enrichment for dense prediction tasks while significantly reducing the computational burden compared to applying attention to the entire feature set.

Inventive Principle:

Principle #16Partial or excessive action

4Reliability

If transformers are combined with convolutional neural network for image classification and object detection, then the performance is improved, but the computational complexity increases quadratically with the number of pixels

Engineering Contradiction:

Improveperformance accuracyVSAvoidcomputational complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The network processes images by segmenting them into multi-scale feature representations at different resolution levels. The transformer architecture is then applied to these segmented features rather than to the full-resolution image, reducing the quadratic computational complexity while maintaining the performance benefits of transformer-based global attention.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS20220391636A1Multi-resolution neural network architecture search space for dense prediction tasks

Publication Date: 2022.12.08 LEMON INC(GB)

AI summary

Systems and methods for searching a search space are disclosed. Some examples may include using a first parallel module including a first plurality of stacked searching blocks and a second plurality of stacked searching blocks to output first feature maps of a first resolution and to output second feature maps of a second resolution. In some examples, a fusion module may include a plurality of searching blocks, where the fusion module is configured to generate multiscale feature maps by fusing one or more feature maps of the first resolution received from the first parallel module with one or more feature maps of the second resolution received from the first parallel module, and wherein the fusion module is configured to output the multiscale feature maps and output third feature maps of a third resolution.