Joint NAS and Multi-Task Dense Prediction Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for multi-task dense predictions (MT-DP) are not optimized for edge computing devices due to limited computational resources and hardware heterogeneity, and existing neural architecture search (NAS) techniques are not suited for complex tasks like MT-DP, leading to inefficiencies in dense prediction models.
Innovation Solution
Joint training and optimization of MT-DP and hardware-aware NAS models, using a base architecture template and sampling neural network components to create candidate architectures that are optimized for specific hardware constraints and tasks, with performance metrics guiding the selection and deployment of models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing MT-DP techniques are used to train large models with powerful computational resources, then model accuracy is improved, but the models are not suitable for edge applications with limited computational resources
Solution Approach 1:
The patent applies parameter changes by modifying the architecture parameters of neural networks to be suitable for edge devices. The system searches for optimal architecture parameters (such as layer configurations, filter sizes, and connectivity patterns) that maintain acceptable accuracy while reducing computational resource consumption. This is achieved through automated architecture search that evaluates multiple candidate architectures against both accuracy metrics and resource constraints.
Solution Approach 2:
The patent segments the model development process into distinct phases: architecture search, candidate generation, evaluation, and selection. The architecture search space is segmented into multiple candidate architectures that can be independently evaluated. This segmentation allows the system to explore different architectural configurations and select those that best balance accuracy and resource efficiency for edge deployment.
2Productivity
If existing NAS techniques focus on simpler tasks like classification, then the search process is faster and easier, but the techniques are not suited for complex tasks like multi-task dense prediction
Solution Approach 1:
The patent implements universality by designing a unified architecture search framework that can handle multiple task types simultaneously. The system searches for architectures that can perform multi-task dense prediction (including semantic segmentation, depth estimation, and surface normal prediction) within a single unified model. This multi-functional approach allows the same architecture search methodology to be applied across different task complexities, from simple classification to complex dense prediction tasks.
Solution Approach 2:
The patent applies dynamics by making the architecture search process adaptive to different task requirements. The search algorithm dynamically adjusts the evaluation criteria and architecture candidates based on the specific task demands. For complex multi-task dense prediction, the system dynamically evaluates architectures based on their ability to handle multiple tasks simultaneously, rather than using static evaluation metrics designed for simpler tasks.
3Manufacturing precision
If machine learning models are designed for one hardware platform, then optimization for that platform is achieved, but the models do not generalize to other hardware platforms
Solution Approach 1:
The patent applies universality by creating hardware-agnostic architecture representations during the search phase. The architecture search framework evaluates candidates based on hardware-abstracted metrics rather than platform-specific performance characteristics. This allows the same architecture to be deployed across different hardware platforms (mobile devices, embedded systems, cloud servers) without requiring retraining or significant modification, achieving both optimization and generalization.
4Use of energy by moving object
If dense prediction models are designed for edge devices, then resource efficiency is improved, but the computational expense of pixel-level predictions remains high
Solution Approach 1:
The patent applies parameter changes by optimizing architectural parameters specifically for resource efficiency in dense prediction tasks. The architecture search modifies parameters such as the number of layers, filter dimensions, and feature map resolutions to reduce computational complexity while maintaining the ability to perform pixel-level predictions. The system identifies parameter configurations that achieve acceptable accuracy with significantly reduced computational expense compared to standard dense prediction architectures.
Data Source
AI summary
Implementations are described herein for performing joint optimization of multi-task learning of dense predictions (MT-DP) and hardware-aware neural architecture search (NAS). In various implementations, a set of tasks to be performed using a resource-constrained edge computing system may be determined. Based on a base multi-task dense-prediction (MT-DP) architecture template, the set of tasks, and a plurality of hardware-based constraints of a target edge computing system, a network architecture search (NAS) may be used to sample candidate MT-DP architecture(s) from a search space of neural network architecture components. Each sampled candidate MT-DP architecture may include a distinct assembly of sampled neural network architecture components applied to the base MT-DP architecture template. Image data may be processed using the candidate MT-DP architecture(s) to determine performance metrics. These performance metrics may be used to jointly train the MT-DP architecture(s) and/or the NAS.