Multi-Task Neural Network Parameter Segmentation for Embedded Vision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large machine learning models for autonomous vehicles are difficult to deploy in embedded vision systems due to limited computational capabilities and memory, and require real-time inference for safety, which is often slow.
Innovation Solution
An over-parameterized multi-task learning neural network architecture is trained with a shared convolutional layer factorized into a shared and task-specific component, allowing for improved optimization and generalization during training, then contracted for efficient inference.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large machine learning models are used for autonomous vehicle perception tasks, then model expressivity and joint performance improve, but inference time increases and computational requirements exceed embedded system capabilities
Solution Approach 1:
The patent segments the large model parameters into task-specific components and shared components. Each task has its own parameter subset while sharing common features through the shared component, enabling specialized performance for each task while reducing overall computational burden through parameter reuse across tasks
Solution Approach 2:
The patent implements multi-functionality by designing a unified model architecture that performs multiple perception tasks (object detection, semantic segmentation, depth estimation) simultaneously. The shared component serves all tasks universally while task-specific components provide specialized functionality, achieving both accuracy and efficiency
2Reliability
If large machine learning models are deployed in embedded vision systems, then model expressivity improves, but memory requirements exceed available embedded memory
Solution Approach 1:
The patent segments the model parameters into task-specific and shared components, storing them separately in memory. This segmentation allows the system to load only necessary parameter subsets for each task rather than loading the entire large model, reducing peak memory requirements while maintaining full model expressivity when all tasks are considered together
Solution Approach 2:
The patent changes the parameter representation by factorizing the weight matrix into task-specific and shared components. This parameter transformation reduces the total number of parameters that need to be stored in memory while preserving the model's expressive capability through the structured factorization
3Productivity
If multi-task learning is used to reduce inference time and improve joint performance, then inference speed and joint performance improve, but model parameter size increases
Solution Approach 1:
The patent segments the parameter space into task-specific and shared components, organizing complexity in a structured manner. This segmentation allows efficient inference by loading only relevant parameter subsets while maintaining the benefits of multi-task learning through the shared component that captures common patterns across all tasks
Data Source
AI summary
A computer-implemented method includes receiving data generated using at least one sensor of a vehicle; and simultaneously performing multiple different prediction tasks on the data using a multi-task neural network, wherein the multi-task neural network comprises at least one shared parameter inference matrix comprising parameters shared between the multiple different prediction tasks, and the at least one shared parameter inference matrix was over-parameterized during training into at least one shared parameter matrix and multiple task-specific parameter matrices, each of the multiple task-specific parameter matrices being associated with a different one of the multiple different tasks.