Shared Dense Robot Vision Network With Task-Specific Heads
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robotic systems require separate networks for each robot vision task, leading to inefficiencies in computation and training, as well as increased complexity and resource usage, making them less flexible and more costly to maintain and update.
Innovation Solution
A shared dense network, such as a Feature Pyramid Network (FPN), is combined with task-specific heads to perform different robot vision tasks, allowing the heavy computation to be done once and enabling quick training of task-specific heads for various tasks, reducing the need for frequent retraining of the dense network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate networks are used for each robot vision task, then task-specific performance is improved, but device complexity and training time increase
Solution Approach 1:
The patent merges multiple task-specific networks into a single shared dense network that processes image data for multiple robot vision tasks simultaneously. This consolidation reduces the overall number of networks from many separate task-specific networks to one shared network with multiple task-specific heads, directly addressing the technical contradiction by lowering device complexity while maintaining task-specific performance through specialized output heads.
Solution Approach 2:
The shared dense network is designed with multi-functionality to handle multiple different robot vision tasks (e.g., object detection, segmentation, tracking) using a single network architecture. This universal network replaces multiple specialized networks, reducing system complexity while maintaining the ability to perform various tasks through task-specific heads that branch from the shared feature extraction layers.
2Measurement precision
If separate networks are used for each robot vision task, then task accuracy is maintained, but training time and computational resources increase
Solution Approach 1:
By combining multiple task-specific networks into one shared dense network, the patent enables simultaneous training for multiple tasks in a single training process. This eliminates the need to train separate networks sequentially, significantly reducing training time while maintaining task accuracy through the task-specific heads that preserve task-related feature learning.
Solution Approach 2:
The shared dense network performs preliminary feature extraction that benefits all downstream tasks. By pre-training the shared network on multiple tasks or on a superset of features, the system establishes a strong foundation that accelerates subsequent task-specific training, reducing overall training time while maintaining high accuracy across all tasks.
3Adaptability or versatility
If multiple separate networks are deployed, then each task can be optimized independently, but processing latency increases
Solution Approach 1:
The patent merges multiple independent network processing pipelines into a single shared dense network that processes image data once for all tasks. This eliminates redundant processing of the same input data through multiple separate networks, reducing processing latency while maintaining task optimization through specialized heads that branch from the shared feature extraction layers.
4Measurement precision
If separate networks are used for each task, then task-specific features are captured, but computational resources and memory usage increase
Solution Approach 1:
The patent combines multiple task-specific networks into one shared dense network, eliminating redundant computation of common features. The shared network extracts features once that benefit all tasks, significantly reducing computational resources and memory usage compared to running multiple separate networks, while maintaining feature extraction quality through task-specific heads.
Data Source
AI summary
A method includes receiving image data representing an environment of a robotic device from a camera on the robotic device. The method further includes applying a trained dense network to the image data to generate a set of feature values, where the trained dense network has been trained to accomplish a first robot vision task. The method additionally includes applying a trained task-specific head to the set of feature values to generate a task-specific output to accomplish a second robot vision task, where the trained task-specific head has been trained to accomplish the second robot vision task based on previously generated feature values from the trained dense network, where the second robot vision task is different from the first robot vision task. The method also includes controlling the robotic device to operate in the environment based on the task-specific output generated to accomplish the second robot vision task.