BEV Multi-Task Vehicle Control for Detection and Drivable Area Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing vehicle control systems face challenges in efficiently utilizing deep neural network-based object detection and image segmentation for autonomous driving, particularly in determining drivable areas, object states, and risk assessment of surrounding vehicles and pedestrians.
Innovation Solution
A method and apparatus utilizing multi-task learning with a shared backbone network for object detection and image segmentation, generating features from a bird's eye view image to control vehicle operations based on detection and segmentation information, including risk level assignment and area classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If separate neural networks are used for object detection and image segmentation, then each task can be optimized independently, but the system complexity and computational resource consumption increase
Solution Approach 1:
The patent combines object detection and image segmentation into a single unified neural network model. The network processes input images through shared feature extraction layers, then branches into separate detection and segmentation heads. This merging approach reduces the total number of parameters and computational resources required compared to running two separate networks, while maintaining high accuracy for both tasks through multi-task learning mechanisms.
Solution Approach 2:
The unified neural network is designed to perform multiple functions simultaneously - both object detection and image segmentation. The shared backbone layers extract general features that are useful for both tasks, while task-specific heads handle the particular requirements of each function. This multi-functional design allows a single model to replace what would traditionally require separate specialized networks.
2Measurement precision
If multiple independent models are deployed for different driving tasks, then each model can achieve high precision for its specific task, but the overall processing time and computational load increase
Solution Approach 1:
The patent merges multiple task-specific models into a single integrated neural network that processes inputs once and generates multiple outputs simultaneously. The shared feature extraction backbone processes the input image a single time, then the features are distributed to different task heads (detection, segmentation, etc.), eliminating the need for multiple separate forward passes through independent models and significantly reducing processing time.
Solution Approach 2:
The unified network enables continuous processing where a single input image flows through the network once to produce all required outputs in one continuous operation. This eliminates the gaps and redundant processing that would occur if multiple independent models were sequentially applied, maintaining continuous useful action from input to all outputs without interruption or repetition.
3Reliability
If comprehensive object detection and segmentation are performed for all scenarios, then safety and control accuracy improve, but the computational resources and system complexity increase
Solution Approach 1:
The patent combines comprehensive detection and segmentation capabilities into a single efficient neural network architecture. By sharing computational resources through common backbone layers and only performing specialized processing where needed in the task-specific heads, the system achieves high reliability for safety-critical functions while minimizing redundant computations and energy consumption compared to running multiple separate comprehensive models.
Data Source
AI summary
A vehicle control method includes generating, using backbone associated with a neural network, a feature based on a bird's eye view (BEV) image obtained by a vehicle, generating, based on the feature and using a first neck associated with the neural network for object detection, detection information indicating a detection result for an object, generating, based on the feature and using a second neck associated with the neural network for image segmentation, a segmentation image, and controlling, based on the detection information and the segmentation image, the vehicle.