Vehicle 3D Bounding Box Estimation From Partial Image Cues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing systems for autonomous driving face inaccuracies in determining 3D outer bounding boxes of vehicles due to incomplete vehicle images, leading to low accuracy in location and orientation estimation.
Innovation Solution
An image processing method that generates 3D outer bounding box location information using two-dimensional bounding frame coordinates, wheel coordinates, and angles, irrespective of image completeness, enhancing accuracy and enabling precise determination of vehicle orientation and dimensions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a neural network is used to output vehicle information based on complete vehicle images, then the processing speed is fast, but the accuracy deteriorates when the vehicle image is incomplete
Solution Approach 1:
The patent segments the vehicle detection task into multiple independent components: 2D bounding box detection, wheel coordinate detection, and angle detection. Each component is detected separately using the neural network, and then integrated through geometric calculation to form the complete 3D outer bounding box. This segmentation allows each sub-task to be optimized independently and makes the system more robust to incomplete images, as each component can be detected from available image portions without requiring the complete vehicle to be visible.
Solution Approach 2:
The patent transitions from direct 3D bounding box detection to a multi-dimensional approach: first detecting 2D bounding box parameters, then wheel coordinates, then rotation angles, and finally integrating these across multiple dimensions to construct the 3D outer bounding box. This dimensional decomposition allows the system to accurately determine vehicle parameters even when the vehicle is not fully visible in the image, as each dimension can be inferred from partial observations.
2Measurement precision
If traditional 2D bounding box detection is used, then the implementation is simple, but the 3D location accuracy deteriorates
Solution Approach 1:
The patent segments the complex 3D localization problem into simpler sub-problems: 2D bounding box detection, wheel coordinate detection, and angle detection. Each sub-problem is solved using relatively simple neural network modules, and the results are integrated through geometric calculation. This segmentation maintains implementation simplicity while achieving accurate 3D localization, as each segment can be processed independently and then combined.
Solution Approach 2:
The patent introduces wheel coordinates and rotation angles as intermediary parameters that bridge the gap between simple 2D bounding box detection and accurate 3D localization. These intermediaries serve as additional measurements that, when combined with the 2D bounding box, enable precise calculation of the 3D outer bounding box through geometric relationships, without requiring complex direct 3D detection algorithms.
3Adaptability or versatility
If complete vehicle images are required for accurate detection, then the detection accuracy is high, but the applicability deteriorates in real-world scenarios
Solution Approach 1:
The patent applies partial action by detecting only the necessary components visible in the image (2D bounding box, wheels, and angle markers) rather than requiring the complete vehicle. The geometric calculation then integrates these partial observations to infer the complete 3D outer bounding box. This approach enables accurate detection even when parts of the vehicle are occluded or not visible, significantly improving applicability to real-world incomplete images.
Solution Approach 2:
The patent replaces the mechanical requirement of complete vehicle visibility with a geometric calculation system. Instead of relying on the physical presence of all vehicle parts in the image, the system uses geometric relationships between detected features (2D bounding box, wheel coordinates, angle markers) to mathematically reconstruct the 3D outer bounding box. This substitution of geometric reasoning for physical completeness enables accurate detection in partial view scenarios.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
This application provides an image processing method, a network training method, and a related device, and relates to image processing technologies in the artificial intelligence field. The method includes: inputting a first image including a first vehicle into an image processing network to obtain a first result output by the image processing network, where the first result includes location information of a two-dimensional 2D bounding frame of the first vehicle, coordinates of a wheel of the first vehicle, and a first angle of the first vehicle, and the first angle of the first vehicle indicates an included angle between a side line of the first vehicle and a first axis of the first image; and generating location information of a three-dimensional 3D outer bounding box of the first vehicle based on the first result. The location information of the three-dimensional 3D outer bounding box of the first vehicle is generated based on the three parameters: the location information of the two-dimensional bounding frame of the second vehicle, the coordinates of the wheel, and the first angle, thereby improving accuracy of the obtained 3D outer bounding box.