3D Vehicle Perception via 2D DNN Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current vehicle perception systems for autonomous driving rely on expensive and time-consuming manual or automated labeling for training datasets, and require a differential global positioning system (DGPS) for accurate 3D object detection, which is not feasible in normal traffic scenarios.
Innovation Solution
A vehicle perception system that uses a combination of 3D and 2D deep neural networks (DNNs) to detect and classify objects. The system generates 3D detection results from perception sensors, transforms these results into 2D bounding boxes, applies a 2D DNN for verification, calculates mean average precision (mAP) values, identifies corner cases, and re-trains the 3D DNN using these cases to improve performance without additional annotated datasets or DGPS.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual labeling is used to create ground truths for training datasets, then the accuracy of 3D object detection is improved, but the time and cost increase significantly
Solution Approach 1:
The system uses the 2D DNN to automatically generate ground truths by comparing its own predictions with the 3D DNN outputs. The 2D DNN serves itself by using its predictions to create training data, eliminating the need for manual human annotators and significantly reducing labeling time and cost while maintaining detection accuracy
Solution Approach 2:
The system implements a feedback loop where the 2D DNN's predictions are used to generate ground truths that are then fed back into the 3D DNN for retraining. This continuous feedback mechanism allows the system to improve its accuracy automatically without external intervention, resolving the contradiction between precision and time investment
2Measurement precision
If automated labeling services are used, then the cost is reduced compared to manual labeling, but the service is expensive and still requires additional infrastructure
Solution Approach 1:
The system eliminates the need for external automated labeling services by using its own 2D DNN to generate ground truths. This self-service approach removes the requirement for additional infrastructure, third-party services, or complex processing pipelines, reducing both cost and device complexity while maintaining detection accuracy
Solution Approach 2:
The 2D DNN serves multiple functions: it performs object detection, generates ground truths for training, and provides verification for the 3D DNN. This multi-functionality eliminates the need for separate automated labeling infrastructure, reducing overall system complexity and cost
3Measurement precision
If DGPS is used for 3D object detection, then the detection accuracy is improved, but the system is only feasible in controlled environments
Solution Approach 1:
The system introduces a 2D DNN as an intermediary that bridges the gap between 3D detection and ground truth generation. This intermediary component allows the system to operate in normal traffic scenarios without DGPS by using the 2D DNN's predictions to create reliable ground truths, thereby improving adaptability while maintaining detection accuracy
Solution Approach 2:
The system changes the approach from using GPS-based spatial parameters (requiring DGPS) to using image-based 2D bounding box parameters. This parameter transformation allows the system to function accurately in normal traffic scenarios without specialized positioning infrastructure, resolving the adaptability issue
4Reliability
If additional annotated training datasets are used, then the performance of 3D DNN is improved, but the cost and time for labeling increase
Solution Approach 1:
The system creates a feedback loop where the 2D DNN's predictions are automatically converted into ground truths and fed back into the 3D DNN for retraining. This feedback mechanism continuously improves the 3D DNN's performance without requiring time-consuming manual annotation, resolving the contradiction between reliability and time investment
Solution Approach 2:
The system performs preliminary action by using the 2D DNN to generate ground truths before retraining the 3D DNN. This preliminary ground truth generation eliminates the need for manual annotation during the training process, allowing the system to improve performance efficiently without time loss
Data Source
AI summary
Vehicle perception techniques include applying a 3D DNN to a set of inputs to generate 3D detection results including a set of 3D objects, transforming the set of 3D objects onto a set of images as a first set of 2D bounding boxes, applying a 2D DNN to the set of images to generate 2D detection results including a second set of 2D bounding boxes, calculating mean average precision (mAP) values based on a comparison between the first and second sets of 2D bounding boxes, identifying a set or corner cases based on the calculated mAP values, and re-training or updating the 3D DNN using the identified set of corner cases, wherein a performance of the 3D DNN is thereby increased without the use of expensive additional manually and/or automatically annotated training datasets.