RGB-D Fusion Obstacle Classification for Autonomous Driving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current obstacle classification methods in autonomous driving suffer from reduced recognition accuracy in special lighting environments, such as backlighting or dark conditions, due to the reliance on grayscale and RGB images.
Innovation Solution
An RGB-D fusion information-based obstacle target classification method that collects images through binocular cameras, generates disparity maps, and fuses depth information with RGB images using a classification model to improve recognition accuracy by selecting candidate categories based on spatial size and confidence levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If grayscale image and RGB image are used for obstacle classification, then the system structure remains simple, but the recognition accuracy deteriorates in special lighting environments
Solution Approach 1:
The patent introduces depth information as a new dimension beyond traditional 2D RGB images. By fusing RGB images with depth maps from binocular cameras, the system creates a 3D spatial understanding of obstacles, enabling accurate classification in challenging lighting conditions where 2D images fail.
Solution Approach 2:
The patent combines multiple types of image data (RGB images and depth maps) into a composite representation. This fusion of different data modalities creates a more robust feature set that maintains recognition accuracy across varying lighting environments, similar to how composite materials combine different properties for enhanced performance.
2Reliability
If depth information is integrated into classification, then recognition accuracy improves in special environments, but the device complexity increases
Solution Approach 1:
The binocular camera system serves multiple functions: it captures both RGB color information and depth information simultaneously. This multi-functionality allows the system to maintain reliability across different lighting conditions without requiring separate specialized devices for each function.
Solution Approach 2:
The patent merges the RGB image processing pipeline with the depth map processing pipeline into a unified classification system. By combining these processing streams and fusing their features, the system achieves improved reliability while managing device complexity through integrated architecture.
3Measurement precision
If RGB-D fusion is implemented, then classification accuracy improves, but the processing complexity increases
Solution Approach 1:
The system performs preliminary processing of both RGB images and depth maps separately before fusion, extracting key features in advance. This preliminary action simplifies the subsequent fusion process by reducing the dimensionality and complexity of the data that needs to be processed together, thereby managing computational complexity while maintaining accuracy.
Data Source
AI summary
An RGB-D fusion information-based obstacle target classification method includes: collecting an original image through a binocular camera within a target range, and acquiring a disparity map of the original image; collecting a color-calibrated RGB image through a reference camera of the binocular camera within the target range; acquiring an obstacle target through disparity clustering in accordance with the disparity map and the color-calibrated RGB image, and acquiring a target disparity map and a target RGB image of the obstacle target; calculating depth information about the obstacle target in accordance with the target disparity map; and acquiring a classification result of the obstacle target through RGB-D channel information fusion in accordance with the depth information and the target RGB image.