Binocular-LiDAR Object Detection for Sparse Point Cloud Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Lidar-based object detection is affected by weather conditions and has limited detection distance, leading to reduced accuracy and difficulty in implementing accurate object detection due to sparse point cloud data.
Innovation Solution
An object detection method that combines binocular images from a dual-lens camera with laser point cloud data from a lidar, converting two-dimensional coordinates into three-dimensional coordinates in a camera coordinate system and merging them to enhance detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If lidar is used for object detection, then three-dimensional point cloud image can be accurately restored, but detection accuracy is reduced in adverse weather conditions and at limited detection distances
Solution Approach 1:
The patent merges binocular camera image data with lidar point cloud data to create a combined detection system. The binocular camera provides visual information that complements the sparse lidar point cloud, especially in adverse weather conditions where lidar performance degrades. This combination allows the system to maintain accurate object detection by utilizing the strengths of both sensing modalities.
2Measurement precision
If lidar is used for object detection, then three-dimensional spatial information can be obtained, but point cloud data becomes sparse making detection difficult
Solution Approach 1:
The patent combines dense image data from binocular cameras with sparse lidar point cloud data. The binocular vision system generates rich visual features and dense pixel information that compensates for the sparsity of lidar point cloud data, enabling more reliable object detection while preserving accurate three-dimensional spatial information.
Solution Approach 2:
The patent uses coordinate transformation and data fusion algorithms as intermediaries to integrate data from binocular cameras and lidar. These intermediary processing steps align the data from different sources into a unified coordinate system and fuse them effectively, allowing the dense visual data to supplement the sparse point cloud data for improved detection.
3Reliability
If only lidar data is used, then three-dimensional detection capability is provided, but detection reliability is low in adverse weather
Solution Approach 1:
The patent merges lidar and binocular camera systems to create a multi-sensor detection platform that adapts to various weather conditions. When lidar performance degrades in fog, rain, or snow, the binocular camera provides alternative visual detection capabilities, maintaining overall system reliability across different environmental conditions.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Improves object detection accuracy by integrating visual point cloud data from binocular images with laser point cloud data, addressing the limitations of lidar in adverse weather conditions and sparse data.
Implementation Method 1
A computing device determines an object region of binocular image, obtains two-dimensional coordinates of m pixel pairs from the object region of the binocular image
Implementation Method 2
The laser point cloud data is data obtained by scanning the to-be-detected object by using a lidar
Implementation Method 3
the lidar is easily affected by weather conditions and has a limited detection distance
Data Source
Figure 1A~1B
Figure 2
Figure 3~4
AI summary
Embodiments of the present invention disclose an object detection method, including: A computing device obtains two-dimensional coordinates of m pixel pairs from an object region of binocular image, where the binocular images include a first image and a second image that are captured by using a dual-lens camera, and one pixel in each pixel pair is from the first image, and the other pixel in each pixel pair is from the second image, and pixels in each pixel pair respectively correspond to a same feature of the object region in each of the first image and the second image; determines three-dimensional coordinates of the to-be-detected object in a camera coordinate system based on the two-dimensional coordinates of the m pixel pairs; converts the three-dimensional coordinates into data in a coordinate system in which laser point cloud data of the to-be-detected object is located, and merges the data and the laser point cloud data into merged point cloud data; and further determines the to be detected object based on the merged point cloud data. By implementing the embodiments of the present invention, problems in the prior art such as low accuracy of object detection or difficulty in implementing object detection can be resolved.