LiDAR-Camera Fusion for Complex-Shape Object Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object detection methods struggle to accurately recognize objects with complex shapes or large sizes due to issues like over-segmentation and non-uniform point cloud distribution, especially in autonomous driving environments.
Innovation Solution
A sensor fusion system using a LiDAR detector for object segmentation, a camera detector for image recognition, and a fusion recognition unit that employs graph-based probability optimization to combine LiDAR and camera data, ensuring accurate object recognition by clustering point clouds and associating object labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LiDAR sensor is used for object recognition, then three-dimensional point cloud data can be obtained, but over-segmentation occurs causing one object to be divided into several point groups
Solution Approach 1:
The patent merges camera image data with LiDAR point cloud data through sensor fusion. The camera provides contextual information about object boundaries and semantics, while the LiDAR provides precise three-dimensional spatial information. By combining these complementary data sources, the system resolves the over-segmentation problem where LiDAR alone divides single objects into multiple point groups, achieving complete and accurate object recognition.
Solution Approach 2:
The patent introduces camera data as an intermediary to bridge the gap in object recognition. The camera captures two-dimensional image information that provides semantic context and boundary information, which serves as a mediator to guide the correct grouping of LiDAR point cloud data. This intermediary information helps distinguish between separate objects and different parts of the same object, preventing erroneous over-segmentation.
2Measurement precision
If stereo camera is used for object detection, then distance information can be obtained, but distance information is non-uniform for objects with irregular shapes including concave or convex parts
Solution Approach 1:
The patent combines stereo camera data with LiDAR point cloud data to compensate for the non-uniform distance information problem. While the stereo camera provides distance information for visible surfaces, it fails to provide uniform distance data for concave or convex parts. The LiDAR sensor complements this by providing uniform three-dimensional distance measurements across all object surfaces, including hidden and irregular areas, through active laser scanning.
3Device complexity
If existing object detection methods are used for complex-shaped objects, then detection process can be simplified, but accurate recognition cannot be achieved
Solution Approach 1:
The patent segments the object recognition problem into two distinct components: visual feature extraction from camera images and spatial structure extraction from LiDAR point clouds. By dividing the complex recognition task into these manageable segments that can be processed independently and then fused, the system achieves accurate recognition of complex-shaped objects without requiring an overly complicated unified detection method.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system effectively recognizes complex-shaped and large-sized objects by integrating LiDAR and camera data, providing precise size, shape, direction, and type information for autonomous driving path planning.
Implementation Method 1
A general method of recognizing an object from information obtained from the LiDAR sensor is to segment points for each object. The information from one or more LiDAR sensors are data expressed as a point cloud (or point group) in a three-dimensional space, which is obtained by emitting a laser and analyzing light reflected from a surface of the subject.
Implementation Method 2
Information obtained from a camera is the intensity appearing in three wavelength bands of the visible light band of a subject projected in the direction perpendicular to the camera.
Implementation Method 3
The radar sensor analyzes an electromagnetic wave reflected from the surface of the subject by emitting an electromagnetic wave instead of laser of the LiDAR sensor
Implementation Method 4
an ultrasonic sensor calculates a distance by measuring reflection from a surrounding object after transmitting an ultrasonic wave
Data Source
AI summary
The present invention relates to a sensor fusion object detection system and method of deriving the object segmentation information including segmented point segments for each object by clustering a point cloud obtained from a LIDAR sensor, deriving object recognition information for each object from an image obtained from a camera sensor, and deriving object point groups using a graph-based probability optimization technique based on a first probability as to whether each point segment calculated based on the object segmentation information and the object recognition information correspond to a particular object and a second probability as to whether two different point segments correspond to the same object and merged as one. According to the object detection system and method according to the present invention, there is an effect that it is possible to accurately detect an object having a complex shape or a large size.