Lidar-Camera Data Fusion Using Rotation-Angle Trigger Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing unmanned devices face challenges in fusing image data with point cloud data to achieve accurate depth and color information integration, which is crucial for effective control and navigation.
Innovation Solution
A system integrating a lidar and multiple image sensors, where a processor synchronizes data acquisition by sending trigger signals to image sensors based on lidar rotation angles, fusing pixel information from images with point cloud data using pose change information to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image data and point cloud data are acquired separately without synchronization, then data acquisition is simple, but the fusion accuracy of depth and color information is poor
Solution Approach 1:
The system performs preliminary actions by acquiring the rotation angle of the laser emitter before triggering the image sensor. The processor obtains the rotation angle from the rotation angle measurer and uses it to determine the corresponding image sensor, ensuring that the image and point cloud data are synchronized before fusion processing.
Solution Approach 2:
The system implements feedback by continuously monitoring the rotation angle of the laser emitter and using this information to dynamically select which image sensor to trigger. The processor receives feedback from the rotation angle measurer and adjusts the trigger signal accordingly, ensuring precise synchronization between lidar and image sensor acquisitions.
2Area of stationary object
If multiple image sensors are used to cover different rotation angle intervals, then the coverage of color information is improved, but the complexity of selecting and synchronizing the correct sensor increases
Solution Approach 1:
The system segments the coverage area into different rotation angle intervals, with each image sensor responsible for a specific interval. The processor divides the rotation angle range and assigns corresponding image sensors to each segment, allowing parallel data acquisition from multiple sensors without interference.
Solution Approach 2:
The processor acts as an intermediary between the rotation angle measurer and the image sensors. It receives the rotation angle, determines which image sensor corresponds to the current rotation angle interval, and sends the appropriate trigger signal, simplifying the synchronization process.
3Measurement precision
If trigger signals are sent based on rotation angle intervals, then data synchronization is improved, but the timing precision requirement increases
Solution Approach 1:
The system performs preliminary calculation of the rotation angle interval before triggering the image sensor. The processor obtains the rotation angle, determines the corresponding interval, and sends the trigger signal in advance, ensuring that the image sensor captures data at the correct moment without timing delays.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution provides accurate fusion of color and depth information, enabling improved obstacle detection, trajectory planning, and high-precision mapping, as well as enhanced training of machine learning models.
Implementation Method 1
a lidar and at least two image sensors. The lidar is configured to acquire a point cloud by rotating a laser emitter
Implementation Method 2
acquire a point cloud by rotating a laser emitter
Implementation Method 3
obtains a rotation angle of the laser emitter determined by a rotation angle measurer of the lidar
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
A data fusion method and apparatus, and an applicable system. The applicable system comprises a vehicle (1), the vehicle (1) is provided with a lidar (2) and at least one image sensor (3), where the lidar (2) acquires a point cloud in by rotating a laser transmitter. A processor (5) is configured to acquire a rotation angle of a laser transmitter of the lidar (2) (S200), select, according to a predetermined correspondence between at least one rotation angle interval and the at least one image sensor (3), an image sensor (3) corresponding to a rotation angle interval in which the obtained rotation angle of the laser emitter is located as a specified image sensor (3) (S202), send a trigger signal to the specified image sensor (3), to enable the specified image sensor (3) to acquire an image (S204), receive the image and a point cloud that is acquired and returned by the lidar (2) within the rotation angle interval in which the obtained rotation angle is located (S206), and fuse information of pixels in the image and information of points in the point cloud according to pose change information of the vehicle (1) in a process of acquiring the image and the point cloud (S208).