Multi-sensor data fusion method, device, electronic device and medium

Abstract

Embodiments of the present disclosure disclose a multi-sensor data fusion method, apparatus, electronic device, and medium. A specific implementation of the method includes: acquiring the point cloud data set in the target area and the image captured by the vehicle-mounted monocular camera; performing feature extraction on the image to generate an image feature map; performing grid division on the target area to generate sub-target areas Set; determine the mean value of point cloud data in each sub-target area to generate point cloud data mean information; generate a set of point cloud data feature information; determine the mapping of point cloud data corresponding to each point cloud data feature information in the image feature map coordinates; generating splicing feature information; generating fusion feature information; generating obstacle detection information; sending the obstacle detection information to the target display terminal and displaying it on the target display terminal. This implementation reduces the waste of computing resources and improves the accuracy of generating obstacle information.

Description

technical field [0001] The embodiments of the present disclosure relate to the field of computer technology, and in particular to a multi-sensor data fusion method, device, electronic equipment and media. Background technique [0002] In the field of autonomous driving, different sensors have different advantages and limitations. Therefore, when detecting obstacles, multiple sensors are often required for cooperative perception. At present, the existing technology often uses different perception algorithms to detect obstacles on the information obtained by each sensor. Then, the identified obstacle information is fused through a fusion algorithm. Furthermore, the final obstacle information is obtained. [0003] However, when the above-mentioned method is used for multi-sensor data fusion, the following technical problems often exist: [0004] First, it is necessary to use different perception algorithms to detect obstacles on the data obtained from different sensors, and...

AI Technical Summary

Problems solved by technology

[0004] First, it is necessary to use different perception algorithms to detect obstacles on the data obtained from different sensors, and then use the fusion algorithm to fuse the obtained multiple obstacle information to generate the final obstacle information. This method requires multiple sets of Algorithms for data processing, the steps are cumbersome, and cause a waste of computing resources;

[0005] Second, when generating the final obstacle information, it is to fuse the obtained multiple obstacle information instead of fusing the data features. Since the obstacle information contains less data features, the final obstacle information may be The information is not accurate enough;

[0006] Third, in the actual situation, the point cloud data on the front and rear sides of the self-driving vehicle during driving is more important, and the point cloud data on both sides of the vehicle body is relatively less important. Point cloud data, without discarding relatively unimportant point cloud data, makes the calculation process contain more unimportant point cloud data, making the calculation efficiency not high enough

Images