Adaptive multi-sensor data fusion method and system based on mutual information

Abstract

The invention discloses an adaptive multi-sensor data fusion method based on mutual information. The method comprises the following steps: receiving an RGB image of a road surface acquired by a camera; receiving point cloud data of the road surface synchronously acquired by a laser radar; preprocessing the point cloud data to obtain dense point cloud data; inputting the RGB image and the dense point cloud data into a pre-established and trained fusion network, and outputting a data fusion result, wherein the fusion network is used for calculating mutual information of the feature tensor and the expected feature of the input data, allocating a fusion weight of the input data according to the mutual information, and outputting a data fusion result according to the fusion weight. According tothe method, an information theory tool of mutual information is introduced, the correlation between the extracted feature of the input data and the expected feature of the fusion network is calculated, the quality of the data quality and the information amount can be reasonably and objectively quantified, a strict mathematical method is used as a theoretical support, and certain interpretabilityis achieved.

Description

technical field [0001] The invention belongs to the technical field of intelligent driving, and in particular relates to an adaptive multi-sensor data fusion method and system based on mutual information. Background technique [0002] Intelligent driving vehicles need to sense and measure the surrounding environment through a variety of sensors to ensure safe driving. The fusion network can fuse data from different sensors to complete driving tasks, including lane line detection, object detection and tracking, etc. However, there may be various problems with the data from different sensors. For example, the camera may have problems such as image overexposure, the influence of rain and fog, and obstacles, and the lidar may have problems such as too sparse point cloud and a long distance from the target. In response to these problems, mutual information calculations can be used to evaluate data quality, and effective information can be weighted by assigning fusion weights to...

AI Technical Summary

Problems solved by technology

However, they are either not adaptive enough to the data or computationally complex

The introduction of mutual information overcomes some difficulties in the existing methods: 1) lack of reasonable quantification tools and support of mathematical methods; 2) the problem of network non-convergence is easy to exist in the training process; 3) fusion weights cannot be adaptively Adjustments are made to inputs from different sensors, i.e. cannot be adjusted in terms of data quality and informativeness

Images