Calibration method integrated with three-dimensional point cloud and two-dimensional image based on neural network

Abstract

The invention discloses a calibration method integrated with a three-dimensional point cloud and a two-dimensional image based on a neural network. The method comprises the steps of acquiring pixel coordinates of the image and voxel coordinates of a laser radar; establishing an N * N matrix in one-to-one correspondence with the pixel coordinate points and the voxel coordinate points as a trainingset; creating a neural network structure, wherein the neural network structure comprises a matrix product kernel and a matrix product layer, the neural network structure comprises an input layer, an external reference product layer and an internal reference product layer, the input layer is a voxel coordinate matrix, the weight of the external reference product layer is an external reference matrix, and the weight of the internal reference product layer is an internal reference matrix; taking the statically calibrated external parameter matrix as an initial value training model; and obtainingan external parameter matrix by using the trained model, and fusing the three-dimensional point cloud and the two-dimensional image according to the obtained external parameter matrix. A more accurateexternal parameter matrix can be obtained, so that integration from the three-dimensional point cloud to the two-dimensional image is more accurate.

Description

technical field [0001] The invention relates to the technical field of sensor information fusion processing, in particular to a neural network-based calibration method and system for 3D point cloud and 2D image fusion. Background technique [0002] In unmanned driving environment perception equipment, lidar and camera have their own advantages and disadvantages. The advantages of the camera are low cost, high color recognition of environmental scenes, and relatively mature technology. The disadvantage is that it is difficult to obtain accurate three-dimensional information, and it is relatively limited by environmental lighting. [0003] The advantage of lidar is that it has a long detection distance, can accurately obtain three-dimensional information of objects, has high stability and good robustness. However, the current cost of lidar is relatively high, and the final form of the product has not yet been determined. [0004] As far as the application characteristics of...

AI Technical Summary

Problems solved by technology

[0015] In order to solve the inaccurate problem of the existing 3D and 2D fusion projection, especially the large deviation of the projection result from 3D to 2D in the state of motion, and it is difficult to achieve commercialization, the present invention provides a neuron-based The calibration method and system for the fusion of 3D point cloud and 2D image of the network can obtain a more accurate external parameter matrix, making the fusion of 3D point cloud to 2D image more accurate

Images