An automatic driving three-dimensional target detection method based on three-mode data source fusion

Abstract

The application relates to an automatic driving three-dimensional target detection method based on three-mode data source fusion, and steps include: using data sources of three modes of an RGB camera, a laser radar and a millimeter wave radar as input; through a region proposal network, taking each adjacent H frame as a group of data, generating a 3D proposal frame and cutting out a region of interest (RoI) of the three modes; performing RoI pooling on the regions of interest of the three modes; through a fusion network, fusing the regions of interest (RoI) of the three modes in the H frame, and using a three-dimensional convolution layer of the H channel to fuse a time dimension to obtain a feature tensor; stretching the obtained feature tensor into one dimension and passing through a full connection layer to obtain a final output. Through time fusion, the application improves the utilization rate of data and improves the training effect of a three-dimensional target detection model without increasing the calculation amount too much; combining the RGB camera, the laser radar and the millimeter wave radar can enhance the anti-interference performance of the automatic driving three-dimensional target detection model.

Images