Cross-attention-based radar-visual fusion data target detection method and system

Abstract

The application discloses a radar-vision fusion data target detection method and system based on cross attention, which comprises the following steps: acquiring a vision image and a point cloud feature map, composing a sample data set, and dividing the sample data set into a training set, a verification set and a test set according to a certain proportion; constructing a target detection model based on a cross attention mechanism; setting an optimizer, a learning rate strategy and a loss function, iteratively training the target detection model using the training set, comparing the prediction results of the target detection model using the verification set, and obtaining a trained target detection model; performing comparative experiments and ablation experiments using the test set to verify the prediction results of the trained target detection model; and performing target detection using the trained target detection model. The application combines a convolutional neural network and a Transformer, utilizes double-branch continuous multiple information interactions, fully gives play to the advantages of multi-modal information fusion at each stage, and realizes adaptive feature extraction between multi-modalities through a cross attention mechanism.

Images