Millimeter wave radar and vision fused three-dimensional target detection method based on attention mechanism

Abstract

The invention provides a millimeter-wave radar and vision fused three-dimensional target detection method based on an attention mechanism. The method comprises a data acquisition and processing step: acquiring millimeter-wave radar point cloud and vision image data with approximately synchronous time; a fusion step: converting millimeter wave radar point cloud data from a radar coordinate system to a camera coordinate system to realize space synchronization, then carrying out preprocessing operation, extracting speed and depth information of a target point cloud in a point cloud frame, constructing radar matrix data, and completing radar information extraction; extracting an image feature map and a radar feature map in the fusion framework through a neural network, and performing feature fusion in combination with an attention mechanism to obtain a fusion feature map; and a detection step: performing up-sampling on the fused feature map, inputting the fused feature map into a branch convolution network, and decoding output information of branch convolution to obtain the category and three-dimensional information of the target. According to the method, visual image data and millimeter wave radar data are efficiently fused, and the accuracy and reliability of three-dimensional target detection in a complex scene are effectively improved.

Description

technical field [0001] The present invention relates to the technical field of target detection by multimodal information fusion, and more particularly, the present invention relates to a three-dimensional target detection method based on the fusion of millimeter-wave radar and vision based on an attention mechanism. Background technique [0002] Object detection is one of the important research directions in the field of computer vision and image processing, which can be used to detect specific categories of object instances in digital images. As an important part of scene understanding, object detection is widely used in many fields of modern society, such as surveillance security, autonomous driving, traffic monitoring, medical image analysis, and drone scene analysis. [0003] The current mainstream target detection algorithms are all based on visual images, and have always been a research hotspot in computer vision, robotics and other related fields, such as R-CNN and Y...

AI Technical Summary

Problems solved by technology

At present, the performance of the two-dimensional target detection method based on visual images has been greatly improved, but there are still some shortcomings and problems that are difficult to solve immediately: 1. The quality of the collected optical camera data will be greatly affected by the external environment. In the case of bad weather, insufficient lighting, or unsatisfactory shooting distance and angle, the collected images will appear blurred, resulting in the loss of semantic information of key object instances in digital images; It can return the pixel coordinates of the target. The distance between pixels is not the distance in the real physical space. Therefore, it lacks the parameter information of the physical world such as depth, size and speed, and has great limitations in practical applications.

These characteristics of 3D target detection have inherent advantages over image-based 2D detection, but there are still many research difficulties in 3D target detection

In addition, since 3D target detection adds more dimensional information, the detection network needs more dimensional regression, which greatly increases the complexity of the network, so there is still a lot of room for improvement in real-time performance of 3D target detection.

Images