Roadside camera and 4D millimeter wave fused target detection method, equipment and medium

Abstract

The invention relates to a roadside camera and 4D millimeter wave fused target detection method, equipment and a medium. The target detection method comprises the following steps: obtaining a first image of a roadside camera and a first point cloud of a 4D millimeter wave radar; projecting the first point cloud to the first image, adding distance information to the first image, and obtaining a second image; taking the second image as the input of a depth completion convolutional network, and obtaining a third image which is a dense depth map; converting the third image into a second point cloud, and encoding to generate a fourth image; inputting the fourth image into a convolutional neural network, and carrying out target detection to obtain a target frame; and mapping the target frame into a 3D frame in a camera coordinate system, and adding speed information on the 3D frame based on the first point cloud. Compared with the prior art, the method has the advantages of long sensing distance, accurate target positioning and the like.

Description

technical field [0001] The invention relates to the technical field of intelligent Internet of Vehicles, in particular to a target detection method, equipment and medium for fusion of roadside cameras and 4D millimeter waves. Background technique [0002] At present, autonomous driving and roadside perception are mainly based on 3D detection algorithms based on lidar. According to the characteristics of lidar, commonly used target detection methods can be divided into four categories: BEV-based, camera view-based, point-wise feature-based, and fusion-based feature-based. LiDAR has the characteristics of not being affected by light, and this technology can also directly obtain accurate three-dimensional information. [0003] However, lidar still has the following shortcomings: [0004] (1) The high cost of lidar hinders the large-scale application of the market to a certain extent. [0005] (2) LiDAR is easily affected by weather. Lasers generally have less attenuation an...

AI Technical Summary

Problems solved by technology

[0004] (1) The high cost of lidar hinders the large-scale application of the market to a certain extent

[0005] (2) LiDAR is vulnerable to weather

In bad weather such as heavy rain, thick smoke, and dense fog, the attenuation increases sharply, and the propagation distance is greatly affected. Moreover, the atmospheric circulation will also cause distortion and vibration of the laser beam, which directly affects the measurement accuracy of the laser radar.

Therefore, in rainy days, foggy days, snowy days, heavy dust, etc., the original data of the lidar will be disturbed, and the detection algorithm will be affected at the same time, resulting in the inability to accurately locate the target object

Images