A 3D vehicle detection method based on multi-sensor fusion

Abstract

The invention discloses a 3D vehicle detection method based on multi-sensor fusion, and the method comprises the steps: Step 1, obtaining the semantic information (i.e., an RGB image) of a vehicle through a camera installed on the vehicle, carrying out the scanning of the surrounding environment of the vehicle through a laser radar located at the top of the vehicle, and obtaining the precise depthinformation (i.e., laser radar point cloud) of the environment; Step 2, preprocessing the laser radar point cloud, taking Z-axis [0, 2.5] m according to the height of the automobile, and equally dividing the point cloud into five height slices along the Z-axis direction; Step 3, generating a 3D vehicle region of interest on the laser radar point cloud; Step 4, performing feature extraction on theprocessed radar point cloud and RGB image and generating corresponding feature maps; Step 5, respectively mapping the 3D vehicle region of interest to feature maps of the point cloud and the RGB image; and Step 6, fusing the feature maps of the mapping part in the step 5, and finally realizing 3D positioning and detection of the vehicle target.

Description

technical field [0001] The invention belongs to the field of automatic driving, and in particular relates to a vehicle 3D detection method based on multi-sensor fusion. Background technique [0002] A smart car is a complex system including perception, decision-making and control technologies. Environmental perception provides fundamental information for path planning and decision-making control, and vehicle detection is an extremely critical task in the autonomous vehicle environment perception system. The mainstream obstacle detection sensor It is the camera and lidar. Now the vision-based vehicle detection has achieved good results. The camera is low in cost and can obtain the texture and color of the target, so it is widely used in intelligent driving. However, the camera is more sensitive to light and shadow parts. , cannot provide accurate and sufficient location information, often leading to problems such as low real-time performance or poor robustness. LiDAR can obt...

AI Technical Summary

Problems solved by technology

[0002] A smart car is a complex system including perception, decision-making, and control technologies. Environmental perception provides fundamental information for path planning and decision-making control, and vehicle detection is an extremely critical task in the autonomous vehicle environment perception system. The mainstream obstacle detection sensor It is the camera and lidar. Now the vision-based vehicle detection has achieved good results. The camera is low in cost and can obtain the texture and color of the target, so it is widely used in intelligent driving. However, the camera is more sensitive to light and shadow parts. , cannot provide accurate and sufficient location information, often leading to problems such as low real-time performance or poor robustness

LiDAR can obtain the target distance and three-dimensional information, the detection distance is long and not affected by light, but the texture and color of the target cannot be determined, so a single sensor cannot meet the needs of autonomous driving

Images