Millimeter wave radar and monocular camera information fusion method for smart car

Abstract

The invention discloses a millimeter wave radar and monocular camera information fusion method for a smart car. The method mainly comprises three stages of in the first stage, firstly installing and adjusting the position of a camera and a millimeter wave radar on the smart car, using a Zhang Zhengyou calibrating method to obtain internal parameters of the camera, and finally projecting an imagingplane of the camera to a vertical view to satisfy the condition that the target information recognized by two sensors is in the same vehicle coordinate system; in the second stage, training an image-based multi-target recognition model by using a deep learning method offline, in the process of driving the smart car online, utilizing the radar and the camera to collect data in real time, converting the identified target information into a unified vehicle coordinate system and eliminating an invalid target; in the third stage, using a global nearest neighbor matching algorithm to judge whetherthe target is consistent or not, and using an extended Kalman filtering wave to track a matched target and an unmatched target to achieve multi-target recognition. The method has high target recognition rate and reliable safety, and can effectively improve the practicality of the smart car to the environment.

Description

technical field [0001] The invention belongs to the field of multi-sensor information fusion, and in particular relates to a millimeter wave radar and camera information fusion method. Background technique [0002] Smart cars add advanced sensors (radars, cameras), controllers, actuators and other devices on the basis of ordinary cars, and realize intelligent information exchange with people, cars, roads, etc. through on-board sensing systems and information terminals, so that Cars have intelligent environmental perception capabilities, which can automatically analyze the safety and dangerous state of the car, and make the car reach the destination according to the wishes of people, and finally achieve the purpose of operating instead of people. At present, the sensors that can identify obstacles around the smart car include cameras, thermal imagers, millimeter-wave radars, lidars, and ultrasonic radars. Cameras are the most widely used sensors in real life. Perceiving the ...

AI Technical Summary

Problems solved by technology

However, this method is essentially only a verification of a single sensor target, and it cannot judge the undetected target, and cannot fully utilize the role of the sensor

Images