Mining area target identification method based on fusion of fisheye camera and laser radar

Abstract

The invention provides a mining area target identification method based on fusion of a fisheye camera and a laser radar. The mining area target identification method comprises the following steps: 1) carrying out information acquisition, distortion correction and data augmentation; 2) feature extraction and fusion: performing feature extraction on the data subjected to distortion correction and data augmentation by using different branches, and then performing fusion output on the features of the two kinds of data through a bilinear interpolation-based method; and 3) target identification: using a classic RPN structure, and performing result output through several important steps of generating an anchor frame, determining a mapping relationship between the anchor frame and a real frame, mapping parameter learning, non-maximum suppression and determining an effective candidate frame according to a score. The advantages of the two are integrated, the recognition performance of unmanned driving in a mining area environment is effectively improved by using a fusion mode, rich, real-time and accurate target pose information is provided for a subsequent decision control module, and unmanned driving landing is assisted.

Description

technical field [0001] The present invention relates to the technical field of environmental perception, and in particular to a mine area target recognition method based on the fusion of a fisheye camera and a laser radar, which is applied to the perception of vehicle driving environment in a mine area scene. Background technique [0002] Autonomous driving technology has developed rapidly in recent years, and the technology has been implemented in some specific scenarios at home and abroad. Because the mining area has the characteristics of a closed environment, relatively fixed transportation roads, and slow vehicle speeds, it has become one of the most suitable scenarios for the implementation of autonomous driving technology that has attracted much attention. As the basis of autonomous driving research, perception technology provides rich environmental information for subsequent control and decision-making modules. Target recognition is an important part of perception t...

AI Technical Summary

Problems solved by technology

[0008] 1. The industrial-grade fisheye camera used in the mining area has a large degree of distortion, and the distortion correction effect of the traditional isometric model is poor. Except for the central part, the pixels on the edge of the image have a large degree of stretching problem

[0009] 2. The distribution of the point cloud directly on the fisheye image will be uneven, especially the point cloud at the edge of the image is more sparse, and the feature extraction effect is very small

[0010] 3. The update frequency of the fisheye camera and lidar is very different, and there is a matching error between the images captured at different frame rates and the point cloud, which makes it impossible to perform subsequent feature extraction and target recognition network training

[0011] 4. The fusion of fisheye camera and lidar is cross-dimensional heterogeneous sensor fusion. The current fusion method has a large degree of information loss and incomplete fusion

Images