Monocular image-oriented three-dimensional object detection method based on three-dimensional reconstruction

Abstract

The invention discloses a monocular image-oriented three-dimensional object detection method based on three-dimensional reconstruction, and belongs to the field of image processing and computer vision. The method comprises the following steps: firstly, converting input data from a two-dimensional image plane into a three-dimensional point cloud space by utilizing an independent module so as to obtain better input representation; performing three-dimensional detection by using a PointNet network as a backbone network to obtain a three-dimensional position, a three-dimensional size and a three-dimensional direction of the object. In order to improve the recognition capability of the point cloud, the invention provides a multi-modal feature fusion module, and RGB information of points and RGBfeatures of ROI are supplemented and embedded into the generated point cloud representation. Compared with a two-dimensional image, the method for deriving the three-dimensional bounding box from thethree-dimensional scene is more efficient; compared with a similar three-dimensional object detection method based on a monocular camera, the method provided by the invention is more efficient.

Description

technical field [0001] The invention belongs to the field of image processing and computer vision, and relates to three-dimensional target detection based on monocular images in outdoor scenes. It specifically relates to a monocular image-based 3D reconstruction-based 3D object detection method, which takes a monocular image as input and outputs the real 3D coordinates, size, and orientation of an object of interest (such as a vehicle, pedestrian, etc.) in the image. A 3D detection method with such information. Background technique [0002] In recent years, with the development of deep learning and computer vision, a large number of two-dimensional object detection algorithms have been proposed and widely used in various visual products. However, for applications such as unmanned driving, mobile robots and virtual reality, two-dimensional detection technology is far from meeting the actual needs. In order to provide more accurate target position and geometric information, ...

AI Technical Summary

Problems solved by technology

[0006] The present invention aims to overcome the deficiencies of the prior art, provides a more accurate 3D object detection method based on a monocular camera, solves the problem of reconstructing 3D space, and can extract 3D semantics well. An independent module converts the input data from a two-dimensional image plane to a three-dimensional point cloud space to obtain a better input representation; in order to improve the recognition ability of point clouds, this invention proposes a multi-modal feature fusion module that complements RGB features Embedded into the generated point cloud representation; then 3D detection using the PointNet network to obtain the 3D position, size and orientation of the object

Images