Three-dimensional target detection method and system based on image restoration

Abstract

The invention relates to a three-dimensional target detection method and system based on image restoration. The method comprises the following steps: obtaining an RGB image and radar point cloud of athree-dimensional target; generating a two-dimensional target detection box on the RGB image according to a two-dimensional target detection algorithm; for the picture with the shielded target, carrying out instance segmentation by adopting an instance segmentation algorithm to obtain a mask at the shielded position of the target, and then calculating a complete mask of the target according to morphological closed operation; converting the radar point cloud into a depth map through a camera matrix, performing image restoration on the shielded part of the target on the depth map, and extractingdepth information of the target in a depth map form according to a complete mask of the target after the restoration is completed; converting the depth information in the depth map form of the targetinto restored point cloud; and inputting the repaired point cloud into a three-dimensional target detection network for three-dimensional target detection. Compared with the prior art, the three-dimensional target detection method plays a role in reducing offset and improving precision in three-dimensional target detection.

Description

Technical field [0001] The present invention relates to the technical field of computer vision and image processing, in particular to a method and system for detecting a three-dimensional target based on image repair. Background technique [0002] As an indispensable task in the field of computer vision, target detection is widely used in fields such as autonomous driving, medical image analysis, and video surveillance. Nowadays, for three-dimensional target detection tasks, algorithms using deep learning frameworks are very common, and deep learning networks need to use a lot of data for training. The current training data generally includes RGB images, depth images and lidar point cloud data. The 3D target detection method based on RGB image lacks 3D geometric information, so it needs more prior information to make up. These limitations make the target detection effect of this kind of method unable to be greatly improved. Although the three-dimensional target detection method...

AI Technical Summary

Problems solved by technology

The 3D target detection method based on RGB images needs more prior information to make up for the lack of 3D geometric information. These limitations have prevented the target detection effect of this type of method from being greatly improved.

Although the 3D object detection method based on RGB-D utilizes the 3D information in the depth map, the reason for the too little information usage leads to a large offset in the 3D frame positioning.

Although the existing 3D target detection method based on RGB image and radar point cloud directly uses the 3D radar point cloud information, since the point cloud scanned by the lidar camera only obtains the points of the object within the camera's field of view, so for background For a single object, the occluded part of the point cloud has a large area of ​​loss, and the point cloud information of the object is extremely incomplete, and these incomplete information will lead to large errors in the 3D target detection frame

Images