Depth image error point removal method combining image semantics and three-dimensional information

Abstract

The invention discloses a depth image error point removal method combining image semantics and three-dimensional information, which comprises the following steps: 1) collecting input data including a color image and a depth image with error points; 2) inputting a color image, and predicting a normal of the color image through a residual convolutional neural network; inputting a depth image with error points, and calculating a normal of the depth image by adopting a principal part analysis method; 3) inputting a color image marked with a potential depth image error point region, and predicting the potential depth image error point region in the color image through a convolutional neural network with cavity convolution; projecting a potential depth image error point region and two normal lines to three dimensions; and 4) calculating two normal differences of each point in the local neighborhood in three dimensions and the local density of each point to eliminate depth error points. The method has the advantages that the quality of the depth image is improved, and the effectiveness of restoring the depth image with noise by application such as depth completion is improved.

Description

technical field [0001] The present invention relates to the technical field of depth image denoising, in particular to a method for removing error points in depth images that combines image semantics and three-dimensional information. Background technique [0002] As an implicit representation of 3D information in a scene, depth images are widely used in many fields. Fields such as human-computer interaction, 3D reconstruction, robot path planning, and virtual reality all use depth images to guide the realization of some functions. There are generally three methods for acquiring depth images, namely time-of-flight method, structured light method and binocular vision method. Among them, the time-of-flight method and the structured light method have relatively low cost and stable real-time performance, and have received extensive attention in daily life applications. At the same time, the depth camera can not only collect depth images, but also collect corresponding color im...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art, and propose a depth image error point removal method that combines image semantics and three-dimensional information, which can effectively solve the problem of depth error points caused by the shooting environment, and at the same time, effectively remove After these depth error points, the accuracy and accuracy of the depth image can be further improved to improve the effect of downstream applications

Images