No-distortion integrated imaging three-dimensional displaying method based on Kinect

Abstract

The invention discloses a no-distortion integrated imaging three-dimensional displaying method based on a Kinect sensor. The method comprises the steps of acquiring a depth image of a three-dimensional scene by means of the Kinect sensor, and performing black hole filling on the depth image obtained by the Kinect according to a rapid stepping method; performing combined bilateral filtering on the depth image after filling according to a corresponding colorful image, and obtaining the depth image with continuous depth change and smooth edge; and finally obtaining a primitive image array of a large-field-depth complex scene by means of an optical field model, thereby realizing no-distortion three-dimensional integrated imaging displaying. According to the primitive image array obtained through the method of the invention, because of an ideal state without influence by scattering and refraction, the image quality is better than that of the primitive image array which is obtained through a microlens array. The no-distortion integrated imaging three-dimensional displaying method can effectively overcome physical restriction of a micro-lens array in recording the large-field-depth scene and furthermore can realize no-distortion three-dimensional integrated imaging displaying for the large-field-depth compliex scene.

Description

technical field [0001] The invention relates to the technical fields of computer vision, three-dimensional reconstruction and stereoscopic display, in particular to an integrated imaging three-dimensional display method that can adapt to large depth of field and complex scenes. Background technique [0002] With the development of optoelectronic devices and video processing technology, integrated imaging 3D display has become a research hotspot in the next generation of true 3D display technology. Compared with holographic three-dimensional display, it has the characteristics of relatively small amount of data, no coherent light source and no harsh environmental requirements, etc. It has become one of the most advanced three-dimensional display methods in the world, and it is also the most promising way to realize three-dimensional TV. There are two implementation schemes for naked-view true 3D display, one is the multi-view scheme, and the other is the "texture + depth" sch...

AI Technical Summary

Problems solved by technology

The first key problem is that due to the reflection, illumination and occlusion of the surface material of the object in the 3D scene, there are a large number of black holes in the depth image acquired by the Kinect sensor.

Especially in the complex background with a large depth of field, because the infrared sensor of the Kinect sensor cannot capture the light reflected from the surface of the object, it is impossible to obtain high-quality continuous depth images in the black hole area that forms the depth image

The second key issue is that there is an edge misalignment between the depth image and the color image due to the inherent optical distortion between the depth camera and the color camera of the Kinect sensor

For example, Telea et al. proposed a fast stepping method in [Animage inpainting technique based on the fast marching method.journal of graphics, gpu, and game tools, 9(1):23–34,2004.], which can effectively repair There is a large black hole area in the image, but this method does not consider the effect of the color image on the depth image repair, and there is still a mismatch between the edge of the repaired depth image and the color image

In addition, Petsching et al. proposed a joint bilateral filter in the literature [Digital photography with flash and no-flash image pairs.ACM Trans.Graph.,23(3):664–672,2004.], combining color images with depth images Perform image denoising and edge smoothing. However, when the black hole area is large, the depth image repaired by this method still has black holes, and its usability is extremely poor.

[0005] Therefore, it is difficult to directly use the color image and depth image obtained by the Kinect sensor or the depth image repaired by a simple method to establish a light field model to achieve undistorted integrated imaging 3D display

Images