A method for real-time filling of holes in kinect depth images

Abstract

Disclosed in the invention is a method for filling a cavity in a Kinect depth image in real time. The method comprises the following steps: step 1, obtaining a color image of a shooting scene and a corresponding depth image by using a Kinect camera and determining a foreground image position and a background image position by using an operating-period Gauss average method and a background differencing method; step 2, constructing a deepest image by using a Kinect depth image and filling a cavity in the deepest image; step 3, replacing the pixel of the deepest image with a background pixel of the depth image, thereby realizing cavity filling of the depth image background; and step 4, with the foreground image position obtained by the step 1, marking the cavity existed in the foreground position of the Kinect depth image and filling the marked cavity in real time. According to the invention, the object edge in the depth video with the filled cavity becomes stable; and the no flicker phenomenon occurs in the depth video. Meanwhile, the cavity filling speed in the depth image and the image stability can be improved effectively.

Description

technical field

[0001] The invention belongs to the technical field of depth image processing, and more specifically relates to a method for real-time filling of holes in a Kinect depth image. Background technique

[0002] Kinect is a Microsoft sensor exhibited by Microsoft in June 2010 (see K. Khoshelham, Sander Oude Elberink.Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications.Sensors 2012, 2012(12), pp. 1437-1454). Kinect 3D stereo camera, which can simultaneously acquire the color image of the shooting scene and the corresponding depth image, because the Kinect camera can easily and cheaply obtain the depth image of the scene in real time, the Kinect camera has been widely used in the fields of 3D scene reconstruction and object segmentation. application.

[0003] The capture frequency of Kinect's depth image is 30fps, and there is a large temporal correlation between frames in the actual depth video, which is of great significance for image comp...

Examples

Embodiment Construction

[0061] A kind of method of the present invention carries out real-time filling to the hole in the Kinect depth image, as figure 1 As shown, the specific implementation steps are as follows:

[0062] Step 1: Use the Kinect camera to obtain the color image of the shooting scene and the corresponding depth image, use the running-time Gaussian average method to extract the background of the color image to obtain the background image, and then use the background difference method to segment the Kinect color image to obtain the foreground image. Determine the position of the foreground image and the position of the background image, the specific steps are as follows:

[0063] 1.1 Use the Kinect camera to obtain the color image of the shooting scene and its corresponding depth image;

[0064] 1.2 Use the running-time Gaussian averaging method to extract the Kinect color image background to obtain the background image, the expression of which is:

[0065] (1)

[0066] Among them...

technical field

[0001] The invention belongs to the technical field of depth image processing, and more specifically relates to a method for real-time filling of holes in a Kinect depth image. Background technique

[0002] Kinect is a Microsoft sensor exhibited by Microsoft in June 2010 (see K. Khoshelham, Sander Oude Elberink.Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications.Sensors 2012, 2012(12), pp. 1437-1454). Kinect 3D stereo camera, which can simultaneously acquire the color image of the shooting scene and the corresponding depth image, because the Kinect camera can easily and cheaply obtain the depth image of the scene in real time, the Kinect camera has been widely used in the fields of 3D scene reconstruction and object segmentation. application.

[0003] The capture frequency of Kinect's depth image is 30fps, and there is a large temporal correlation between frames in the actual depth video, which is of great significance for image comp...

Examples

Embodiment Construction

[0061] A kind of method of the present invention carries out real-time filling to the hole in the Kinect depth image, as figure 1 As shown, the specific implementation steps are as follows:

[0062] Step 1: Use the Kinect camera to obtain the color image of the shooting scene and the corresponding depth image, use the running-time Gaussian average method to extract the background of the color image to obtain the background image, and then use the background difference method to segment the Kinect color image to obtain the foreground image. Determine the position of the foreground image and the position of the background image, the specific steps are as follows:

[0063] 1.1 Use the Kinect camera to obtain the color image of the shooting scene and its corresponding depth image;

[0064] 1.2 Use the running-time Gaussian averaging method to extract the Kinect color image background to obtain the background image, the expression of which is:

[0065] (1)

[0066] Among them...

Images