Virtual viewpoint image post-processing method

Abstract

The invention discloses a virtual viewpoint image post-processing method. The virtual viewpoint image post-processing method comprises the following steps that a color image reference frame and a range image reference frame of an image frame to be processed are selected; a window, containing a cavity of the image frame to be processed, to be processed is obtained, and all matching windows which are similar to the window to be processed are searched for within the search range of the color image reference frame and the range image reference frame; the window which is the most similar to the window to be processed is selected and serves as the best matching window, and the cavity of the window to be processed is filled up with a pixel of the position, corresponding to the cavity of the window to be processed, of the best matching window. According to the virtual viewpoint image post-processing method, space-time relevant information of a reference viewpoint image in the process of drawing is utilized, the cavity is effectively filled up through the method that the position of the pixel which is matched with the cavity, caused by occlusion or exposure or rounding calculation, of the image to be processed according to the searching and matching method is selected from the selected color image reference frame and the range image reference frame, and therefore the quality of a virtual viewpoint image is improved.

Description

technical field [0001] The invention relates to image processing, in particular to a post-processing method for virtual viewpoint images. Background technique [0002] With the rapid development of digital media technology and the continuous improvement of living standards, people have put forward higher requirements for video entertainment programs, and traditional flat (two-dimensional) videos viewed from a fixed viewpoint can no longer meet people's needs. In such a situation, three-dimensional TV (3DTV) / free-viewpoint video (FVV) came into being. 3D TV and free-viewpoint video increase the depth information of scene objects and enhance the visual quality. Realism, so that the image is no longer limited to the display screen, as if it can walk out of the display screen, making the audience feel that the scene is oncoming, as if they are in the scene, and feel a real sense of depth and immersion. [0003] A key technology in 3D TV and free-view video is the generation of ...

AI Technical Summary

Problems solved by technology

In the process of 3D image transformation, many holes will be generated in the virtual view point image due to occlusion, exposure or rounding operations, etc., as well as the edge mutation of the object in the depth map obtained through depth estimation and inaccurate partial depth values. Problems such as resampling and false contours affect the subjective and objective quality of virtual viewpoint images

[0006] In view of the above-mentioned problems in the virtual viewpoint image, in the post-processing method of the virtual viewpoint image, some people introduce depth information as an auxiliary in the process of hole filling, and add the depth information to the calculation of the priority of the area to be repaired and the characterization of the cost function. There is still a lot of distortion due to padding errors

At the same time, when the depth information is used, the hole in the drawn virtual viewpoint depth image is not filled first, resulting in a large distortion of the depth information used, so that the judgment of priority and the calculation of the cost function have a certain impact

In addition, if the hole needs to be filled is a sequence with more intense movement, after filling it with weighted average filling, a certain "fuzzy" area may be formed in the hole area, which will affect the subjective quality

Images