Processing method for color aliasing and shadow part in three dimensional video virtual viewpoint synthesis

Abstract

The invention belongs to the field of a 3D (three dimensional) video and relates to a processing method for a color aliasing and shadow part in three dimensional video virtual viewpoint synthesis. The method is realized by selectively mapping the color aliasing and shadow part corresponding to the left-right viewpoint depth map boundary part and judging whether the color aliasing and shadow part is remained or not. According to the method, the distortion of the synthesis virtual viewpoint boundary position caused by the color aliasing and shadow part is eliminated and the subjective quality and objective quality of the virtual viewpoints are improved.

Description

technical field [0001] The invention belongs to the field of 3D video, and relates to a method for processing color aliasing and shadow parts in virtual viewpoint synthesis of 3D video. Background technique [0002] Traditional binocular stereoscopic video and multi-viewpoint video can no longer meet people's increasingly high audio-visual needs because they cannot achieve self-adjustment of depth. In contrast, a new generation of 3D video systems based on depth data and viewpoint synthesis can use depth-image-based rendering (DIBR) technology to synthesize virtual viewpoint images, and adjust the final displayed left and right views according to user needs The size of the parallax can improve the user's visual experience, so it has received extensive attention from academia and industry. [0003] Virtual view synthesis maps the color image of the 3D scene taken at one viewpoint to the image plane of another virtual viewpoint through 3D projection based on depth images (3D ...

AI Technical Summary

Problems solved by technology

[0004] However, because the color image boundary captured by the camera generally does not change sharply, but has an aliasing part with a span of several pixels, that is, the mixture of the foreground color and the background color (which can be considered as the result of the low-pass filtering of the foreground color and the background color ), and due to the limitations of the depth acquisition technology, the quality of the depth image at the edge of the object is generally poor, so the boundary of the object edge in the color image and the depth image cannot be highly consistent, resulting in boundary synthesis distortion

In addition, traditional lossy coding may blur the boundary between color and depth, exacerbating the edge distortion of synthetic viewpoints

On the other hand, when some scenes are collected, it cannot be completely assumed that the scene objects are Lambertian, and there may be shadows around the objects. After the viewpoint mapping, the pixels in the shadows will deviate from the objects, which will affect the background and generate background noise.

[0005] At present, for the virtual viewpoint synthesis algorithm of the border aliasing part of the color image, such as Xuyuan Xu et al. proposed to assign the depth value of the border aliasing part of the color image as the foreground depth value for virtual viewpoint synthesis, which can guarantee the foreground color to the background color. Smooth transition, but at the boundary from the foreground to the background, the aliased part of the left view will coincide with the correct color information of the right view, resulting in color errors after view blending (at the boundary from the background to the foreground, the right view's The aliased part will coincide with the correct color information of the left view, resulting in wrong color after view fusion), such as figure 1 and 2 shown

Ce Zhu et al. proposed a mismatch suppression alignment enhancement method for the border area, which includes suppressing the mapping of the color aliasing part, which can effectively reduce the border distortion, but at the same time because the color aliasing part (which contains part of the foreground information) cannot be mapped To the virtual viewpoint, the foreground object in the virtual viewpoint will be lost at the boundary

On the other hand, in the traditional view synthesis algorithm, because the foreground difference between the left and right viewpoints at the boundary of the depth map is not considered, part of the correct foreground information of the left and right viewpoints is fused with the aliased part after viewpoint fusion, resulting in wrong fusion results. Reduce the quality of the final composite view

Images