Free-Viewpoint Image Synthesis Depth Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing free-viewpoint image synthesis methods based on DIBR suffer from issues like ghosts, holes, and distortions due to low-quality depth maps, particularly at the edges and uneven depth distribution, which affect the quality of composite images.
Innovation Solution
A method involving depth map correction, forward and back-projection, optical flow alignment, and weighting calculation to synthesize high-quality free-viewpoint images, which includes correcting depth maps based on color maps, forward-projecting depth maps to a virtual viewpoint, back-projecting color maps, aligning images using optical flow, and performing weighting and blending to produce a composite image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If DIBR-based free-viewpoint image synthesis is used, then multi-view viewing needs are met, but ghosts, holes, and distortions appear in composite images due to low-quality depth maps
Solution Approach 1:
The patent applies preliminary action by correcting the depth map before using it in the DIBR synthesis process. Specifically, the depth map is corrected based on color map information to fix edge depth errors and uneven depth distribution beforehand. This preliminary correction ensures that the subsequent synthesis process produces high-quality composite images without ghosts, holes, or distortions, while still maintaining the multi-view viewing capability.
2Reliability
If depth map correction is performed, then image synthesis quality improves, but processing time increases
Solution Approach 1:
The patent applies local quality by focusing the depth map correction process on specific problem areas rather than processing the entire depth map uniformly. The correction is applied locally at edges and regions with uneven depth distribution, where the most significant errors occur. This targeted approach corrects the most critical defects while minimizing unnecessary processing time in regions that are already accurate.
3Measurement precision
If optical flow algorithm is applied to align images, then alignment precision improves, but computational complexity increases
Solution Approach 1:
The patent applies feedback by using the optical flow algorithm to align the left and right images in the color map, then using this alignment information to correct the depth map. The corrected depth map is then used to generate the virtual viewpoint image, and any remaining misalignments can be iteratively corrected. This feedback loop ensures high alignment precision while managing computational complexity through iterative refinement rather than a single complex operation.
Data Source
AI summary
The embodiment of the present specification discloses an image synthesis method, apparatus and device for free-viewpoint. The method includes: correcting a first depth map based on a first color map and the first depth map inputted, wherein the first color map includes a left image and a right image, and the first depth map includes a left image and a right image; forward-projecting the second depth map to a virtual viewpoint position based on the pose of a virtual viewpoint to obtain a third depth map, where the third depth map is a projection map located at the virtual viewpoint position; back-projecting the left and right color maps closest to the virtual viewpoint position in the first color map to the virtual viewpoint position based on the third depth map to obtain a second color map, where the second color map is a projection view of the first color map at the virtual viewpoint position; correcting the second color map by using an optical flow algorithm to obtain a third color map, where the optical flow algorithm implements aligning the left and right images of the second color map; performing weighting calculation and blendering on the third color map to obtain a composite image.