Novel View Image Control for Thin Objects and Occlusion Errors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing novel view synthesis techniques face issues with inaccurate depth estimation and occlusion region filling, leading to side effects such as thin objects disappearing and limited viewpoint movement, resulting in poor stereoscopic image quality.
Innovation Solution
An electronic apparatus and method that identifies foreground and background regions using depth maps, predicts side effects based on object thickness and density, and controls viewpoint movement paths to minimize these effects by refining depth maps and adjusting occlusion regions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Area of stationary object
If a large viewpoint movement path is used to generate novel view images, then the field of view and coverage are improved, but depth estimation accuracy deteriorates and occlusion region filling errors increase
Solution Approach 1:
The patent segments the image into foreground and background regions using depth information, then applies different processing strategies to each segment. The foreground region uses a reduced viewpoint movement path to maintain depth estimation accuracy, while the background region can utilize a larger viewpoint movement path to expand field of view, thus resolving the contradiction between coverage and precision.
Solution Approach 2:
The patent applies local quality by using different viewpoint movement paths for different regions of the image. Specifically, the foreground region uses a first (reduced) viewpoint movement path while the background region uses a second (larger) viewpoint movement path, allowing each region to have optimized quality characteristics appropriate to its requirements.
2Measurement precision
If viewpoint movement path is reduced to minimize side effects, then depth estimation accuracy is improved, but field of view and image coverage are limited
Solution Approach 1:
The patent divides the image into foreground and background segments and applies different viewpoint movement strategies to each. This segmentation allows the system to maintain high depth estimation accuracy in the foreground while still achieving adequate field of view through background processing with larger viewpoint movements.
Solution Approach 2:
The patent implements local quality by optimizing viewpoint movement path differently for foreground and background regions. The foreground region receives a reduced viewpoint movement path for high precision, while the background region utilizes a larger viewpoint movement path for broader coverage, achieving overall system optimization.
3Reliability
If thin objects are present in the input image, then processing complexity increases and side effects are amplified, but complete viewpoint movement control is needed to maintain image quality
Solution Approach 1:
The patent performs preliminary action by identifying thin objects in the input image before generating the novel view image. Based on this preliminary identification, the system determines whether to apply a reduced viewpoint movement path, preventing side effects related to thin objects while avoiding unnecessary processing complexity for images without such objects.
Solution Approach 2:
The patent applies parameter changes by dynamically adjusting the viewpoint movement path parameters based on the presence of thin objects. When thin objects are detected, the system changes the viewpoint movement path to a reduced path, thereby maintaining image quality while adapting processing complexity to the actual content requirements.
4Area of stationary object
If occlusion region filling is performed with large viewpoint movement, then coverage of occluded areas is improved, but filling errors increase due to inaccurate depth estimation
Solution Approach 1:
The patent segments the occlusion region handling into foreground and background processing. The foreground occlusion regions use a reduced viewpoint movement path to maintain filling accuracy, while background occlusion regions can utilize larger viewpoint movements for better coverage, thus resolving the contradiction between coverage and filling precision.
Solution Approach 2:
The patent applies local quality by using different viewpoint movement paths for foreground and background occlusion regions. This allows foreground occlusion filling to maintain high precision with reduced viewpoint movement, while background occlusion regions achieve better coverage through larger viewpoint movements, optimizing both coverage and accuracy locally.
Data Source
AI summary
An electronic apparatus is disclosed. The electronic apparatus includes: a memory storing at least one instruction and at least one processor, comprising processing circuitry, individually and/or collectively, configured to: identify a foreground region and a background region included in an input image based on a depth map corresponding to the input image, and generate a novel view image by converting a viewpoint based on the foreground region; identify side effect prediction information including at least one from among whether an object of less than or equal to a specified thickness is included in the input image or an object density degree based on the depth map, and generate the novel view image by controlling a viewpoint movement path based on the identified side effect prediction information.