Predicted Motion Vector Generation Using Depth Information
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing methods struggle to accurately predict motion vectors for target blocks, especially when they are at boundary portions of foreground objects, leading to deteriorated prediction precision due to occlusion and disparity differences between foreground and background blocks.
Innovation Solution
An image processing device and method that utilizes a depth image with depth information to generate a predicted motion vector for a target block, employing a disparity information use prediction standard to improve prediction accuracy by selecting the most relevant disparity information from surrounding blocks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a predicted motion vector is calculated from disparity vectors of surrounding blocks in a color image, then encoding can be performed, but prediction precision deteriorates when target blocks are at boundary portions of foreground objects due to occlusion and disparity differences
Solution Approach 1:
The patent introduces a depth image as an intermediary to mediate the selection of reference blocks for predicted motion vector calculation. The depth image provides depth information that helps identify foreground blocks among surrounding blocks, allowing the system to select only relevant foreground blocks for prediction while excluding background blocks affected by occlusion or disparity differences. This intermediary depth information resolves the contradiction by enabling accurate block classification without requiring complex analysis of the color image alone.
Solution Approach 2:
The patent applies local quality by treating foreground blocks and background blocks differently in the prediction process. Instead of uniformly using all surrounding blocks, the system uses depth information to identify and selectively apply disparity vectors only from foreground blocks that share similar depth characteristics with the target block. This localized approach ensures that prediction is performed using only relevant blocks, thereby maintaining high prediction precision at boundary portions where occlusion and disparity differences would otherwise degrade performance.
2Quantity of substance
If disparity vectors from all surrounding blocks are used for prediction, then more reference information is available, but prediction accuracy decreases when foreground and background blocks are mixed
Solution Approach 1:
The patent changes the parameter used for block selection from spatial proximity alone to a combination of spatial proximity and depth information. By introducing depth as an additional parameter, the system can filter surrounding blocks to include only those that are both spatially adjacent and depth-consistent with the target block. This parameter change allows the system to maintain a sufficient number of reference blocks while ensuring they all belong to the same foreground object, thereby preserving prediction accuracy.
Solution Approach 2:
The patent segments the set of surrounding blocks into foreground blocks and background blocks using depth information from the depth image. This segmentation allows the system to separately process and select only the relevant foreground blocks for predicted motion vector calculation. By dividing the reference block set based on depth characteristics, the system avoids mixing foreground and background blocks, thus maintaining high prediction accuracy while still utilizing multiple reference blocks.
Data Source
AI summary
The present technique relates to an image processing device, an image processing method and a program which can calculate a vector which is precisely predicted as a predicted motion vector of a disparity vector. A predicted motion vector generation unit 243 generates a predicted motion vector of a disparity vector which represents a disparity of a target block of an operation target of a viewpoint #2 color image C#2 different from a point of view #1 with respect to a color image C#1 of the point of view #1, from a disparity information image D#2 of the point of view #2 which has disparity information related to a disparity of each pixel of the color image C#2 of the point of view #2 as a pixel value according to a depth information use prediction standard. The present invention is applicable to, for example, generate a predicted motion vector of a disparity vector used to generate a predicted image upon encoding or decoding of a plurality of viewpoint images.