CG Image Channel Separation for MR Temporal Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In mixed reality (MR) systems, the combination of captured and computer graphics (CG) images often results in temporal mismatches, causing discomfort for users due to delays in CG image rendering, and can also disrupt the depth relationship between objects in the images, leading to incorrect spatial alignment.
Innovation Solution
An image processing apparatus that separates CG images into color and combining information channels, performs pixel interpolation for color channels, and pixel replacement for depth and alpha channels to correct temporal mismatches and maintain accurate depth relationships, ensuring seamless integration with captured images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If image correction is performed to cancel the delay of the CG image, then the temporal alignment between CG image and captured image is improved, but the depth relationship between objects in the captured image and CG image becomes incorrect
Solution Approach 1:
The patent segments the CG image into multiple channels (color channels and combining information channels including depth and alpha channels). Different correction methods are applied to different channels: pixel interpolation for color channels and pixel replacement for combining information channels. This segmentation allows temporal alignment correction without compromising depth relationship accuracy.
Solution Approach 2:
The patent applies different correction qualities to different parts of the image data. Color channels receive pixel interpolation for smooth temporal alignment, while combining information channels (depth and alpha) receive pixel replacement to preserve exact depth relationships. This local quality differentiation resolves the contradiction between temporal alignment and depth accuracy.
2Loss of time
If pixel interpolation is applied to correct the CG image, then temporal alignment is improved, but the depth information and alpha channel accuracy deteriorate
Solution Approach 1:
The patent separates the CG image into color channels and combining information channels (depth and alpha). Pixel interpolation is applied only to color channels for temporal alignment, while pixel replacement is used for combining information channels to preserve precision. This segmentation prevents degradation of depth and alpha channel accuracy.
Solution Approach 2:
Different correction qualities are applied locally to different channels: pixel interpolation for color channels and pixel replacement for combining information channels. This ensures that temporal alignment is achieved where needed without compromising the precision of depth and alpha information.
3Loss of time
If the CG image is corrected using conventional image correction, then the delay is reduced, but the combining information (depth and alpha channels) becomes inaccurate
Solution Approach 1:
The patent segments the CG image into color channels and combining information channels, applying different correction methods to each. This segmentation allows conventional pixel interpolation to be used for color channels while protecting the combining information channels from degradation through pixel replacement.
Solution Approach 2:
The patent applies different correction qualities to different parts of the image data: pixel interpolation for color channels and pixel replacement for combining information channels. This local differentiation resolves the contradiction between reducing delay and maintaining combining information accuracy.
Data Source
AI summary
An image processing apparatus includes an acquisition unit configured to acquire a captured image obtained by an imaging apparatus capturing a real world, a motion acquisition unit configured to acquire motion information on the imaging apparatus in a space of the real world, a CG generation unit configured to generate a CG image including combining information related to combining with the captured image, based on the motion information on the imaging apparatus, a CG correction unit configured to separate the CG image including the combining information into an image channel and a combining information channel, correct the image channel by pixel interpolation according to the motion information on the imaging apparatus, and correct the combining information channel by pixel replacement according to the motion information on the imaging apparatus, and a combining unit configured to combine the captured image and the CG image corrected by the CG correction unit.