Multi-Scale Color Transfer for Occlusion-Resistant AR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Color Transfer techniques in Augmented Reality environments face issues with occlusion and scale changes, leading to incorrect or less-than-ideal color adjustments of virtual objects due to inaccurate color difference measurements.
Innovation Solution
The method employs color measurement features at multiple scales, utilizing an illumination invariant correlation method to exclude occluding object colors and adjust virtual object colors based on accurate measurements, ensuring robustness against occlusion and scale changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If known Color Transfer technique is used to adjust virtual object colors, then color adaptation to lighting conditions is achieved, but measurement precision deteriorates due to occlusion and scale changes
Solution Approach 1:
The patent segments the image target into multiple regions and extracts color measurement features from different spatial locations. By dividing the image into multiple measurement regions rather than using a single global color measurement, the system can identify and exclude occluded regions while maintaining accurate color transfer from visible regions. This segmentation approach directly addresses the occlusion problem by allowing selective use of valid color measurements.
Solution Approach 2:
The patent extends color measurement from a single scale to multiple scales by extracting color features at different resolutions or depths. This multi-scale approach allows the system to capture color information that is invariant to scale changes in the image target, thereby maintaining measurement precision even when the target appears at different sizes. The multi-dimensional feature extraction resolves the scale change sensitivity issue.
2Adaptability or versatility
If color difference measurements are performed using known technique, then color transfer function can be derived, but reliability deteriorates when occluding objects are present
Solution Approach 1:
The patent applies local quality by extracting color measurement features from specific local regions of the image target rather than using global average colors. By identifying regions with reliable color information and excluding occluded regions, the system maintains robust color transfer even when parts of the image are occluded. Each local region contributes differently to the overall color transfer based on its reliability.
Solution Approach 2:
The patent performs preliminary analysis to identify occluded regions and select reliable color measurement features before deriving the color transfer function. By pre-screening measurement regions and excluding those affected by occlusion or scale changes, the system ensures that only reliable color differences are used in the transfer function derivation, thereby maintaining adaptability to lighting changes while ensuring measurement robustness.
3Device complexity
If single-scale color measurement is used, then processing simplicity is maintained, but measurement precision deteriorates under scale changes
Solution Approach 1:
The patent creates a multi-scale color measurement system that serves multiple functions: it accurately measures colors at different scales, identifies occluded regions, and provides robust color transfer regardless of target size variations. This universal approach handles both scale changes and occlusion with a single integrated methodology, maintaining measurement precision while managing complexity through unified processing.
Data Source
Figure 1
Figure 2
Figure 3A~3C
AI summary
Disclosed is a method and apparatus for using color measurement features at multiple scales for a Color Transfer technique. In one embodiment, the functions implemented include: resizing a ground truth image target frame to a plurality of different scales; selecting one or more color measurement features from the ground truth image target frame at each of the plurality of different scales; making a color measurement for each color measurement feature in the ground truth image target frame; and adjusting colors of a virtual object in an augmented frame based at least in part on the color measurements.