AR Occlusion Handling via Pixel Classification and Alpha Matting

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for handling occlusion in augmented reality lack robustness, particularly in poor environmental conditions such as low light or insufficient color contrast, and in dynamic scenes.

Innovation Solution

The method involves generating a pixel classification image that includes an occluding object, with foreground, background, and unknown pixels, using color and depth information. This image is then used to initiate an alpha matting process, creating an alpha mask to blend virtual objects with the foreground, thereby rendering a final augmented reality image with proper occlusion.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional occlusion handling methods are used, then the system is simpler to implement, but the accuracy and robustness deteriorate in poor environmental conditions and dynamic scenes

Engineering Contradiction:

Improverobustness of occlusion handlingVSAvoidcomplexity of processing system

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent segments the image processing into distinct stages: initial pixel classification based on color probability distributions, depth-based filtering, and alpha matting for unknown pixels. This segmentation allows each stage to focus on specific aspects of occlusion handling, improving robustness while maintaining manageable system complexity through modular processing

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent changes multiple parameters simultaneously: it uses both color space parameters (foreground color probability distribution) and depth parameters (normalized depth image) to classify pixels. By adjusting and combining these different parameter types, the system achieves improved reliability in various environmental conditions without requiring an overly complex device architecture

Inventive Principle:

Principle #35Parameter changes

2Measurement precision

If pixel classification and alpha matting are used, then the accuracy of occlusion handling is improved, but the computational time and resources increase

Engineering Contradiction:

Improveaccuracy of pixel classificationVSAvoidprocessing time for occlusion rendering

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The patent performs preliminary pixel classification using color probability distributions and depth information before the alpha matting process. By pre-classifying pixels into foreground, background, and unknown categories, the system reduces the computational burden during the final rendering stage, maintaining high accuracy while reducing overall processing time

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent applies alpha matting selectively only to unknown pixels that require blending, rather than processing all pixels uniformly. This partial action approach maintains measurement precision for critical areas while reducing unnecessary computational overhead on already-classified foreground and background pixels, thereby optimizing the balance between accuracy and processing time

Inventive Principle:

Principle #16Partial or excessive action

3Reliability

If color and depth information are processed, then the robustness in low light and low contrast conditions is improved, but the use of computational resources increases

Engineering Contradiction:

Improverobustness in poor environmental conditionsVSAvoidcomputational energy consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent merges color information (foreground color probability distribution) with depth information (normalized depth image) to create a combined classification approach. This merging allows the system to leverage complementary strengths of both data types - color for object identification and depth for spatial relationships - improving robustness in challenging environmental conditions while optimizing computational energy usage through integrated processing

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS12307732B2Methods for handling occlusion in augmented reality applications using memory and device tracking and related apparatus

Publication Date: 2025.05.20 TELEFONAKTIEBOLAGET LM ERICSSON (PUBL)

AI summary

A method performed by a device for occlusion handling in augmented reality is provided. The device can generate at least one pixel classification image in a frame including an occluding object and having foreground, background, and unknown pixels. Generation of the at least one pixel classification image can include (1) calculating an initial foreground pixel probability image, and an initial background pixel probability image, and (2) calculating a normalized depth image based on depth information of the occluding object. The device can obtain an alpha mask to blend a virtual object and the foreground of the at least one pixel classification image based on determining a color of the unknown pixels. The device can render a final composition of an augmented reality image containing the virtual object occluded by the occluding object based on applying the alpha mask to pixels in the at least one pixel classification image.