Head-Mounted Display Inpainting for Real-World View Obstructions

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing head-mounted display (HMD) devices struggle to provide an immersive augmented reality experience by effectively inpainting objects obstructing the user's view in real environments, leading to a diminished sense of presence and engagement.

Innovation Solution

The HMD device employs a method and system to detect objects in a real environment using depth information, identify target objects based on spatial proximity, and inpaint regions corresponding to these objects, thereby enhancing the immersive experience by removing visual obstructions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If object detection and inpainting are performed in real-time for augmented reality, then immersive experience is improved, but processing time and computational complexity increase

Engineering Contradiction:

Improveimmersive experienceVSAvoidprocessing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary object detection and depth information extraction before the inpainting operation. By detecting objects and obtaining their depth information in advance, the system prepares the necessary data structures and identifies target objects for inpainting, reducing the actual processing time when inpainting is needed.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The processing pipeline is segmented into distinct stages: object detection, depth information extraction, target object identification, and inpainting execution. This segmentation allows each stage to be optimized independently and enables parallel processing where possible, improving overall processing efficiency.

Inventive Principle:

Principle #1Segmentation

2Measurement precision

If depth information is obtained for all detected objects, then inpainting accuracy is improved, but computational load increases

Engineering Contradiction:

Improveinpainting accuracyVSAvoidcomputational load

Core Design Contradiction:

Measurement precisionVSPower

Solution Approach 1:

The system applies different processing quality levels to different objects based on their relevance. Depth information is obtained with high precision for target objects that require inpainting, while other detected objects receive minimal or no depth processing. This local quality approach maintains inpainting accuracy for critical objects while reducing overall computational load.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

Instead of obtaining depth information for all detected objects with equal precision, the system performs partial action by focusing computational resources only on objects that are likely to be inpainted (target objects). This selective approach reduces unnecessary computational expenditure while maintaining sufficient accuracy for the actual inpainting tasks.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS20250370266A1Head-mounted display device and operation method of the same

Publication Date: 2025.12.04 SAMSUNG ELECTRONICS CO LTD

AI summary

Provided are a head-mounted display (HMD) device and an operation method of the head-mounted display (HMD) device. The method may include obtaining an original image by capturing a real environment, detecting at least one object included in the original image, obtaining depth information of the detected at least one object using the original image, identifying a target object from among the detected at least one object, based on depth information, inpainting a region corresponding to the identified target object in the original image and displaying the inpainted image.