AR Surface Segmentation via Rectangle Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional head-mounted display units are limited by processing power and resource constraints, resulting in rudimentary and inconvenient user interfaces for augmented reality applications.
Innovation Solution
The implementation of augmented reality surface segmentation using rectangle tracking and finger tracking techniques, which utilize a head-mounted camera to identify reference objects and render user interfaces correctly aligned with physical surfaces, enabling more intuitive and functional interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional head-mounted display units are used with basic processing, then device complexity is reduced, but user interface functionality and intuitiveness deteriorate
Solution Approach 1:
The patent introduces an intermediary processing system that separates complex AR processing from the head-mounted display unit. A server or remote computing device handles the computationally intensive tasks of surface segmentation, rectangle tracking, and UI rendering, while the HMD unit focuses on capturing images and displaying results. This mediator approach enables sophisticated user interfaces without overloading the wearable device's processing capabilities.
Solution Approach 2:
The system creates a virtual copy of the physical workspace by detecting surfaces and rectangles in the real world, then rendering corresponding virtual UI elements on these detected surfaces. This copying approach allows the system to provide rich, context-aware user interfaces that mirror the physical environment without requiring complex 3D modeling or reconstruction of the entire scene.
2Adaptability or versatility
If advanced surface segmentation and tracking techniques are implemented, then user interface alignment and functionality improve, but processing power requirements increase
Solution Approach 1:
The patent divides the complex task of AR processing into distinct segments: (1) image capture by the HMD camera, (2) surface detection and segmentation, (3) rectangle tracking and identification, (4) UI rendering. Each segment can be processed independently, with the most computationally intensive segments (surface segmentation, rectangle tracking) performed on a remote server rather than on the power-constrained HMD device.
Solution Approach 2:
Instead of performing complete 3D reconstruction and full scene understanding, the system applies partial action by focusing only on detecting surfaces and rectangles that are sufficient for the intended UI functionality. This selective processing approach provides adequate surface alignment capability without the excessive processing power requirements of complete environmental reconstruction.
3Manufacturing precision
If rectangle tracking and reference object identification are used, then virtual object alignment accuracy improves, but computational complexity increases
Solution Approach 1:
The system employs self-service mechanisms where detected rectangles automatically serve as reference objects for UI alignment without requiring manual calibration or user intervention. Once rectangles are detected in the scene, they autonomously provide the geometric constraints needed to align virtual UI elements with the physical surface, enabling high alignment accuracy through automated geometric reasoning.
Data Source
AI summary
Methods, systems, computer-readable media, and apparatuses for providing intuitive, functional, and convenient ways of enabling a user of a head-mounted display unit or another augmented reality enabled device to interact with various user interfaces and other features provided by such a unit or device are presented. In some embodiments, a computing device, such as a head-mounted display unit, may receive camera input of a scene. Subsequently, the computing device may identify at least one reference object in the scene, for example, based on detecting one or more rectangles in the received camera input. The computing device then may receive input that defines a surface segment relative to the at least one reference object. Thereafter, the computing device may render the surface segment.