XR Landmark Rendering for Precise Touch Interface Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing XR devices face imprecision and inconvenience in receiving user inputs due to imprecise traditional controllers and mid-air gestures lacking tactile feedback, and display interface devices with XR devices suffer from desynchronization and visual artifacts.
Innovation Solution
Utilizing a display interface device that displays recognizable landmark patterns, allowing an XR device to overlay a virtual interface over these patterns, providing precise tactile feedback and improving input precision by aligning with interface elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional video game controllers are used for XR input, then users can provide inputs to control virtual objects, but the input precision is insufficient for fine scrolling or sliding adjustments
Solution Approach 1:
The patent creates a virtual copy of a physical display interface (touchscreen, trackpad, or mouse) within the XR environment. This virtual interface replicates the visual and interaction characteristics of the physical device, allowing users to perform precise input operations. The virtual interface elements (buttons, sliders, scroll areas) are rendered to match the aspect ratio and dimensions of the original physical display, enabling accurate mapping of user inputs.
Solution Approach 2:
The patent introduces a camera as an intermediary device that captures the physical display interface and transmits its visual output to the XR device. This intermediary system bridges the physical and virtual environments, allowing the XR system to receive and process inputs from the physical display interface with high precision. The camera acts as a mediator that translates physical display states into virtual reality representations.
2Ease of operation
If mid-air gestures are used for XR input, then users can avoid carrying controllers, but the input precision deteriorates due to lack of tactile feedback and inconsistent hand tracking
Solution Approach 1:
The patent creates a virtual representation of a physical display interface (such as a touchscreen or trackpad) within the XR environment. This virtual interface copy provides tactile feedback through haptic actuators and visual cues, enabling users to interact with virtual elements as if touching a physical surface. The virtual interface maintains the dimensional and aspect ratio properties of the original physical display, ensuring accurate input mapping.
Solution Approach 2:
The patent replaces the mechanical hand-tracking system with an optical-capture-based input system. Instead of relying on computer vision to track hand movements in mid-air, the system uses a camera to capture the physical display interface and translates its visual output directly into virtual reality. This substitution eliminates the need for complex hand-tracking algorithms and provides more consistent and precise input detection.
3Reliability
If display interface devices are integrated with XR devices, then users can receive tactile feedback, but visual artifacts and desynchronization occur between the captured and displayed images
Solution Approach 1:
The patent implements a dynamic rendering system that continuously adjusts the virtual interface representation based on real-time camera capture data. The system processes each video frame independently, dynamically mapping the captured display interface to the corresponding virtual interface elements. This dynamic approach ensures that the virtual interface remains synchronized with the physical display state, adapting to changes in perspective, lighting, and device orientation.
Solution Approach 2:
The patent transforms the visual parameters of the captured display interface (such as brightness, contrast, color space, and resolution) to match the characteristics of the XR display device. By adjusting these parameters, the system eliminates visual artifacts and ensures accurate color and brightness representation. The aspect ratio and dimensional parameters of the virtual interface are calibrated to correspond precisely with the physical display interface.
Data Source
AI summary
Systems and techniques are described for providing a virtual interface for an XR device using a display interface device. A display interface device can display a landmark pattern, such as a quick response (QR) code, on its display. The display can be in the field of view of a camera of the XR device, so that camera captures images depicting the displayed landmark pattern. The XR device can generate, and display on its own display, output images based on the captured images, for instance by overlaying a virtual interface over the landmark pattern. The pose and/or size of the virtual interface in the output images can be based on the pose and/or size of the landmark pattern and/or display in the captured images. The display interface device can receive inputs through its display interface, such as touch-based inputs, which the display interface device can identify for the XR device.