Physical Input Device Overlays for Lower-Complexity CGR Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies struggle to effectively integrate physical input devices with computer-generated reality environments, limiting user interaction and functionality.
Innovation Solution
A method is provided for detecting a physical input device in a real environment using an image sensor and displaying a representation of the input device in a CGR environment, updating the representation based on input data received through a communication bus, and enhancing the device's functionality with virtual features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If physical input devices are integrated with CGR environments, then user interaction and functionality are enhanced, but device complexity increases
Solution Approach 1:
The patent uses an intermediary system comprising image sensors, processors, and communication buses that mediate between physical input devices and CGR environments. This intermediary layer detects physical devices, tracks their positions, and facilitates interaction without requiring direct integration, thereby reducing overall system complexity while enhancing adaptability.
Solution Approach 2:
The system creates virtual representations or copies of physical input devices within the CGR environment. These virtual copies interact with the physical devices through image sensor detection and processing, allowing the physical devices to function within the virtual environment without direct physical integration, thus maintaining simplicity while enabling versatility.
2Ease of operation
If physical input devices are displayed in CGR environment, then user interaction is improved, but power consumption increases
Solution Approach 1:
The system employs periodic action by activating image sensors and updating virtual representations only when necessary - such as when physical devices enter detection zones or when user interaction is anticipated. This periodic operation reduces continuous power consumption while maintaining ease of operation during active use periods.
Solution Approach 2:
Physical input devices utilize their own inherent capabilities (such as touch sensitivity, button mechanisms) to trigger interactions without requiring additional power-consuming components. The system leverages the devices' existing functionality combined with image sensor detection, avoiding the need for power-intensive integration solutions.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances user interaction and efficiency in CGR environments by customizing the functionality of physical input devices with virtual features, reducing clutter and improving operability while conserving power.
Implementation Method 1
detecting a physical input device in a real environment using the image sensor
Data Source
Figure 1A
Figure 1B
Figure 2A
AI summary
The present disclosure relates to techniques for displaying representations of physical input devices and overlaying visual features on the representations of physical input devices in a computer-generated reality (CGR) environment. The techniques include displaying a virtual application in a CGR environment and, in response to detecting an input field in the displayed virtual application, displaying at least a portion of the displayed application on a representation of a physical input device. The at least a portion of the displayed application includes the detected input field. In response to detecting an input received at the physical input device, the input field is updated with the input, and the updated input field is displayed.