Simulated Reality Display Device Sharing via Gesture Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current simulated reality systems lack the ability to share real-world computing devices, such as smartphones and tablets, among users in a seamless and interactive manner.
Innovation Solution
The implementation of a display device sharing mechanism in a shared simulated reality setting, where users can share and control display devices through gesture inputs, allowing one user to project and interact with the display of another user's device, enabling collaborative interaction and control across distances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If display devices are shared among users in SR settings, then co-presence experience and collaboration are improved, but device complexity and control management become more challenging
Solution Approach 1:
The patent creates virtual copies of display devices in the simulated reality environment. Each user's physical display device is replicated as a virtual object in the SR setting, allowing multiple users to interact with the same device instance simultaneously. This copying approach enables seamless sharing while maintaining individual user control through virtual representations, resolving the contradiction between improved co-presence and manageable complexity.
Solution Approach 2:
The system introduces a virtual representation as an intermediary between the physical display device and multiple users. This virtual object acts as a mediator that receives inputs from one user and transmits them to the actual device, while also allowing other users to interact with the same virtual interface. This intermediary layer simplifies control management by centralizing the interaction point while maintaining versatility for multiple users.
2Ease of operation
If gesture-based control is implemented for device sharing, then ease of operation is improved, but measurement precision and gesture recognition accuracy become more challenging
Solution Approach 1:
The patent extends gesture recognition into the virtual three-dimensional space of the simulated reality environment. Instead of relying solely on two-dimensional screen gestures, the system captures and interprets three-dimensional hand movements and gestures in physical space, mapping them to virtual device interactions. This dimensional extension improves ease of operation by allowing natural hand gestures while enhancing measurement precision through multi-axis motion tracking.
Solution Approach 2:
The system implements real-time feedback mechanisms where the virtual representation immediately responds to user gestures with visual and/or haptic feedback. This feedback loop helps users calibrate their gestures and understand recognition accuracy, allowing them to adjust their movements for more precise control. The feedback system continuously refines gesture recognition by learning from user interactions, improving both ease of operation and measurement precision over time.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
In one implementation, a method of providing display device sharing and interactivity in simulated reality is performed at a first electronic device including one or more processors and a non-transitory memory. The method includes obtaining a gesture input to a first display device in communication with the first electronic device from a first user, where the first display device includes a first display. The method further includes transmitting a representation of the first display to a second electronic device in response to obtaining the gesture input. The method additionally includes receiving an input message directed to the first display device from the second electronic device, where the input message includes an input directive obtained by the second electronic device from a second user. The method also includes transmitting the input message to the first display device for execution by the first display device.