Interactive Virtual Objects in XR Replays With Pose Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices with near-eye displays struggle to provide immersive extended reality experiences that allow users to interact with virtual objects in a way that mimics real-world interactions, particularly in multiuser communication sessions where maintaining spatial relationships and user roles is challenging.
Innovation Solution
The electronic device incorporates sensors, displays, processors, and memory to enable extended reality environments with interactive virtual objects, allowing users to switch between different viewing modes and roles, such as presenter or audience, while maintaining spatial relationships and enabling communication based on user input and sensor feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the viewpoint is positioned within the three-dimensional environment for immersive viewing, then the immersion experience is improved, but the ability to observe and interact with the virtual environment from external perspectives is limited
Solution Approach 1:
The system dynamically switches between first viewing mode (viewpoint within the three-dimensional environment) and second viewing mode (viewpoint external to the representation). The processor detects user inputs and automatically transitions between viewing modes, allowing the display perspective to be flexible and adaptive rather than fixed.
Solution Approach 2:
The near-eye display device provides multiple viewing modes that serve different functions: the first viewing mode enables immersive experience for presence, while the second viewing mode enables observation and interaction from external perspectives. Both modes are integrated into a single device, making it universally capable of different interaction scenarios.
2Adaptability or versatility
If users can directly communicate with all participants in multiuser sessions, then the interaction capability is improved, but the management of user roles and spatial relationships becomes complex
Solution Approach 1:
The system applies different communication rules to different user roles within the same multiuser session. Presenter role users have different communication permissions compared to audience role users. The processor determines communication capabilities based on the specific role assigned to each user, allowing localized quality control over interaction patterns.
Solution Approach 2:
The system provides feedback to users about their current role and communication permissions. The processor monitors user inputs and adjusts communication capabilities based on role assignments, informing users of their interaction boundaries and enabling appropriate communication behaviors within the structured role system.
3Ease of operation
If the replay allows modification of virtual object poses, then the interactivity is improved, but the complexity of controlling and managing the replay state increases
Solution Approach 1:
The system records the sequence of user inputs and virtual object interactions during the live extended reality session. When entering replay mode, these pre-recorded inputs are processed to automatically reproduce the original session flow, including virtual object manipulations, while allowing users to intervene and modify poses when needed.
Solution Approach 2:
The processor acts as an intermediary between the replay playback and user control inputs. It manages the replay state, processes user modifications to virtual object poses, and reconciles these changes with the recorded session data, simplifying the user's interaction while maintaining complex state management internally.
Data Source
AI summary
An electronic device such as a head-mounted device may present extended reality content such as a representation of a three-dimensional environment. The representation of the three-dimensional environment may be changed between different viewing modes having different immersion levels in response to user input. The three-dimensional environment may represent a multiuser communication session. A multiuser communication session may be saved and subsequently viewed as a replay. There may be an interactive virtual object within the replay of the multiuser communication session. The pose of the interactive virtual object may be manipulated by a user while the replay is paused. Some multiuser communication sessions may be hierarchical multiuser communication sessions with a presenter and audience members. The presenter and audience members may receive generalized feedback based on the audience members during the presentation. A participant may have their role changed between audience member and presenter during the presentation.