Mediated Reality Virtual Controller Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mediated reality technologies, such as augmented and virtual reality, face challenges in seamlessly integrating user interactions with virtual control elements and sound control within the virtual environment, limiting intuitive and immersive user experiences.
Innovation Solution
A method that enables the display and interaction with virtual control elements within a virtual visual scene, representing physical controller devices, and simultaneous rendering of sound control elements to allow users to control corresponding sound scenes, using user actions to determine the point of view and interactions within the virtual environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If virtual control elements are displayed in the virtual visual scene to enable user interaction, then user interaction capability is improved, but system complexity increases
Solution Approach 1:
The patent creates virtual copies of physical controller elements within the virtual visual scene. These virtual control elements replicate the appearance and functionality of physical controls, allowing users to interact with virtual representations that mirror real-world devices. This copying approach enables intuitive interaction without requiring complex new interface paradigms.
Solution Approach 2:
The virtual control elements serve as intermediaries between the user and the virtual environment. By displaying virtual representations of physical controllers within the VR/AR scene, the system provides a mediating interface that bridges physical user actions and virtual environment responses, simplifying the interaction model.
2Loss of information
If control elements are displayed only when users look at physical controllers, then information overload is reduced, but interaction responsiveness decreases
Solution Approach 1:
The system performs preliminary detection of user actions (such as picking up or approaching a physical controller) and pre-loads or pre-displays the corresponding virtual control elements in the virtual visual scene. This preliminary action ensures that control elements are ready for immediate interaction when the user looks at them, eliminating delays while maintaining a clean interface when controllers are not in use.
Solution Approach 2:
The display of virtual control elements is made dynamic rather than static. Elements appear and disappear based on real-time detection of user actions and gaze direction. This dynamic behavior allows the interface to adapt to user needs, showing controls only when relevant while maintaining rapid responsiveness to user interactions.
3Measurement precision
If virtual control elements represent physical controller portions, then spatial awareness is improved, but rendering complexity increases
Solution Approach 1:
The virtual control elements are rendered with local quality corresponding to their physical counterparts. Each virtual element represents a specific portion of a physical controller with appropriate visual characteristics, positioning, and scaling. This localized rendering approach maintains spatial accuracy without requiring complete reconstruction of entire controller assemblies, optimizing rendering efficiency.
Data Source
Figure 1A~2B
Figure 1C~3B
Figure 4~5B
AI summary
According to various, but not necessarily all, embodiments of the invention there is provided a method comprising: enabling display to a user of a virtual visual scene; in dependence upon detection of an occurrence of a predetermined action by a user, starting display to the user of one or more control visual elements in the virtual visual scene while the virtual visual scene is being displayed to the user; and enabling interaction by the user with the one or more control visual elements within the displayed virtual visual scene to enable control while the virtual visual scene is being displayed to the user, wherein the one or more control visual elements displayed in the virtual visual scene represent a controller device physically used by the user , within the virtual visual scene, wherein the predetermined action by the user is in relation to a first portion of the physical controller device physically used by the user, and wherein the one or more control visual elements displayed in the virtual visual scene represent at least the first portion of the physical controller device within the virtual visual scene.