Virtual Assistant Objects for In-Headset VR Interaction Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users interacting with automated assistants in virtual environments face challenges in controlling the assistant due to audio interference from the VR headset or the need to remove the headset to view responses on a display device.
Innovation Solution
An automated assistant that renders virtual objects in the virtual environment, allowing users to control the assistant through these objects using spoken or gestural inputs, with features that indicate operations or warm words, and can affect both virtual and physical environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If the automated assistant responds through a standalone speaker, then the user can hear the response, but the audio may be interrupted by VR headset audio and the user cannot view visual responses
Solution Approach 1:
The patent introduces a virtual object as an intermediary interface between the user and the automated assistant. This virtual object, rendered in the VR environment, serves as a mediator that consolidates both visual and auditory response capabilities within the VR headset, eliminating the need for separate standalone speakers and display devices. The virtual object acts as a unified communication channel that resolves the conflict between hearing and seeing assistant responses.
Solution Approach 2:
The patent merges the visual display function and audio output function into a single virtual object within the VR environment. By combining these previously separate functions (standalone display device and standalone speaker) into one integrated virtual interface, the system allows users to simultaneously access both visual and auditory information through the VR headset without interference or the need to remove the headset.
2Loss of information
If the user removes the VR headset to view the display device, then the user can see the assistant response, but the user must repeat their spoken utterance and disrupts the immersive experience
Solution Approach 1:
The patent transitions the assistant interaction from the physical dimension (external display devices and speakers) to the virtual dimension (objects within the VR environment). By rendering the assistant interface as a virtual object in three-dimensional space, the system allows users to access visual and auditory information within the immersive VR environment, eliminating the need to exit the VR experience or repeat utterances across multiple devices.
Solution Approach 2:
The virtual object serves as an intermediary that captures and processes user inputs directly within the VR environment, eliminating the need for users to remove the headset or repeat utterances to external devices. The virtual object mediates the entire interaction cycle—receiving spoken commands, processing them, and delivering both visual and auditory responses—within the immersive VR space.
3Extent of automation
If the automated assistant uses a separate device to fulfill requests, then the assistant can perform operations, but the user cannot control the assistant through the virtual environment
Solution Approach 1:
The patent makes the virtual object multi-functional by enabling it to both receive user inputs (spoken and gestural) and execute assistant operations within the VR environment. The virtual object serves as a universal interface that combines control functions and operation execution, allowing users to interact with the automated assistant entirely through the virtual environment without needing to switch to external devices for control or execution.
Solution Approach 2:
The virtual object acts as an intermediary that bridges the gap between user gestures/inputs in the VR environment and the automated assistant's operational capabilities. It receives control inputs from the user within the virtual space and translates them into actionable commands for the assistant, enabling full control and operation execution through the virtual environment without requiring external device interaction.
Data Source
AI summary
Implementations set forth herein relate to an automated assistant that can be accessible via a virtual environment for controlling features of the virtual environment and/or devices in a physical environment of the user. When the automated assistant is invoked, the automated assistant can materialize in the virtual environment according to any request that the automated assistant has been invoked to fulfill. For example, depending on the request from the user, the automated assistant can cause rendering of a virtual object for fulfilling the request and/or controlling an ongoing operation of the automated assistant. When the virtual object is rendered to control an operation of the automated assistant, or another application, the virtual object can include a virtual feature that the user can interact with to control the virtual environment and/or devices in a physical environment of the user.