Virtual Object Response via Machine Learning Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual reality, augmented reality, and mixed reality applications have limited and unnatural responses to user interactions, as they are pre-programmed and do not recognize inputs such as blowing, clapping, whistling, or shouting, restricting the range and realism of virtual object movements.
Innovation Solution
A system utilizing a machine learning model, specifically a generative adversarial network (GAN) like CycleGAN, predicts and generates natural responses of virtual objects to various user interactions, including touch, sound, and air pressure, allowing for unlimited direction and force movements within virtual environments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If pre-programmed response movements are used for virtual objects, then the software application can provide structured and controllable interactions, but the response realism and movement diversity are limited
Solution Approach 1:
The patent replaces the traditional mechanical pre-programmed response system with a machine learning-based predictive system. Instead of using hardcoded if-then logic to determine virtual object movements, the system employs a trained machine learning model that predicts movements based on patterns learned from real-world data, enabling more diverse and realistic responses while maintaining reliability through the model's training on structured datasets
Solution Approach 2:
The patent changes the parameters of the virtual object response system by transitioning from fixed discrete movement options to continuous predicted movements. The machine learning model outputs probabilistic predictions about movement direction, distance, and type, allowing the system to generate responses across a continuous spectrum rather than selecting from predefined discrete options, thereby increasing response diversity
2Device complexity
If pre-programmed response movements are used for virtual objects, then the application logic remains simple and manageable, but the user interaction realism and naturalness deteriorate
Solution Approach 1:
The patent applies preliminary action by training the machine learning model in advance using extensive datasets of real-world interactions and their corresponding outcomes. This pre-training phase allows the model to learn complex patterns and relationships before deployment, so that during actual use, the system can generate realistic responses without requiring complex real-time processing logic, thus maintaining manageable application complexity while improving interaction realism
3Ease of operation
If traditional input recognition methods are used, then the software can recognize standard digital inputs, but it cannot recognize natural physical interactions like blowing, clapping, whistling, or shouting
Solution Approach 1:
The patent applies universality by designing a unified machine learning-based input recognition system that can handle multiple types of inputs through a single framework. Instead of creating separate recognition modules for different input types (touch, sound, air pressure), the system uses a general-purpose predictive model that can process various input modalities, enabling the software to recognize both standard digital inputs and natural physical interactions like blowing, clapping, whistling, and shouting
Data Source
AI summary
Provided is a system and method for moving a virtual object within virtual space in response to an external input supplied by a user. A machine learning model may predict a movement of the virtual object and implement such movement in a next frame of the virtual space. An example operation may include one or more of receiving a measurement of an external input of a user with respect to a virtual object displayed in virtual space, predicting, via execution of a machine learning model, a movement of the virtual object in the virtual space in response to the external input of the user based on the measurement of the external input of the user, and moving the virtual object in the virtual space based on the predicted movement of the virtual object by the machine learning model.