Adapting Sensory Datastreams in VR via Machine Learning

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current augmented and virtual reality environments lack the ability to adapt sensory input streams in response to individual users' emotional states and sensitivities, leading to inconsistent user experiences due to varying sensitivities and personal preferences.

Innovation Solution

A system and method using machine learning and reinforcement learning to process sensory data streams, determining desired emotional states, and adjusting the intensity of sensory inputs such as sound, vision, smell, taste, and touch based on user reactions, allowing for adaptive and personalized experiences without user feedback.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If standardized sensory datastreams are used in VR/AR environments, then device complexity is reduced and ease of operation is improved, but user experience consistency deteriorates due to varying individual sensitivities and preferences

Engineering Contradiction:

Improveease of operationVSAvoidadaptability

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The system dynamically adjusts sensory datastream parameters (intensity, volume, brightness, etc.) based on real-time monitoring of user physiological responses and emotional states. The standardized datastreams are transformed into personalized experiences through continuous adaptation, resolving the contradiction between operational simplicity and user-specific adaptability.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The invention changes multiple sensory parameters simultaneously (audio volume, visual brightness, haptic intensity, olfactory concentration) based on detected user state. By modifying these parameters in response to physiological measurements, the system maintains ease of operation while achieving high adaptability to individual user sensitivities and preferences.

Inventive Principle:

Principle #35Parameter changes

2Productivity

If sensory datastream intensity is increased to enhance user experience, then user engagement is improved, but harmful effects increase for users with hyperesthesia or sensitivity conditions

Engineering Contradiction:

Improveuser engagementVSAvoidharmful effects

Core Design Contradiction:

ProductivityVSObject-affected harmful factors

Solution Approach 1:

The system implements continuous feedback loops where user physiological responses (heart rate, skin conductance, respiration) are monitored and used to adjust sensory datastream intensity in real-time. This feedback mechanism prevents harmful effects by automatically reducing intensity when stress or discomfort is detected, while maintaining high engagement during optimal states.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system takes preliminary anti-action by detecting early signs of user stress or discomfort through physiological sensors and preemptively adjusting sensory intensity before harmful effects occur. This prevents hyperesthesia-related issues by counteracting increasing sensitivity trends before they become problematic.

Inventive Principle:

Principle #9Preliminary anti-action

3Adaptability or versatility

If user feedback is collected to personalize sensory experiences, then adaptability is improved, but system complexity and user burden increase

Engineering Contradiction:

ImproveadaptabilityVSAvoidsystem complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system performs self-service by automatically collecting and processing user physiological data without requiring explicit user feedback or input. Sensors embedded in the VR/AR device continuously monitor heart rate, skin conductance, and respiration, and the machine learning model automatically adjusts sensory parameters, eliminating the need for user surveys or manual adjustments while achieving high adaptability.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The invention introduces physiological sensors and machine learning models as intermediaries between the user and the sensory datastreams. These intermediaries automatically translate physiological states into appropriate sensory adjustments, reducing system complexity by replacing complex user feedback mechanisms with automated sensor-based control.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20240342429A1Adaptation of senses datastreams in virtual reality and augmented reality environments

Publication Date: 2024.10.17 TELEFONAKTIEBOLAGET LM ERICSSON (PUBL)

AI summary

In one aspect, a computer-implemented method of adapting a sensory datastream is provided. The method includes obtaining a raw sensory datastream from a source, wherein the raw sensory datastream comprises input for a sensory actuator of a user device. The method includes obtaining state information, wherein the state information comprises information indicating a first state of a user. The method includes predicting, using a machine learning model, a desired second state of the user based on the obtained state information. The method includes determining an action to adapt the raw sensory datastream based on the desired second state of the user. The method includes adapting the raw sensory datastream in accordance with the determined action and the first state of the user to create a processed sensory datastream. The method includes providing the processed sensory datastream to the sensory actuator of the user device.