Video Game Sequence Emotion Prediction Using Multimodal Biosignals
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video game technologies fail to accurately predict and adapt to the emotional state of players due to noisy dermo-galvanic signals from skin sensors, which are disturbed by artifacts at the interface, leading to suboptimal gaming experiences.
Innovation Solution
A method using neural networks to process audio and video streams, combined with biosignal analysis, to generate timestamped descriptors and predict emotional states, allowing for personalized game adaptation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If dermo-galvanic sensors are used to detect player emotional state, then emotional detection capability is improved, but signal quality deteriorates due to noise and artifacts from skin interface
Solution Approach 1:
The patent introduces an intermediary processing layer (signal filtering and normalization algorithms) between the dermo-galvanic sensors and the emotional state analysis system. This intermediary component removes noise and artifacts from the skin interface while preserving the underlying emotional signal patterns, thereby resolving the contradiction between maintaining sensor contact for detection and eliminating interface-induced noise.
Solution Approach 2:
The patent replaces direct reliance on noisy mechanical/electrical skin contact sensors with a computational approach using machine learning algorithms that process and interpret the sensor data. Instead of depending solely on clean physical signals, the system substitutes mechanical signal quality requirements with computational signal processing capabilities, allowing emotional detection despite noisy physical interfaces.
2Measurement precision
If multiple biosignal processing methods are integrated for accurate emotional prediction, then prediction accuracy is improved, but system complexity increases
Solution Approach 1:
The patent merges multiple biosignal processing methods (dermo-galvanic signal processing, video analysis, audio processing) into a unified machine learning framework. By combining these diverse signal sources and processing approaches into a single integrated system, the patent achieves improved emotional prediction accuracy while managing complexity through unified architecture rather than separate independent systems.
Solution Approach 2:
The patent creates a multi-functional system where a single machine learning framework handles multiple types of biosignals (electrical skin responses, video facial expressions, audio vocal cues). This universal processing approach allows one system to perform multiple emotional detection functions simultaneously, improving accuracy through multi-modal analysis while avoiding the complexity of separate dedicated systems for each signal type.
Data Source
AI summary
A method is provided for automatically predicting the emotional effect produced by a video game sequence, comprising labeling sequences of the game by automatically generating descriptors at time sequences of the game, the labeling comprising applying digital processing to the audio stream of the video game sequence using a neural network architecture and an NLP coding layer, to extract a first series of timestamped descriptors, and applying digital processing to the video stream to provide a second series of timestamped descriptors for characterizing the scenes of each image of the video stream, and transmitting them as M-tuples to a neural network. The method also comprises processing biosignals to extract timestamped signals and transmit them as N-tuples to a neural network and processing the M-tuples corresponding to the timestamped descriptors and the N-tuples to provide at least one indicator predicting the emotional state brought about by a type of audiovisual sequence.