Sentiment-Aware In-Game Overlays for Hearing-Impaired Players

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Hearing impaired players often miss audio cues in video games, leading to an incomplete gaming experience due to inadequate sub-titling, which affects their ability to understand the game context and urgency, making it difficult for them to enjoy the game as non-impaired players do.

Innovation Solution

Implementing overlay logic in game engines to detect user sentiment during gameplay and generate in-game effects, such as modified sub-titling and animations, to convey the intended sentiment and context, using artificial intelligence to predict player preferences and personalize in-game experiences.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If sub-titling is used to convey audio information to hearing impaired players, then accessibility is improved, but the sub-titling is inadequate or cumbersome and fails to convey urgency and context in a timely manner

Engineering Contradiction:

ImproveaccessibilityVSAvoidtimeliness of information conveyance

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The system segments the audio information into multiple channels: visual subtitles for basic information, color-coded overlays for emotional context, and icon-based urgency indicators. This segmentation allows hearing impaired players to receive information simultaneously across different visual modalities without the cumulative burden of traditional text-only subtitles.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent employs color changes in the overlay display to convey different types of information and urgency levels. Different colors indicate different emotional states or urgency levels, allowing players to quickly grasp context without reading lengthy text. This visual encoding provides timely information conveyance that text alone cannot achieve.

Inventive Principle:

Principle #32Color changes

2Adaptability or versatility

If traditional audio effects are used in video games, then the gaming experience is immersive and engaging, but hearing impaired players cannot fully experience or appreciate the audio effects

Engineering Contradiction:

Improvegaming experience accessibilityVSAvoidaudio information loss

Core Design Contradiction:

Adaptability or versatilityVSLoss of information

Solution Approach 1:

The system introduces visual overlays as an intermediary medium that translates audio information into visual representations. These overlays act as a bridge between the audio effects and hearing impaired players, conveying the same emotional and contextual information that audio would normally provide, thus eliminating the loss of information.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent substitutes the acoustic channel with a visual channel for conveying game context and urgency. Instead of relying on audio mechanics, the system uses visual overlays, color changes, and icon animations to transmit information, making the gaming experience accessible to hearing impaired players without sacrificing immersion or engagement.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Adaptability or versatility

If overlay logic detects user sentiment and generates in-game effects, then the game becomes more engaging and intense for impaired players, but the system complexity increases

Engineering Contradiction:

Improvepersonalization of gaming experienceVSAvoidoverlay logic complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The overlay logic is designed as a multi-functional system that simultaneously performs sentiment detection, context analysis, and visual effect generation. By consolidating these functions into a single overlay component, the system achieves personalized gaming experiences without proportionally increasing overall system complexity. The overlay serves multiple purposes: it translates audio to visual, detects player state, and generates contextual effects.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The overlay logic operates autonomously by automatically detecting user sentiment and generating appropriate visual effects without requiring manual player input or configuration. The system self-adjusts based on real-time sentiment analysis and game context, providing personalized experiences while minimizing the complexity burden on the player and simplifying the development process.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS20250213982A1User sentiment detection to identify user impairment during game play providing for automatic generation or modification of in-game effects

Publication Date: 2025.07.03 SONY INTERACTIVE ENTERTAINMENT LLC

AI summary

A method including executing game logic of a video game to generate a plurality of video frames for a game play of the video game by a player, wherein game state data is generated during the executing the game logic. The method including determining a current context in the game play of the video game based on the game state data. The method including determining a user sentiment of the player towards the current context in the game play of the video game. The method including determining that the user sentiment of the player is inconsistent with an expected user sentiment for the current context. The method including generating an in-game effect for the current context. The method including presenting the in-game effect simultaneous with one or more video frames associated with the current context in the game play of the video game.