Multimodal Speech Input Using Timed Controller Cues

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing input methods for entertainment systems, such as videogame consoles, are limited in their ability to efficiently and accurately convert spoken language into text, particularly for users with strong accents, atypical speech, or in noisy environments, leading to unreliable speech-to-text recognition.

Innovation Solution

An input system and method that utilizes phonetic input structures based on syllables, allowing users to input words phonetically using handheld controllers, combined with dynamic phoneme selection and physical input enhancements to improve speech recognition accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If speech-to-text recognition is used for text input, then text input efficiency is improved, but recognition accuracy deteriorates for users with strong accents, atypical speech, or in noisy environments

Engineering Contradiction:

Improvetext input efficiencyVSAvoidrecognition accuracy

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system combines speech input with phonetic input methods, merging two different input approaches into a unified system that leverages the strengths of both while mitigating their individual weaknesses

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system introduces phonetic intermediaries that bridge the gap between spoken language and text representation, providing an alternative pathway for text input that is less susceptible to accent and noise issues

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If traditional virtual keyboard navigation is used, then text input reliability is maintained, but text input speed deteriorates

Engineering Contradiction:

Improvetext input reliabilityVSAvoidtext input speed

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system segments the text input process into phonetic units that can be selected and combined more efficiently than individual characters, breaking down the complex task of character-by-character selection into manageable phonetic components

Inventive Principle:

Principle #1Segmentation

3Productivity

If speech recognition is used, then text input speed is improved, but ease of operation deteriorates for users with speech difficulties

Engineering Contradiction:

Improvetext input speedVSAvoidease of use for users with speech difficulties

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The system creates a universal text input interface that supports multiple input modalities including both speech and phonetic input, making it accessible to users with different abilities and preferences

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS20250384887A1Input system and method

Publication Date: 2025.12.18 SONY INTERACTIVE ENTERTAINMENT LLC

AI summary

An input system comprises a spoken input processor configured to receive spoken inputs from a microphone operably coupled to the input system; a physical input processor configured to receive physical inputs from a peripheral device operably coupled to the input system, the physical inputs having a timing relative to the spoken inputs; and a speech recognition processor configured to recognise speech from the speech inputs; wherein the speech recognition processor uses at least some of the received physical inputs, and their timing relative to the spoken inputs, as part of the speech recognition process.