Enhanced Caption Generation for Hearing Assistance

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing captioning systems for hard of hearing users struggle to provide clear and concise real-time captions during voice communications, especially in complex conversations with discontinuous speech and lack of context, leading to confusion and difficulty in understanding intended meanings.

Innovation Solution

A system that generates enhanced captions by simplifying complex words, providing context, and offering summary-type captions to improve comprehension, along with communication augmentation by providing additional information and initiating supplemental activities based on the conversation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If verbatim captions are provided in real-time, then accuracy of transcription is improved, but comprehension clarity deteriorates due to complex words and lack of context

Engineering Contradiction:

Improvetranscription accuracyVSAvoidcomprehension clarity

Core Design Contradiction:

Measurement precisionVSEase of operation

Solution Approach 1:

The caption is divided into two distinct components: verbatim transcription for accuracy and enhanced summary for clarity. The enhanced summary segment separately processes the caption to provide simplified, context-rich representations, allowing users to access both precise transcription and improved comprehension without compromise

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

An intermediary processing layer is introduced between the raw verbatim caption and the user's comprehension. This intermediary enhances the caption by generating summary-type representations that maintain accuracy while improving clarity through context addition and language simplification

Inventive Principle:

Principle #24Intermediary (Mediator)

2Speed

If caption processing is performed in real-time, then responsiveness is improved, but processing complexity increases due to continuous analysis requirements

Engineering Contradiction:

Improvecaption responsivenessVSAvoidprocessing complexity

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

The system performs preliminary actions by pre-processing captions to identify complex words, discontinuous speech patterns, and context gaps before presentation. This advance preparation enables the system to respond quickly to user needs while managing processing complexity through proactive rather than reactive analysis

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The caption processing system dynamically adjusts its complexity based on real-time conditions. It monitors speech patterns, user interaction, and context to modulate the level of enhancement applied, providing detailed processing when needed and simplified processing when context is sufficient, thus managing computational resources efficiently

Inventive Principle:

Principle #15Dynamics

3Ease of operation

If enhanced captions with simplification are provided, then comprehension clarity is improved, but transcription fidelity deteriorates

Engineering Contradiction:

Improvecomprehension clarityVSAvoidtranscription fidelity

Core Design Contradiction:

Ease of operationVSMeasurement precision

Solution Approach 1:

The caption output is segmented into verbatim and enhanced portions, allowing the system to maintain full transcription fidelity in the verbatim segment while applying simplification and context enhancement in the summary segment. This segmentation enables users to access both accurate transcription and improved comprehension

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different quality levels are applied locally to different portions of the caption. The verbatim portion maintains high fidelity for accuracy-critical segments, while the enhanced summary applies localized simplification and context enhancement to improve comprehension without compromising overall transcription accuracy

Inventive Principle:

Principle #3Local quality

4Productivity

If context and additional information are added to captions, then communication effectiveness is improved, but caption length and processing time increase

Engineering Contradiction:

Improvecommunication effectivenessVSAvoidcaption processing time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system applies partial enhancement by adding context and additional information selectively rather than universally. It identifies when context addition is beneficial (e.g., for complex words or discontinuous speech) and applies enhancement only in those cases, avoiding unnecessary processing time for straightforward captions

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS20240308252A1Caption modification and augmentation systems and methods for use by hearing assisted users

Publication Date: 2024.09.19 ULTRATEC INC

AI summary

A system and method for facilitating communication between an assisted user (AU) and a hearing user (HU) includes receiving an HU voice signal as the AU and HU participate in a call using AU and HU communication devices, transcribing HU voice signal segments into verbatim caption segments, processing each verbatim caption segment to identify an intended communication (IC) intended by the HU upon uttering an associated one of the HU voice signal segments, for at least a portion of the HU voice signal segments (i) using an associated IC to generate an enhanced caption different than the associated verbatim caption, (ii) for each of a first subset of the HU voice signal segments, presenting the verbatim captions via the AU communication device display for consumption, and (iii) for each of a second subset of the HU voice signal segments, presenting enhanced captions via the AU communication device display for consumption.