Speech Recognition Correction Using Prior Edits and Fulfillment Data

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing speech recognition systems often misrecognize certain words or phrases due to variations in speech among users, leading to prolonged interactions and erroneous responses, necessitating manual corrections through higher latency interfaces.

Innovation Solution

Generate an alternate speech recognition hypothesis by replacing misrecognized terms with alternate terms based on user corrections and fulfillment data, using machine learning models to determine the likelihood of misrecognition and the feasibility of automated assistant actions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If automatic speech recognition is used to process spoken utterances, then speech processing speed is improved, but recognition accuracy deteriorates due to variations in user speech characteristics

Engineering Contradiction:

Improvespeech processing speedVSAvoidrecognition accuracy

Core Design Contradiction:

SpeedVSMeasurement precision

Solution Approach 1:

The system performs preliminary actions by capturing correction data from prior interactions where users manually corrected speech recognition errors. This correction data is stored and later applied to generate alternate hypotheses when similar misrecognitions are detected, allowing the system to learn from past errors proactively rather than reacting to each error individually during real-time processing

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback mechanisms by using fulfillment data from previous speech recognition attempts to identify and correct errors. When a speech recognition hypothesis leads to unsuccessful fulfillment (e.g., the assistant cannot execute the command), the system captures this feedback and uses it to generate corrected alternate hypotheses for future processing

Inventive Principle:

Principle #23Feedback

2Measurement precision

If the system repeatedly attempts to recognize misrecognized phrases, then recognition accuracy is improved, but interaction duration increases

Engineering Contradiction:

Improverecognition accuracyVSAvoidinteraction duration

Core Design Contradiction:

Measurement precisionVSDuration of action of moving object

Solution Approach 1:

Instead of repeatedly attempting recognition in real-time, the system performs preliminary correction by generating alternate hypotheses based on pre-captured correction data and fulfillment feedback. This allows the system to prepare corrected versions of misrecognized phrases in advance, reducing the need for repeated real-time recognition attempts and extending interaction time

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system creates copies of corrected speech recognition hypotheses based on patterns from prior interactions. When a misrecognition is detected, the system copies the correction pattern from similar past cases to generate alternate hypotheses, avoiding the need to re-ask the user or repeatedly attempt recognition

Inventive Principle:

Principle #26Copying

3Measurement precision

If manual keyboard interface is used to correct speech recognition errors, then recognition accuracy is improved, but system latency increases

Engineering Contradiction:

Improverecognition accuracyVSAvoidsystem latency

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system replaces the mechanical manual typing process with automated computational correction. Instead of requiring users to physically type corrections via keyboard, the system automatically generates corrected hypotheses by processing audio data, comparing it against correction data from prior interactions, and producing alternate speech recognition hypotheses that match the user's intended meaning

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Measurement precision

If alternate speech recognition hypotheses are generated for all terms, then recognition accuracy is improved, but computational resource consumption increases

Engineering Contradiction:

Improverecognition accuracyVSAvoidcomputational resource consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The system applies local quality by selectively generating alternate hypotheses only for specific terms that have been identified as misrecognitions through fulfillment data analysis. Rather than processing all terms uniformly, the system focuses computational resources only on the local problematic areas where errors occurred, significantly reducing overall computational consumption while maintaining accuracy where needed

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS20250292772A1Correcting speech recognition errors by considering prior user edits and/or assessing fulfillment data

Publication Date: 2025.09.18 GOOGLE LLC

AI summary

Implementations relate to correcting a speech recognition hypothesis based on prior correction(s) made by a user and/or fulfillment data associated with fulfilling a request embodied in the speech recognition hypothesis. A candidate speech recognition hypothesis can be generated in response to the user providing a spoken utterance to an application, such as an automated assistant. When a confidence metric for the candidate speech recognition hypothesis does not satisfy a threshold, one or more terms of the candidate speech recognition hypothesis can be compared to correcting data. The correcting data can indicate whether the user previously corrected any term(s) present in the candidate speech recognition hypothesis and, if so, correct the term(s) accordingly. Fulfillment data generated for the candidate hypothesis and/or for the corrected hypothesis can also be processed to determine whether to utilize the candidate hypothesis or the corrected hypothesis in responding to the user.