On-Device ASR Personalization Using Pseudo-Corrections

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Inaccurate speech recognition in on-device automatic speech recognition (ASR) models can occur due to resource constraints and is exacerbated by the forgetting of corrections over time, leading to inefficiencies and increased memory usage.

Innovation Solution

The generation and utilization of pseudo-corrections to prevent forgetting in on-device ASR models by periodically updating the model based on user corrections and pseudo-corrections, even after the original correction's time-to-live (TTL) has lapsed.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If corrections are cached for an elongated duration to prevent forgetting, then speech recognition accuracy is improved, but memory consumption increases

Engineering Contradiction:

Improvespeech recognition accuracyVSAvoidmemory consumption

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent changes the temporal parameter of correction retention by introducing a time-to-live (TTL) mechanism. Corrections are cached for a specific duration rather than indefinitely, allowing the system to balance accuracy improvement with memory management. After the TTL expires, corrections are discarded, preventing unbounded memory growth while maintaining accuracy during the useful period.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The system implements periodic updates to the ASR model using cached corrections. Rather than continuously accumulating corrections indefinitely, the model is periodically retrained with accumulated corrections up to their TTL expiration, creating a rhythmic cycle of data accumulation and model update that manages memory consumption systematically.

Inventive Principle:

Principle #19Periodic action

2Quantity of substance

If corrections are discarded after TTL to free memory, then memory usage is reduced, but the ASR model forgets corrections leading to inaccurate speech recognition

Engineering Contradiction:

Improvememory usageVSAvoidspeech recognition accuracy

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

The system performs preliminary actions by accumulating and caching corrections during their TTL period before they are needed for model updates. This preparation phase allows corrections to be readily available when the ASR model requires retraining, ensuring that accuracy is maintained without requiring long-term memory retention of individual corrections.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Instead of retaining the original correction data indefinitely, the system creates copies of corrections in a cache with limited lifetime. These cached copies are used to periodically update the ASR model, and once their TTL expires, they are discarded. This copying mechanism allows the system to manage memory usage while preserving the essential information needed for model improvement.

Inventive Principle:

Principle #26Copying

3Measurement precision

If on-device ASR model is made more robust to improve accuracy, then speech recognition precision is improved, but device processor and memory resources are exceeded

Engineering Contradiction:

Improvespeech recognition precisionVSAvoidprocessor and memory resources

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent segments the speech recognition system into two parts: a lightweight on-device ASR model for real-time processing and a cloud-based training system for model improvement. The on-device model remains computationally efficient while the cloud system handles complex training operations using accumulated corrections, allowing accuracy improvement without exceeding device resource constraints.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The cached corrections act as an intermediary between user corrections and the ASR model. Rather than directly updating the model with every correction (which would require excessive resources), the system accumulates corrections in a cache and uses them periodically for model updates. This intermediary mechanism enables gradual model improvement while managing computational resources efficiently.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentEP4416721B1Generation and utilization of pseudo-correction(s) to prevent forgetting of personalized on-device automatic speech recognition (ASR) model(s)

Publication Date: 2026.03.18 GOOGLE LLC

Figure 1
Figure 2
Figure 3

AI summary

On-device processor(s) of a client device may store, in on-device storage and in association with a time to live (TTL) in the on-device storage, a correction directed to ASR processing of audio data. The correction may include a portion of a given speech hypothesis that was modified to an alternate speech hypothesis. Further, the on-device processor(s) may cause an on-device ASR model to be personalized based on the correction. Moreover, and based on additional ASR processing of additional audio data, the on-device processor(s) may store, in the on-device storage and in association with an additional TTL in the on-device storage, a pseudo-correction directed to the additional ASR processing. Accordingly, the on-device processor(s) may cause the on-device ASR model to be personalized based on the pseudo-correction to prevent forgetting by the on-device ASR model.