Hybrid Speech Recognition Model Update via Cloud OOV Extraction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing speech recognition systems for connected devices face challenges such as unreliable network connectivity, power consumption issues, privacy concerns, and degraded user experience due to limited local resources and reliance on remote server processing.

Innovation Solution

Implementing a hybrid approach that combines local and cloud-based Automated Speech Recognition (ASR) systems, where the client device uses a compact language model updated dynamically with high-confidence user data and Out-Of-Vocabulary (OOV) words identified by cloud ASR, to enhance speech recognition accuracy and efficiency while conserving power and maintaining user privacy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If cloud-based speech recognition systems are used to provide good accuracy independent of acoustics, vocabulary, and grammar, then speech recognition accuracy is improved, but network connectivity reliability and power consumption are worsened

Engineering Contradiction:

Improvespeech recognition accuracyVSAvoidnetwork connectivity reliability

Core Design Contradiction:

Measurement precisionVSReliability

Solution Approach 1:

The system divides speech recognition into two segments: local processing for basic recognition and cloud processing for enhancement. The local device performs initial speech-to-text conversion using a compact model, then selectively sends only difficult or uncertain recognitions to the cloud server for improvement, rather than relying entirely on cloud services.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The local device acts as an intermediary between the user and the cloud server. It pre-processes speech signals locally, makes initial recognition decisions, and only forwards data to the cloud when necessary, reducing dependency on continuous network connectivity while still benefiting from cloud-based accuracy improvements.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

Engineering Contradiction:

Improvespeech recognition accuracyVSAvoidpower consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The system applies partial action by using the compact local model for most speech recognition tasks and only invoking the more resource-intensive cloud service when local confidence is low or for specific enhancement needs. This partial reliance on cloud services maintains accuracy improvement while significantly reducing overall power consumption compared to continuous cloud processing.

Inventive Principle:

Principle #16Partial or excessive action

3Adaptability or versatility

If remote servers are used to convert speech to text, then speech recognition capability is improved, but device complexity and data transmission requirements are worsened

Engineering Contradiction:

Improvespeech recognition capabilityVSAvoiddevice complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The system extracts the essential speech recognition functionality into a compact local model that can operate independently, while using the cloud server only for supplemental enhancement. This extraction of core functionality to the local device reduces overall system complexity and data transmission requirements while maintaining improved speech recognition capability.

Inventive Principle:

Principle #2Taking out (Extraction)

4Use of energy by moving object

If local speech recognition systems with compact language models are used, then power consumption and device resources are conserved, but speech recognition accuracy is worsened

Engineering Contradiction:

Improvepower consumptionVSAvoidspeech recognition accuracy

Core Design Contradiction:

Use of energy by moving objectVSMeasurement precision

Solution Approach 1:

The system merges the advantages of both local and cloud-based approaches by combining the compact local model for efficient basic recognition with selective cloud server enhancement. This merging allows the system to achieve both power conservation through local processing and accuracy improvement through cloud collaboration.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The system uses feedback from local speech recognition confidence scores to determine when cloud enhancement is needed. When local confidence is low or when encountering out-of-vocabulary words, the system requests correction from the cloud server, creating a feedback loop that maintains accuracy while minimizing unnecessary cloud processing and power consumption.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS10325590B2Language model modification for local speech recognition systems using remote sources

Publication Date: 2019.06.18 HYUNDAI MOTOR CO LTD

AI summary

A language model is modified for a local speech recognition system using remote speech recognition sources. In one example, a speech utterance is received. The speech utterance is sent to at least one remote speech recognition system. Text results corresponding to the utterance are received from the remote speech recognition system. A local text result is generated using local vocabulary. The received text results and the generated text result are compared to determine words that are out of the local vocabulary and the local vocabulary is updated using the out of vocabulary words.