Hierarchical Speech Recognition Model Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud-based automatic speech recognition (ASR) systems require significant computing power and energy, leading to inefficiencies in mobile devices, as they rely on large models to cover all language scenarios, resulting in high error rates and latency in speech-to-text conversion.
Innovation Solution
Implementing a hierarchical speech recognition processing system where context data is transmitted between devices, allowing a subset speech recognition model to be used on a local device for initial processing, with the option to send unrecognized utterances to a higher processing level, such as a cloud, for further recognition and action execution, utilizing personalized language and acoustic models adapted to the user's context.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If cloud-based ASR systems use large speech recognition models to cover all language scenarios, then recognition accuracy is improved, but energy consumption and computing power requirements increase significantly
Solution Approach 1:
The speech recognition model is segmented into multiple levels: a smaller first speech recognition model deployed on the mobile device for common scenarios, and a larger second speech recognition model on the cloud server for complex scenarios. This segmentation allows the mobile device to perform most recognition tasks locally with low energy consumption, while only escalating uncertain cases to the cloud, thus resolving the contradiction between accuracy and energy usage.
Solution Approach 2:
Instead of deploying the complete large-scale speech recognition model on the mobile device, only a partial model (first speech recognition model) covering common language scenarios is deployed locally. This partial deployment reduces the model size and energy requirements while maintaining adequate accuracy for most cases, with the option to supplement with cloud-based recognition when needed.
2Reliability
If cloud-based ASR systems use large speech recognition models to cover all language scenarios, then recognition accuracy is improved, but processing latency increases
Solution Approach 1:
The speech recognition system is segmented into local and cloud components, enabling parallel processing where common speech recognition tasks are handled locally by the first speech recognition model without cloud round-trip delays. Only unrecognized or complex utterances are transmitted to the cloud, significantly reducing overall processing latency while maintaining high accuracy.
Solution Approach 2:
The first speech recognition model is pre-deployed on the mobile device with pre-trained language and acoustic models, enabling immediate local processing of speech inputs without waiting for cloud server responses. This preliminary setup allows the device to handle common recognition tasks instantly, reducing latency for the majority of speech inputs.
3Use of energy by moving object
If a smaller speech recognition model is used on the mobile device, then energy consumption is reduced, but recognition accuracy for complex scenarios deteriorates
Solution Approach 1:
The system introduces an intermediary mechanism where the mobile device's first speech recognition model acts as a preliminary filter, handling simple recognition tasks locally. When the model encounters uncertain or complex scenarios, it mediates by transmitting the speech input to the cloud-based second speech recognition model for final recognition, thus maintaining high accuracy while minimizing energy consumption.
Solution Approach 2:
The system replaces the mechanical approach of running a single large model on the mobile device with a hybrid architecture that substitutes cloud-based computational power for local processing of complex cases. This substitution allows the mobile device to maintain a small, energy-efficient local model while leveraging cloud resources only when necessary, resolving the accuracy-energy tradeoff.
Data Source
AI summary
A method and device for recognizing an utterance. The method includes transmitting context data associated with a first device to a second device. A first speech recognition model is received from the second device. The first speech recognition model is a subset of a second speech recognition model present at the second device. The first speech recognition model is based on the context data. It is determined whether the utterance can be recognized at the first device based on the first speech recognition model. If the utterance cannot be recognized at the first device, then at least a portion of the utterance is sent to the second device. If the utterance can be recognized at the first device, then an action associated with the recognized utterance is performed.