Hybrid Speech Recognition with Local Dialogue Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech recognition systems for portable and embedded devices face challenges such as high computational intensity, bandwidth requirements, battery consumption, and lower quality recognition due to large language and acoustic models, while cloud-based systems suffer from high latency and battery consumption.
Innovation Solution
A hybrid system combining a local small ASR engine with a remote large ASR engine, where the remote engine trains and updates the local engine with frequently used dialogues and acoustic models, allowing offline customization and improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large language and acoustic models are used for speech recognition, then recognition quality is improved, but computational intensity and battery consumption increase
Solution Approach 1:
The speech recognition system is segmented into two parts: a compact local ASR engine for frequently used dialogues and a remote cloud-based ASR engine for comprehensive processing. The local engine handles common queries offline with minimal energy consumption, while the remote engine processes complex queries when needed, thus reducing overall battery consumption while maintaining recognition quality.
Solution Approach 2:
The system implements local customization by training the local ASR engine with user-specific frequently used dialogues and acoustic models. This creates a specialized local recognition system optimized for the individual user's speech patterns and commonly used commands, improving recognition quality for those specific cases while consuming less energy than running a full large-scale model locally.
2Measurement precision
If cloud-based ASR systems are used, then recognition accuracy is improved, but latency and network dependence increase
Solution Approach 1:
The system performs preliminary action by pre-training the local ASR engine with frequently used dialogues and acoustic models before actual speech recognition tasks. This preparation enables the local engine to immediately process common queries without network communication, eliminating latency for those cases while maintaining accuracy through pre-loaded language and acoustic models.
Solution Approach 2:
The local ASR engine acts as an intermediary between the user and the remote cloud-based ASR engine. It first attempts to process speech queries locally, and only when the query is not recognized or falls outside the pre-trained vocabulary does it forward the query to the remote engine, thus reducing overall latency by handling simple cases locally.
3Loss of time
If local ASR processing is implemented, then latency is reduced, but model size and device complexity increase
Solution Approach 1:
The system extracts only the essential components needed for local processing: a compact language model, acoustic model, and frequently used dialogue database. By taking out only these critical elements from the full cloud-based system and storing them locally, the patent achieves low-latency processing for common queries without requiring the complete large-scale model architecture.
4Use of energy by moving object
If frequently used dialogues are processed locally, then battery consumption is reduced, but recognition accuracy for uncommon queries decreases
Solution Approach 1:
The local ASR engine serves as an intermediary that handles frequently used dialogues with high accuracy using pre-trained models, thereby conserving battery energy. When the local engine encounters unrecognized or uncommon queries, it acts as a gateway to forward these queries to the remote cloud-based ASR engine, which provides comprehensive recognition capabilities, thus maintaining overall system accuracy while reducing energy consumption for common cases.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A speech processing system and a method therefor is provided. The speech processing system may capture one or more speech signals. Each of the one or more speech signals may include at least one dialogue uttered by a user. Dialogues may be extracted from the one or more speech signals. Frequently uttered dialogues may be identified over a period of time. The frequently uttered dialogues may be a set of dialogues that are uttered by the user a number of times during the period of time more than other dialogues uttered by the user during the period of time. A local language model and a local acoustic model may be generated based on, at least in part, the frequently uttered dialogues. The one or more speech signals may be processed based on, at least in part, the local language model and the local acoustic model.