Hybrid Speech Recognition Controller for Mobile Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile electronic devices face limitations in processing power and memory, restricting their ability to perform efficient automatic speech recognition (ASR) locally, and are often constrained to support only a single language, making it difficult to handle multilingual utterances and user privacy concerns regarding speech data transmission.
Innovation Solution
A hybrid speech processing system that combines local embedded ASR and server-based ASR engines, with a controller determining when to send audio or recognized text to the server for processing based on analysis, semantic categories, and user privacy settings, enabling efficient multilingual support and data privacy control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If speech processing is performed locally on the mobile device, then processing speed is improved, but speech recognition accuracy deteriorates due to limited processing power and memory
Solution Approach 1:
The speech processing system is segmented into two parts: local embedded ASR processing for immediate response and cloud-based ASR processing for high accuracy. The controller divides speech data into different processing paths based on the determined processing type, sending complex or uncertain speech to the cloud while handling simple speech locally.
Solution Approach 2:
The controller acts as an intermediary between the local embedded ASR and the cloud-based ASR system. It determines which processing path to use by analyzing speech characteristics, confidence scores, and network conditions, then routes speech data accordingly to balance speed and accuracy.
2Measurement precision
If speech data is transmitted to the server for processing, then speech recognition accuracy is improved, but user privacy deteriorates due to potential exposure of sensitive speech information
Solution Approach 1:
Different speech data receives different processing treatments based on its characteristics. Sensitive or private speech is processed locally with higher privacy protection, while non-sensitive speech is transmitted to the cloud for more accurate recognition. The system applies different quality levels of processing to different portions of speech data.
Solution Approach 2:
The system uses feedback from confidence scores and speech analysis to dynamically adjust whether to transmit data to the cloud. When the embedded ASR confidence score is high, speech is processed locally; when it's low or the speech appears sensitive, the controller determines to send it to the cloud with appropriate privacy considerations.
3Device complexity
If a single language is supported for speech recognition, then device complexity is reduced, but adaptability deteriorates due to inability to handle multilingual utterances
Solution Approach 1:
The hybrid ASR system provides universal multilingual support by combining local embedded ASR for common languages with cloud-based ASR for less common languages and multilingual scenarios. The controller identifies the language or languages being used and routes to the appropriate processing path, enabling the system to handle multiple languages without significantly increasing local device complexity.
Data Source
AI summary
Methods and apparatus for selectively performing speech processing in a hybrid speech processing system. The hybrid speech processing system includes at least one mobile electronic device and a network-connected server remotely located from the at least one mobile electronic device. The mobile electronic device is configured to use an embedded speech recognizer to process at least a portion of input audio to produce recognized text. A controller on the mobile electronic device determines whether to send information from the mobile electronic device to the server for speech processing. The determination of whether to send the information is based, at least in part, on an analysis of the input audio, the recognized text, or a semantic category associated with the recognized text.