On-Device Language Model for Real-Time Transcription Error Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models for speech recognition suffer from issues such as missing words, accuracy problems, and lapses, limiting their effectiveness for real-time transcription in applications like meeting notes or accessibility for hearing impairments.
Innovation Solution
A user electronic device is equipped with a language model that runs on-device, enabling near real-time error filtering, speaker identification, and summarization. This approach processes raw audio data to generate live transcriptions, corrects errors, and inserts speaker identifiers, all while maintaining user privacy and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If a speech transcriber is used to generate live transcription in real-time, then transcription speed is improved, but transcription accuracy deteriorates due to missing words and errors
Solution Approach 1:
A language model acts as an intermediary between the speech transcriber and the final transcription output. The language model receives the live transcription tokens and processes them to correct errors, fill in missing words, and improve overall accuracy while maintaining real-time performance capabilities
2Measurement precision
If cloud-based language models are used for transcription correction, then transcription accuracy is improved, but latency increases and user privacy is compromised
Solution Approach 1:
The language model is deployed to run locally on the user's device, enabling it to service itself and perform transcription correction without needing to send data to external cloud servers. This self-contained approach eliminates network latency and keeps all processing local to the device
3Measurement precision
If cloud-based language models are used for transcription correction, then transcription accuracy is improved, but user privacy and data security deteriorate due to data transmission requirements
Solution Approach 1:
The language model is extracted from cloud-based infrastructure and embedded directly into the local device. This extraction removes the need for data to leave the device, eliminating the privacy and security risks associated with transmitting sensitive audio and transcription data over networks
4Productivity
If existing speech recognition models are used, then basic transcription functionality is provided, but transcription reliability deteriorates due to frequent errors and lapses
Solution Approach 1:
The system combines a speech transcriber model with a language model into a composite transcription system. The speech transcriber handles real-time audio-to-text conversion while the language model layer adds correction and refinement capabilities, creating a more reliable overall system that leverages the strengths of both model types
Data Source
AI summary
A user electronic device comprising: one or more microphones configured to capture raw audio data; and one or more processors and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising: receiving the raw audio data captured by the one or more microphones; processing the raw audio data using a speech transcriber to generate a live transcription of the raw audio data that comprises a plurality of text tokens; processing the raw audio data to generate a speaker identification output that identifies, for each of the text tokens, a respective speaker for each of the text tokens in the live transcription; and processing a first input comprising (i) a first input prompt and (ii) an input text generated from the live transcription using a language model neural network to generate a modified transcription.