Online Meeting Speech Interruption Handling with Concurrent Text Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Network issues during online meetings or calls can cause frame loss, leading to non-recoverable information loss and adversely impacting user experience, as users have to guess or ask for repeated speech, disrupting communication flow and wasting time.
Innovation Solution
Implementing a system with a meeting manager and an interruption manager that generates a text stream through speech-to-text conversion, forwarding it concurrently with the media stream, and displaying or synthesizing speech using metadata and user-specific speech models to handle interruptions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If speech audio stream is transmitted over network during online meeting, then real-time communication is enabled, but network issues cause frame loss and information loss
Solution Approach 1:
The patent creates a text copy of the speech audio stream through speech-to-text conversion. This text representation serves as a backup that can be displayed when the audio stream is lost due to network issues, allowing users to read the transcribed speech instead of hearing it, thus maintaining communication continuity despite network failures.
Solution Approach 2:
The system performs speech-to-text conversion in advance and transmits the text stream concurrently with the audio stream. By preparing the text representation beforehand and having it ready for transmission, the system can immediately switch to displaying text when audio is lost, without waiting for network failure detection and recovery.
2Reliability
If speech is converted to text and transmitted concurrently, then communication continuity is maintained during interruptions, but system complexity increases
Solution Approach 1:
The text stream serves multiple functions: it acts as a backup during network interruptions, provides closed captions for hearing-impaired users, and offers a transcription record for later review. This multi-functionality justifies the added complexity by providing multiple benefits from a single speech-to-text conversion process.
3Measurement precision
If user-specific speech models are used for synthesis, then speech quality and user experience are improved, but processing requirements and complexity increase
Solution Approach 1:
The system uses user-specific speech models selectively rather than always. The metadata includes indicators that allow the receiving device to determine when to apply user-specific synthesis versus using default synthesis. This partial application reduces processing energy while maintaining high speech quality when it matters most.
Data Source
Figure 1
Figure 2
Figure 3A~3C
AI summary
A device for communication includes one or more processors configured to receive, during an online meeting, a speech audio stream representing speech of a first user. The one or more processors are also configured to receive a text stream representing the speech of the first user. The one or more processors are further configured to selectively generate an output based on the text stream in response to an interruption in the speech audio stream.