Voice-Synthesized Word Replacement for Packet-Loss Audio Calls
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing communication platforms face challenges in maintaining continuous audio streams during periods of poor network connectivity or congestion, leading to dropped packets and disrupted real-time communication sessions due to packet retransmission delays and invalid data.
Innovation Solution
Implementing a system that records audio at the user device and uses machine learning to detect missing words, predicting and synthesizing them in the user's voice to generate a continuous audio stream without relying on packet retransmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If packet retransmission is used to handle lost audio packets, then audio completeness may be improved, but communication delay increases and real-time continuity deteriorates
Solution Approach 1:
The system performs preliminary actions by detecting packet loss in advance and immediately generating replacement audio content using speech-to-text conversion and text-to-speech synthesis, rather than waiting for retransmission. This preliminary intervention maintains audio continuity without the delays associated with traditional packet retransmission protocols.
Solution Approach 2:
The patent introduces an intermediary processing layer that converts lost audio packets into text representations and then synthesizes replacement audio. This intermediary text-based approach acts as a mediator between the lost audio data and the need for continuous audio playback, enabling faster regeneration than direct audio retransmission.
2Loss of information
If packet retransmission is implemented, then data loss may be reduced, but network congestion worsens and productivity decreases
Solution Approach 1:
The system extracts the essential information from lost audio packets by converting them to text representations. By extracting only the critical semantic content rather than retransmitting the entire audio packet, the system reduces network load while preserving communication effectiveness, thereby improving overall productivity.
Solution Approach 2:
Instead of retransmitting original audio packets, the system creates synthetic copies of the missing audio content based on text conversions. These synthesized copies are inserted into the audio stream to replace lost segments, maintaining information completeness without the overhead of original packet retransmission.
3Loss of energy
If traditional packet loss handling is used, then network bandwidth is conserved, but audio stream continuity breaks and communication quality deteriorates
Solution Approach 1:
The patent replaces the traditional mechanical packet retransmission system with an information-processing approach using speech-to-text and text-to-speech conversion. This substitution allows the system to handle packet loss through computational methods rather than network retransmission, maintaining audio continuity while being more efficient with network bandwidth.
Data Source
AI summary
A server generates a continuous audio stream during a real-time communication session. The server obtains a first audio stream from a user device connected to the real-time communication session and detects speech data in the first audio stream. The server converts the speech data to text data that includes one or more words. The server determines that the text data is missing a word based on a context of the one or more words. The server synthesizes a predicted word for replacing the missing word in a voice of a user of the user device and combines the synthesized word with the first audio stream to generate the continuous audio stream. The server transmits the continuous audio stream to other user devices connected to the real-time communication session.