Pseudo Video Generation for Voice-Video Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing communication systems experience delays between voice processing (recognition, translation, and synthesis) and real-time video, leading to a perceived delay that affects interactivity and the meaning of conversations.
Innovation Solution
A transmission device is used on the talker side to process voice data, generate pseudo video data with a playback time length corresponding to the delay caused by voice processing, and transmit this pseudo video data to the receiver side, along with real-time video and information data, to mask the delay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If voice processing (recognition, translation, and synthesis) is performed to enable multi-language communication, then communication capability between persons with different mother languages is improved, but time gap occurs between processed voice information and real-time video
Solution Approach 1:
The system performs preliminary actions by generating agreeable response information in advance based on predicted conversation flow, and prepares pseudo video data beforehand. This allows the system to output responses without waiting for complete voice processing, thereby reducing the perceived time gap while maintaining multi-language communication capability.
Solution Approach 2:
The system introduces an intermediary mechanism by generating agreeable response information that mediates between the delayed processed voice and real-time video. This intermediary content fills the time gap and makes the receiver unaware of the delay, resolving the contradiction between processing time and synchronization.
2Stability of the object's composition
If video is intentionally delayed to synchronize with processed voice, then voice-video synchronization is improved, but delay from start of conversation to start of video playback increases
Solution Approach 1:
The transmission device performs preliminary action by transmitting agreeable response information before the delayed video data. This preliminary transmission ensures that the receiver starts receiving content immediately without waiting for the delayed video, thereby reducing the overall delay while maintaining voice-video synchronization through the pseudo video data.
Solution Approach 2:
The system inverts the conventional approach by not delaying the video stream itself, but instead generating and transmitting pseudo video data that mimics the delayed video content. This inversion allows the real video to be transmitted without delay while the pseudo video fills in the gaps, resolving the contradiction between synchronization and start delay.
3Ease of operation
If agreeable response information is transmitted to reduce perceived delay, then interactivity is improved, but time gap may occur between video and agreeable response or subtitle information
Solution Approach 1:
The system uses feedback by monitoring the timing relationships between video data, agreeable response information, and subtitle information. Based on this feedback, the transmission device adjusts the timing and content of pseudo video data generation to maintain synchronization while preserving the interactivity benefits of agreeable response transmission.
Solution Approach 2:
The system applies dynamics by making the pseudo video data generation adaptive and flexible. The transmission device dynamically adjusts the content and timing of pseudo video based on the actual delay characteristics and synchronization requirements, allowing the system to maintain both interactivity and synchronization stability under varying conditions.
Data Source
AI summary
A transmission device used on a receiver side in a communication system for communication between a talker and a receiver includes a processing unit that processes voice data obtained from speech of the talker to generate information data corresponding to the voice data, a pseudo video generation unit that generates pseudo video data having a playback time length corresponding to a length of a delay time caused by the processing unit, and a transmission unit that transmits the pseudo video data to the receiver side and then transmits video data during the speech of the talker and the information data to the receiver side.