Embedding Audio in Video Frames for Synchronization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In network environments, especially in virtual desktop infrastructure, it is challenging to maintain synchronization between audio and video data due to separate logical channels and varying processing times for video frames, leading to potential delays and desynchronization.
Innovation Solution
Embedding audio within video frame data in an encoded video stream, allowing both audio and video to be transmitted and received synchronously, with the audio being parsed and routed to an audio buffer while the video is processed by a GPU, ensuring synchronized output.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If audio and video data are transmitted over separate logical channels, then network protocol flexibility is improved, but synchronization between audio and video deteriorates
Solution Approach 1:
The patent merges audio data into the video frame data structure by embedding audio samples within video NAL units. This combining approach allows both audio and video to be transmitted through the same logical channel, ensuring they experience identical network conditions and processing delays, thereby maintaining synchronization while preserving network protocol flexibility
2Manufacturing precision
If video frames are processed with varying processing times, then video quality is improved, but audio-video synchronization deteriorates
Solution Approach 1:
The patent applies preliminary action by embedding audio data into video frames at the source before transmission. This ensures that audio and video are coupled together in advance, so that regardless of varying video processing times at the destination, both streams experience the same delays and arrive synchronized, preserving synchronization while allowing flexible video processing
Data Source
AI summary
Synchronization of audio and video data can be enforced by embedding audio within video frame data. In a network environment in which audio and video data are generated on a source computing device and then transferred over the network to a destination computing device for output, the source computing device can be configured to encode the video data into an encoded video stream. As part of this encoding, the source computing device can also embed audio in the encoded video stream in a manner that associates the audio with corresponding video frames. The encoded video stream with the embedded audio can then be transferred over the network. The destination computing device can be configured to parse the encoded video stream to remove the audio and route it to an audio buffer and then route the encoded video stream to a GPU.