Video Transmission Frame Selection for Latency Stabilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video transmission technologies face challenges in stabilizing latency and jitter due to dynamic source-channel mismatches, particularly in real-time wireless communication scenarios like unmanned aerial vehicle applications, where sudden changes in channel bandwidth or source encoding cause fluctuations in transmission time.
Innovation Solution
The method involves dividing an image/video sequence into subsequences, encoding them separately, and selecting frames for transmission based on each subsequence's bitstream size and current channel bandwidth, with error frames reconstructed using linear interpolation, ensuring stable and fault-tolerant transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional rate control algorithms are used to stabilize average bit rate, then overall average latency is controlled, but per-frame latency and jitter cannot be resolved due to dynamic source-channel mismatch
Solution Approach 1:
The video stream is segmented into multiple channels with different code rates. The transmitting terminal divides the encoded video data into multiple channels, each with different coding rates, allowing the receiving terminal to select appropriate channels based on current channel conditions, thereby resolving per-frame latency and jitter issues that conventional average-based rate control cannot address
Solution Approach 2:
The system dynamically adapts to changing channel conditions by selecting from multiple pre-encoded channels with different code rates. Instead of using a fixed or slowly-adapting average bit rate, the system can immediately switch between channels to match current bandwidth availability, resolving the dynamic source-channel mismatch problem
2Quantity of substance
If the channel bandwidth is reduced by one half, then transmission capacity decreases, but the transmission latency is doubled accordingly
Solution Approach 1:
The video data is pre-encoded into multiple channels with different code rates before transmission. When channel bandwidth changes, the receiving terminal can immediately select the appropriate pre-encoded channel without waiting for re-encoding, thus avoiding latency doubling when bandwidth is reduced
3Adaptability or versatility
If a camera moves abruptly or an object in camera view has large motion, then encoded bitstream size is suddenly doubled, but transmission latency is doubled accordingly
Solution Approach 1:
The video stream is divided into multiple channels with different code rates to handle varying motion complexity. When abrupt motion occurs causing bitstream size to double, the system can switch to a higher code rate channel that was pre-encoded, avoiding the latency penalty of real-time rate adjustment
4Reliability
If multiple channels of coded data are prepared for the same data, then fault tolerance is improved, but device complexity increases
Solution Approach 1:
The video data is segmented into multiple channels with different code rates using separate encoding processes. This segmentation provides fault tolerance by allowing selection of appropriate channels based on channel conditions, while the modular structure manages encoding complexity through independent channel encoders
Data Source
Figure 1~2
Figure 3~4
AI summary
A method for processing video data is provided. The method includes down-sampling temporally an image sequence to form a plurality of subsequences; encoding the plurality of subsequences separately to form a plurality of encoded subsequences; and selecting frames for transmission according to a size of each frame in the plurality of encoded subsequences and a channel bandwidth. An unmanned aerial vehicle and computer-readable storage medium are also provided.