Tensor Sub-Tensor Encoding for Loss-Resilient Real-Time Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current real-time video streaming systems face challenges in achieving loss resilience and compression efficiency due to packet losses, leading to video stuttering and quality degradation, especially in long-latency networks, as existing methods like FEC and error concealment either add excessive redundancy or fail to maintain quality under high data missing rates.
Innovation Solution
A video encoder framework using autoencoders decomposes frames into sub-tensors, applies per-packet entropy encoding, and employs selective state resynchronization to ensure decodability with partial packet loss, maintaining quality through random-like masking and efficient redundancy management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If FEC (forward error correction) is used to ensure loss resilience, then reliability is improved, but device complexity and redundancy increase excessively
Solution Approach 1:
The patent segments the video frame into multiple independent sub-tensors, where each sub-tensor is encoded separately into packets. This segmentation allows the receiver to decode complete frames even when some packets are lost, as each sub-tensor can be independently reconstructed. This resolves the contradiction by providing loss resilience through segmentation rather than through excessive redundancy added by FEC.
2Reliability
If traditional error concealment methods are used, then reliability is improved, but manufacturing precision and quality maintenance deteriorate under high data missing rates
Solution Approach 1:
The patent applies preliminary action by encoding multiple sub-tensors with different levels of detail and importance before transmission. The encoder distributes packets across multiple sub-tensors, ensuring that even if some packets are lost, the receiver can reconstruct the frame using the remaining sub-tensors. This preliminary distribution of information across multiple segments ensures quality maintenance under high data missing rates without requiring complex error concealment.
3Reliability
If redundancy is added to ensure decodability with packet loss, then reliability is improved, but compression efficiency deteriorates
Solution Approach 1:
The patent applies local quality by encoding different sub-tensors with different levels of redundancy based on their importance and the network conditions. Not all sub-tensors are encoded with the same redundancy level; instead, the system adapts the encoding parameters for each sub-tensor to achieve optimal compression efficiency while maintaining sufficient redundancy for loss-resilient decoding. This localized approach to quality control resolves the contradiction between reliability and compression efficiency.
Data Source
AI summary
Systems, methods, and computer program products are provided for streaming video over a network. In various embodiments, a source video including at least a source frame is read. The source frame is encoded into a corresponding tensor representation by a machine learning model. The corresponding tensor representation is decomposed into a plurality of sub-tensors. Each of the plurality of sub-tensors is encoded into a corresponding packet and transmitted via a network from a source node to a receiver node.