Live Video Streaming with Integrated Encoding and Transmission Semantics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional live video streaming over IP networks faces challenges in balancing reliability and real-time delivery due to packet loss and corruption, leading to poor user experience, especially in lossy conditions, as existing techniques suffer from delayed recovery and poor reactivity to changing channel conditions.
Innovation Solution
A method and system for live video streaming with integrated encoding and transmission semantics using a reference encoder and delta encoder to generate full and delta frames, with a transmitter unit that periodically transmits packets with specific headers and region of interest information, categorizing packets as critical or non-critical for adaptive communication semantics, allowing for quick recovery and efficient bitrate management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If TCP with backward error correction is used for reliable transport, then packet loss recovery is improved, but real-time delivery performance deteriorates due to delay in recovery
Solution Approach 1:
The video stream is segmented into I-frames and P-frames with different protection strategies. I-frames are protected with forward error correction codes while P-frames use lighter protection, allowing selective recovery based on frame type and reducing overall recovery delay
Solution Approach 2:
Forward error correction codes are pre-applied to critical I-frames before transmission, enabling immediate recovery at the receiver without waiting for retransmission, thus reducing recovery delay while maintaining reliability
2Loss of energy
If traditional GOP-based encoding is used, then video compression efficiency is improved, but reactivity to changing channel conditions deteriorates due to slow reaction time
Solution Approach 1:
The encoding scheme dynamically adapts between I-frame and P-frame generation based on real-time channel conditions. When channel quality degrades, the system increases I-frame frequency or switches to more robust encoding modes, enabling fast reaction to changing conditions while maintaining compression efficiency
Solution Approach 2:
Encoding parameters such as frame rate, quantization levels, and GOP structure are dynamically adjusted based on channel conditions. The system changes these parameters in real-time to optimize the balance between compression efficiency and adaptability to varying network environments
3Reliability
If all packets are transmitted with confirmable communication semantics, then transmission reliability is improved, but real-time delivery performance deteriorates due to increased overhead and latency
Solution Approach 1:
Different communication semantics are applied locally to different packet types based on their criticality. I-frames use confirmable semantics for high reliability, while P-frames use unconfirmable semantics for faster delivery, optimizing the balance between reliability and speed for each packet type
Solution Approach 2:
Instead of applying confirmable semantics to all packets, the system applies it partially only to critical I-frames. This partial application maintains sufficient reliability for important data while reducing overall overhead and improving real-time delivery performance for the video stream
Data Source
AI summary
This disclosure relates generally to method and system for live video streaming with integrated encoding and transmission semantics. The system receives a set of frames associated with a live video stream encoded to generate a set of data fragments using a reference encoder and a delta encoder. Transmitter unit of the live video streaming protocol transmits each packet of the set of full frames and the set of delta frames in sequence with a payload specific header based on a packet mode. Further, the receiver unit receives each packet of the full frames and each packet of the delta frames based on the packet mode to reconstruct an original sequence from the foreground pixels by estimating a total number of packets expected at each frame interval and loss incurred in each packet of the set of full frames and the set of delta frames.