Video Streaming Buffer Control for Ultra-Low Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current MJPEG solutions fail to provide ultra-low latency and reliable real-time video streaming over networks with varying bandwidth conditions, leading to increased latency and potential connection drops due to bandwidth limitations and TCP flow control.
Innovation Solution
An apparatus comprising a frame generator, data buffer, and control unit that adapts the frame flow by adjusting framerate and encoding quality based on real-time bandwidth measurements, using a proportional or proportional-derivative controller to maintain a predefined buffer size, ensuring data is transmitted according to available network bandwidth without client-side modifications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If MJPEG encoding is used to achieve ultra-low latency, then latency is reduced, but bandwidth consumption increases beyond available network capacity
Solution Approach 1:
The system dynamically adapts the frame flow characteristics (framerate, resolution, or compression) based on real-time network conditions. The control unit continuously monitors available bandwidth and adjusts encoding parameters to match current network capacity, allowing the system to maintain ultra-low latency when bandwidth permits while reducing quality requirements when bandwidth is limited.
Solution Approach 2:
The invention changes physical parameters of the video stream (such as framerate, resolution, or compression level) to adapt to varying network conditions. By modifying these parameters in real-time, the system can fit the video stream within available bandwidth constraints while minimizing latency impact.
2Reliability
If TCP flow control is implemented to manage bandwidth, then network reliability improves, but buffer overflow occurs causing increased latency
Solution Approach 1:
The control unit implements a feedback mechanism that continuously monitors the data buffer size and network conditions. When the buffer approaches overflow, the system receives feedback about the current state and automatically adjusts the frame flow rate or encoding parameters to prevent overflow, thereby maintaining both reliability and low latency.
Solution Approach 2:
The system takes preliminary action by proactively adjusting the frame flow characteristics before the buffer completely overflows. By anticipating potential buffer overflow conditions and adjusting parameters in advance, the system prevents latency increases while maintaining reliable transmission.
3Reliability
If network send buffer is increased to handle bandwidth variations, then transmission reliability improves, but buffered frames become outdated increasing client-side latency
Solution Approach 1:
Instead of using a statically large buffer, the system employs dynamic buffer management where the effective buffer size and frame flow rate are continuously adjusted based on real-time network conditions. This allows the system to maintain transmission reliability during bandwidth variations while minimizing the amount of time frames spend buffered, thus reducing client-side latency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An apparatus (100) comprising: - a frame generator (101) configured to receive a video (105) and to transform the video (105) into a frame flow (106), comprising a flow of frames sequentially generated at a framerate, where a frame is encoded individually in an image file (214) at an encoding quality level; - a data buffer (102) configured to receive the frame flow (106), and temporarily store received data; - a control unit (104) configured to obtain a size measure (108) of the data buffer (102), and adapt the frame flow (106) based on the size measure (108); - a transmitter unit (103) configured to transmit data from the data buffer (102) via a network connection (107) to a video client (200) across said network.