Video Latency Reduction via Frame Elimination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Video latency issues in real-time media events, particularly due to high CPU intensity and network burden from video data handling, result in jerky video and audio desynchronization, limiting the quality of real-time conferences.
Innovation Solution
Minimizing video latency by capturing video at a first frame rate and eliminating frames prior to processing, allowing the processor to handle the video as if it were captured at a lower second frame rate, thereby reducing end-to-end latency without overburdening the CPU or network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If video capture rate is increased to reduce latency, then video latency is reduced, but CPU capacity and network resources are consumed
Solution Approach 1:
The patent captures video at a higher frame rate than necessary for processing (e.g., capturing at 60 fps but processing at 30 fps), using only the needed portion of the captured data. This allows the system to reduce latency by having frames ready faster while not processing all captured frames, thus avoiding excessive CPU and network resource consumption.
Solution Approach 2:
The patent extracts and eliminates certain video frames from the capture stream before processing. By removing unnecessary frames (e.g., eliminating every other frame), the system reduces the data volume that needs CPU-intensive processing and network transmission, while maintaining the ability to reduce latency through higher capture rates.
2Loss of time
If video capture rate is increased to reduce latency, then video latency is reduced, but network resources are burdened
Solution Approach 1:
The system captures video at a higher frame rate to reduce latency but processes and transmits at a lower effective rate. This partial use of captured frames allows latency reduction without proportionally increasing network data volume, as not all captured frames are transmitted.
Solution Approach 2:
The patent removes certain frames from the video stream before network transmission. By eliminating unnecessary frames (e.g., through frame dropping or selective filtering), the network data volume is reduced while the capture rate remains high enough to provide low-latency processing for the frames that are transmitted.
3Reliability
If all captured video frames are processed, then video quality is maintained, but CPU capacity is consumed
Solution Approach 1:
The patent processes only a subset of captured frames at the full processing quality level. By selecting which frames to fully process and which to simplify or eliminate, the system maintains acceptable video quality while significantly reducing CPU capacity consumption compared to processing every captured frame at full quality.
Solution Approach 2:
The system applies different processing quality levels to different frames based on their importance. Critical frames (e.g., those containing important visual information or occurring at key moments) receive full processing quality, while less critical frames receive reduced processing or are eliminated, optimizing the balance between video quality and CPU consumption.
Data Source
AI summary
Procedures for minimizing video latency using different frame rates are described. In an implementation, a video captured at a first frame rate is obtained. At least one frame of the video is eliminated and the resultant video, with the eliminated at least one frame is processed as if the video were captured at a slower frame rate than the first frame rate.