Video Encoding Scene Switch Detection for Low-Latency IDR Insertion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud scenarios with high real-time requirements, such as cloud gaming, the latency caused by pre-processing each image frame for scene switches in video encoding reduces user experience due to the fast encoding speed of cloud servers.
Innovation Solution
Determine whether a scene switch occurs during the encoding process, encoding the next image frame into an IDR frame only when a scene switch is detected, eliminating the need for pre-processing and reducing latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If pre-processing is performed before encoding each image frame to detect scene switches, then scene switch detection accuracy is improved, but processing latency increases
Solution Approach 1:
The patent applies preliminary action by performing scene switch detection during the encoding process itself rather than before encoding. The encoder detects scene switches by analyzing prediction modes and pixel differences during frame encoding, allowing the system to prepare for IDR frame insertion without requiring separate pre-processing steps, thus reducing latency while maintaining detection accuracy
Solution Approach 2:
The patent maintains continuity of useful action by integrating scene switch detection into the continuous encoding workflow. The detection process occurs concurrently with encoding operations, utilizing already-computed prediction modes and pixel data during the encoding process, ensuring that the useful action of encoding continues without interruption while scene switch detection proceeds simultaneously
2Loss of time
If scene switch detection is performed during encoding, then processing latency is reduced, but detection complexity increases
Solution Approach 1:
The patent applies self-service by utilizing encoding process data (prediction modes, pixel differences) that are already generated during normal encoding operations. The scene switch detection mechanism serves itself by reusing existing encoding intermediates rather than requiring separate detection hardware or additional processing resources, thus reducing overall system complexity while achieving low-latency detection
Solution Approach 2:
The patent demonstrates universality by making the encoding process serve dual functions: both compressing the video data and detecting scene switches. The same encoding operations that produce compressed output also generate the information needed for scene switch detection, allowing one process to perform multiple functions and reducing the need for separate detection mechanisms
Data Source
AI summary
An image encoding method and apparatus, a real-time communication method, a device, a storage medium, and a program product. The method includes: obtaining a first image frame in a video stream, encoding the first image frame into a first non-instantaneous decoding refresh (non-IDR) frame, and encoding a next image frame of the first image frame into an instantaneous decoding refresh (IDR) frame based on a determination, according to the first non-IDR frame, that a scene switch occurs.


