Video Encoding Scene Switch Detection for Low-Latency IDR Insertion

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

In cloud scenarios with high real-time requirements, such as cloud gaming, the latency caused by pre-processing each image frame for scene switches in video encoding reduces user experience due to the fast encoding speed of cloud servers.

Innovation Solution

Determine whether a scene switch occurs during the encoding process, encoding the next image frame into an IDR frame only when a scene switch is detected, eliminating the need for pre-processing and reducing latency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If pre-processing is performed before encoding each image frame to detect scene switches, then scene switch detection accuracy is improved, but processing latency increases

Engineering Contradiction:
Improvescene switch detection accuracyVSAvoidprocessing latency
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent applies preliminary action by performing scene switch detection during the encoding process itself rather than before encoding. The encoder detects scene switches by analyzing prediction modes and pixel differences during frame encoding, allowing the system to prepare for IDR frame insertion without requiring separate pre-processing steps, thus reducing latency while maintaining detection accuracy

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent maintains continuity of useful action by integrating scene switch detection into the continuous encoding workflow. The detection process occurs concurrently with encoding operations, utilizing already-computed prediction modes and pixel data during the encoding process, ensuring that the useful action of encoding continues without interruption while scene switch detection proceeds simultaneously

Inventive Principle:
Principle #20Continuity of useful action

2Loss of time

If scene switch detection is performed during encoding, then processing latency is reduced, but detection complexity increases

Engineering Contradiction:
Improveprocessing latencyVSAvoidencoding process complexity
Core Design Contradiction:
Loss of timeVSDevice complexity

Solution Approach 1:

The patent applies self-service by utilizing encoding process data (prediction modes, pixel differences) that are already generated during normal encoding operations. The scene switch detection mechanism serves itself by reusing existing encoding intermediates rather than requiring separate detection hardware or additional processing resources, thus reducing overall system complexity while achieving low-latency detection

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The patent demonstrates universality by making the encoding process serve dual functions: both compressing the video data and detecting scene switches. The same encoding operations that produce compressed output also generate the information needed for scene switch detection, allowing one process to perform multiple functions and reducing the need for separate detection mechanisms

Inventive Principle:
Principle #6Universality (Multi-functionality)

Data Source

PatentUS12470722B2Image encoding method, real-time communication method, device, storage medium, and program product
Publication Date: 2025.11.11 TENCENT TECHNOLOGY (SHENZHEN) CO LTD
  • US12470722B2 patent drawing
  • US12470722B2 patent drawing
  • US12470722B2 patent drawing

AI summary

An image encoding method and apparatus, a real-time communication method, a device, a storage medium, and a program product. The method includes: obtaining a first image frame in a video stream, encoding the first image frame into a first non-instantaneous decoding refresh (non-IDR) frame, and encoding a next image frame of the first image frame into an instantaneous decoding refresh (IDR) frame based on a determination, according to the first non-IDR frame, that a scene switch occurs.