Special Reference Picture Reconstruction for Video Stream Switching

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing video streaming technologies face challenges in efficiently adjusting bit rates in real-time due to the mismatch of reconstructed reference pictures during stream switching, leading to degraded video quality and increased computational complexity, especially when using scalable video coding.

Innovation Solution

The method involves decoding and reconstructing special reference pictures (SRP) from identified reconstructed reference pictures, using difference pictures to compensate for mismatches, and encoding these SRPs with dedicated flags to facilitate flexible stream switching without substantial bit rate overhead, suitable for any GOP structure and number of reference pictures.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If stream switching is performed at inter-coded pictures (P or B pictures), then real-time bit rate adjustment is achieved, but the mismatch of reconstructed reference pictures causes degraded video quality

Engineering Contradiction:
Improvebit rate adjustment speedVSAvoidvideo reconstruction quality
Core Design Contradiction:
SpeedVSReliability

Solution Approach 1:

The patent applies preliminary action by constructing special reference pictures (SRP) in advance before stream switching occurs. The SRP is built by replacing reference pictures in the display picture buffer (DPB) with corresponding reference pictures from the target stream, ensuring that when switching happens, the reference pictures are already prepared and match, thus preventing reconstruction errors while enabling fast switching.

Inventive Principle:
Principle #10Preliminary action

2Adaptability or versatility

If Random Access Points (RAP) are assigned frequently to enable prompt stream switching, then switching capability is improved, but encoding I/IDR pictures introduces substantial bit rate overhead

Engineering Contradiction:
Improvestream switching capabilityVSAvoidbit rate overhead
Core Design Contradiction:
Adaptability or versatilityVSLoss of energy

Solution Approach 1:

The patent applies local quality by making only the reference pictures local to the target stream while keeping the current picture encoding unchanged. Instead of re-encoding the entire current picture as an I/IDR frame, only the reference pictures used for prediction are replaced with SRP containing references from the target stream. This localized modification enables stream switching without the substantial bit rate overhead of frequent intra-coded pictures.

Inventive Principle:
Principle #3Local quality

3Adaptability or versatility

If scalable video coding is used to enable real-time bit rate adjustments, then adaptability to bandwidth changes is improved, but computational complexity and compression efficiency are degraded

Engineering Contradiction:
Improvebit rate adaptabilityVSAvoidencoder and decoder complexity
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent introduces special reference pictures (SRP) as an intermediary mechanism that bridges the current stream and target stream. The SRP acts as a mediator containing reference pictures from the target stream, allowing the decoder to switch streams by simply replacing references in the DPB without implementing complex scalable video coding structures. This intermediary approach achieves bit rate adaptability with minimal additional complexity.

Inventive Principle:
Principle #24Intermediary (Mediator)

4Adaptability or versatility

If multiple versions of the same video sequence are encoded for bit rate adjustment, then stream switching flexibility is improved, but the mismatch of reconstructed reference pictures during switching degrades quality

Engineering Contradiction:
Improvestream switching flexibilityVSAvoidpicture reconstruction accuracy
Core Design Contradiction:
Adaptability or versatilityVSReliability

Solution Approach 1:

The patent applies copying by creating special reference pictures that copy reference picture data from the target stream into the current stream's decoding context. When stream switching is needed, the DPB is updated by copying reference pictures from the target stream's DPB into the current stream's DPB as SRP. This copying mechanism ensures that the reference pictures match the target stream, maintaining reconstruction accuracy while enabling flexible switching between multiple video versions.

Inventive Principle:
Principle #26Copying

Data Source

PatentEP2959688B1Coding and decoding of a picture block
Publication Date: 2022.05.18 INTERDIGITAL VC HOLDINGS INC
  • EP2959688B1 patent drawingFigure 1~2
  • EP2959688B1 patent drawingFigure 3~4
  • EP2959688B1 patent drawingFigure 5

AI summary

A method for decoding a picture block is disclosed. The decoding method comprises: - decoding (10) at least one stream S_diff into decoded data and into one information for identifying a reconstructed reference picture in a decoder picture buffer; - reconstructing (12) a special reference picture from at least the identified reconstructed reference picture and from the decoded data; - replacing (14) in the decoder picture buffer one reconstructed reference picture by the special reference picture; and - reconstructing (16) the picture block from at least the special reference picture.