Blending Intra and Inter Coded Video Frames to Reduce Beating Artifacts

Solution Overview

Problem

Modern video coding techniques that rely on intra-coding frames for resilience against transmission errors can introduce artifacts, such as 'beating' artifacts, when these frames are displayed alongside inter-coded frames, impairing the perceived quality of decoded video.

Innovation Solution

A method where a decoder identifies and blends content from intra-coded frames with inter-coded frames at common temporal instances, using metadata to determine correlations and adjust frame rendering to alleviate these artifacts.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If intra-coded frames are used at regular intervals for resilience against transmission errors, then reliability is improved, but beating artifacts appear that worsen perceived video quality

Engineering Contradiction:

Improveresilience against transmission errorsVSAvoidbeating artifacts

Core Design Contradiction:

ReliabilityVSObject-affected harmful factors

Solution Approach 1:

The patent merges the content of multiple decoded frames (including intra-coded and inter-coded frames) that correspond to the same temporal instance to generate a composite frame. This combining approach allows the system to maintain the reliability benefits of intra-coded frames while mitigating their visual artifacts through blending with adjacent frames.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent creates a composite video frame by combining pixel data from multiple source frames with different coding characteristics. This composite approach类似于creating a composite material, where the strengths of different frame types (intra and inter coded) are combined to produce a final output that avoids the weaknesses of individual frame types.

Inventive Principle:

Principle #40Composite materials

2Reliability

If intra-coded frames are coded without reference to other frames, then transmission error resilience is improved, but display discontinuities occur that worsen visual quality

Engineering Contradiction:

Improvetransmission error resilienceVSAvoiddisplay continuity

Core Design Contradiction:

ReliabilityVSStability of the object's composition

Solution Approach 1:

The patent merges the content of multiple decoded frames (including intra-coded and inter-coded frames) that correspond to the same temporal instance to generate a composite frame. This combining approach allows the system to maintain the reliability benefits of intra-coded frames while mitigating their visual artifacts through blending with adjacent frames.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent processes multiple frames to achieve visual homogeneity in the final output. By blending frames from different coding types, the system creates a more uniform visual appearance across frame boundaries, reducing the perceptibility of coding-related discontinuities.

Inventive Principle:

Principle #33Homogeneity

3Loss of energy

If frames are coded on an intra-basis at regular intervals, then bandwidth compression is reduced, but beating artifacts are introduced that worsen perceived quality

Engineering Contradiction:

Improvebandwidth compression efficiencyVSAvoidbeating artifacts

Core Design Contradiction:

Loss of energyVSObject-affected harmful factors

Solution Approach 1:

The patent merges the content of multiple decoded frames (including intra-coded and inter-coded frames) that correspond to the same temporal instance to generate a composite frame. This combining approach allows the system to maintain the reliability benefits of intra-coded frames while mitigating their visual artifacts through blending with adjacent frames.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent converts the harmful beating artifacts into a benefit by using the intra-coded frames as one component in a multi-frame blend. The presence of these frames, which would normally cause artifacts, is transformed into a useful element that provides both error resilience and contributes to the composite image that reduces visual artifacts when blended with other frames.

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Data Source

PatentUS11323730B2Temporally-overlapped video encoding, video decoding and video rendering techniques therefor

Publication Date: 2022.05.03 APPLE INC

AI summary

Techniques for video exchange are disclosed in which a video source sends redundant copies of video coded at a given presentation time. A decoder may determine, from metadata stored with a first decoded frame, whether other decoded frames have been designated as correlated to the first decoded frame. If so, the decoder may fetch other decoding frames using time indicator values of other decoded frames in metadata stored with the first decoded frame. When other decoded frame(s) are found, the decoder may blend content of the first coded frame and the other decoded frame(s). The decoder may render the blended frame. When one such coded frame is an intra-coded frame, the proposed technique may alleviate beating artifacts by blending recovered data of the intra-coded frame with recovered data of another frame, which may be inter-coded.