Video Stream Header Preloading for Low-Latency Image Decoding

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In systems requiring low delay for streaming, the preprocessing of header interpretation and decoded image generation are sequential, leading to delays in displaying the input stream, as preprocessing takes time and hinders reductions in delay.

Innovation Solution

An image processing apparatus that generates an encoded stream by adding part of the header information of a second access unit to supplemental enhancement information (SEI) of a first access unit, allowing for parallel processing and reducing the delay in decoding.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of time

If preprocessing and decoded image generation are performed sequentially, then processing simplicity is maintained, but decoding delay increases

Engineering Contradiction:

Improvedecoding delayVSAvoidprocessing structure complexity

Core Design Contradiction:

Loss of timeVSDevice complexity

Solution Approach 1:

The patent applies preliminary action by embedding header information of the second access unit into the SEI of the first access unit during encoding. This allows the decoder to perform preprocessing of the second AU's header in advance (parallel to decoding the first AU), rather than waiting for sequential processing, thereby reducing overall decoding delay without significantly increasing system complexity

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent enables continuous useful action by allowing the decoder to process the second access unit's header information continuously alongside the decoding of the first access unit. The embedded header information in SEI allows preprocessing to continue without interruption, maintaining steady workflow and reducing idle time between processing stages

Inventive Principle:

Principle #20Continuity of useful action

2Productivity

If preprocessing is performed before decoded image generation, then decoding accuracy is ensured, but real-time processing capability is reduced

Engineering Contradiction:

Improvereal-time processing capabilityVSAvoiddecoding accuracy

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

By embedding the second AU's header information in the first AU's SEI, the patent enables preliminary processing of subsequent data while current data is being decoded. This maintains decoding accuracy through proper header processing while improving real-time processing capability through parallel operation

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The SEI message acts as an intermediary carrier that transports the second AU's header information to the decoder. This intermediary mechanism allows the header information to be available in advance without disrupting the normal decoding workflow, ensuring both accuracy and real-time performance

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS12200260B2Image processing apparatus and method

Publication Date: 2025.01.14 SONY SEMICON SOLUTIONS CORP

AI summary

Provided is an image processing apparatus and method for shortening a delay from inputting of a stream till generation of a decoded image. The image processing apparatus generates an encoded stream by adding at least part of header information about a second access unit (AU) to be processed immediately after a first AU or information regarding at least part of the header information, to supplemental enhancement information (SEI) about the first AU.