Interleaved Video Bitstream Structure for Machine Task Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding technologies struggle to efficiently handle machine tasks, as they are primarily optimized for human perception and may not meet the specific requirements of machine consumption, leading to increased complexity and suboptimal performance.
Innovation Solution
A bitstream structure is designed for machine tasks, interleaving coded information according to a video coding standard with restoration data, allowing for separate processing of machine-specific information, such as sequence-level and frame-level data, using extensions within the video coding standard framework.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If video coding is optimized for human perception, then visual quality for human viewing is improved, but performance for machine tasks deteriorates
Solution Approach 1:
The bitstream is segmented into two distinct portions: a first portion containing coded information according to video coding standards, and a second portion containing restoration data specifically for machine tasks. This segmentation allows the system to optimize different parts of the bitstream for different purposes - human perception and machine task performance respectively.
Solution Approach 2:
Restoration data specifically tailored for machine tasks is extracted and placed in a separate second portion of the bitstream. This extracted data includes sequence-level and picture-level restoration information that is specifically designed to enhance machine task performance without being constrained by human perception requirements.
2Measurement precision
If separate bitstreams are used for human consumption and machine tasks, then performance for each purpose is improved, but system complexity increases
Solution Approach 1:
Two previously separate bitstreams - one optimized for human consumption and another for machine tasks - are merged into a single interleaved bitstream structure. The coded information and restoration data are interleaved at the bitstream level, allowing both human and machine processing needs to be met from a single integrated stream, thereby reducing system complexity.
Solution Approach 2:
The single interleaved bitstream serves multiple functions simultaneously: it provides coded information for human video decoding while also containing restoration data for machine task processing. This multi-functional design eliminates the need for separate processing pipelines and reduces overall system complexity.
3Measurement precision
If restoration data is added for machine tasks, then machine task performance is improved, but bitstream size increases
Solution Approach 1:
Rather than providing complete redundant information for both human and machine processing, the patent provides partial restoration data - specifically sequence-level and picture-level restoration information - that is sufficient to enhance machine task performance without duplicating all the detailed coded information. This partial action approach adds only the necessary minimal data required for machine task improvement.
Data Source
AI summary
An aspect of the disclosure provides a method of video decoding. For example, a bitstream is received. The bitstream includes coded information of a video for a machine task, the coded information includes a first portion and a second portion that are interleaved, the first portion is coded according to a video coding standard, and the second portion includes restoration data for the machine task. The coded information is reordered to generate a sub stream of the bitstream having the first portion. First reconstructed pictures of the video are generated from the sub stream according to the video coding standard. The restoration data is determined from the second portion. Second reconstructed pictures for the machine task are generated based on the first reconstructed pictures and the restoration data.