Spatial Extrapolation Prompt Signaling in Video Coding Bitstreams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding standards, such as ITU-T H.264, ITU-T H.265, ITU-T H.266, and ECM, lack efficient methods for signaling spatial extrapolation text prompts, which are crucial for enhancing video coding techniques beyond their capabilities.
Innovation Solution
Incorporating techniques for signaling spatial extrapolation text prompts through neural-network post-filter characteristics messages, parsing syntax elements to derive text string prompts, and determining prompt string positions for generating spatial extrapolation image areas.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If spatial extrapolation text prompts are added to enhance video coding capabilities, then video quality and compression performance are improved, but the complexity of the coding system increases
Solution Approach 1:
The patent embeds text prompt signaling within existing video coding bitstream structures. The text prompt is nested inside SEI messages or parameter sets that are already part of the H.264/H.265/H.266 coding framework, allowing the new functionality to be integrated without creating a completely separate system. This nesting approach enables spatial extrapolation text prompts to be conveyed through existing infrastructure, reducing the increase in system complexity.
Solution Approach 2:
The patent introduces syntax elements and signaling mechanisms as intermediaries between the text prompt generation system and the video decoding system. These intermediaries include specific syntax elements in SEI messages that carry text prompt information, allowing seamless integration of text-based spatial extrapolation guidance into the existing video coding pipeline without requiring fundamental changes to the core coding architecture.
2Productivity
If text prompt signaling mechanisms are implemented for spatial extrapolation, then compression performance is improved, but the bitstream complexity increases
Solution Approach 1:
The patent segments the text prompt signaling into distinct syntax elements within the bitstream structure. By separating the text prompt information into dedicated fields within SEI messages or parameter sets, the system can efficiently convey spatial extrapolation guidance without creating a monolithic complex structure. This segmentation allows for targeted processing and reduces overall bitstream complexity by organizing data in a modular fashion.
Solution Approach 2:
The patent designs the text prompt signaling mechanism to be universally applicable across different video coding scenarios. The same syntax elements and SEI message structures can be used for various types of spatial extrapolation tasks, making the solution multi-functional. This universality reduces bitstream complexity by avoiding the need for separate specialized signaling mechanisms for different applications, as a single unified approach can handle multiple cases.
Data Source
AI summary
A device may be configured to perform spatial extrapolation based on information included in a neural-network post-filter characteristics message. In one example, a neural-network post-filter characteristics message includes a syntax element specifying a text string prompt used for generating contents of a spatial extrapolation image area and a syntax element having a value specifying auxiliary input data includes character values derived from the text string prompt The device may be configured to determine a prompt string position and derive the text string prompt based on the prompt string position.