Hybrid INR Spatiotemporal Context Coding for Lower Video Bitrate
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current approaches for encoding and decoding data using implicit neural representation (INR) networks are inefficient in exploiting spatial and temporal redundancies across latent variables, leading to high complexity and bitrate in data compression.
Innovation Solution
The proposed method involves training an INR network to produce data regions from latent variables, determining distributions of these variables using a shaped context region, and coding them into a bitstream, while also encoding the network parameters, thereby optimizing the encoding process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If current approaches are used to estimate latent variables' distributions based on context, then the INR network can represent the signal, but spatial and temporal redundancies are not effectively exploited leading to high bitrate
Solution Approach 1:
The patent segments the context construction process into distinct spatial and temporal components. Spatial context is constructed from neighboring latent variables in the current frame, while temporal context is constructed from corresponding latent variables in reference frames. This segmentation allows independent optimization of each context type and enables selective usage based on content characteristics, reducing overall complexity while improving redundancy exploitation.
Solution Approach 2:
The patent introduces a temporal dimension to the traditional spatial context construction. By constructing context from both spatial neighbors (current frame) and temporal references (previous frames), the system moves from 2D spatial context to 3D spatiotemporal context. This dimensional expansion enables exploitation of temporal redundancies alongside spatial redundancies, reducing bitrate without proportionally increasing complexity.
2Loss of information
If spatial and temporal redundancies are not effectively exploited, then the encoding process is simpler, but the bitrate remains high
Solution Approach 1:
The patent implements feedback mechanisms where the constructed spatial and temporal contexts are used to guide the entropy coding process. The context models provide probability distributions for latent variables, and the actual latent variable values feed back into the context construction for subsequent variables. This feedback loop enables adaptive exploitation of redundancies, improving encoding efficiency and reducing bitrate through more accurate probability modeling.
Solution Approach 2:
The patent dynamically adjusts coding parameters based on the constructed contexts. The context models generate variable-specific probability distributions that change parameters such as prediction values and probability ranges for entropy coding. This parameter adaptation allows the encoder to match the coding strategy to the local characteristics of each latent variable, maximizing redundancy exploitation and improving encoding efficiency.
3Loss of information
If a shaped context region is used to determine distributions, then spatial and temporal redundancies are better exploited, but the context construction becomes more complex
Solution Approach 1:
The patent employs dynamic context region construction where the shape and extent of the context region are adaptively determined based on content characteristics. Rather than using fixed context regions, the system adjusts the spatial and temporal extents of context regions dynamically, expanding them when strong redundancies are detected and contracting them when redundancies are weak. This dynamic approach improves compression efficiency while keeping processing complexity manageable through content-adaptive adjustments.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Apparatuses and methods are disclosed for encoding and decoding data. Techniques disclosed provide for the encoding of a data region of a frame. The encoding includes training an INR network to produce the data region from latent variables. And, determining distributions of the latent variables using respective contexts, constructed based on latent variables located within a shaped context region. Then, coding into a bitstream the latent variables based on the determined distributions and further coding into the bitstream the parameters of the trained INR network. Techniques disclosed also provide for decoding the data region. The decoding includes decoding from the bitstream the parameters of the INR network, determining the distributions of the latent variables using the respective contexts, and decoding from the bitstream the latent variables based on the determined distributions. Using the parameters of the INR network, the INR network produces the data region from the latent variables.