Separated Neural Restoration Parameters for Efficient Video Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding systems face challenges in efficiently separating and transmitting feature restoration parameters, particularly in neural network-based systems, leading to inefficiencies in compression and decoding processes.
Innovation Solution
The proposed solution involves encoding and transmitting neural network restoration parameters separately, using indicators and supplemental enhancement information to facilitate decoding, allowing for efficient separation and transmission of feature restoration parameters, including neural network-based and principal component analysis types, with specific parameters like structure and weight parameters, and tensors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If neural network restoration parameters are transmitted together with video data, then decoding can be performed, but transmission bandwidth and processing complexity increase
Solution Approach 1:
The patent separates neural network restoration parameters into distinct components (structure parameters, weight parameters, tensor data) and transmits them through different channels or at different stages. The video data contains basic encoding information while neural network parameters are transmitted separately via supplemental enhancement information (SEI) messages or dedicated parameter streams, reducing the complexity of processing a single monolithic data structure.
Solution Approach 2:
The patent extracts critical neural network parameters (such as restoration type indications, feature restorer type indications, and essential weight parameters) and transmits them separately from the main video bitstream. This extraction allows the decoder to obtain necessary restoration parameters without processing the entire neural network model data, thereby reducing computational complexity while maintaining decoding accuracy.
2Reliability
If all neural network parameters are transmitted, then model restoration is complete, but transmission bandwidth increases
Solution Approach 1:
The patent extracts and transmits only the essential neural network parameters needed for effective video restoration, such as restoration type indications, feature restorer type indications, and critical weight parameters. Less critical parameters or parameters that can be inferred by the decoder are not transmitted, thereby reducing transmission bandwidth while maintaining sufficient model restoration completeness for practical applications.
Solution Approach 2:
The patent transmits a partial set of neural network parameters that provides sufficient restoration quality for practical purposes, rather than transmitting the complete model. The decoder uses received parameters (such as structure parameters and selected weight parameters) to reconstruct a functional restoration model that achieves acceptable performance without requiring all original model parameters.
3Productivity
If feature reduction is applied to NN models, then compression efficiency improves, but decoding complexity increases
Solution Approach 1:
The patent performs feature reduction and parameter extraction in advance during the encoding phase, transforming the neural network model into a compressed representation with essential parameters (structure parameters, reduced weight parameters, feature restorer type indications). The decoder receives this pre-processed data and only needs to perform straightforward reconstruction operations, thereby achieving high compression efficiency without significantly increasing decoding complexity.
Data Source
AI summary
Systems, methods, and instrumentalities are disclosed for separating the coding and transmission of feature restoration parameters in a feature compression bitstream. In examples, the decoder may obtain a neural network (NN) feature restorer type indication (e.g., fcm_nn_restorer_type) in video data. The NN feature restorer type indication may be configured to indicate an NN feature restorer type associated with an NN model. Based on the NN feature indication type indication, the decoder may obtain a feature parameter associated with the NN feature restorer type. Based on the feature parameter associated with the NN feature restorer type, the NN model may be decoded.