Neural Image Compression With Blockwise Adaptive Online Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural network-based video or image coding frameworks are limited to specific types of compression frameworks, requiring increased computing memory and cost, leading to overall lower performance.
Innovation Solution
A method for content-adaptive online training of multiple blocks in neural image compression, where an input image is split into blocks, a subset with a shared pattern is selected, and a neural network is preprocessed to generate updated parameters, optimizing the E2E NIC framework for improved compression performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional neural network-based coding frameworks are used to accommodate various types of frameworks, then adaptability is improved, but computing memory cost increases
Solution Approach 1:
The input image is divided into multiple blocks, and the neural network processes each block independently through content-adaptive online training. This segmentation allows the system to handle various framework types without requiring the entire system to be reconfigured, reducing memory overhead while maintaining adaptability.
Solution Approach 2:
The patent applies content-adaptive online training where different blocks of the image receive customized neural network processing based on their local characteristics. Each block is trained independently with its own parameters, allowing the system to adapt to different content types locally without increasing global memory requirements.
2Adaptability or versatility
If conventional neural network-based coding frameworks are used to accommodate various types of frameworks, then adaptability is improved, but rate-distortion loss increases
Solution Approach 1:
The neural network undergoes content-adaptive online training before actual compression, where it learns optimal parameters for different image blocks. This preliminary training action enables the network to be better prepared for the specific content it will compress, improving rate-distortion performance while maintaining framework adaptability.
Solution Approach 2:
The patent dynamically adjusts neural network parameters through online training based on the specific content of each image block. By changing parameters adaptively rather than using fixed parameters, the system achieves better rate-distortion tradeoff while accommodating different framework types.
3Productivity
If the neural network is trained on all blocks, then compression performance is improved, but computing resources increase
Solution Approach 1:
Instead of training the neural network on all blocks uniformly, the patent applies content-adaptive online training only to selected blocks that benefit most from it. This partial action approach achieves sufficient compression performance while significantly reducing the computing resources required compared to training all blocks.
Solution Approach 2:
The image is segmented into multiple blocks, and the neural network is applied selectively to different blocks based on their content characteristics. This segmentation allows the system to optimize compression performance for critical blocks while using simpler methods for others, reducing overall computing resource consumption.
Data Source
AI summary
Content-adaptive online training for end-to-end (E2E) neural image compression (NIC) using a neural network performed by at least one processor, is provided, including receiving an input image, to an E2E NIC framework, splitting the input image into a plurality of blocks, selecting a subset of blocks from the plurality of blocks, the subset of blocks sharing a same pattern, preprocessing a neural network of the E2E NIC framework, wherein the preprocessed neural network is applied to the selected subset of blocks, computing updated parameters using the preprocessed neural network, and generating an updated E2E NIC framework, based on the updated parameters.