Neural Image Compression Weight Entropy for Smaller Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image compression methods face challenges in minimizing the size of neural network weights during the learning process, as existing processes require transmitting pre-learned model weights, leading to inefficiencies.
Innovation Solution
A neural network-based image compression method that updates model weights by incorporating both image reconstruction performance and estimated entropy, using a substitution function with a uniform distribution to minimize entropy, and adjusts the weight parameters based on a combined loss function.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If pre-learned model weights are transmitted for image compression, then image compression functionality is enabled, but the size of transmitted weights becomes large
Solution Approach 1:
The patent changes the parameter being optimized from pure image reconstruction quality to a combined loss function that includes both reconstruction quality and entropy of weights. By minimizing entropy, the weight values become more concentrated and representable with fewer bits, directly reducing the quantity of substance (weight size) while maintaining functionality
Solution Approach 2:
The patent replaces the conventional mechanical approach of directly transmitting fixed weights with an information-theoretic approach using entropy estimation and minimization. This substitution allows the system to achieve more efficient weight representation by leveraging statistical properties of the weight distribution
2Quantity of substance
If neural network weights are minimized for compression, then weight size is reduced, but image reconstruction performance may deteriorate
Solution Approach 1:
The patent merges two previously separate objectives into a single unified loss function: image reconstruction quality (PSNR/SSIM) and entropy of weights. This combination allows the optimization process to simultaneously consider both reconstruction performance and weight compactness, ensuring that weight minimization does not come at the expense of image quality
Solution Approach 2:
The patent implements feedback by continuously monitoring both reconstruction quality and entropy during the learning process. The loss function provides real-time feedback that guides weight updates, ensuring that any reduction in weight size is compensated by maintaining or improving reconstruction performance through adaptive optimization
Data Source
AI summary
A method of learning a neural network-based image compression model according to the disclosed embodiment, may include receiving a learning target image as an input; encoding the input image through the neural network-based image compression model configured to include a weight parameter, and decoding the encoded image through the neural network-based image compression model; calculating an entropy estimation value for a network model weight of the neural network-based image compression model; calculating a reconstruction performance value by comparing qualities of the learning target image and the decoded image; and learning the neural network-based image compression model by updating the weight parameter of the neural network-based image compression model based on the entropy estimation value for the network model weight and the reconstruction performance value. Accordingly, it is possible to minimize the size of the weight of the neural network-based image compression model.