Content-Aware Image Compression Using Specialized Block Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep image/video compression methods struggle with optimizing compression efficiency for image blocks with varying attributes, leading to sub-optimal performance when processing image blocks with different variances, objects, or textures, and are resource-intensive due to large model sizes needed for diverse image types.
Innovation Solution
Implementing a content-aware dataset generation and model selection process that groups image blocks with similar attributes into datasets, trains specific neural networks for these groups, and reduces model capacity through techniques like sparsification and pruning, while using autoregressive and hierarchical priors models for efficient compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If deep image compression methods are trained on randomly cropped patches, then training simplicity is maintained, but compression performance on specific image blocks deteriorates
Solution Approach 1:
The patent segments the image database into multiple groups based on content attributes (e.g., natural images, text, graphics). Each segment is trained separately with a dedicated model, allowing each model to specialize in specific image types rather than using a single generic model trained on random patches. This segmentation resolves the contradiction by maintaining training organization while significantly improving compression performance for each image category.
2Adaptability or versatility
If a single large model is used to handle diverse image attributes, then versatility is improved, but model size and computational resources increase
Solution Approach 1:
Instead of one large universal model, the patent divides the system into multiple smaller specialized models, each handling a specific image attribute group. This segmentation reduces the size of each individual model while collectively maintaining versatility across diverse image types.
Solution Approach 2:
The system achieves universality through a collection of specialized models rather than a single multi-functional model. Each model is optimized for specific image attributes, and the system as a whole can handle diverse image types by selecting the appropriate specialized model, reducing overall computational burden and model size requirements.
3Productivity
If multiple specialized models are trained for different image attributes, then compression efficiency is improved, but system complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-grouping images into attribute-based categories and pre-training specialized models for each group before actual compression tasks. This preparation work is done once during system setup, and then the pre-trained models can be efficiently selected and applied during compression operations, improving efficiency without proportionally increasing operational complexity.
Solution Approach 2:
The patent introduces an intermediary component (the attribute-based grouping system and model selection mechanism) that manages the complexity of multiple specialized models. This intermediary layer handles the coordination between diverse image inputs and appropriate models, making the system easier to manage despite having multiple specialized components.
4Use of energy by moving object
If model capacity is reduced through sparsification and quantization, then computational resources are reduced, but model performance may deteriorate
Solution Approach 1:
The patent applies local quality by reducing model capacity specifically for handling certain image attribute types where full precision may not be necessary. Different levels of model compression can be applied to different image groups based on their specific requirements, optimizing the balance between computational resources and performance for each category rather than uniformly compressing all models.
Data Source
AI summary
A system may receive an input image block, and input the input image block into multiple models which may be trained using a plurality of different datasets of image blocks. Each model of the multiple models may be trained using a dataset having similar attributes. The system may determine a model having a highest compression efficiency from among the multiple models, and encode the input image block using the determined model.