Vision Transformer Image Coding for Key-Region Compression Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image/video encoding technologies, based on convolutional neural networks, uniformly encode all areas of an image, failing to distinguish between key and non-key areas, leading to loss of important information and inflexible compression control, and are inadequate for machine vision tasks.
Innovation Solution
An image encoding method that partitions the image into patches, calculates gradient values to identify key areas, and uses a vision transformer model for selective and controllable compression, retaining dense information and discarding sparse areas, enabling flexible bit rate control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If uniform encoding is applied to all image areas, then the encoding process is simple, but important information is lost and compression control is inflexible
Solution Approach 1:
The image is divided into multiple patches, and each patch is further segmented into key areas and non-key areas based on gradient values. This segmentation enables selective encoding where key areas are encoded with higher precision and non-key areas with lower precision, resolving the contradiction between simple encoding and information preservation.
Solution Approach 2:
Different encoding qualities are applied to different regions of the image. Key areas with high gradient values (indicating important information) are encoded with higher quality, while non-key areas with low gradient values are encoded with lower quality. This local differentiation maintains important information while reducing overall encoding complexity.
2Device complexity
If uniform encoding is applied to all image areas, then the encoding process is straightforward, but compression control becomes inflexible
Solution Approach 1:
The encoding system dynamically adjusts the encoding parameters and quality levels based on the content characteristics of different image regions. By calculating gradient values and identifying key areas, the system can adaptively control compression ratios for different regions, providing flexible compression control while maintaining process simplicity through automated decision-making.
3Loss of information
If all image patches are encoded, then complete information is retained, but compression efficiency decreases
Solution Approach 1:
The method extracts and identifies key areas from the image based on gradient values, then selectively encodes only these key areas with high precision while applying lower precision encoding to non-key areas. This extraction approach maintains essential information completeness while significantly improving compression efficiency by avoiding redundant encoding of less important regions.
Solution Approach 2:
Non-key areas with low gradient values are discarded from high-quality encoding and processed with lower quality encoding, while key areas are recovered and encoded with high precision. This selective discarding and recovering strategy optimizes the balance between information completeness and compression efficiency.
Data Source
AI summary
An image coding method and apparatus, an image decoding method and apparatus, a readable medium, and an electronic device are disclosed. The image coding method includes: obtaining an original image, and performing block processing to obtain a plurality of image blocks; calculating a gradient value of a pixel in each image patch, and screening for important region blocks according to the gradient values of the pixels; and inputting the important region patches and position information of the important region patches in the original image into a visual conversion model for coding so as to generate a bit stream.