Learned Lossy Image Compression for Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer vision algorithms require uncompressed images for processing, which increases storage and bandwidth needs, as most image compression methods, like JPEG, necessitate decoding before analysis, leading to inefficiencies in processing and storage.
Innovation Solution
A system using a learned image compression model that includes an encoder and decoder, trained with machine learning, to compress images and enable object detection directly on compressed representation data, reducing storage and processing demands by performing partial decoding for object detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional image compression methods like JPEG are used, then storage and bandwidth are reduced, but decoding is required before computer vision processing, increasing processing time and complexity
Solution Approach 1:
The system segments the image processing pipeline by separating compression from traditional decoding requirements. The encoder generates compressed representations that preserve essential features for computer vision tasks without requiring full reconstruction, allowing parallel processing of compression and vision tasks.
Solution Approach 2:
The patent introduces an intermediary representation format between the compressed image and the computer vision algorithm. This intermediate representation maintains the compressed structure while being directly compatible with vision algorithms, eliminating the need for traditional decoding to uncompressed formats.
2Quantity of substance
If images are compressed using traditional methods, then storage efficiency improves, but image quality and processing speed for computer vision tasks deteriorate due to decoding requirements
Solution Approach 1:
The compression model is pre-trained with knowledge of computer vision task requirements. During training, the model learns to preserve features important for vision tasks while compressing the image, so that when compression is applied, the output is already optimized for both storage efficiency and subsequent vision processing speed.
Solution Approach 2:
The system changes the parameter space by using learned compression representations instead of traditional compression formats. The encoder learns optimal compression parameters specific to computer vision tasks, creating a representation that balances storage efficiency with processing speed by maintaining task-relevant features in compressed form.
3Reliability
If full decoding is performed on compressed images, then image quality is maintained, but storage and processing demands increase
Solution Approach 1:
Instead of performing complete decoding to restore full image quality, the system performs partial decoding or uses the compressed representation directly for vision tasks. This partial action approach maintains sufficient quality for vision processing while avoiding the full processing overhead of traditional decoding.
Data Source
AI summary
Techniques for performing learnt image compression and object detection using compressed image data are described. A system may perform image compression using an image compression model that includes an encoder, an entropy model, and a decoder. The encoder, the entropy model, and the decoder may be jointly trained using machine learning based on training data. After training, the encoder and the decoder may be separated to encode image data to generate compressed image data or to decode compressed image data to generate reconstructed image data. In addition, the system may perform object detection using a compressed object detection model that processes compressed image data generated by the image compression model. For example, the compressed object detection model may perform partial decoding using a single layer of the decoder and perform compressed object detection on the partially decoded image data.