Codec Network Joint Training for ML Task Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image encoding and decoding technologies are not optimized for machine learning tasks, leading to redundant information that is not effectively removed, which affects the performance of machine learning tasks on decoded reconstructed images.
Innovation Solution
A joint training method is employed for the codec network and the task execution network using a joint loss function that includes a loss function for the task execution network and a bitrate function for the feature bitstream, ensuring that the codec network is optimized for the machine learning task while constraining the bitrate.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing image encoding and decoding technologies are used, then the encoding and decoding process can be completed, but redundant information is not effectively removed which affects machine learning task performance
Solution Approach 1:
The patent changes the optimization parameters of the codec network by introducing a task-specific loss function that combines reconstruction loss with task performance loss. This parameter change enables the network to optimize for both compression quality and machine learning task performance simultaneously, effectively removing redundant information that does not contribute to the task.
Solution Approach 2:
The codec network is designed to serve multiple functions: traditional image compression and reconstruction, plus optimization for specific machine learning tasks. By making the codec network multi-functional through joint training with the task execution network, it can remove redundant information while preserving task-critical features.
2Reliability
If the codec network is optimized for machine learning tasks, then task performance improves, but the bitrate constraint may be violated
Solution Approach 1:
The joint loss function dynamically balances reconstruction quality and task performance while incorporating bitrate constraints. By changing the optimization parameters to include all three factors (reconstruction, task performance, and bitrate), the system achieves task optimization without violating bitrate limits.
Solution Approach 2:
The joint training process provides feedback between the codec network and task execution network, allowing the system to learn how to compress images in a way that maintains task performance within bitrate constraints. The feedback loop enables continuous optimization of the compression parameters based on task performance metrics.
Data Source
AI summary
An image processing method, an encoding method, an image processing apparatus, and an encoding apparatus are provided. The image processing method includes the following operations. An encoded bitstream is received from a trained encoder. The encoded bitstream is decoded by a trained decoder to obtain a decoded reconstructed image. The decoded reconstructed image or the decoded reconstructed image subjected to image post-processing is processed by a trained task execution network to perform or complete a machine learning task. The trained encoder and the trained decoder belong to a trained codec network.