Neural Post-Filter for Decoded Data in Machine Vision Codecs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multimedia systems face challenges in optimizing the performance of machine vision tasks due to inefficiencies in data compression and decoding processes, particularly in the use of traditional codecs that do not leverage learned neural networks effectively.
Innovation Solution
Implementing a post-processing filter trained on auxiliary information obtained from traditional codecs, utilizing a learned neural network to enhance the performance of machine vision tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional codecs are used for data compression and decoding, then device complexity is reduced and ease of operation is maintained, but machine vision task performance is insufficient
Solution Approach 1:
The system segments the processing pipeline by separating traditional codec functions from neural network enhancement functions. The traditional codec handles compression and decoding, while a dedicated neural network component processes auxiliary information to enhance machine vision tasks. This segmentation allows each component to be optimized independently, maintaining codec simplicity while adding AI-driven performance improvements.
Solution Approach 2:
The patent introduces an intermediary neural network component that processes auxiliary information from the traditional codec and generates enhanced representations for machine vision tasks. This intermediary layer acts as a bridge between the traditional codec output and the requirements of machine vision algorithms, improving task performance without requiring fundamental changes to the codec structure.
2Reliability
If learned neural networks are integrated into the codec, then machine vision task performance is improved, but device complexity and computational requirements increase
Solution Approach 1:
The system applies partial neural network processing by selectively enhancing specific aspects of the decoded data that are most beneficial for machine vision tasks, rather than fully replacing the traditional codec with a neural network-based system. This partial action approach achieves performance improvements while limiting the increase in device complexity to only the necessary neural network components.
3Productivity
If auxiliary information from traditional codecs is processed through learned neural networks, then data compression efficiency is improved, but processing time and computational resources increase
Solution Approach 1:
The neural network processes auxiliary information during the encoding phase, preparing enhanced representations in advance before the decoding and machine vision task execution. This preliminary action allows the decoded data to be ready for immediate use in machine vision tasks, reducing the processing time required during the actual vision task execution.
Data Source
AI summary
A method includes receiving auxiliary information and/or at least one auxiliary feature, the at least one auxiliary feature being based on the auxiliary information; wherein the auxiliary information comprises information available within a decoder, or the auxiliary information comprises a bitstream output from data encoded using an encoder; receiving decoded data generated using the decoder; and generating filtered data with at least one filter using the auxiliary information and/or the at least one auxiliary feature via applying the filter to the decoded data; wherein the at least one filter comprises a learned filter; wherein the filtered data is configured to be used for at least one machine task performed using a model.