Neural Network Image Encoding for Visual Sensitivity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
AI-based compression methods face challenges in effectively compressing images or videos with regions of high visual sensitivity, leading to noticeable distortion, particularly in areas like human faces, which affects the quality of the compressed output.
Innovation Solution
The method involves using multiple trained neural networks to identify and separately encode regions of high visual sensitivity, performing quantization, and transmitting these representations alongside the main image encoding, allowing for more precise bit allocation and improved reconstruction quality through techniques like Poisson blending and hyper-latent representations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If lossless compression is used, then all original information can be recovered, but data quantity reduction is limited
Solution Approach 1:
The patent applies different compression strategies to different regions of the image based on visual sensitivity. High visual sensitivity regions (like faces) use lossless or high-quality lossy compression to preserve information, while low sensitivity regions use aggressive lossy compression to reduce data quantity. This resolves the contradiction by making compression quality spatially variable rather than uniform.
Solution Approach 2:
The image is segmented into multiple regions with different visual sensitivities using a neural network. Each region is then processed independently with appropriate compression settings. This segmentation allows the system to recover information where needed while reducing overall data quantity through aggressive compression in less critical areas.
2Quantity of substance
If uniform lossy compression is applied, then data quantity is reduced, but distortion in high visual sensitivity areas increases
Solution Approach 1:
The patent implements spatially varying compression quality by identifying high visual sensitivity regions and applying milder compression to these areas while using stronger compression in low sensitivity regions. This ensures that distortion remains acceptable in critical areas while achieving overall data reduction.
Solution Approach 2:
The compression parameters (such as quantization strength) are dynamically adjusted based on the visual sensitivity of each region. High sensitivity regions receive parameter settings that preserve quality, while low sensitivity regions use parameters that maximize compression. This parameter adaptation resolves the contradiction between data reduction and quality preservation.
3Productivity
If AI-based compression is used, then compression efficiency improves, but performance on high visual sensitivity content deteriorates
Solution Approach 1:
The patent segments the image processing into two stages: first identifying high visual sensitivity regions using a neural network, then applying region-specific compression. This segmentation allows the AI to maintain high efficiency while improving quality in critical areas through targeted processing.
Solution Approach 2:
The neural network performs preliminary identification of high visual sensitivity regions before the main compression process. This preliminary action enables the compression algorithm to allocate bits appropriately, ensuring high quality output in critical regions while maintaining overall compression efficiency.
Data Source
AI summary
A method for lossy image or video encoding, transmission and decoding, the method comprising the steps of: receiving an input image at a first computer system; encoding the input image using a first trained neural network to produce a latent representation; identifying one or more regions of the input image associated with high visual sensitivity; encoding the one or more regions of the input image associated with high visual sensitivity using a second trained neural network to produce one or more region latent representations; performing a quantization process on the latent representation and the one or more region latent representations; transmitting the result of the quantization process to a second computer system; decoding the result of the quantization process to produce an output image, wherein the output image is an approximation of the input image.