Adaptive Image Region Encoding for ML Precision Under Bitrate Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cascaded machine learning models in image processing face inefficiencies due to reduced detail in smaller image regions, leading to precision and accuracy limitations, and increasing resolution or bitrate requirements for improved performance.
Innovation Solution
Adaptive encoding of image regions based on output data from machine learning models, adjusting encoding quality settings to maintain or improve performance without increasing resolution and bitrate for the entire frame.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the resolution or bitrate of the camera feed is increased to capture more detail in smaller image regions, then the precision and accuracy of machine learning models is improved, but the computational resources, storage requirements, and bandwidth requirements increase
Solution Approach 1:
The patent applies local quality by encoding different regions of the image at different quality levels. Specifically, regions of interest (such as detected objects or areas requiring detailed analysis) are encoded at higher quality settings, while other regions use lower quality settings. This allows machine learning models to receive sufficient detail in critical areas without requiring the entire image to be encoded at high resolution, thereby reducing overall computational and storage resources while maintaining model precision where it matters most.
2Reliability
If the resolution of the camera feed is increased to maintain detail in smaller image regions, then the quality of machine learning model output is improved, but the hardware requirements and processing time increase
Solution Approach 1:
The system encodes specific regions of interest within the image at higher quality settings rather than increasing the resolution of the entire image. This localized approach ensures that machine learning models receive sufficient detail in critical areas for reliable output, while avoiding the need to process and store high-resolution data for the entire image, thereby reducing hardware requirements and processing time.
3Productivity
If the bitrate of the camera feed is increased to improve image quality for machine learning analysis, then the performance of machine learning models is improved, but the storage and bandwidth requirements increase
Solution Approach 1:
The patent implements local quality encoding where different bitrate settings are applied to different regions of the image based on their importance. Regions containing objects of interest or areas requiring detailed machine learning analysis are encoded at higher bitrates, while other regions use lower bitrates. This selective approach maintains model performance in critical areas while significantly reducing overall storage and bandwidth requirements compared to encoding the entire image at high bitrate.
Data Source
AI summary
In various examples, properties may be determined for image regions, where the image regions are indicated by output data generated using MLMs. An encoder may use the properties to generate encoded images using encoding quality settings for the image regions. When an encoded image is decoded and applied to the MLMs, corresponding output data may indicate an image region which is likely to correspond to an encoded image region of the encoded image, and which may be applied to at least one MLM. Thus, the properties for encoding an image region to an encoded image can be adapted to control the visual quality of an image region determined from a decoded version of the encoded image. The properties may be determined based at least on performance metric values for the MLMs or based at least on a ranking of the image regions.