ROI Extraction Using Class Activation Maps and MOS Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing ROI extraction methods lack accuracy and reliability in ensuring even human gaze distribution across objects and proper class prioritization, leading to inconsistent and unreliable ROI extraction.
Innovation Solution
A processor-based ROI extraction method that generates distorted input images using machine learning models trained on MOS datasets, determines quality scores, and extracts ROIs using class activation maps with thresholding, allowing for accurate ROI selection based on pixel values and distortion adjustments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If saliency detection method or deep learning feature recognition is used for ROI extraction, then the extraction process can be automated, but the accuracy and reliability of ROI extraction deteriorates due to uneven human gaze distribution and unknown class priorities
Solution Approach 1:
The patent introduces an intermediary process between automated detection and final ROI extraction: human gaze tracking data is collected and used to train a machine learning model that predicts optimal ROIs. This intermediary training phase allows the system to learn from human behavior patterns, making the automated extraction reliable while maintaining automation.
Solution Approach 2:
The patent performs preliminary action by collecting human gaze data and training the machine learning model before actual ROI extraction. This pre-training phase establishes the foundation for accurate automated extraction, ensuring that when the system operates autonomously, it does so with high reliability based on learned human preferences.
2Quantity of substance
If deep learning recognizer extracts multiple objects as ROI, then comprehensive object detection is achieved, but the problem arises that priorities of classes are unknown and classes may not be actually included in ROI
Solution Approach 1:
The patent implements feedback by using human gaze data as ground truth to train the machine learning model. The model learns from the feedback of where humans actually look and focus attention, enabling it to prioritize classes accurately. This feedback loop ensures that when multiple objects are detected, they are prioritized according to actual human interest rather than arbitrary algorithms.
3Measurement precision
If quality score determination using MOS dataset is implemented, then ROI extraction accuracy is improved, but processing time and computational complexity increase
Solution Approach 1:
The patent performs preliminary action by pre-training the machine learning model on MOS datasets before actual ROI extraction operations. This one-time computationally intensive training phase is done in advance, allowing subsequent ROI extractions to be fast and efficient. The time cost is shifted from operational phase to setup phase, maintaining high precision while reducing processing time during actual use.
Data Source
AI summary
An apparatus with region of interest (ROI) extraction includes: a processor configured to: generate an input image by distorting an original image comprising one or more objects; determine, based on the original image, a quality score of the input image using a machine learning model that is trained based on a mean opinion score (MOS) dataset; generate a class activation map for the input image based on the quality score of the input image; and extract an ROI from the original image based on the class activation map.