Saliency-Guided Image Quality Scoring With Region Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image/video quality assessment (I/VQA) techniques face challenges in accurately predicting video quality without relying on original images, particularly due to computational burdens and inefficiencies in end-to-end learning, especially when dealing with in-the-wild videos and random sampling of sub-regions.
Innovation Solution
An electronic apparatus employing neural network models to generate saliency maps and quality scores, segmenting images into sub-regions based on user gaze data to identify regions of interest, and using these sub-regions for quality assessment, thereby reducing computational load and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If end-to-end learning is used for image/video quality assessment, then model accuracy can be improved, but computational burden and training time increase significantly
Solution Approach 1:
The patent segments the image into multiple sub-regions and processes each sub-region separately through the neural network model. This segmentation approach reduces the computational burden on the model while maintaining quality assessment accuracy, as the model only needs to process smaller sub-region features rather than the entire large-resolution image at once.
2Measurement precision
If the entire image is processed for quality assessment, then comprehensive quality evaluation is achieved, but processing time and computational resources increase
Solution Approach 1:
The image is divided into multiple sub-regions, and the neural network model processes each sub-region independently. This segmentation enables faster processing while maintaining comprehensive quality evaluation, as parallel processing of smaller sub-regions is more efficient than processing the entire image sequentially.
Solution Approach 2:
The patent focuses on processing key sub-regions that contain important visual information rather than uniformly processing the entire image. This partial action approach maintains quality assessment accuracy while reducing overall processing time and computational resources required.
3Measurement precision
If high-resolution images are assessed in full, then detailed quality metrics are obtained, but computational complexity and processing time increase
Solution Approach 1:
High-resolution images are segmented into smaller sub-regions, which reduces the computational complexity of processing while preserving detailed quality metrics within each sub-region. The neural network model processes these smaller sub-regions more efficiently, obtaining detailed quality information without the overwhelming complexity of processing the entire high-resolution image at once.
Data Source
AI summary
An electronic apparatus may include: a memory storing: a first neural network model (NNM) trained to output a saliency map for an image and a second NNM trained to output a quality score for an image; and a processor connected to the memory and configured to: obtain the saliency map including a saliency value of each pixel of a plurality of pixels included in a first image through the first neural network model based on the first image, identify a plurality of first sub-regions respectively corresponding to a plurality of regions included in the first image based on the saliency map, and obtain the quality score for the first image through the second neural network model based on the identified plurality of first sub-regions, wherein the quality score is based on a plurality of first quality scores respectively corresponding to the identified plurality of first sub-regions.