Siamese Transformer Image Quality Prediction Across Unlabeled Domains
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image quality assessment models face challenges in accurately judging image quality across different domains due to the need for costly manual labeling and the performance drop when tested on out-of-distribution datasets, with unsupervised domain adaptation techniques often sacrificing source accuracy or requiring overlapping distortions.
Innovation Solution
A Siamese transformer network with shared parameters is used to predict image quality differences between image pairs, leveraging pseudo-label ensembling to adapt to unsupervised domains and maintain high accuracy without additional fine-tuning, using pair-wise learning and transformer blocks to capture subtle quality differences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual labeling is used to train image quality assessment models, then measurement precision is improved, but loss of time and loss of substance increase due to expensive and time-consuming labeling processes
Solution Approach 1:
The system uses the model itself to generate pseudo-labels for unlabeled images, creating a self-service labeling mechanism. The pre-trained model predicts quality scores for unlabeled images, and these predictions are used as pseudo-labels for further training, eliminating the need for manual labeling of every image in the dataset.
Solution Approach 2:
The system changes the training parameter from requiring manual ground truth labels to using predicted pseudo-labels. By modifying the labeling parameter from human-annotated scores to model-generated predictions, the system reduces time loss while maintaining training effectiveness through iterative refinement of these pseudo-labels.
2Adaptability or versatility
If domain adaptation techniques are used to improve performance on out-of-distribution datasets, then adaptability is improved, but source accuracy deteriorates or additional fine-tuning is required
Solution Approach 1:
The system performs preliminary domain adaptation by training on a diverse set of labeled images from multiple domains before deployment. This pre-adaptation to various domains in advance allows the model to maintain source accuracy while being prepared for out-of-distribution inputs, avoiding the need for fine-tuning when deployed on new domains.
Solution Approach 2:
The system creates a universal model that functions across multiple domains simultaneously by training on diverse data from different image domains. This multi-functional training approach allows the single model to maintain high accuracy on source domains while adapting to out-of-distribution datasets without requiring domain-specific fine-tuning.
3Measurement precision
If more labeled data is collected to improve model accuracy, then measurement precision is improved, but loss of substance increases due to costly data collection and labeling
Solution Approach 1:
The system creates copies of labeled data by generating pseudo-labels for unlabeled images using the pre-trained model. Instead of collecting and labeling new data, the system copies the labeling process by having the model predict labels for similar unlabeled images, significantly reducing data collection costs while expanding the effective training dataset.
Solution Approach 2:
The system performs self-service data augmentation by generating its own training labels through model predictions. The model serves itself by producing pseudo-labels that are then used to retrain and improve the model, eliminating the need for external data collection and manual labeling resources.
Data Source
AI summary
A method includes obtaining, using at least one processing device of an electronic device, a specified image. The method also includes identifying, using the at least one processing device, a reference image and a corresponding reference image label. The method further includes inputting, using the at least one processing device, the specified image and the reference image to a Siamese transformer network trained to predict an image quality difference between an input image pair. The method also includes predicting, using the Siamese transformer network, an image quality difference between the specified image and the reference image. In addition, the method includes adding, using the at least one processing device, the corresponding reference image label to the predicted image quality difference to obtain an image quality score of the specified image.