Image Tag Relevance Prediction via Cluster Balance and Voting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image tagging systems face inefficiencies due to cluster imbalance and user tagging biases, leading to inaccurate image retrieval and tagging, especially for rare or specific tags, as they rely on computationally challenging classifiers and biased annotators.
Innovation Solution
An enhanced clustering technique, such as modified recursive KMeans, is applied to balance clusters, and a diversity-based voting framework is used to mitigate user tagging biases, enabling more accurate tag prediction and retrieval by considering similarity in tag sequences and annotators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional text-based image search techniques are used, then the search system can retrieve images based on keyword tags, but the accuracy of tag prediction is poor due to cluster imbalance and user tagging biases
Solution Approach 1:
The system performs preliminary actions by collecting and analyzing tag sequences from multiple annotators before making final tag predictions. This includes gathering training data with annotations, computing tag sequence similarities, and establishing voting frameworks in advance to improve prediction accuracy while accounting for annotator biases
Solution Approach 2:
The system implements feedback mechanisms by using tag sequence voting where multiple annotators' predictions are aggregated. The final tag assignment considers the consensus across annotators, with feedback loops that refine predictions based on aggregated voting results and tag sequence similarities
2Adaptability or versatility
If a large number of sample images are collected for each possible tag to train classifiers, then the system can handle more tags, but the computational complexity becomes challenging when the number of tags exceeds 10000
Solution Approach 1:
The system segments the large-scale tagging problem into manageable components by processing tag sequences from multiple annotators separately and then aggregating results through voting. This divides the computational task of handling 10000+ tags into smaller, parallelizable units based on annotator contributions rather than processing all tags simultaneously
Solution Approach 2:
The system uses copying by leveraging tag sequences from multiple annotators as redundant information sources. Instead of training separate classifiers for each tag, it copies the annotation process across multiple annotators and uses voting to aggregate results, reducing computational complexity while maintaining versatility
3Productivity
If similar images are annotated by similar annotators, then the tagging process is efficient, but tagging biases are introduced that corrupt keyword tags even when images have differences
Solution Approach 1:
The system uses feedback through tag sequence voting to counteract tagging biases. By aggregating predictions from multiple annotators and comparing tag sequences, the system identifies and corrects biased annotations, maintaining efficiency while improving accuracy through consensus-based feedback mechanisms
Solution Approach 2:
The system applies local quality by treating each annotator's tag sequences individually and computing similarities locally rather than applying uniform tagging. This allows the system to account for individual annotator characteristics and biases while maintaining overall tagging efficiency through localized analysis
4Measurement precision
If large sets of data are clustered to group similar data points, then the classifier can distinguish between tags, but cluster imbalance occurs where majority of data points are in the same cluster leaving other clusters with few or no data points
Solution Approach 1:
The system performs preliminary actions by collecting comprehensive tag sequences from multiple annotators before clustering and voting. This preliminary data collection ensures adequate representation across all tag clusters, preventing imbalance by gathering sufficient examples for rare tags before the classification and voting processes
Data Source
AI summary
Embodiments of the present invention provide an automated image tagging system that can predict a set of tags, along with relevance scores, that can be used for keyword-based image retrieval, image tag proposal, and image tag auto-completion based on user input. Initially, during training, a clustering technique is utilized to reduce cluster imbalance in the data that is input into a convolutional neural network (CNN) for training feature data. In embodiments, the clustering technique can also be utilized to compute data point similarity that can be utilized for tag propagation (to tag untagged images). During testing, a diversity based voting framework is utilized to overcome user tagging biases. In some embodiments, bigram re-weighting can down-weight a keyword that is likely to be part of a bigram based on a predicted tag set.