Interactive Annotation With Feature-Balanced Sample Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional annotation methods for machine learning, particularly in supervised learning, impose a heavy burden on users due to the need for labeling large datasets, and semi-supervised learning can lead to feature imbalances, resulting in inaccurate label estimation.
Innovation Solution
An annotation method involving a computer-driven process that includes feature extraction, dimensionality reduction, interactive labeling, semi-supervised learning, and supervised learning to balance feature presentation and select reliable data groups, using methods like k-means or DBSCAN for clustering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If semi-supervised learning is used to label data, then user workload is reduced, but feature imbalance occurs in sample data selection
Solution Approach 1:
The system calculates an evaluation value for each data point based on the distribution of sample data in the feature value space, and uses this evaluation value to determine the next sample data to present to the user. This feedback mechanism ensures that sample data with higher evaluation values (indicating under-represented features) are selected, preventing feature imbalance while maintaining annotation efficiency through semi-supervised learning
Solution Approach 2:
The system transforms the sample data selection process from random or uniform selection to selection based on evaluation values calculated in the feature value space. By changing the selection parameter from uniform probability to evaluation-based probability, the system prioritizes data points that contribute to balancing feature representation, thereby resolving the contradiction between efficiency and accuracy
2Measurement precision
If a user labels all data manually, then label accuracy is high, but annotation workload is heavy
Solution Approach 1:
The system applies semi-supervised learning where only a portion of the data (sample data with high evaluation values) is manually labeled by the user, while the remaining data is labeled using the learned model. This partial action approach maintains high label accuracy for critical samples while significantly reducing the time and effort required compared to manual labeling of all data
Solution Approach 2:
The learned model acts as an intermediary between manual labeling and final dataset creation. The model is trained on manually labeled sample data and then used to automatically label the remaining data, serving as a bridge that preserves accuracy while reducing manual workload
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A sample-data presentation step of presenting sample data 9s to a user, a labelling step in which a user labels the sample data 9s, and an evaluation value calculation step of calculating an evaluation value in a feature value space S are repeated. Then, in the sample-data presentation step in second and subsequent series of the steps, data 9 with the highest evaluation value calculated in the clustering step in a previous series is presented as the sample data 9s. In this manner, in the sample-data presentation step in second and subsequent series, the data 9 having a different feature from that of the sample data 9s presented in the sample-data presentation step in the previous series can be presented. This can prevent features of the sample data 9s to be presented to the user, from being imbalanced.