Feature-Space Sample Selection for Balanced Data Annotation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional annotation methods for machine learning, particularly in supervised learning, impose a heavy burden on users due to the need for manual labeling of large datasets, and semi-supervised learning can lead to imbalanced sample data selection, resulting in inaccurate label estimation.
Innovation Solution
An annotation method involving a computerized process that includes feature extraction, dimensionality reduction, interactive labeling, semi-supervised learning, and supervised learning to select and label data groups with high reliability, using methods like k-means or DBSCAN for clustering to balance feature presentation and improve labeling efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If semi-supervised learning is used to label data, then user workload is reduced, but sample data becomes imbalanced causing inaccurate label estimation
Solution Approach 1:
The system calculates an evaluation value for each data point based on the distribution of sample data in the feature value space, and uses this evaluation value to determine the selection order of subsequent sample data. This feedback mechanism ensures that data points with higher evaluation values (indicating better representativeness) are selected first, preventing imbalance in sample data while maintaining efficient labeling through semi-supervised learning
Solution Approach 2:
The system transforms the sample data into a feature value space and calculates evaluation values based on distribution characteristics. By changing the parameter selection criterion from random or uniform sampling to evaluation-value-based sampling, the system achieves both efficient labeling and accurate label estimation
2Measurement precision
If manual annotation is performed for all data, then labeling accuracy is maintained, but user workload becomes heavy
Solution Approach 1:
The system performs feature extraction and creates a feature value space before the labeling process. By pre-calculating evaluation values and determining the selection order of sample data in advance, the system prepares the optimal sampling strategy beforehand, enabling efficient semi-supervised learning while maintaining labeling accuracy
Solution Approach 2:
Instead of requiring manual annotation of all data, the system performs manual labeling only on a strategically selected subset of sample data with high evaluation values. The remaining data is labeled through semi-supervised learning, achieving acceptable accuracy with reduced user workload
Data Source
AI summary
A sample-data presentation step of presenting sample data to a user, a labelling step in which a user labels the sample data, and an evaluation value calculation step of calculating an evaluation value in a feature value space are repeated. Then, in the sample-data presentation step in second and subsequent series of the steps, data with the highest evaluation value calculated in the clustering step in a previous series is presented as the sample data. In this manner, in the sample-data presentation step in second and subsequent series, the data having a different feature from that of the sample data presented in the sample-data presentation step in the previous series can be presented. This can prevent features of the sample data to be presented to the user, from being imbalanced.