Image Dataset Clustering for Machine-Learning Bias Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning datasets suffer from bias, leading to skewed outcomes and low accuracy due to overrepresentation of certain classes and underrepresentation of others, particularly in vision-based perception tasks with limited datasets, without a solid metric to effectively visualize and measure bias.
Innovation Solution
A system and method to generate metadata annotations sensitive to nuances, optimize clusters and subclusters, determine bias metrics, and train machine-learning programs using a less biased training set, incorporating techniques like principal component analysis and k-means clustering to reduce dimensionality and improve dataset representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional machine learning datasets are used, then training can proceed with available data, but the model suffers from biased outcomes and low accuracy due to overrepresentation of certain classes
Solution Approach 1:
The patent segments the dataset into multiple clusters based on feature similarity, then further divides clusters into subclusters. This segmentation allows identification of overrepresented and underrepresented classes, enabling targeted data collection to balance the dataset and reduce bias while improving model accuracy.
Solution Approach 2:
The patent applies principal component analysis to reduce dimensionality of image data while preserving essential features. This dimensionality transformation enables efficient processing and bias detection across different data dimensions, allowing the system to identify representation imbalances that would be difficult to detect in the original high-dimensional space.
2Reliability
If data augmentation is applied to mitigate bias, then dataset representation improves, but computational complexity and processing time increase
Solution Approach 1:
The patent performs clustering and bias analysis on the existing dataset before training the model. By identifying underrepresented classes and generating targeted synthetic data in advance through data augmentation, the system avoids the need for complex iterative retraining processes, reducing overall computational complexity while improving dataset representation.
Solution Approach 2:
The system automatically detects bias in the dataset, identifies which classes need augmentation, and generates synthetic data for those specific classes without requiring manual intervention. This self-service approach streamlines the process and reduces computational overhead compared to comprehensive data augmentation of all classes.
3Measurement precision
If comprehensive metadata annotation is performed to capture nuances, then model training quality improves, but annotation time and resource requirements increase
Solution Approach 1:
The patent generates metadata annotations that capture local nuances and features specific to each image and cluster, such as lighting conditions, background elements, and object characteristics. This localized annotation approach focuses computational resources on capturing the most discriminative features for each data point, improving measurement precision while managing annotation time through automated processing.
Data Source
AI summary
A computer includes a processor and a memory storing instructions executable by the processor to receive a dataset of images; extract feature data from the images; optimize a number of clusters into which the images are classified based on the feature data; for each cluster, optimize a number of subclusters into which the images in that cluster are classified; determine a metric indicating a bias of the dataset toward at least one of the clusters or subclusters based on the number of clusters, the numbers of subclusters, distances between the respective clusters, and distances between the respective subclusters; and after determining the metric, train a machine-learning program using a training set constructed from the clusters and the subclusters.