GAN Dataset Generation for Algorithmic Fairness Evaluation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for evaluating fairness in Machine Learning (ML) models lack consistent and systematic evaluation practices, rely on small and unrealistic datasets, and fail to account for diverse bias patterns, leading to inconsistent and non-reproducible results.
Innovation Solution
A method and system for generating anonymized datasets with predetermined biases using Generative Adversarial Networks (GANs) to create realistic and diverse datasets for evaluating algorithmic fairness, allowing for fine-grained control over bias types and scenarios, thereby enhancing the robustness of fairness evaluations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If small existing datasets (e.g., COMPAS, UCI Adult) are used for fairness evaluation, then the evaluation can be performed with available data, but the datasets are unrealistic and lead to inconsistent and non-reproducible results
Solution Approach 1:
The patent uses Generative Adversarial Networks (GANs) to create synthetic copies of real-world datasets that preserve the statistical properties and bias patterns of original data while removing sensitive information. This allows reproduction of realistic fairness evaluation scenarios without relying on small, inconsistent existing datasets
Solution Approach 2:
The patent systematically varies parameters such as bias strength, group size ratios, and feature distributions in generated datasets to create multiple versions with controlled characteristics. This enables consistent reproduction of different fairness challenge scenarios while maintaining realistic data properties
2Adaptability or versatility
If diverse bias patterns are introduced to evaluate model robustness, then the evaluation becomes more comprehensive, but the complexity of dataset generation and evaluation increases
Solution Approach 1:
The patent segments bias introduction into distinct, modular components: group size disparity, prevalence disparity, and distinct conditional class separability. Each bias type can be independently controlled and evaluated, making the complex evaluation process more manageable and systematic
Solution Approach 2:
The GAN-based dataset generation framework serves multiple functions: it can generate datasets with different bias patterns, remove sensitive information for privacy protection, and maintain realistic statistical properties. This multi-functional approach reduces overall system complexity despite the comprehensive evaluation capabilities
3Reliability
If GANs are used to generate anonymized datasets, then privacy is protected and large realistic datasets are created, but the generation process is computationally intensive and time-consuming
Solution Approach 1:
The patent performs preliminary actions by pre-training GANs on real-world data to learn underlying distributions before actual fairness evaluation. Once trained, the GANs can rapidly generate synthetic datasets with controlled bias patterns, reducing generation time for subsequent evaluations
Solution Approach 2:
The patent optimizes GAN generation parameters such as batch size, number of epochs, and randomness seeds to balance privacy protection with generation speed. By carefully tuning these parameters, the system achieves both strong anonymization and reasonable generation times
Data Source
Figure 1~2
Figure 3~4
Figure 5~6A
AI summary
Computer system and method for obtaining a generated dataset with a predetermined bias, from a real-world tabular dataset, for evaluating algorithmic fairness of a machine learning model, comprising training a Generative Adversarial Network, GAN, model on the real-world dataset for generating a dataset as a anonymized reconstructed dataset of the real-world dataset, and sampling of the trained GAN model to obtain the generated dataset. Introducing the predetermined bias in the generated dataset may comprise appending one or more columns generated to introduce bias into the dataset; selecting one or more columns and sampling the generated dataset according to a predetermined distribution of the selected column or columns for introducing bias into the dataset; or by adapting a value function of the GAN for introducing the bias in the training of the GAN model on the real-world dataset.