Covariate Shift Re-weighting for Fair Transfer Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models often propagate biases present in training data, especially in high-stakes applications where protected attributes are unavailable or unrecorded, leading to fairness issues in risk assessments and other regulated domains.
Innovation Solution
The implementation of transfer learning solutions using covariate shift formulations to correct source data by re-weighting it based on related datasets with accessible protected attributes, addressing scenarios where protected attributes are missing in either the training or test data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine learning models are trained on available data without protected attributes, then model training can proceed, but fairness and bias reduction deteriorate
Solution Approach 1:
The patent introduces protected attributes from a related dataset as an intermediary to mediate between the source training data and target test data. This mediator enables the model to learn fairness-aware representations even when protected attributes are unavailable in the primary dataset, resolving the contradiction between training capability and prediction fairness
Solution Approach 2:
The patent segments the data into source domain (with labels but without protected attributes) and target domain (with protected attributes but without labels), then applies transfer learning to bridge them. This segmentation allows the model to leverage protected attributes from the target domain to correct biases in the source domain training process
2Reliability
If protected attributes are used to correct bias, then fairness improves, but data availability and complexity increase
Solution Approach 1:
The patent performs preliminary action by pre-processing the source training data with covariate shift correction using protected attributes from the target domain before actual model training. This preliminary correction reduces bias in advance, allowing the model to achieve fairness without requiring protected attributes during the main training process
Solution Approach 2:
The patent creates a corrected version of the source data by copying and re-weighting training examples based on covariate shift estimates derived from target domain data with protected attributes. This copying approach allows fairness correction without directly modifying the original data structure or requiring permanent storage of combined datasets
3Reliability
If covariate shift correction is applied to re-weight source data, then fairness improves, but computational complexity increases
Solution Approach 1:
The patent changes the parameter distribution of the source data by applying covariate shift correction through re-weighting examples based on estimated density ratios. This parameter transformation adjusts the importance of different training examples to compensate for distributional differences, achieving fairness through parameter modification rather than complex architectural changes
Data Source
AI summary
A method of utilizing a computing device to correct source data used in machine learning includes receiving, by the computing device, first data. The computing device corrects the source data via an application of a covariate shift to the source data based upon the first data where the covariate shift re-weighs the source data.