Modified Split Training for Latent Bias Reduction in Inference Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Inference models often introduce latent bias due to limited or biased training data, leading to inaccurate predictions that can negatively impact computer-implemented services by perpetuating undesired discrimination.
Innovation Solution
A modified split training method is employed to create a multipath inference model, where one path predicts the bias feature and the other path predicts the target feature, with the shared body portion being frozen to reduce latent bias, resulting in an updated inference model with reduced bias.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional training methods are used with limited training data, then the inference model can be trained quickly, but latent bias is introduced into the model
Solution Approach 1:
The training process is segmented into two distinct phases: (1) initial training of the shared body portion with frozen weights, and (2) subsequent training of path-specific portions with the shared body frozen. This segmentation allows the model to learn general features first, then specialize without introducing bias, resolving the contradiction between training efficiency and bias reduction.
Solution Approach 2:
The shared body portion acts as an intermediary that is trained first and then frozen. This frozen shared body serves as a bias-free foundation that mediates between the limited training data and the final prediction paths, allowing subsequent path-specific training to occur without introducing latent bias into the core feature representations.
2Measurement precision
If the inference model is trained to improve prediction accuracy, then the model may capture more patterns, but latent bias from training data is amplified
Solution Approach 1:
The harmful bias component is extracted and isolated in the path-specific portions of the model, while the shared body portion is kept frozen and bias-free. This extraction allows the model to achieve high prediction accuracy through path-specific learning without allowing bias to propagate through the entire model architecture.
Solution Approach 2:
Different parts of the model have different training characteristics: the shared body portion is trained once and frozen to maintain high quality unbiased representations, while path-specific portions are trained separately to achieve local optimization for specific prediction tasks. This local quality approach allows accuracy improvement in paths without compromising the bias-free nature of the shared foundation.
3Device complexity
If a single-path inference model is used, then the model structure is simple, but the model exhibits latent bias and reduced fairness
Solution Approach 1:
The model is segmented into a shared body portion and multiple path-specific portions. This segmentation maintains relative structural simplicity while enabling fair predictions through the frozen shared body that prevents bias propagation to multiple prediction paths, thereby improving fairness without excessive complexity.
4Reliability
If the shared body portion is frozen during training, then latent bias is reduced, but the training process becomes more complex
Solution Approach 1:
The shared body portion is trained in advance and then frozen before path-specific training begins. This preliminary action establishes a bias-free foundation that simplifies subsequent training by preventing bias propagation, making the overall process more manageable despite the additional training phase.
Data Source
AI summary
Methods, systems, and devices for providing computer-implemented services are disclosed. To provide the computer-implemented services, inference models used by data processing systems may be managed to reduce the likelihood of the inference models provide inferences indicative of bias features. The inference models may be managed using modified split training. The inferences provided by the inference models may be less likely to include latent bias thereby reducing bias in computer-implemented services provided using the inferences.