Vertical Federated Random Forest With Diversity Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models lack transparency and fail to provide diversity scores while maintaining data privacy in vertical federated learning settings, particularly in hiring processes, leading to potential biases and reduced workforce diversity.
Innovation Solution
Integrate diversity score computation into Random Forest classifiers trained in a vertical federated learning setting, using Gaussian distribution fitting to maintain privacy and provide explainability through local diversity scores for new observations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If machine learning models are used to evaluate job candidates quickly and effectively, then hiring efficiency is improved, but model transparency deteriorates
Solution Approach 1:
The patent segments the machine learning model into multiple decision trees (random forest), where each tree provides a separate explanation path. This allows the system to maintain high hiring efficiency through automated ML evaluation while improving transparency by breaking down the complex model into interpretable segments that show specific candidate characteristics and their impact on decisions.
Solution Approach 2:
The patent introduces diversity scores as an intermediary metric that mediates between the black-box ML model and stakeholders. These scores translate complex model predictions into understandable diversity metrics, enabling transparency without compromising the efficiency of automated hiring decisions.
2Object-affected harmful factors
If data privacy is maintained in vertical federated learning, then information security is improved, but model explainability deteriorates
Solution Approach 1:
The patent applies vertical federated learning where different organizations hold different feature segments (columns) of the data. Each organization trains models on their local data segments without accessing other organizations' data, maintaining privacy while collectively training a comprehensive model. The system aggregates results to provide explainability without requiring data centralization.
Solution Approach 2:
The patent uses diversity scores as intermediaries that enable explainability in the federated learning framework. These scores are computed from the distributed model predictions and provide transparency about candidate evaluation diversity without requiring access to the actual training data, thus maintaining privacy while improving explainability.
3Object-affected harmful factors
If vertical federated learning is used to maintain data privacy, then information security is improved, but model complexity increases
Solution Approach 1:
The patent implements a universal random forest framework that works across multiple organizations in the federated learning setting. The same basic random forest algorithm is applied locally at each organization and then aggregated, providing a unified approach that handles privacy requirements without proportionally increasing complexity. The diversity score computation uses the same universal logic across all organizations.
Data Source
AI summary
One example method includes receiving a random forest classifier model that comprises a group of decision trees, wherein the random forest classifier model is created using a vertical federated framework, providing new observations, not included in a set of training observations, to a trained random forest classifier model, wherein the random forest classifier model is trained in the vertical federated framework, and wherein the training is performed using the set of training observations as input to the random forest classifier model, and generating, by the trained random forest classifier model, one or more diversity scores pertaining to the new observations.