Multi-Tenant Bias Detection for Data-Driven ML Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data-driven models designed for a single tenant often fail to perform effectively when applied to tenants with different characteristics, leading to biased and inaccurate results due to unaccounted-for variations in tenant data features.
Innovation Solution
A bias detection technique that involves aggregating datasets from multiple tenants, training a classification model to distinguish between them, and assessing accuracy rates to determine if the model can be safely deployed or requires adjustments for the new tenant.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a data-driven model is designed for a single tenant, then the model achieves high accuracy for that specific tenant, but the model becomes incompatible and inaccurate when applied to new tenants with different characteristics
Solution Approach 1:
The patent applies preliminary action by performing bias detection before deploying a model to a new tenant. The system proactively identifies potential compatibility issues by comparing the new tenant's data characteristics against the training tenant's characteristics, allowing adjustments to be made before the model goes into production, thus preventing inaccurate predictions from occurring in the first place
Solution Approach 2:
The patent introduces an intermediary mechanism - a bias detection system that acts as a mediator between the trained model and the new tenant's data. This intermediary layer analyzes data characteristics, identifies biases, and determines whether the model is suitable for deployment, serving as a buffer that prevents direct application of potentially incompatible models
2Adaptability or versatility
If manual comparison and analysis of tenant features is performed, then the model can be adjusted for different tenants, but the process becomes time-consuming and complex due to the large number of properties
Solution Approach 1:
The patent replaces the manual mechanical process of comparing tenant features with an automated computational system. The bias detection system automatically analyzes data characteristics, compares tenants, and identifies biases without human intervention, substituting the time-consuming manual analysis with an efficient automated algorithmic approach
Solution Approach 2:
The patent changes the parameter of analysis from examining individual properties one by one to analyzing the multidimensional distribution of feature values as a whole. By transforming the approach from property-level to distribution-level analysis, the system efficiently captures the essential differences between tenants without requiring manual inspection of each property
3Measurement precision
If all possible properties are analyzed to identify relevant features, then complete bias detection is achieved, but the complexity of the analysis increases significantly
Solution Approach 1:
The patent extracts only the essential information needed for bias detection - the multidimensional distribution of feature values - from the complete set of all possible properties. By taking out and focusing on the distributional characteristics rather than analyzing every individual property in detail, the system achieves accurate bias detection while maintaining manageable complexity
Data Source
AI summary
A first dataset associated with input data provided to a data-driven machine-learning model (MLM) that provides a service to a first tenant is obtained. A second dataset is obtained for a second tenant. Each set of input data from the datasets are labeled with an identifier for the first tenant or the second tenant. The first dataset and second dataset are aggregated into a training dataset to train a classification MLM to classify each set of input data as originating from the first tenant or the second tenant. The classification MLM is tested for accuracy. Based on the accuracy of the classification MLM, a determination is provided as to whether the data-driven MLM can be used with the second tenant without adjustments to the data-driven MLM or as to whether adjustments to the data-driven MLM are needed before using the data-driven MLM.