Normalizing Flows for Privacy-Preserving AUC Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for calculating the Area Under the Receiver Operating Characteristic (AUC) curve for binary classification models are hindered by the need to access sensitive user data, which violates data privacy regulations, especially in scenarios where data is stored on client computing devices and cannot be collected or shared with a central server.
Innovation Solution
The proposed solution involves an integration-based AUC computation model, normalizing flows, and the FedAdam model, which allow for AUC estimation without sharing raw data by training normalizing flows models on client devices and using federated learning to update and aggregate model parameters, enabling AUC calculation using probability density functions and cumulative density functions without exposing user data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional AUC calculation methods are used, then accurate performance evaluation is achieved, but sensitive user data must be accessed and shared, violating data privacy regulations
Solution Approach 1:
The patent introduces normalizing flows as an intermediary mechanism that transforms the AUC calculation process. Instead of directly accessing sensitive user data, the system uses trained normalizing flow models that have learned the underlying data distributions. These models serve as mediators that enable AUC computation through probability density function evaluations without requiring raw data exposure, thus resolving the contradiction between measurement accuracy and data privacy protection
Solution Approach 2:
The patent creates copies of the data distribution characteristics through normalizing flow models. Rather than sharing actual user data, the system trains local models on client devices that capture the essential statistical properties of the data. These model copies enable centralized AUC calculation while the original sensitive data remains securely stored on client devices, eliminating privacy violations while maintaining calculation accuracy
2Reliability
If data is stored on client computing devices to protect privacy, then data security is improved, but the ability to collect and share data for AUC calculation is reduced
Solution Approach 1:
The patent extracts the essential information needed for AUC calculation from the sensitive user data through local model training. The normalizing flow models are trained on client devices to learn the data distributions, and only the learned parameters and probability density function evaluations are transmitted to the central server. This extraction process separates the useful analytical information from the sensitive raw data, enabling AUC computation while maintaining data security and preventing information loss
Solution Approach 2:
The patent replaces the mechanical data collection and sharing system with a model-based computational system. Instead of physically collecting and transferring user data to the central server, the system uses distributed local model training and model parameter aggregation. This substitution transforms the data-centric approach into a model-centric approach, where the mechanical act of data transfer is replaced by model training and parameter exchange, thereby maintaining data security while enabling analysis
Data Source
AI summary
The present disclosure describes techniques for determining performance of a classifier. A first machine learning model and a second machine learning model may be trained by aggregating updates to the first machine learning model and the second machine learning model received from a plurality of client computing devices. A cumulative distribution function (CDF) associated with a distribution of the positive samples in the user data may be estimated using the trained first machine learning model. A probability density function (PDF) associated with a distribution of the negative samples in the user data may be estimated using the trained second machine learning model. An integration-based computation of an area under the receiver operating characteristic curve (AUC) of the classifier may be performed using the PDF and the CDF.