Encrypted eCDF Similarity Testing for Private Federated Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for determining data distribution similarity in federated learning require sharing of data samples, which is not feasible due to privacy and security concerns.
Innovation Solution
A system and method for performing a privacy-preserving distribution similarity test using a server that generates non-sensitive testing values and encrypted eCDF values, allowing clients to conduct a KS test without revealing their data, using public and private keys for encryption and decryption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional KS test is used to determine data distribution similarity, then measurement precision of distribution similarity is improved, but data privacy is compromised due to required data sharing
Solution Approach 1:
The patent introduces encrypted eCDF values as an intermediary representation that allows distribution comparison without exposing raw data. The server computes eCDF values from encrypted data using homomorphic encryption, enabling the KS test to be performed on transformed data that preserves privacy while maintaining measurement precision for distribution similarity assessment
Solution Approach 2:
The patent transforms the data from raw samples to eCDF values, changing the parameter representation. This transformation allows the distribution similarity test to operate on cumulative distribution functions rather than individual data points, maintaining statistical accuracy while preventing direct observation of sensitive data values
2Reliability
If data samples are shared between collaborating parties for distribution testing, then reliability of distribution similarity test is improved, but security is worsened due to exposure of sensitive data
Solution Approach 1:
The patent creates encrypted copies of the data distribution characteristics through eCDF values. Instead of sharing actual data samples, each party generates encrypted representations of their data distribution that can be compared for reliability assessment while maintaining security through cryptographic protection
Solution Approach 2:
The encrypted eCDF values serve as a secure intermediary that enables reliable distribution comparison without direct data exposure. The server acts as a mediator that processes these encrypted representations to determine distribution similarity, ensuring both reliability of the test and security of the underlying data
3Object-affected harmful factors
If encryption is applied to preserve privacy during distribution testing, then data privacy is improved, but device complexity increases due to cryptographic operations
Solution Approach 1:
The server performs multiple functions including key management, homomorphic encryption operations, eCDF computation from encrypted data, and coordination of the KS test. This multi-functional approach consolidates cryptographic complexity into a single server component, reducing the burden on client devices while maintaining strong privacy protection
Data Source
AI summary
Method and servers for performing a privacy-preserving distribution similarity test between first and second data being available to a first and a second client respectively. The method includes generating a target upper boundary value, a target lower boundary value, and a target step size based on first and second boundary values and step-sizes received from the first and second clients receptively, generating non-sensitive testing values, causing generation of first and second Empirical Cumulative Distribution Function (eCDF) values by the first and second client respectively based on the non-sensitive testing values and on the first and second data respectively, acquiring first encrypted eCDF values from the first client, providing encrypted statistic data to the first client, acquiring a result of a privacy-preserving distribution similarity test from the first client, and identifying the first client and the second client as suitable participants for the federated learning task.