Black-Box Neural Network Privacy Auditing with Score Binning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks lack effective methods for auditing privacy preservation, particularly in scenarios where the training data is not shared due to confidentiality concerns, and there is a risk of compromised privacy levels without explicit knowledge of the underlying data distribution.
Innovation Solution
A method and apparatus for auditing neural network privacy by randomly selecting test and training data, calculating a score function difference, binning the results, and estimating privacy levels using total variation distance, allowing for blackbox analysis without requiring detailed network knowledge.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If differential privacy is implemented by adding random noise to training data, then privacy protection is improved, but the accuracy of the neural network model deteriorates
Solution Approach 1:
The patent implements a feedback mechanism where the privacy auditor evaluates the neural network's output on test data and compares it against the training data distribution. This feedback loop allows for detecting privacy violations and adjusting the noise addition strategy to balance privacy protection with model accuracy.
Solution Approach 2:
The patent changes the parameter ε (epsilon) of differential privacy dynamically. The privacy auditor adjusts the noise addition intensity based on the evaluated privacy level, allowing the system to optimize between privacy protection and model accuracy by modifying the differential privacy parameter.
2Object-affected harmful factors
If the training data is kept confidential and not shared, then data security is improved, but the ability to audit privacy preservation deteriorates
Solution Approach 1:
The patent introduces a privacy auditor as an intermediary that does not require access to the actual training data. Instead, the auditor uses the neural network's output on test data and compares it with the training data distribution to infer privacy preservation levels, enabling auditing without data sharing.
Solution Approach 2:
The patent creates a copy of the training data distribution by using test data that represents the same underlying distribution. This copy allows the privacy auditor to evaluate privacy preservation without requiring access to the actual sensitive training data, maintaining data security while enabling auditing.
3Device complexity
If the neural network operates as a black box without detailed knowledge of its internals, then system complexity is reduced, but the ability to verify privacy compliance deteriorates
Solution Approach 1:
The patent extracts the essential information needed for privacy verification from the neural network's output behavior rather than requiring internal knowledge. By focusing on the input-output relationship and comparing test data predictions against training data distribution, the system verifies privacy compliance without dissecting the complex model internals.
Data Source
AI summary
Selecting, at random, test data from a test dataset and training data from a training dataset to obtain an input dataset;inputting the input data into a neural network;determining, for each of the input data, a value of a score function, wherein the score function is an arbitrary loss function;sorting the values of the score function into plural bins;for each of the bins:calculating a proportion of the training data among the input data for which the values of the score function fall in the respective bin;calculating a proportion of the test data among the input data for which the values of the score function fall in the respective;subtracting the proportion of the test data from the proportion of the training data to obtain a distance for the respective bin;calculating a total distance by summing the distances for the bins being larger than 0.