Label Noising Model for Machine Learning Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing approaches fail to provide satisfactory label differential privacy for training machine-learned models, especially in regression tasks, where maintaining statistical indistinguishability of label data is crucial but often compromised due to noise-induced errors.
Innovation Solution
A method involving a label noising model that bin raw input labels and randomly samples from these bins based on a probability distribution, optimized for regression tasks using a quantization loss function, ensuring differential privacy by generating noised label data that better aligns with the raw data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If label data is noised to achieve differential privacy, then data privacy is improved, but noise-induced errors increase and model performance deteriorates
Solution Approach 1:
The patent segments the label space into multiple bins, where each bin represents a range of label values. Instead of adding noise to individual label values, the system assigns labels to bins and then samples from these bins. This segmentation approach reduces noise-induced errors because the binning process groups similar values together, and the sampling from bins introduces less variability than direct noise addition to individual labels.
Solution Approach 2:
The patent changes the parameter representation from continuous label values to discrete bin indices. By transforming the label space into binned categories and using a probability distribution over these bins, the system achieves differential privacy with reduced noise impact. The probability distribution parameters (first probability for correct bin, second probability for other bins) control the trade-off between privacy and accuracy.
2Reliability
If traditional noise addition is used for label differential privacy, then privacy protection is achieved, but statistical indistinguishability is compromised
Solution Approach 1:
The patent divides the label space into discrete bins, which creates natural groupings that preserve statistical properties. When sampling from these bins according to a probability distribution, the resulting noised labels maintain better statistical indistinguishability compared to traditional noise addition, because the binning structure preserves the underlying data distribution patterns while still providing privacy protection.
Solution Approach 2:
The patent introduces bin indices as an intermediary between the original label data and the noised output. Instead of directly adding noise to labels, the system first maps labels to bins and then samples from these bins. This intermediary binning process acts as a mediator that preserves statistical properties while achieving differential privacy, improving statistical indistinguishability compared to direct noise addition.
Data Source
AI summary
An example method is provided for conducting differentially private communication of training data for training a machine-learned model. Initial label data can be obtained that corresponds to feature data. A plurality of label bins can be determined to respectively provide representative values for initial label values assigned to the plurality of label bins. Noised label data can be generated, based on a probability distribution over the plurality of label bins, to correspond to the initial label data, the probability distribution characterized by, for a respective noised label corresponding to a respective initial label of the initial label data, a first probability for returning a representative value of a label bin to which the respective initial label is assigned, and a second probability for returning another value. The noised label data can be communicated for training the machine-learned model.