Relaxed Projection for Synthetic Data Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods face challenges in accurately answering a large number of statistical queries while maintaining differential privacy, as simple perturbation techniques like Gaussian or Laplacian noise addition can lead to scaling errors and are computationally difficult to improve, especially for complex query classes like k-way marginals.
Innovation Solution
The generation of synthetic datasets that closely match noisy answers from perturbed queries, using a continuously relaxed projection mechanism to iteratively adjust and optimize the synthetic dataset, allowing for the use of standard optimization tools and maintaining privacy through randomized rounding or normalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If simple perturbation techniques (Gaussian or Laplacian noise addition) are used to answer statistical queries, then differential privacy is maintained, but scaling errors increase and accuracy deteriorates as the number of queries increases
Solution Approach 1:
The patent creates a synthetic copy of the private dataset that preserves statistical properties and correlations. Instead of directly querying the private dataset with noise addition, the synthetic dataset is constructed to mimic the private data's characteristics, allowing accurate queries without direct access to sensitive information. This copying approach maintains privacy while improving query accuracy by avoiding cumulative noise errors.
Solution Approach 2:
The patent transforms the discrete private data into a continuous relaxed space for optimization, then converts back to discrete synthetic data. By changing the parameter space from discrete to continuous during the synthesis process, standard optimization tools can be applied to minimize the difference between query answers on synthetic and private datasets, thereby improving accuracy while maintaining privacy constraints.
2Measurement precision
If synthetic datasets are constructed to improve query accuracy, then measurement precision improves, but device complexity and computational difficulty increase
Solution Approach 1:
The patent replaces complex combinatorial optimization mechanisms with standard continuous optimization tools. By relaxing the discrete data synthesis problem into a continuous space, gradient-based optimization methods can be applied instead of exhaustive search or complex discrete optimization algorithms. This substitution reduces computational complexity while maintaining the ability to generate accurate synthetic datasets.
Solution Approach 2:
The patent divides the synthetic data generation process into distinct phases: initialization in continuous relaxed space, iterative optimization using gradient descent, and final conversion back to discrete data. This segmentation allows each phase to be optimized independently using appropriate methods, reducing overall computational complexity compared to attempting to solve the entire problem in one step.
3Productivity
If the number of statistical queries increases, then the utility of the private dataset improves, but scaling errors increase and accuracy deteriorates
Solution Approach 1:
The patent performs preliminary synthesis of a complete synthetic dataset before any queries are executed. This pre-computed synthetic dataset is then used to answer multiple queries without requiring additional noise addition for each query. The preliminary action of creating the synthetic dataset once allows subsequent queries to be answered accurately without accumulating noise errors, thereby maintaining precision across many queries.
Data Source
AI summary
An algorithm releases answers to very large numbers of statistical queries, e.g., k-way marginals, subject to differential privacy. The algorithm answers queries on a private dataset using simple perturbation, and then attempts to find a synthetic dataset that most closely matches the noisy answers. The algorithm uses a continuous relaxation of the synthetic dataset domain which makes the projection loss differentiable, and allows the use of efficient machine learning optimization techniques and tooling. Rather than answering all queries up front, the algorithm makes judicious use of a privacy budget by iteratively and adaptively finding queries for which relaxed synthetic data has high error, and then repeating the projection. The algorithm is effective across a range of parameters and datasets, especially when a privacy budget is small or a query class is large.