Private synthetic data generation method for mixed distribution data

Abstract

The invention relates to a privacy synthetic data generation method for mixed distribution data, and belongs to the technical field of computer data privacy and data release. According to the method, distributed marginal measurement and a distribution estimation algorithm based on a graph model are used, and the problem of marginal measurement of mixed distribution data and challenges brought by high-dimensional distribution estimation are solved. The algorithm is fused into a selection-measurement-generation normal form, a synthetic data set of mixed distribution data is generated under differential privacy protection, and efficient issuing and sharing of the data are achieved. According to the method, the privacy release of distributed multi-party data is realized, and the development of data-driven research can be promoted.

Description

Technical Field

[0001] The present invention relates to a privacy synthetic data generation method for mixed distributed data, belonging to the technical field of computer data privacy and data publishing. Background Art

[0002] Amidst the rapid advancement of information technology and the booming era of big data, data-driven research has become a key force. In particular, the rise of large-scale language models, such as Chat-GPT and DeepSeek, has brought unprecedented convenience and revolutionary advancements to numerous disciplines. However, this trend has also been accompanied by increasing challenges in data privacy, data scarcity, and data silos, which have become significant obstacles to the further development of data-driven research. Traditional centralized AI training models face challenges such as user privacy protection, data compliance, and the division of responsibilities between institutions. Therefore, how to promote collaborative data sharing and use among multiple parties while protecting individual privacy and enhancing the value of data learning and usage has become an increasingly prominent issue in the digital age.

[0003] The continuous advancement of privacy protection technology has built a solid line of defense for data security. The concept of differential privacy proposed by Dwork and others provides a solid mathematical foundation for individual privacy protection in the field of data statistical analysis, enabling everyone to systematically explore the delicate balance between the utility of information release and individual privacy protection. This method ensures that the analysis results of adjacent data sets (that is, data sets that differ only in the information of a single participant) are highly consistent, which means that compared with individuals who do not participate in data analysis, participants under the differential privacy mechanism will not face additional privacy leakage risks. Currently, differential privacy technology has been widely used in enterprises and government agencies. For example, Apple has announced the introduction of differential privacy technology in its iOS system to anonymously collect users' health and activity data in order to optimize product functions and performance.

[0004] Synthetic data generation technology combined with differential privacy protection provides effective safeguards for data publishing and sharing. By providing virtual data that mimics the statistical characteristics of the original data, this technology allows for an effective trade-off between data quality and privacy assurance. This type of approach originated from the research of Hardt et al., who applied a differential privacy index mechanism based on a multiplicative weight mechanism to select the optimal query and constructed a select-measure-generate framework to generate high-quality synthetic data for data publishing and sharing. Based on this structure, researchers have proposed a variety of techniques to improve the quality of synthetic data. For example, Aydore proposed RAP, which uses a gradient-based method to generate synthetic data. Ryan et al. proposed distribution estimation based on graphical models to effectively address the computational burden brought by data dimensionality. However, current applications of the select-measure-generate framework mainly focus on direct operations on single datasets.

[0005] Multi-party data publishing is a major technical challenge in the field of privacy computing. It requires analyzing and processing raw datasets stored by multiple parties while maintaining differential privacy, and outputting a new dataset (or data distribution) that can be subsequently used in various downstream tasks such as machine learning and statistical analysis. Currently, research on collaborative multi-party data publishing can be categorized into horizontally partitioned datasets and vertically partitioned datasets, depending on how the raw datasets are constructed. In recent years, there have been some pioneering works on horizontally partitioned data. Alhadidi et al. and Hong et al. focused on differentially private integrated publishing of horizontally partitioned data, but their efforts were limited by the limited number of participants and the generalization capabilities of downstream tasks. Cheng et al. proposed a sequential update method for differentially private Bayesian networks (DP-SUBNs) and overcame these challenges with ideal data utility. Several researchers have also studied vertically partitioned data. Mohammed et al. proposed DistDiffGen, a method for publishing differentially private data for vertically partitioned datasets. Similarly, the limitations on the number of participants and the generalization capabilities of downstream tasks are also reflected in DistDiffGen. Based on this, Tang et al. proposed a differentially private latent tree (DPLT) method to address these limitations and demonstrated its superiority through extensive experiments.

[0006] However, existing work on multi-party data distribution patterns is relatively simplistic and idealistic. Horizontal and vertical data partitioning, respectively, assume that each party possesses all entries or attributes of the complete dataset. This data distribution pattern is rarely encountered in real-world applications. Typically, a participant only holds a subset of the complete dataset and some of its attributes, similar to how data is distributed across multiple parties in a hybrid manner, including horizontal and vertical partitioning. Currently, no research has explored the publication and sharing of hybrid distributed data. This issue has important theoretical and practical significance for ensuring efficient data sharing and collaboration, and urgently requires attention and research. Summary of the Invention

[0007] The purpose of this invention is to develop a collaborative publishing and sharing mechanism for distributed private data from multiple parties. The goal is to find a practical data publishing algorithm that can publish the relevant features of a dataset as a whole while preserving privacy, making it suitable for subsequent research queries. This universal preprocessing method for private datasets can collaboratively construct publicly releasable datasets from multiple original private datasets, making it easier for researchers to use data and promoting the development of artificial intelligence in fields such as healthcare while complying with legal regulations on privacy protection.

[0008] This paper innovatively proposes a method for generating private synthetic data for mixed-distribution data. Its innovations include using distributed marginal measurement and a distribution estimation algorithm based on a graphical model to address the marginal measurement problem of mixed-distribution data and the challenges posed by high-dimensional distribution estimation. By integrating these two algorithms into the select-measure-generate paradigm, synthetic datasets of mixed-distribution data are generated under differential privacy protection, enabling efficient data publishing and sharing.

[0009] The technical solutions adopted by the present invention are as follows.

[0010] A method for generating private synthetic data for mixed distributed data, comprising the following steps:

[0011] Step 1: Initialize workload and parameters.

[0012] First, initialize the workload, define a set of marginals of interest, and construct synthetic data based on the distribution of these marginals.

[0013] Then, the data distribution and parameter variables of each stage are initialized, including privacy budget, number of iterations, and privacy allocation parameters.

[0014] Step 2: Select Measure and Update.

[0015] First, in the selection stage, a discrete sampling method (such as an exponential mechanism) is used to sample the marginal distribution, and the fractional equation is designed to ensure that the marginal distribution obtained by sampling has the maximum approximation error.

[0016] Then, a distributed marginal measurement algorithm is used to aggregate the measurement results of the sampled marginals from the data of each participant. During the aggregation process, the Gaussian mechanism of differential privacy is used to protect the privacy of the measurement results to prevent the leakage of sensitive information.

[0017] Finally, in the update phase, the data distribution is updated using the measurement results of the sampled margins through the graphical model estimation algorithm, and the ideal data distribution is obtained after iteration.

[0018] Step 3: Generate private synthetic data.

[0019] Based on the data distribution obtained above, a randomized sampling method is used to generate new data points from the original data in proportion or according to set rules, forming tabular data that approximately matches the data distribution. By adjusting the sampling parameters, low error and diversity are ensured.

[0020] Beneficial effects

[0021] Compared with the prior art, the present invention has the following advantages:

[0022] 1. This paper studies the real-world scenario of mixed distributed data for the first time and proposes a high-quality differentially private synthetic data generation algorithm for publishing mixed distributed data with privacy requirements.

[0023] 2. This paper utilizes distributed marginal measurements and a distribution estimation algorithm based on a graphical model to address the problem of measuring privacy margins for mixed-distribution data, as well as the challenge of estimating the distribution of high-dimensional data using low-dimensional marginal distributions. This method can generate fake data that maintains the statistical characteristics of mixed-distribution data while satisfying the definition of differential privacy. This paper effectively enables the private publication of distributed multi-party data and promotes the development of data-driven research. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is the overall implementation process of the method of the present invention. DETAILED DESCRIPTION

[0025] The present invention will be further described in detail below with reference to the accompanying drawings.

[0026] like Figure 1Figure 2 illustrates the joint publishing process for mixed-distribution data. Each participant measures the margin of their local data and perturbs the measurement results with a Gaussian perturbation under differential privacy protection. The perturbed measurements from each participant are then aggregated to obtain a privacy-margin measurement, which is then passed to the administrator. Finally, the administrator feeds the privacy-margin measurement into a privacy-sensitive synthetic data generation algorithm, which iteratively updates the data distribution using the select-measure-generate paradigm. The resulting data distribution is then used to generate new synthetic data for data publishing and sharing.

[0027] Specifically, a method for generating privacy-preserving synthetic data for mixed distributed data includes the following steps:

[0028] Step 1: Initialize workload and parameters.

[0029] Workload is the set of marginal distributions that synthetic data is expected to preserve well, and it is also a metric for evaluating the effectiveness of different mechanisms. By initializing the workload and various parameters, we can ensure the stable operation of the proposed method and generate high-quality synthetic data.

[0030] Specifically, step 1 includes the following steps:

[0031] Step 1.1: Initialize the workload.

[0032] We first predefine a set of marginal sets for synthetic data. Margin (also known as marginal query) is a key statistic that encapsulates the low-dimensional structure represented in the high-dimensional data distribution.

[0033] Specifically, the margin r of a set of attributes is essentially a histogram that counts all combinations of values ​​in that set of attributes. The ratio of this count to the overall number is called the marginal distribution. In this step, a three-dimensional margin of all attributes in the data is typically constructed as the workload. Finally, the margins in the workload are removed, and the purified workload is used as the input to the mechanism to generate synthetic data specifically tailored to the marginal distribution of interest.

[0034] It's important to note that a large margin means that the attribute values ​​used to construct it contain a large number of combinations, but the counts of most combinations are 0. This not only reduces computational efficiency but also fails to contribute to the data synthesis process. Therefore, we set the hyperparameter Maxsize to limit the capacity of each margin. When the number of combinations exceeds this parameter, the margin is deleted from the original workload.

[0035] Step 1.2: Initialize parameters.

[0036] First, initialize the parameter variables, including the privacy budget, number of iterations, and privacy allocation parameter. The privacy budget determines how well the synthetic data generation method protects sensitive information in the original data. This parameter should be set based on the actual situation. A larger value can lead to more privacy leakage, while a smaller value can affect the quality of the synthetic data. The number of iterations determines the number of updates to the data distribution. Too few iterations can cause the estimated data distribution to differ significantly from the original data distribution. The privacy allocation parameter determines the amount of privacy budget allocated during the selection and measurement phases.

[0037] Step 1.3: Initialize the data distribution.

[0038] Before generating synthetic data, a uniform distribution is sampled across the entire data domain as the initial distribution, as demonstrated by Liu et al. However, this approach suffers from the significant drawback that the required computational resources grow exponentially with the dimensionality of the private data. In this approach, a graphical model is employed to address this problem, leveraging an unoptimized initial graphical model to estimate the margins in the workload and using these estimates as the initial distribution. This approach significantly reduces the computational resource requirements, achieving significant time and space savings. It not only optimizes the initialization process but also demonstrates a heuristic response.

[0039] Step 2: Select Measure and Update.

[0040] Distributed marginal measurement and graphical model-based distribution estimation algorithms are used to address the marginal measurement problem of mixed partitioned data and the challenges posed by high-dimensional distribution estimation, respectively. These two algorithms are integrated into the selection-measurement-generation paradigm.

[0041] Specifically, step 2 includes the following steps:

[0042] Step 2.1: Distributed Margin Measurement.

[0043] We measure marginal queries on mixed partitioned datasets by decomposing marginal queries.

[0044] Specifically, the mixed partitioned data is considered as a horizontal partition between K participants and a vertical partition between S secondary participants. For the horizontally partitioned data, the noisy measurement results are added together to obtain the measurement result of the entire data.

[0045] For a given marginal query r, the secondary party D i,j {1≤i≤K,1≤j≤S} faces two different scenarios when responding: one is that the data attributes it holds completely contain the attributes of the marginal query r, which means that their intersection is equal to the query attributes; the other is that the intersection does not completely cover all the attributes of the marginal query, resulting in the intersection being less than the attributes of r. j{1≤j≤S}, to solve the two scenarios described above, where r j The attributes of correspond to the intersection of the data attributes held by the current participant and the original marginal query r. The core of the algorithm is to carefully divide the corresponding marginal queries for each local dataset to ensure that they are perfectly aligned with the specific attributes.

[0046] On this basis, the concept of indicator vectors is introduced to accurately express the response of the local dataset to its marginal query and to protect the privacy of these vectors. j , we will get an indicator vector v i,j , whose length is the same as the number of records in the local data. This vector represents the subquery r j For local data D i,j The actual measurement value of is represented by a vector value of 1, indicating that the query condition matches, and a vector value of 0 indicates that the query condition does not match. Furthermore, due to the semi-trusted setting, the indicator vector is perturbed using a Gaussian mechanism under differential privacy. This ensures the security of the query response submitted to the semi-honest administrator.

[0047] Finally, the marginal query r of the vertically partitioned data original is answered by computing the inner product of the S perturbation indicator vectors.

[0048] The whole process is shown in Algorithm 1.

[0049]

[0050]

[0051] Among them, v i,j Represents the local dataset D i,j The indicator vector generated for the marginal query r, represents its corresponding noise version, Represents r j In D i,j On the original marginal measure, represents the distributed marginal measure of D.

[0052] Step 2.2: Distribution estimation based on graphical models.

[0053] A graphical model-based estimation algorithm is used to predict the overall distribution of synthetic data. The algorithm cleverly transforms the distribution update into an optimization problem. With the help of gradient descent strategy, it only uses low-dimensional marginal distribution information to iteratively update the overall distribution of the data, thereby greatly reducing the computational burden brought by high data dimensions. Its core is to extract noise margins from sensitive data in the form of a tuple list around an optimization problem. As input, the goal is to output a distribution p that best reflects the noise margin of the input. This optimization problem is expressed as follows:

[0054]

[0055] in, Represents the optimal data distribution, U={p|p(x)≥0&∑ x∈Ω p(x)=n} is a set of probability distributions over the domain Ω. Indicates that i The marginal measurement of the data distribution p is given above. This estimation algorithm is regarded as a black box, and an interface for updating the data distribution is provided in this mechanism.

[0056] Step 2.3: Select-Measure-Generate Paradigm.

[0057] Distributed marginal measurement and graphical model-based distribution estimation algorithms are integrated into the selection-measurement-generation paradigm, and a differentially private hybrid generation algorithm for generating hybrid partitioned data is developed.

[0058] The whole process is shown in Algorithm 2.

[0059]

[0060]

[0061] Among them, Pr represents probability, represents the distributed marginal measurement; M r represents the original marginal measure.

[0062] Step 3: Private synthetic data generation.

[0063] In this step, based on the obtained data distribution P t , a randomized sampling method is used to generate new data points from the original data in proportion or according to set rules to form tabular data that approximately matches the data distribution, and low error and diversity are ensured by adjusting the sampling parameters.

[0064] Step 3.1: Construct the original data schema.

[0065] First, analyze the distribution of the raw data. This involves calculating statistical indicators such as the frequency, mean, median, and standard deviation of each feature to identify key patterns and trends in the data. Then, design a tabular template to ensure that the generated data accurately reflects these statistical characteristics.

[0066] Step 3.2: Generate synthetic data.

[0067] The randomization method is used to distribute P from the data tThese samples will be selected based on pre-set data distribution rules.

[0068] Furthermore, in order to maintain low error and consistency of data distribution, the sampling ratio and random seed can be adjusted to ensure that the generated data is both diverse and faithfully reflects the original data.

[0069] In practice, this process can be implemented using the random library in Python or the sampling methods in scikit-learn. For example, you can choose to draw samples from each category or feature with a certain probability and generate new data points based on the set statistical parameters. In addition, to ensure the quality of the generated data, you can set some stopping conditions, such as stopping sampling after reaching a predetermined sample size.

Claims

1. A method for generating private synthetic data for mixed distributed data, characterized in that: Distributed marginal measurement and graphical model-based distribution estimation algorithms are used to address the marginal measurement problem of mixed distribution data and the challenges posed by high-dimensional distribution estimation, respectively. These two algorithms are integrated into the selection-measurement-generation paradigm to generate synthetic datasets of mixed distribution data under differential privacy protection. The following steps are included: Step 1: Workload and parameter initialization; First, the workload is initialized; a set of marginal sets of interest are defined, and synthetic data is constructed based on the distribution of these marginals; then, the data distribution and the parameter variables of each stage are initialized, including the privacy budget, the number of iterations, and the privacy allocation parameter; Step 2: Select - Measure - Update; First, in the selection phase, a discrete sampling method is used to sample the marginal distribution, and a fractional equation is designed to ensure that the sampled marginal distribution has the maximum approximation error. Then, a distributed marginal measurement algorithm is used to aggregate the measurement results of the sampled marginals from the data of each participant. During the aggregation process, the Gaussian mechanism of differential privacy is used to protect the privacy of the measurement results. Finally, in the update phase, the measurement results of the sampled marginals are used to update the data distribution through a graphical model estimation algorithm. After iteration, the ideal data distribution is obtained. Step 3: Generate privacy synthetic data; Based on the data distribution obtained above, a random sampling method is used to generate new data points from the original data in proportion or according to set rules to form tabular data that approximately matches the data distribution.

2. The method for generating private synthetic data for mixed distributed data according to claim 1, characterized in that: In step 1, a set of margins is predefined for the synthetic data. The margins encapsulate the low-dimensional structure represented in the high-dimensional data distribution. The marginal r of a set of attributes is essentially a histogram that counts all the combinations of values ​​in the set of attributes. The ratio of this count to the overall number is called the marginal distribution. A 3D marginal of the overall attributes in the data is constructed as a workload. Finally, the marginals in the workload are removed and the purified workload is used as input to the mechanism to generate synthetic data tailored to the marginal distribution of interest. The capacity of each margin is limited by setting the hyperparameter Maxsize. When the number of combinations is greater than this parameter, the margin is deleted from the original workload.

3. The method for generating private synthetic data for mixed distributed data according to claim 1, characterized in that: In step 1, an unoptimized initial graphical model is used to estimate the margins in the workload and these estimates are used as the initial distribution.

4. The method for generating private synthetic data for mixed distributed data according to claim 1, characterized in that: In step 2, distributed marginal measurement is performed to measure marginal queries on the mixed partitioned dataset by decomposing marginal queries; The mixed partitioned data is considered as a horizontal partition between K participants and a vertical partition between S secondary participants among them; for the horizontally partitioned data, the noisy measurement results are added together to obtain the measurement result of the entire data; For a given marginal query r, the secondary party D i,j {1≤i≤K,1≤j≤S} faces two different scenarios when responding: one is that the data attributes it holds completely contain the attributes of the marginal query r, which means that their intersection is equal to the query attributes; the other is that the intersection does not completely cover all the attributes of the marginal query, resulting in the intersection being less than the attributes of r; dividing the marginal query r into S subqueries r j {1≤j≤S}, where r j The attributes of correspond to the intersection of the data attributes held by the current participant and the original marginal query r; Introducing the indicator vector, for each subquery r j , we will get an indicator vector v i,j , whose length is the same as the number of records in the local data; this vector represents the subquery r j For local data D i,j The actual measurement value of , where vector value = 1 indicates that the query condition matches, and vector value = 0 indicates that the query condition does not match. The Gaussian mechanism under differential privacy is used to perturb the indicator vector; Finally, the original marginal query r of the vertically partitioned data is answered by computing the inner product of the S perturbation indicator vectors; Use a graph-based estimation algorithm to predict the overall distribution of synthetic data; An optimization problem to extract noise margins from sensitive data in the form of a list of tuples As input, the goal is to output a distribution p that best reflects the noise margin of the input; this optimization problem is expressed as follows: in, Represents the optimal data distribution, U={p|p(x)≥0&∑ x∈Ω p(x)=n} is the set of probability distributions over the domain Ω; Indicates that i The marginal measure of the data distribution p above; Integrate distributed marginal measurement and distribution estimation algorithms based on graphical models into the selection-measurement-generation paradigm, and adopt a differentially private hybrid generation algorithm for generating mixed partitioned data: Input: mixed partition data D = (D 1,1 ,D 1,2 ,...,D K,S ); workload W; privacy budget ρ; number of iterations T; Output: data distribution p T ; Initialize data distribution p0; r t =ρ / T; for iterationt=1,...,Tdo: Select a marginal query r from W using the differential privacy exponential mechanism t , the related functions are as follows: Measuring Margins with Differential Privacy Use graph-based algorithms to estimate data distribution: end for According to p T Generate synthetic data Among them, Pr represents probability, represents the distributed marginal measurement; M r represents the original marginal measure.

5. The method for generating private synthetic data for mixed distributed data according to claim 1, characterized in that: In step 3, the sampling ratio and random seed are adjusted to ensure that the generated data is both diverse and faithfully reflects the original data.

Images