Differential privacy synthetic data publishing method based on maximum weight matching

Abstract

The application discloses a differential privacy synthetic data publishing method based on maximum weight matching, which comprises the following steps: firstly, pre-processing user data collected by a reliable third-party server; then, using a graph model method to perform information representation on the processed data attributes, so as to obtain an attribute association graph; then, selecting a suitable low-dimensional marginal set according to a maximum weight matching algorithm; subsequently, adding noise to the low-dimensional marginal set, wherein the noise satisfies a differential privacy definition; then, performing post-processing on the low-dimensional marginal set with noise, so as to obtain a standardized low-dimensional marginal set; performing data synthesis according to the low-dimensional marginal set, so that the synthetic data set is as similar as possible to the original data set in statistical information; and finally, publishing the synthetic data set. By using the technical method, the computational complexity can be reduced while ensuring the utility of the synthetic data set, and the method has better utility for high-dimensional data.

Description

Technical Field

[0001] This invention belongs to the field of information security, specifically a differential privacy synthetic data publishing method based on maximum weight matching. Background Technology

[0002] With the rapid development of information technology, online registration, online shopping, and online travel have gradually become integrated into people's lives. Various organizations (such as hospitals and bus stations) can easily obtain detailed user information, accumulating vast amounts of user data. Statistical analysis of this data can provide effective data support for subsequent tasks such as predictive analysis, bringing immense research value. To meet the needs of research and innovation, relevant organizations release the data they obtain, which often includes individual privacy information. If privacy data is not properly protected, it is easy to leak user privacy data, leading to incalculable losses. Therefore, research on privacy-protected data release methods is essential.

[0003] Methods for protecting data privacy are known as disclosure restrictions. These techniques aim to protect sensitive information while releasing data to the public so that researchers can conduct statistical analysis. A common method for protecting privacy in released data is anonymization, which involves removing sensitive information from the data. However, many studies have shown that simply removing sensitive information is not an effective way to protect privacy. Attackers can still obtain sensitive information by identifying and analyzing other attributes, and therefore cannot provide strong privacy protection for the data release process.

[0004] A reliable privacy protection solution currently available is the differential privacy model. This model is indifferent to the attacker's background knowledge and achieves privacy protection by adding appropriate noise to the query or analysis results, providing reliable privacy guarantees for data publication. The differential privacy model has a clear mathematical definition and valid proof. Simply put, it involves adding or deleting a record from adjacent datasets without affecting the computational results. In other words, after processing data using the differential privacy model, the probability of personal information being identified is very low.

[0005] In recent years, numerous research schemes for privacy-preserving data publishing based on differential privacy models have emerged, such as those employing Haar wavelet transform, histogram methods, and partitioning-based approaches. However, these methods are designed for specific tasks, requiring specialized knowledge from third-party central servers for processing, and posing challenges to fully utilizing the data. Therefore, synthetic data publishing schemes based on differential privacy have been proposed. Synthetic datasets can approximate the original data, replacing it for analysis tasks, thus protecting privacy while maintaining accuracy. However, current synthetic data publishing schemes based on differential privacy methods often suffer from high computational complexity and introduce significant noise into high-dimensional datasets, rendering the synthesized datasets unusable. Summary of the Invention

[0006] This invention provides a method for publishing differential privacy-inspired synthetic data based on maximum weight matching. By applying the maximum weight matching algorithm to a probabilistic graphical model constructed from the original dataset, a low-dimensional distribution is obtained. Appropriate noise is added to the low-dimensional distribution based on differential privacy methods, and the data is then synthesized after post-processing. The synthesized dataset is then published, thereby achieving privacy protection for user information and improving the utility of the synthesized dataset.

[0007] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows.

[0008] A differential privacy synthetic data publishing method based on maximum weight matching is proposed. This method obtains a low-dimensional marginal set by applying the maximum weight matching algorithm to the probabilistic graphical model constructed from the original dataset. After adding appropriate noise to the low-dimensional distribution based on differential privacy methods, the data is synthesized through post-processing and then published as a synthetic dataset. This approach can protect the privacy of user information and improve the utility of the synthetic dataset.

[0009] The method includes the following steps:

[0010] S1. The server aggregates the collected user data to obtain an initial dataset, and then constructs a weighted probabilistic graphical model based on the dataset.

[0011] S2. Based on the generated probabilistic graphical model, the server applies the maximum weight matching algorithm to obtain a set of highly correlated low-dimensional marginal sets.

[0012] S3. Based on the definition of differential privacy model and the reasonable allocation method of privacy budget, add appropriate Gaussian noise to the low-dimensional marginal set;

[0013] S4. Adding noise can cause negative probability data and data inconsistency, so post-processing is required.

[0014] S5. Using the low-dimensional marginal set that has been noise-added and post-processed, a high-dimensional dataset is synthesized to approximate the original dataset in terms of statistical information. Finally, the server publishes the synthesized dataset.

[0015] In step S1, the correlation coefficient can be calculated based on the relationships between attributes, and used as the weight value of the probability graph. Given a dataset D, an attribute graph G(V,E) is generated, where V = {V1,V2,…,V...}. d} represents an attribute node with a total of d attributes. In graph G(V,E), the weight of the edge connecting two nodes is the correlation coefficient of the attributes represented by the two nodes. The correlation coefficient is calculated as follows:

[0016]

[0017] Among them, V i V j Let Pr[V] represent the i-th and j-th attribute nodes respectively. i V j ] represents attribute V i and V j The joint probability, Pr[V i ],Pr[V j ] respectively represent attribute V i and attribute V j The probability of.

[0018] In step S2, applying the maximum weight matching algorithm to the constructed probabilistic graphical model includes the following process:

[0019] S21. Initialize the low-dimensional marginal set M as an empty set;

[0020] S22. Select the edge with the largest weight value in the probabilistic graphical model G, indicating that the two attribute nodes it connects have a high correlation. Use this attribute node pair as the selected low-dimensional edge m. i Add to set M, and remove the nodes with these two attributes and all edges connecting them from graph G;

[0021] S23. Repeat step S22 until there are no attribute nodes in the attribute graph G, then the low-dimensional margin with the maximum weight can be obtained.

[0022] In step S3, allocating the privacy budget and adding noise to the low-dimensional margins includes the following process:

[0023] S31. Allocate privacy budgets by distributing them according to the expected squared difference of the noise scale to obtain different privacy proportions. Appropriate noise is added to different low-dimensional distributions. This optimization problem can be expressed as:

[0024]

[0025] stq1+q2+…+q k =1 and 0≤q i i = 1, 2, ..., k

[0026] in, Let {q1, q2, ..., q} represent the size of the domain corresponding to the k low-dimensional boundaries. k} indicates the privacy budget percentage for the corresponding view.

[0027] S32, (ε,δ)-Differential Privacy Definition: Given a random mechanism A, S A Let A be the set of all possible outputs of A. Given two adjacent datasets D and D', for ε > 0 and δ ≥ 0, if the following inequality is satisfied, then the random mechanism A is said to satisfy (ε,δ)-differential privacy.

[0028] Pr[A(D)∈S A ]≤e ε Pr[A(D')∈S A ]+δ

[0029] S33. Definition of Zero-Set Differential Privacy (ρ-zCDP). Suppose there is a random mechanism A. Given two adjacent datasets D and D', if for all α∈(1,∞), the following inequality is satisfied, then the random mechanism A is said to be zero-set differential privacy (ρ-zCDP).

[0030]

[0031] Among them, D α (A(D)||A(D') is the α-Rényi divergence between the two distributions A(D) and A(D'), representing the privacy loss random variable.

[0032] S34. Theorem: If a random mechanism A satisfies ρ-zCDP, then for any δ>0, the random mechanism A satisfies... - Differential privacy.

[0033] S35. Definition of Gaussian Mechanism: Given f:X n →R is a sensitive query. For the input dataset D, the Gaussian mechanism A satisfies the following equation:

[0034] A(D)=f(D)+N(0,σ 2 )

[0035] Where σ is the noise scale, which can be obtained according to the definition. Δ f This indicates the sensitivity of the sensitive query f.

[0036] S36. Based on the above definitions and theorems, the calculation method for the added noise scale can be obtained as follows:

[0037]

[0038] for i = 1, ..., k

[0039] In step S4, since adding noise will cause the statistical values ​​to become decimals and may even result in negative numbers, it is necessary to post-process the noisy low-dimensional distribution. This process includes the following steps:

[0040] S41. Non-negativity processing: The count of each unit in the noise distribution must be non-negative. Correcting negative count units can improve efficiency.

[0041] To maintain the noise scale, negative counts are collected and denoted as negative_sum. All cells with negative counts are set to 0. Positive count cells are arranged in ascending order, and the total negative count nagetive_sum is consumed starting from the smallest positive count until nagetive_sum is 0.

[0042] S42. Normalization: Since the count values ​​after adding noise are not integers and the total count may vary, resulting in the total count not being equal to the number of records, normalization is required to improve accuracy.

[0043] Dividing the current count by the total count gives its proportion in the total count, resulting in a normalized value, where the sum of the proportions equals 1. Multiplying this proportion by the number of records in the original data gives the final count. Since decimals may occur, the integer and decimal parts of the count can be separated. The decimal parts are summed, and the resulting integer value is added to the largest integer part.

[0044] In step S5, the process of synthesizing the dataset based on the low-dimensional noise distribution includes the following steps:

[0045] S51. Initialize a synthetic dataset D syn Synthesize based on the target distribution (i.e., the noise distribution obtained from the above steps);

[0046] S52. For cells whose initial count value is less than the target distribution count value, add min{c t -c s ,αc s} records, where c t c represents the target count value. s This represents the initial count value, and α represents the decay factor, calculated as follows: α0 represents the initial value, k represents the decay rate, t represents the number of iterations, and s represents the step size.

[0047] S53. For cells whose initial count value is greater than the target distribution count value, reduce min{c}. s -c t ,βc s} records, similarly, c t c represents the target count value. s This represents the initial count value, and β is calculated as follows:

[0048] Beneficial effects: Compared with the prior art, the substantial progress and significant feature of this invention is that by applying the maximum weight matching algorithm to obtain a highly correlated low-dimensional marginal set, it can maximize the global correlation score, reduce the computational complexity while ensuring the utility of the synthetic dataset, and has better utility for high-dimensional data. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating an example of the present invention. Detailed Implementation

[0050] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0051] This invention provides a differential privacy data publishing method based on maximum weighting, see reference. Figure 1 The specific implementation steps are described below.

[0052] In step S1, the server aggregates the collected data to obtain an initial dataset, and constructs a weighted probabilistic graphical model based on the dataset. Given a dataset D, it generates an attribute graph G(V,E), where V = {V1,V2,…,V...} d} represents an attribute node with a total of d attributes. In graph G(V,E), the weight of the edge connecting two nodes is the correlation coefficient of the attributes represented by the two nodes. The correlation coefficient is calculated as follows:

[0053]

[0054] Among them, V i V j Let Pr[V] represent the i-th and j-th attribute nodes respectively. i V j ] represents attribute V i and V j The joint probability, Pr[V i ],Pr[V j ] respectively represent attribute V i and attribute V j The probability of.

[0055] In step S2, the server applies the maximum weight matching algorithm based on the generated probabilistic graphical model to obtain highly correlated low-dimensional margins, including the following process:

[0056] S21. Initialize the low-dimensional marginal set M as an empty set;

[0057] S22. Select the edge with the largest weight value in the probabilistic graphical model G, indicating that the two attribute nodes it connects have a high correlation. Use this attribute node pair as the selected low-dimensional edge m. i Add to set M, and remove the nodes with these two attributes and all edges connecting them from graph G;

[0058] S23. Repeat step S22 until there are no attribute nodes in the attribute graph G, then the low-dimensional margin with the maximum weight can be obtained.

[0059] In step S3, based on the differential privacy model and a reasonable allocation of the privacy budget, appropriate Gaussian noise is added to the low-dimensional margin, including the following process:

[0060] S31. Allocate privacy budgets by distributing them according to the expected squared difference of the noise scale to obtain different privacy proportions. Appropriate noise is added to different low-dimensional distributions. This optimization problem can be expressed as:

[0061]

[0062] stq1+q2+…+q k =1 and 0≤q i i = 1, 2, ..., k

[0063] in, Let {q1, q2, ..., q} represent the size of the domain corresponding to the k low-dimensional boundaries. k} indicates the privacy budget percentage for the corresponding view.

[0064] S32, (ε,δ)-Differential Privacy Definition: Given a random mechanism A, S A Let A be the set of all possible outputs of A. Given two adjacent datasets D and D', for ε > 0 and δ ≥ 0, if the following inequality is satisfied, then the random mechanism A is said to satisfy (ε,δ)-differential privacy.

[0065] Pr[A(D)∈S A ]≤e ε Pr[A(D')∈S A ]+δ

[0066] S33. Definition of Zero-Set Differential Privacy (ρ-zCDP). Suppose there is a random mechanism A. Given two adjacent datasets D and D', if for all α∈(1,∞), the following inequality is satisfied, then the random mechanism A is said to be zero-set differential privacy (ρ-zCDP).

[0067]

[0068] Among them, D α (A(D)||A(D') is the α-Rényi divergence between the two distributions A(D) and A(D'), representing the privacy loss random variable.

[0069] S34. Theorem: If a random mechanism A satisfies ρ-zCDP, then for any δ>0, the random mechanism A satisfies... - Differential privacy.

[0070] S35. Definition of Gaussian Mechanism: Given f:X n →R is a sensitive query. For the input dataset D, the Gaussian mechanism A satisfies the following equation:

[0071] A(D)=f(D)+N(0,σ 2 )

[0072] Where σ is the noise scale, which can be obtained according to the definition. Δ f This indicates the sensitivity of the sensitive query f.

[0073] S36. Based on the above definitions and theorems, the calculation method for the added noise scale can be obtained as follows:

[0074]

[0075] In step S4, the noisy low-dimensional distribution is post-processed, specifically including the following steps:

[0076] S41. Non-negativity processing: The count of each unit in the noise distribution must be non-negative. Correcting negative count units can improve efficiency.

[0077] To maintain the noise scale, negative counts are collected and denoted as negative_sum. All cells with negative counts are set to 0. Positive count cells are arranged in ascending order, and the total negative count nagetive_sum is consumed starting from the smallest positive count until nagetive_sum is 0.

[0078] S42. Normalization: Since the count values ​​after adding noise are not integers and the total count may vary, resulting in the total count not being equal to the number of records, normalization is required to improve accuracy.

[0079] Dividing the current count by the total count gives its proportion in the total count, resulting in a normalized value, where the sum of the proportions equals 1. Multiplying this proportion by the number of records in the original data gives the final count. Since decimals may occur, the integer and decimal parts of the count can be separated. The decimal parts are summed, and the resulting integer value is added to the largest integer part.

[0080] In step S5, high-dimensional data is approximated using a privacy-preserving low-dimensional distribution. The server then publishes the resulting synthetic dataset, including the following process:

[0081] S51. Initialize a synthetic dataset D syn Synthesize based on the target distribution (i.e., the noise distribution obtained from the above steps);

[0082] S52. For cells whose initial count value is less than the target distribution count value, add min{c t -c s ,αc s} records, where c t c represents the target count value. s This represents the initial count value, and α represents the decay factor, calculated as follows: α0 represents the initial value, k represents the decay rate, t represents the number of iterations, and s represents the step size.

[0083] S53. For cells whose initial count value is greater than the target distribution count value, reduce min{c}. s -c t ,βc s} records, similarly, c t c represents the target count value. s This represents the initial count value, and β is calculated as follows:

[0084] Example

[0085] An experiment was conducted using the method provided in this invention. The dataset used in the experiment was Adult, a dataset from UCI, which recorded the personal information of 45,222 users, including age, education level, salary, etc. This invention obtains a highly correlated low-dimensional distribution based on the information in the original dataset, and then uses the low-dimensional distribution to approximate and synthesize a high-dimensional dataset. To measure the privacy protection effect, different privacy budgets were set for calculation in the experiment. The privacy budgets were set to 0.4, 0.8, 1.2, 1.6, and 2.0. In the experiment, the server used a dataset synthesis algorithm based on the collected dataset to obtain the synthesized dataset for publication.

[0086] The server evaluates the synthetic dataset based on the SVM classification results and the error of the k-way margins between the original and synthetic datasets. The experimental results of the differential privacy synthetic data publishing method based on maximum weight matching on this dataset are shown in Tables 1 and 2. To ensure the SVM classification results and avoid the influence of randomness on the experimental results, the synthetic dataset is subjected to five-fold cross-validation. Accuracy (ACC) is used as the evaluation criterion for the experiment. For the experiment of statistically analyzing the error of the k-way margins between the original and synthetic datasets, different k values ​​are used for validation. In the experiment, k = 2, 3, and 4 are selected. Based on k = 2, 3, and 4, 400 different margins are randomly selected, their L1 errors are calculated, and the average value is calculated to obtain the error results of the k-way margins between the synthetic and original datasets under different privacy budgets.

[0087] Table 1. Experimental results of SVM classification under different privacy budgets.

[0088]

[0089]

[0090] Table 2. Error results of k-way margins under different privacy budgets.

[0091]

[0092] As shown in Table 1, the accuracy does not change significantly with the privacy budget, but it generally increases with the increase of the privacy budget, and the accuracy can reach above 0.7, ensuring a good classification result. Furthermore, although the results of the five experiments differ slightly, the fluctuations are very small. Table 2 summarizes the error results of the k-way margins of the synthetic dataset and the original dataset under different privacy budgets. The results show that the margin error of 2-way is within 0.1, which is very small compared to the 2-way margin selected in the original dataset. The margin errors of 3-way and 4-way are also within 0.25, indicating relatively high accuracy, and the overall error decreases with the increase of the privacy budget.

Claims

1. A differential privacy-preserving synthetic data publishing method based on maximum weight matching, characterized in that, The method obtains a low-dimensional marginal set by applying the maximum weight matching algorithm to the probabilistic graphical model constructed from the original dataset. After adding appropriate noise to the low-dimensional marginal set based on the differential privacy method, the data is synthesized through post-processing and then the synthesized dataset is published. This achieves privacy protection for user information and improves the utility of the synthesized dataset. The method includes the following processing steps: S1. The server aggregates the collected user data to obtain an initial dataset, and then constructs a weighted probabilistic graphical model based on the dataset. S2. Based on the generated probabilistic graphical model, the server applies the maximum weight matching algorithm to obtain a set of highly correlated low-dimensional marginal sets. S3. Based on the definition of differential privacy model and the reasonable allocation method of privacy budget, add appropriate Gaussian noise to the low-dimensional marginal set; S31. Allocate privacy budgets. Distribute privacy proportions based on the expected squared difference of the noise scale. Add appropriate noise to different low-dimensional marginal sets. The optimization problem is expressed as: in, express The size of the domain corresponding to a low-dimensional marginal set This indicates the percentage of the privacy budget allocated to the corresponding view; S32. Differential Privacy Definition: A random mechanism is provided. , for The set of all possible outputs in two adjacent datasets and Above, for If the following inequalities are satisfied, then it is called a random mechanism. It is to satisfy - Differential privacy; S33, Zero-Centralized Differential Privacy Definition: Let there be a random mechanism Given two adjacent datasets and Above, for all If the following inequalities are satisfied, then it is called a random mechanism. It is zero-centralized differential privacy The following expression exists: in, yes and Between the two distributions - Rényi Divergence represents a random variable that results in privacy loss; S34, Theorem: If the random mechanism Satisfying zero-centralized differential privacy Then for any >0, random mechanism A It is to satisfy - Differential privacy; S35. Definition of Gaussian Mechanism: Given It is a sensitive query for the input dataset. Gaussian mechanism The following equations must be satisfied: in, It is a noise scale, which can be obtained according to the definition. , Indicates a sensitive query Sensitivity; S36. Based on the above definitions and theorems, the calculation method for the added noise scale is as follows: S4. Handle issues such as negative probability data and data inconsistency that occur after adding noise; S5. Using the low-dimensional marginal set that has been noise-added and post-processed, a high-dimensional dataset is synthesized to approximate the original dataset in terms of statistical information. Finally, the synthesized dataset is published by the server.

2. The differential privacy synthetic data publishing method based on maximum weight matching according to claim 1, characterized in that, The data aggregation in step S1 includes calculating the correlation coefficient based on the association between attributes. The calculation process is as follows: Given dataset Generate property graph , Represents attribute nodes, with a total of Attributes, attribute diagram The weight of an edge connecting two nodes is the correlation coefficient of the attributes represented by the two nodes. The expression for calculating the correlation coefficient is as follows: in, They represent the first and the Each attribute node Representing attributes and The joint probability, Representing attributes respectively and attributes The probability of.

3. The differential privacy synthetic data publishing method based on maximum weight matching according to claim 1, characterized in that, Step S2, solving for the low-dimensional marginal set, includes the following process: S21. Initialize and select a low-dimensional marginal set. M It is an empty set; S22. Select Attribute Map The edge with the largest weight value indicates that the two attribute nodes it connects have a high correlation. This attribute node pair is used as the selected low-dimensional edge set. Add to In the collection, and delete the attribute graph. These two attribute nodes and all the edges connected to them; S23. Repeat step S22 until the attribute graph is obtained. If there are no attribute nodes, then the low-dimensional marginal set with the maximum weight is obtained.

4. The differential privacy synthetic data publishing method based on maximum weight matching according to claim 1, characterized in that, Step S5, which synthesizes the dataset based on the low-dimensional noise distribution, includes: S51. Initialize a synthetic dataset The synthesis is performed based on the target distribution, i.e., synthesized from the noise distribution; S52. For cells whose initial count value is less than the target distribution count value, add... This record contains... Indicates the target count value. Indicates the initial count value. This represents the attenuation factor, calculated as follows: , Indicates the initial value. k Indicates the attenuation rate. t Indicates the number of iterations. s Indicates the step size; S53. For cells whose initial count value is greater than the target distribution count value, reduce... The same record. c t Indicates the target count value. c s Indicates the initial count value. The calculation method is as follows .

Images