Genome data generation method based on differential privacy

Abstract

The present invention belongs to the field of information security technology and provides a method for generating genomic data based on differential privacy. The present invention generates data by training a generative adversarial network. In each round of iteration, constraints are first determined based on the distribution of real data at imbalanced sites. The conditional vector and latent variables are used as inputs to the generator, and the conditional data is generated and the loss is calculated to update the generator parameters. An adaptive privacy budget allocation strategy is then adopted to adjust the allocated privacy budget based on the gradient size and privacy loss. The model satisfies the differential privacy mechanism by adding noise to the discriminator gradient according to the allocated privacy budget. Finally, the gradient of the total cost function is calculated to determine whether the optimal privacy-utility trade-off point has been reached. If the gradient is greater than 0, the discriminator is updated based on the noisy gradient and a new round of iteration begins. Otherwise, the training is terminated and the generated model and generated data at this point are output.

Description

Technical Field

[0001] The present invention relates to a genome data generation method based on differential privacy, and belongs to the field of information security technology. Background Art

[0002] Single nucleotide polymorphisms (SNPs) are the most common type of variation, with each individual having approximately 100,000 SNP sites. SNP data is widely used in bioinformatics analysis. Since the invention of high-throughput sequencing technology, as the cost of gene sequencing continues to decline, big data analysis of SNP data has attracted people's attention. However, due to the limitations of sequencing costs and the sensitivity of SNP data that hinder data sharing, the application of SNP data often faces the problem of data shortage. Using generative models to generate data with the same distribution as real data is an effective way to address the data shortage problem. Generative Adversarial Networks (GANs) are currently one of the most promising methods for generating high-quality data. Currently, some people have used GANs to generate SNP data on chromosome segments to solve the data shortage problem.

[0003] During GAN training, the discriminator can overfit to the training set data, resulting in the generated data being closer to the training set data than to non-training set data. Therefore, the membership of the training data in GANs is vulnerable, allowing attackers to infer the membership of the target in the GAN training set through membership inference attacks. When using GANs to generate SNP data for medical or disease research, leaking the membership of training set samples can reveal sensitive information such as an individual's medical history. For example, knowing that a person's data is used in AIDS clinical research can lead to the conclusion that the individual has AIDS. Based on this leaked membership, attackers can launch other attacks, such as data analysis and attribute inference, further infringing on the victim's privacy. In addition to privacy concerns, the availability of generated data must also be considered during generation. Some privacy protection measures come at the cost of significantly reduced data availability. Imbalance in allele classes in SNP data can also lead to the generator failing to adequately represent low-frequency minor alleles, potentially causing mode collapse and reducing the availability of generated data.

[0004] There are relatively few existing SNP data generation models, and most lack privacy protection measures to effectively defend against membership inference attacks. Some generative models employ privacy-preserving methods based on differential privacy to combat membership inference attacks, but these methods lack consideration for data availability. Excessive noise and allele class imbalance can lead to low usability of the generated data. Summary of the Invention

[0005] To effectively address the privacy leaks and insufficient data availability during SNP data generation, this paper proposes a differentially private SNP data generation method that is resistant to membership inference attacks. This method first uses conditional generation to ensure that the generated data maintains a distribution similar to the real data while allowing the generator to fully learn the distribution of imbalanced data. It then employs an adaptive privacy budget allocation strategy, dynamically adjusting the budget allocation based on the gradient and privacy loss. Privacy protection for the generated model is achieved by adding noise to the discriminator gradient. Finally, a total cost function is used to select an appropriate total privacy budget to achieve a good privacy-usability trade-off.

[0006] The technical solution of the present invention:

[0007] A method for generating genomic data based on differential privacy, the steps are as follows:

[0008] Define variables:

[0009] Table 1 Commonly used variables and their descriptions

[0010]

[0011] (1) Train the generative model. Use the generator to generate data using conditional generation. Calculate the loss function of the generator used for conditional generation and the loss function of the discriminator used to evaluate the generated data based on the generated data. Update the generator and discriminator based on their respective loss functions. The specific steps for conditional generation of SNPs data are as follows:

[0012] (1.1) The real SNPs data held by the data holder is used as the training data set for the generative model. First, the allele frequency of each SNP site in the training data set is calculated, and the imbalanced SNPs sites whose allele frequency is lower than the threshold are found;

[0013] When generating SNPs site data under certain conditions, a SNPs site is randomly selected from the unbalanced SNPs sites with equal probability, and then the value of the SNPs site is randomly selected according to the allele frequency, and the value of the SNPs site is used as a constraint condition;

[0014] (1.2) The constraints and random Gaussian noise, i.e., latent variables, are used as inputs to the generator, and data is generated in a conditional manner.

[0015] The discriminator selects data from the training set of the generative model that meets the constraints in step (1.1), that is, the corresponding SNPs site values ​​are the same as the constraints, and randomly samples from them. The collected real data and the data generated using the current generator conditions are used as the input of the discriminator; the discriminator uses the wassertein distance in WGAN as the loss function Among them, P r represents the real data distribution, P g Represents the generated data distribution; calculates the loss function L of the discriminator D The gradient of , adds noise according to the differential privacy protection strategy, and updates the discriminator parameters;

[0016] Then calculate the loss function L of the generator G , calculate the gradient of the generator's loss function with respect to its parameters according to the chain rule and backpropagate to update the generator parameters;

[0017] Generator loss function: L G =L D +αL B +βL C

[0018] in is the Bhattacharyya distance, which is used to represent the genetic distance between the generated data and the real data; P r (x ik ) and P g (x ik ) are the frequencies of the kth allele at the i-th SNPs site in the real data and the generated data, respectively, q is the total number of alleles at the i-th site, and n is the total number of SNPs sites; is a conditional penalty term to force the generator to learn to generate according to the constraints; α and β are the weight coefficients corresponding to the Bhattacharyya distance and the conditional penalty term respectively;

[0019] Privacy protection is provided for the generative model. A differential privacy mechanism is implemented by adding noise to the discriminator gradients, and an appropriate total privacy budget is selected to achieve a good privacy-utility trade-off.

[0020] (1.2) Adopt an adaptive privacy budget allocation strategy to adjust the allocated privacy budget according to the gradient size and privacy loss:

[0021] First, before training begins, an initial privacy budget ∈ 0 is selected. During training, the average gradient size of t iterations is calculated, and the reconstruction error is detected to calculate the privacy loss. The privacy budget is adjusted every t iterations based on the gradient size and privacy loss:

[0022]

[0023] Among them, E(||g0(x)||2) is the expectation of the 2-norm of the gradient of the discriminator loss function between two adjustments, E(||g0(x)||2) is the expectation of the 2-norm of the initial gradient, and P loss is the privacy loss under membership inference attack, r is the budget growth rate;

[0024] Privacy loss P loss It is defined based on the difference in the probabilities of samples in the training set and non-training set in the training set, and is calculated by the reconstruction error, which is the minimum distance between the target sample and the reconstructed sample. The formula is defined as follows:

[0025]

[0026] Among them, x t is the training set sample, x v Is a non-training set sample; l t is the reconstruction error of the target training sample, l v is the average reconstruction error of the non-training set;

[0027] By adding variance to the gradient of the discriminator Gaussian noise to satisfy (∈,δ) differential privacy; where p is the sampling ratio, n d is the number of discriminator iterations in one iteration, ∈ is the allocated privacy budget, and δ is the relaxation term of differential privacy;

[0028] (1.3) The loss function L of the discriminator D As the availability loss of generated data U loss， When the availability of generated data meets the requirements, according to the privacy loss P loss and availability loss U loss To calculate the total cost function C: Where a and b are the weights of availability and privacy; the gradient of the total cost C is used to determine whether to terminate the iteration. If the gradient is greater than 0, the iteration continues; otherwise, the training is terminated and the generated model and generated data at this time are output.

[0029] The present invention has the following beneficial effects: Generating genomic data using generative models can alleviate the problem of data shortage. However, directly using generative models can pose privacy and security issues, particularly the threat of membership inference attacks. Inappropriate privacy protection measures and allele class imbalance can seriously reduce the usability of the generated data. Therefore, the present invention proposes a genomic data generation method based on differential privacy.

[0030] The present invention uses conditional GAN ​​to generate SNPs data, improves the allele class imbalance problem of SNPs data, and thus obtains SNPs generated data with higher usability.

[0031] A differential privacy method is proposed to flexibly allocate the privacy budget according to the gradient size and privacy loss during the training process. While protecting the privacy of training data, it can guide data owners to reasonably allocate the privacy budget and improve the availability of generated SNPs data.

[0032] In addition, the present invention uses a total cost function to guide the generative model to find the optimal privacy-utility trade-off point during the training process, thereby obtaining a better privacy-utility trade-off. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a structural diagram of the genomic data differential privacy generation method described in the present invention.

[0034] Figure 2 Flowchart of the process for generating SNPs data for the conditions described in the present invention.

[0035] Figure 3 This is a flowchart of the differential privacy protection of the generative model described in the present invention. DETAILED DESCRIPTION

[0036] In order to express the purpose, technical solutions and advantages of the present invention more clearly, the present invention will be further described in detail below through embodiments and drawings.

[0037] A genomic data generation method based on differential privacy includes how to conditionally generate SNPs data and how to perform differential privacy protection on the generation model.

[0038] Reference Figure 2 The specific operation process of conditional generation of SNPs data is as follows:

[0039] Step 1. 1000 SNPs on chromosome 1 from 2500 individuals in the 1000 Genomes Project Phase 3 were selected as the training set to train the generative model. The allele frequency (F) of each SNP in the training dataset was calculated. A threshold of μ = 20% was set, and SNPs were screened. Sites with allele frequencies below the threshold μ were identified as imbalanced sites.

[0040] Step 2. Initialize the constraint condition cond to a zero vector. Randomly select a site from the imbalanced sites with equal probability, and then randomly select the value of the constraint condition cond at the site according to the allele frequency.

[0041] Step 3. The conditional vector cond and the latent variable (random Gaussian noise) z are used as input to the generator to generate SNPs data. In the generator, batch normalization and ReLU activation function are used, including two hidden layers and two fully connected layers.

[0042] Step 4. The discriminator selects samples that meet the constraints from the training set of the entire generative model and randomly collects m samples from them. The m real data collected and the m data generated using the current generator conditions are used as the input of the discriminator. The generated data conditional distribution P(x g |cond) and the true data conditional distribution P(x r |cond) as the wassertein distance of the discriminator loss function

[0043] Step 5. Calculate the loss function of the generator: L G =L D +αL B +βL C .

[0044] in is the Bhattacharyya distance, which is used to represent the genetic distance between the generated data and the real data; P r (x ik ) and P g (x ik ) are the frequencies of the kth allele at the i-th SNPs site in the real data and the generated data, respectively, q is the total number of alleles at the i-th site, and n is the total number of SNPs sites; is a conditional penalty term to force the generator to learn to generate according to the constraints; α and β are the weight coefficients corresponding to the Bhattacharyya distance and the conditional penalty term respectively;

[0045] The gradients are then calculated and the generator parameters are updated.

[0046] Reference Figure 3 The specific operation process of how to perform differential privacy protection on the generative model is as follows:

[0047] Step 6. Check whether the number of iterations since the last update of the privacy budget is equal to t. If it is equal to t, execute steps 7-9. Otherwise, execute step 10.

[0048] Step 7. Calculate the average magnitude of the 2-norm of the discriminator gradient over t iterations.

[0049] Step 8. Calculate the reconstruction error and use it to find the privacy loss P in the face of membership inference attacks loss , the formula for privacy loss is as follows:

[0050]

[0051] where x t is the training set sample, x vIt is a non-training set sample. t is the reconstruction error of the target training sample, l v is the average reconstruction error of the non-training set.

[0052] Step 9. Allocate the privacy budget for the next t iterations based on the gradient size and privacy loss: where g w (x) is the gradient of the discriminator loss function between two adjustments, g0 is the initial gradient, P loss is the privacy loss under membership inference attack, and r is the budget growth rate.

[0053] Step 10. Calculate the discriminator gradient by adding the variance to the gradient of the discriminator according to the assigned ∈ Gaussian noise to satisfy (∈,δ) differential privacy; where p is the sampling ratio, n d is the number of discriminator iterations in one iteration, ∈ is the allocated privacy budget, and δ is the relaxation term of differential privacy.

[0054] Step 11. Use the combination theorem of differential privacy to calculate the total privacy budget consumed and calculate the current privacy loss P respectively. loss and availability loss U loss , privacy loss P loss and availability loss U loss The calculation formula is as follows:

[0055]

[0056]

[0057] Step 12. Calculate the total cost function C: The gradient of the total cost C is used to determine whether the iteration needs to be terminated. If the gradient is greater than 0, the discriminator parameters are updated according to the noisy gradient, and then the process returns to step 1 to continue the iteration. Otherwise, the training is terminated and the generated model and generated data are output.

[0058] Through the above steps, we obtained a generative model that can generate genomic data with a distribution similar to that of the training dataset, which is the 1000 SNPs data of chromosome 1 of 2500 individuals in the 1000 Genomes Project Phase 3. Comparing the minor allele frequencies (MAF) of the generated data and the real data, the results show that the MAF of the two are similar. Calculating the F between the generated data and the real data ST , the result should be lower than 0.10, indicating that the genetic distance between the generated data and the real data is close. In summary, the generated data maintains the consistency of the distribution and genetic structure with the real data and has high availability. Membership inference attack on the generated model, under the same availability loss U lossUnder this circumstance, the accuracy of membership inference attack should be lower than that of other existing generation methods, proving that the present invention can achieve a better privacy-utility balance.

Claims

1. A method for generating genomic data based on differential privacy, characterized in that: Here are the steps: (1) Train the generative model. Use the generator to generate data using conditional generation. Calculate the loss function of the generator used for conditional generation and the loss function of the discriminator used to evaluate the generated data based on the generated data. Update the generator and discriminator based on their respective loss functions. The specific steps for conditional generation of SNPs data are as follows: (1.1) The real SNPs data held by the data holder is used as the training data set for the generative model. First, the allele frequency of each SNP site in the training data set is calculated, and the imbalanced SNPs sites whose allele frequency is lower than the threshold are found; When generating SNPs site data under certain conditions, a SNPs site is randomly selected from the unbalanced SNPs sites with equal probability, and then the value of the SNPs site is randomly selected according to the allele frequency, and the value of the SNPs site is used as a constraint condition; (1.2) The constraints and random Gaussian noise, i.e., latent variables, are used as inputs to the generator, and data is generated in a conditional manner. The discriminator selects data from the training set of the generative model that meets the constraints in step (1.1), that is, the corresponding SNPs site values ​​are the same as the constraints, and randomly samples from them. The collected real data and the data generated using the current generator conditions are used as the input of the discriminator; the discriminator uses the wassertein distance in WGAN as the loss function Among them, P r represents the real data distribution, P g Represents the generated data distribution; calculates the loss function L of the discriminator D The gradient of , adds noise according to the differential privacy protection strategy, and updates the discriminator parameters; Then calculate the loss function L of the generator G , calculate the gradient of the generator's loss function with respect to its parameters according to the chain rule and backpropagate to update the generator parameters; Generator loss function: L G =L D +αL B +βL C in is the Bhattacharyya distance, which is used to represent the genetic distance between the generated data and the real data; P r (x ik ) and P g (x ik ) are the frequencies of the kth allele at the i-th SNPs site in the real data and the generated data, respectively, q is the total number of alleles at the i-th site, and n is the total number of SNPs sites; is a conditional penalty term to force the generator to learn to generate according to the constraints; α and β are the weight coefficients corresponding to the Bhattacharyya distance and the conditional penalty term respectively; Privacy protection is provided for the generative model. A differential privacy mechanism is implemented by adding noise to the discriminator gradients, and an appropriate total privacy budget is selected to achieve a good privacy-utility trade-off. (1.3) Adopt an adaptive privacy budget allocation strategy to adjust the allocated privacy budget according to the gradient size and privacy loss: First, before training begins, an initial privacy budget ∈ 0 is selected. During training, the average gradient size of t iterations is calculated, and the reconstruction error is detected to calculate the privacy loss. The privacy budget is adjusted every t iterations based on the gradient size and privacy loss: Among them, E(||g w (x)||2) is the expectation of the 2-norm of the gradient of the discriminator loss function between the two adjustments, E(||g0(x)||2) is the expectation of the 2-norm of the initial gradient, and P loss is the privacy loss under membership inference attack, r is the budget growth rate; Privacy loss P loss It is defined based on the difference in the probabilities of samples in the training set and non-training set in the training set, and is calculated by the reconstruction error, which is the minimum distance between the target sample and the reconstructed sample. The formula is defined as follows: Among them, x t is the training set sample, x v Is a non-training set sample; l t is the reconstruction error of the target training sample, l v is the average reconstruction error of the non-training set; By adding variance to the gradient of the discriminator Gaussian noise to satisfy (∈,δ) differential privacy; where p is the sampling ratio, n d is the number of discriminator iterations in one iteration, ∈ is the allocated privacy budget, and δ is the relaxation term of differential privacy; (1.4) The loss function L of the discriminator D As the availability loss of generated data U loss , when the availability of generated data meets the requirements, according to the privacy loss P loss and availability loss U loss To calculate the total cost function C: Where a and b are the weights of availability and privacy; the gradient of the total cost C is used to determine whether to terminate the iteration. If the gradient is greater than 0, the iteration continues; otherwise, the training is terminated and the generated model and generated data at this time are output.

Images