Data differential privacy handling optimization method for domain deep learning

Abstract

The invention relates to the technical field of data differential privacy handling, and discloses a data differential privacy handling optimization method for domain deep learning, and the method comprises the steps: constructing a domain privacy data set into a deep learning loss function; initializing a privacy budget for generating field privacy data; calculating the contribution gradient of the domain privacy data in the deep learning loss function training process, and carrying out dynamic optimization adjustment on the privacy budget of the domain privacy data based on the contribution gradient; and performing privacy protection on the field privacy data by using the privacy budget after the dynamic optimization adjustment. According to the method, contribution gradients of privacy data in different fields are calculated on the basis of gradients of deep learning model parameters in a training process, then dynamic optimization adjustment is performed on privacy budget on the basis of the contribution gradients and differential privacy change rates of the field privacy data, and under the condition of ensuring a model training effect, the privacy budget is optimized and adjusted. And obtaining a privacy budget for performing privacy protection on the field privacy data, and realizing privacy protection of the field privacy data.

Description

Technical Field

[0001] The present invention relates to the technical field of data differential privacy processing, and in particular to an optimization method for data differential privacy processing in domain deep learning. Background Art

[0002] With the rapid development of deep learning technology, artificial intelligence (AI) has achieved remarkable application results in many fields, especially in the fields of healthcare, finance, retail, social networks, etc. A large amount of sensitive data is used to train deep learning models to improve the prediction accuracy and generalization ability of the models. However, with the popularization of artificial intelligence, data privacy issues have become more serious. Especially in application scenarios involving personal privacy information, the risk of data leakage and privacy protection have become one of the main bottlenecks of technology application. In order to protect the privacy of users, differential privacy has been proposed and widely used as a powerful data privacy protection technology. Differential privacy adds noise to the data set to minimize the impact of individual data on the final result, thereby avoiding the problem of data leakage. Specifically, differential privacy ensures that it is impossible to distinguish whether a single data individual participated in the training process in the output result, avoiding the possibility of inferring sensitive information through the model. The application of differential privacy not only effectively protects personal data, but also enables multi-party data to be jointly learned and shared under the premise of ensuring privacy, thereby promoting deep learning applications in multiple fields. However, the effect of differential privacy often depends on the privacy budget, that is, how much privacy protection resources are allocated during the entire training process. Excessive use of the privacy budget will reduce the effectiveness of the model, while insufficient budget may lead to privacy leakage. Therefore, how to reasonably allocate the privacy budget to ensure that the privacy protection needs at different stages are met without affecting the training efficiency and accuracy of the model has become an urgent problem to be solved. Summary of the invention

[0003] In view of this, the present invention provides a data differential privacy processing optimization method for domain deep learning, which optimizes the configuration of the privacy budget based on the training effect of the deep learning model parameters to achieve privacy data protection.

[0004] To achieve the above objectives, the present invention provides a method for optimizing data differential privacy processing for deep learning in the field, comprising the following steps: S1: Obtain a domain privacy dataset, and construct the domain privacy dataset as a deep learning loss function, wherein the domain privacy dataset consists of domain privacy data and predicted classification results of the domain privacy data.

[0005] S2: Initialize the privacy budget for generating domain private data.

[0006] S3: Calculate the contribution gradient of domain privacy data during the deep learning loss function training process, and dynamically optimize and adjust the privacy budget of domain privacy data based on the contribution gradient.

[0007] S4: Use the privacy budget adjusted by dynamic optimization to protect the privacy of domain privacy data, train the deep learning loss function based on the privacy-protected domain privacy data until the deep learning loss function converges, obtain the optimal deep learning model parameters after domain privacy protection, and build a deep learning model based on the optimal deep learning model parameters.

[0008] As a further improvement method of the present invention: Optionally, the personal sensitive domains include medical field, financial field, consumer field and social communication field, and the domain privacy data set includes N groups of domain privacy data, where N represents the number of domain privacy data in the domain privacy data set, and the representation form of the nth group of domain privacy data is: .

[0009] in: Represents the nth group of domain privacy data; Sequentially indicate Privacy data in the medical, financial, consumer and social communication fields; Privacy data in the field The predicted classification result is .

[0010] Optionally, based on the domain privacy dataset, the deep learning loss function expression constructed is: : .

[0011] in: represents the deep learning model parameters, T represents transposition, represents the L2 norm; Represents deep learning model parameters The deep learning model constructed, the output domain privacy data The predicted classification results.

[0012] Optionally, initialize and generate the deep learning model parameters , the deep learning loss function including domain privacy data and excluding domain privacy data is calculated as the initial contribution deviation of domain privacy data. The initial contribution deviation of the nth group of domain privacy data is: .

[0013] in: represents the initial contribution deviation of the nth group of domain privacy data, represents the L1 norm, represents the deep learning loss function that does not include the nth group of domain privacy data.

[0014] Based on the initial contribution deviation of the domain privacy data, the privacy budget of the domain privacy data is initialized and generated. The privacy budget of the nth group of domain privacy data generated by the initialization is: .

[0015] in: represents the privacy budget of the nth group of domain privacy data generated by initialization, Indicates the total amount of the preset privacy budget.

[0016] Optionally, the deep learning model parameters are trained iteratively during the deep learning loss function training process, the maximum training iteration of the deep learning model parameters is Max, and based on the change value of the deep learning loss function before and after the training iteration, the training gradient of the deep learning model parameters after each training iteration is generated, where the deep learning model parameters after the tth training iteration are , , deep learning model parameters The training gradient is Based on the training gradient, the contribution gradient of each group of domain privacy data in the deep learning loss function training process is calculated. The contribution gradient of the nth group of domain privacy data in the tth training iteration process is: .

[0017] in: represents the contribution gradient of the nth group of domain privacy data in the tth training iteration, represents the preset control coefficient, It represents the privacy budget dynamically optimized and adjusted for the nth group of domain privacy data after the t-1th training iteration.

[0018] Optionally, the privacy budget of the domain privacy data is dynamically optimized and adjusted based on the contribution gradient, wherein the privacy budget The dynamic optimization adjustment formula is: .

[0019] in: Representing Privacy Budget The differential privacy change rate of Indicates the preset contribution gradient threshold.

[0020] Optionally, the privacy budget after dynamic optimization adjustment is used to protect the privacy of domain privacy data. Domain privacy data The formula for privacy protection is: .

[0021] in: The mean is 0 and the standard deviation is A random number in a normal distribution, e represents the random noise intensity control parameter; Based on the privacy budget The protection result of the privacy data of the nth group of domains.

[0022] Optionally, the deep learning loss function is trained based on the privacy-protected domain privacy data to obtain the training iteration results of the deep learning model parameters until the training gradient of the deep learning model parameters is less than a preset gradient threshold. If the training gradient of the deep learning model parameters is less than the preset gradient threshold, the deep learning loss function converges. The training iteration formula of the deep learning model parameters is: ; .

[0023] in: represents the learning rate of the t+1th training iteration, Represents the deep learning model parameters after the t+1th training iteration.

[0024] In order to solve the above problem, the present invention provides an electronic device, the electronic device comprising: A memory storing at least one instruction; Communication interface, enabling electronic equipment to communicate; and A processor executes instructions stored in the memory to implement the preferred method for data differential privacy processing in the above-mentioned domain deep learning.

[0025] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the preferred method for data differential privacy processing in deep learning in the above-mentioned field.

[0026] Compared with the prior art, the present invention proposes a method for optimizing data differential privacy treatment for domain deep learning, which has the following advantages: First, this scheme proposes a privacy protection method for training data in domain deep learning model training. Based on the differential privacy method, the impact of whether domain privacy data is included on the deep learning loss function is quantified to obtain the initial privacy budget of the domain privacy data. The greater the impact of the domain privacy data on the deep learning loss function, the higher the initial privacy budget and the lower the degree of protection of the domain privacy data, thereby avoiding excessive impact on the model training effect of the deep learning model. In addition, privacy protection of domain privacy data can effectively ensure that even if an individual in the domain privacy data set is removed or added, the query result will not change significantly, thereby protecting the personal privacy information in the domain privacy data set from being leaked and avoiding privacy leakage.

[0027] At the same time, this scheme proposes a dynamic optimization adjustment method for the privacy budget. Based on the gradient of the deep learning model parameters during the training process, the contribution gradient of privacy data in different fields is calculated. The higher the contribution gradient, the greater the role of the privacy data in the field in the training process. Then, the privacy budget is dynamically optimized and adjusted based on the contribution gradient and the differential privacy change rate of the field privacy data. While ensuring the model training effect, the privacy budget for privacy protection of the field privacy data is obtained. The privacy budget is used to generate random numbers to perform differential privacy treatment on the field privacy data, and the deep learning model parameters are trained iteratively by gradient descent based on the changes in the privacy budget. The greater the increase in the privacy budget, the lower the privacy protection level of the current field privacy data, and the privacy budget needs to be increased and the step size of the deep learning model parameters needs to be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A flowchart of a preferred method for data differential privacy processing in domain deep learning provided by one embodiment of the present invention.

[0029] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0030] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0031] The embodiment of the present application provides a preferred method for handling data differential privacy in deep learning in the field. The execution subject of the preferred method for handling data differential privacy in deep learning in the field includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided in the embodiment of the present application. In other words, the preferred method for handling data differential privacy in deep learning in the field can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0032] Reference Figure 1 , Embodiment 1 of the present invention is: A method for optimizing differential privacy treatment of data in domain deep learning, comprising the following steps: S1: Obtain a domain privacy dataset, and construct the domain privacy dataset as a deep learning loss function, wherein the domain privacy dataset consists of domain privacy data and predicted classification results of the domain privacy data.

[0033] The personal sensitive fields include the medical field, the financial field, the consumer field and the social communication field. The domain privacy data set includes N groups of domain privacy data, where N represents the number of domain privacy data in the domain privacy data set, and the representation form of the nth group of domain privacy data is: .

[0034] in: Represents the nth group of domain privacy data; Sequentially indicate Privacy data in the medical, financial, consumer and social communication fields; Privacy data in the field The predicted classification result is .

[0035] As an embodiment of the present invention, the predicted classification results of the domain privacy data are in the form of a classification matrix, including predicted classification results of multiple personal sensitive fields, wherein the predicted classification results in the medical field are risk assessment results of common major diseases, the predicted classification results in the financial field are credit risk assessment results, the predicted classification results in the consumer field are consumer behavior prediction results, and the predicted classification results in the social communication field are social behavior prediction results. The consumer behavior prediction results are purchase probabilities of different types of goods, and the social behavior prediction results are attention probabilities of different social sectors.

[0036] Specifically, privacy data in the medical field includes health status, diagnosis results, treatment plans, genomic data, and drug use history; privacy data in the financial field includes bank account information, credit card transaction data, and insurance data information; privacy data in the consumer field includes shopping history data; and privacy data in the social communication field includes social dynamics and social interest data.

[0037] Based on the domain privacy dataset, the deep learning loss function expression constructed is: : .

[0038] in: represents the deep learning model parameters, T represents transposition, represents the L2 norm; Represents deep learning model parameters The deep learning model constructed, the output domain privacy data The predicted classification results.

[0039] S2: Initialize the privacy budget for generating domain private data.

[0040] Initialize and generate the deep learning model parameters , the deep learning loss function including domain privacy data and excluding domain privacy data is calculated as the initial contribution deviation of domain privacy data. The initial contribution deviation of the nth group of domain privacy data is: .

[0041] in: represents the initial contribution deviation of the nth group of domain privacy data, represents the L1 norm, represents a deep learning loss function that does not include the nth group of domain privacy data; specifically, The expression is: .

[0042] in: represents the domain privacy data set that does not contain the nth set of domain privacy data, and x represents the domain privacy data set y represents any domain privacy data in , and y represents the predicted classification result of domain privacy data x.

[0043] Based on the initial contribution deviation of the domain privacy data, the privacy budget of the domain privacy data is initialized and generated. The privacy budget of the nth group of domain privacy data generated by the initialization is: .

[0044] in: represents the privacy budget of the nth group of domain privacy data generated by initialization, Indicates the total amount of the preset privacy budget.

[0045] S3: Calculate the contribution gradient of domain privacy data during the deep learning loss function training process, and dynamically optimize and adjust the privacy budget of domain privacy data based on the contribution gradient.

[0046] The deep learning model parameters are trained iteratively during the deep learning loss function training process, the maximum training iteration of the deep learning model parameters is Max, and based on the change value of the deep learning loss function before and after the training iteration, the training gradient of the deep learning model parameters after each training iteration is generated, where the deep learning model parameters after the tth training iteration are , , deep learning model parameters The training gradient is Based on the training gradient, the contribution gradient of each group of domain privacy data in the deep learning loss function training process is calculated. The contribution gradient of the nth group of domain privacy data in the tth training iteration process is: .

[0047] in: represents the contribution gradient of the nth group of domain privacy data in the tth training iteration, represents the preset control coefficient, It represents the privacy budget dynamically optimized and adjusted for the nth group of domain privacy data after the t-1th training iteration.

[0048] Specifically, the training gradient g t The calculation expression is: ; .

[0049] in: After privacy protection, the deep learning model parameters are expressed is the deep learning loss function of the variable, Based on the privacy budget Domain privacy data protection results.

[0050] The privacy budget of the domain privacy data is dynamically and optimally adjusted based on the contribution gradient, wherein the privacy budget The dynamic optimization adjustment formula is: .

[0051] in: Representing Privacy Budget The differential privacy change rate of g represents the preset contribution gradient threshold.

[0052] Specifically, the calculation formula of the differential privacy change rate is: ; .

[0053] in: Indicates not included Domain privacy data collection, , u represents the predicted classification result of domain privacy data i; Indicates not included The deep learning loss function is Based on the privacy budget The protection result of the privacy data of the nth group of domains.

[0054] S4: Use the privacy budget adjusted by dynamic optimization to protect the privacy of domain privacy data, train the deep learning loss function based on the privacy-protected domain privacy data until the deep learning loss function converges, obtain the optimal deep learning model parameters after domain privacy protection, and build a deep learning model based on the optimal deep learning model parameters.

[0055] The privacy budget adjusted by the dynamic optimization is used to protect the privacy of the domain privacy data. Domain privacy data The formula for privacy protection is: .

[0056] in: The mean is 0 and the standard deviation is A random number in a normal distribution, e represents the random noise intensity control parameter; Based on the privacy budget The protection result of the privacy data of the nth group of domains.

[0057] The deep learning loss function is trained based on the privacy-protected domain privacy data to obtain the training iteration results of the deep learning model parameters until the training gradient of the deep learning model parameters is less than the preset gradient threshold. If the training gradient of the deep learning model parameters is less than the preset gradient threshold, the deep learning loss function converges. The training iteration formula of the deep learning model parameters is: ; .

[0058] in: represents the learning rate of the t+1th training iteration, Represents the deep learning model parameters after the t+1th training iteration.

[0059] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0060] It should be noted that the serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0061] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0062] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for optimizing data differential privacy treatment for domain deep learning, characterized in that: The method comprises: S1: Obtain a domain privacy dataset, and construct the domain privacy dataset as a deep learning loss function, wherein the domain privacy dataset consists of domain privacy data and predicted classification results of the domain privacy data, and the domain privacy data includes privacy data in a variety of personal sensitive fields; The deep learning loss function uses the deep learning model parameters as variables and the domain privacy data set as input to train the deep learning model parameters until the deep learning loss function converges, thereby obtaining the optimal deep learning model parameters that enable the prediction and classification results of the deep learning model to be optimal; The deep learning model takes user data as input and the user's predicted classification result as output, wherein the user data is the user's data information in various personal sensitive areas; S2: Initialize the privacy budget for generating domain privacy data; S3: Calculate the contribution gradient of domain privacy data in the deep learning loss function training process, and dynamically optimize and adjust the privacy budget of domain privacy data based on the contribution gradient; S4: Use the privacy budget adjusted by dynamic optimization to protect the privacy of domain privacy data, train the deep learning loss function based on the privacy-protected domain privacy data until the deep learning loss function converges, obtain the optimal deep learning model parameters after domain privacy protection, and build a deep learning model based on the optimal deep learning model parameters.

2. The optimal method for handling data differential privacy in domain deep learning as claimed in claim 1, characterized in that: The personal sensitive fields include the medical field, the financial field, the consumer field and the social communication field. The domain privacy data set includes N groups of domain privacy data, where N represents the number of domain privacy data in the domain privacy data set, and the representation form of the nth group of domain privacy data is: ； in: Represents the nth group of domain privacy data; Sequentially indicate Privacy data in the medical, financial, consumer and social communication fields; Privacy data in the field The predicted classification result is .

3. The optimal method for handling data differential privacy in domain deep learning as claimed in claim 2, characterized in that: Based on the domain privacy dataset, the deep learning loss function expression constructed is: : ； in: represents the deep learning model parameters, T represents transposition, represents the L2 norm; Represents the domain privacy data output by the deep learning model built based on the deep learning model parameter θ The predicted classification results.

4. The optimal method for handling data differential privacy in domain deep learning as claimed in claim 3, characterized in that: Initialize and generate the deep learning model parameters , the deep learning loss function including domain privacy data and excluding domain privacy data is calculated as the initial contribution deviation of domain privacy data. The initial contribution deviation of the nth group of domain privacy data is: ； in: represents the initial contribution deviation of the nth group of domain privacy data, represents the L1 norm, represents the deep learning loss function that does not include the nth group of domain privacy data; Based on the initial contribution deviation of the domain privacy data, the privacy budget of the domain privacy data is initialized and generated. The privacy budget of the nth group of domain privacy data generated by the initialization is: ； in: represents the privacy budget of the nth group of domain privacy data generated by initialization, Indicates the total amount of the preset privacy budget.

5. The optimal method for handling data differential privacy in domain deep learning as claimed in claim 1, characterized in that: The deep learning model parameters are trained iteratively during the deep learning loss function training process, the maximum training iteration of the deep learning model parameters is Max, and based on the change value of the deep learning loss function before and after the training iteration, the training gradient of the deep learning model parameters after each training iteration is generated, where the deep learning model parameters after the tth training iteration are , , deep learning model parameters The training gradient is Based on the training gradient, the contribution gradient of each group of domain privacy data in the deep learning loss function training process is calculated. The contribution gradient of the nth group of domain privacy data in the tth training iteration process is: ； in: represents the contribution gradient of the nth group of domain privacy data in the tth training iteration, represents the preset control coefficient, It represents the privacy budget dynamically optimized and adjusted for the nth group of domain privacy data after the t-1th training iteration.

6. The optimal method for handling data differential privacy in domain deep learning as claimed in claim 5, characterized in that: The privacy budget of the domain privacy data is dynamically and optimally adjusted based on the contribution gradient, wherein the privacy budget The dynamic optimization adjustment formula is: ； in: Representing Privacy Budget The differential privacy change rate of g Indicates the preset contribution gradient threshold.

7. The optimal method for handling data differential privacy in domain deep learning as claimed in claim 6, characterized in that: The privacy budget adjusted by the dynamic optimization is used to protect the privacy of the domain privacy data. Domain privacy data The formula for privacy protection is: ； in: The mean is 0 and the standard deviation is A random number in a normal distribution, e represents the random noise intensity control parameter; Based on the privacy budget The protection result of the privacy data of the nth group of domains.

8. The optimal method for handling data differential privacy in domain deep learning as claimed in claim 1, characterized in that: The deep learning loss function is trained based on the privacy-protected domain privacy data to obtain the training iteration results of the deep learning model parameters until the training gradient of the deep learning model parameters is less than the preset gradient threshold. If the training gradient of the deep learning model parameters is less than the preset gradient threshold, the deep learning loss function converges. The training iteration formula of the deep learning model parameters is: ； ； in: represents the learning rate of the t+1th training iteration, Represents the deep learning model parameters after the t+1th training iteration.

Images