Privacy auditing method for artificial intelligence decision model

Abstract

The invention discloses a privacy auditing method for an artificial intelligence decision model, and the method comprises the following steps: S1, constructing a training set, and building a non-differential privacy artificial intelligence model and a plurality of differential privacy artificial intelligence models through the training set; s2, establishing a relation between a differential privacy artificial intelligence model parameter and a privacy budget; and S3, estimating a real privacy budget, and completing auditing. The invention provides a more universal DP AI model privacy budget auditing method based on the perspective of Bayesian. In particular, the DP AI model is considered as a privacy budget as a sample in a distribution of parameters. According to the auditing method, various DP AI models including the neural network model, the logistic regression model and the like can be audited in the same mode, and compared with an existing auditing method, the auditing method has wider universality.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a privacy audit method for an artificial intelligence decision model. Background Art

[0002] Artificial Intelligence (AI) decision models use AI technology to make choices and behaviors based on collected data, rules, and goals, providing users with valuable decision-making references. However, when using AI models to discover data patterns, there is a risk of privacy leakage. Important information inadvertently leaked by AI may cause unauthorized access to sensitive information. Therefore, when using AI technology to make decisions, it is necessary to evaluate the privacy leakage risk of AI models to avoid inadvertent data leakage and ensure data privacy security. Differential privacy technology can prevent AI models from leaking sensitive information. Differential Privacy (DP) is a popular privacy protection technology that provides a rigorous mathematical framework. Differential privacy ensures the statistical consistency of data by adding noise. The statistical consistency of data prevents accurate inference of individual information, thereby ensuring that individual data is not leaked. AI models can be trained using differential privacy mechanisms, which can prevent information leakage of AI models. For example, Differential Privacy Stochastic Gradient Descent (DPSGD) is a differential privacy version of Stochastic Gradient Descent (SGD) and is widely used to train AI models. Designing and implementing differentially private AI models is subtle and error-prone, and further improvements are needed in the performance of audit methods.

[0003] Existing privacy budget auditing methods for differential privacy artificial intelligence (DP AI) models are not general enough and not precise enough. For example, in recent auditing studies, the privacy budget of DP AI models is audited by assuming and instantiating attackers with different capabilities. However, the existing auditing method is not general enough because it only applies to AI models trained with DPSGD. In order to improve the versatility of the auditing method, researchers proposed a framework that can audit the privacy budget of various DP AI models. However, the auditing accuracy of these frameworks can be further improved. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a privacy audit method for an artificial intelligence decision model, comprising the following steps: S1. Constructing a training set, and establishing a non-differential privacy artificial intelligence model and multiple differential privacy artificial intelligence models through the training set; S2. Establishing the connection between the differential privacy artificial intelligence model parameters and the privacy budget; S3. Estimate the real privacy budget and complete the audit.

[0005] Furthermore, the S1 step specifically samples the model from a Bayesian perspective, and trains multiple neural network models optimized using differential privacy stochastic gradient descent and a single neural network model optimized using stochastic gradient descent.

[0006] Furthermore, the S2 step specifically includes the following sub-steps: S21. Assume W0 is an AI model not trained with the DP mechanism, and assume W1, W2, ..., Wn are n AI models trained with the DP mechanism and the corresponding added noises are R1, R2, ..., Rn; S22. Construct a parameter difference sequence and approximate the noise sequence by the variance of the parameter difference sequence.

[0007] Furthermore, the parameter difference sequence constructed in step S22 is specifically: {W1 - W0, W2 - W0, ..., Wn - W0}; the parameter sequence is specifically: {R1, R2, ..., Rn}.

[0008] Furthermore, the S22 step specifically includes the following sub-steps: when the noise Ri tends to 0, the impact of the noise Ri on the differential privacy artificial intelligence model Wi is small, that is, Wi tends to W0.

[0009] Furthermore, in step S3, the model noise is used as a parameter to estimate the true privacy budget through Rényi differential privacy if and only if the distribution of any two adjacent input data D and data D' satisfies D ∞ (f(D)||f(D'))≤ ϵ; where ϵ is the privacy budget, f(D) and f(D') represent the mechanism of training using data D or D', respectively, and D ∞ () represents the Rayleigh divergence with parameter α = ∞.

[0010] The present invention provides a privacy audit method for an artificial intelligence decision model, which has the following beneficial effects: Based on the Bayesian perspective, this paper proposes a more general DP AI model privacy budget audit method. In particular, the DP AI model is regarded as a sample in a distribution with the privacy budget as a parameter. This audit method can audit various DP AI models in the same way, including neural network models, logistic regression models, etc., and has a wider versatility than existing audit methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0012] Figure 1 A flow chart of the method provided by the present invention; Figure 2 The present invention provides a flowchart of the method. DETAILED DESCRIPTION

[0013] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0014] The following is a detailed description of the implementation method of the present invention in conjunction with the accompanying drawings. Only some embodiments are described, not all embodiments. For the purpose of clarity, representations and descriptions that are not related to the present invention are omitted in the drawings and descriptions.

[0015] In order to have a clearer understanding of the technical features, purposes and beneficial effects of the present invention, the technical solution of the present invention is now described in detail below. Obviously, the implementation cases described are part of the embodiments of the present invention, not all of the embodiments, and cannot be understood as limiting the scope of the implementation of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the protection scope of the present invention.

[0016] like Figure 1 , Figure 2 As shown, the present invention provides a privacy audit method for an artificial intelligence decision model, comprising the following steps: S1. constructing a training set, and establishing a non-differential privacy artificial intelligence model and multiple differential privacy artificial intelligence models through the training set, wherein the differential privacy artificial intelligence model is sampled from a parameter distribution with a privacy budget as a parameter; S2. establishing a connection between the parameters of the differential privacy artificial intelligence model and the privacy budget; S3. estimating the true privacy budget and completing the audit.

[0017] Specifically, step S1 samples the model from a Bayesian perspective and trains multiple neural network models optimized using differential privacy stochastic gradient descent and a single neural network model optimized using stochastic gradient descent.

[0018] Since the parameters of the artificial intelligence model come from distribution sampling, the model is sampled from a Bayesian perspective. The model parameters are taken as direct calculation objects to ensure the universality of the proposed audit method. In this step, the Bayesian perspective is introduced to train multiple neural network models optimized by differential privacy stochastic gradient descent and a single neural network model optimized by stochastic gradient descent.

[0019] From a Bayesian perspective, the neural network model parameter w follows a certain probability distribution, which means that the parameters of the neural network can be represented by a specific distribution. This method of representing the model by parameter distribution ensures the universality of the audit method. Therefore, the neural network model can be considered as a sample from a specific distribution. Specifically, training multiple models provides model parameters of nodes at the same position, which can describe the parameter space of the neural network. This means that training a neural network model is equivalent to sampling from a distribution. In turn, these model parameters can be used to describe this distribution. Changes in the types of differential privacy AI models can be reflected by the different forms and parameters of the probability distribution function.

[0020] Step S2 specifically includes the following sub-steps: S21. Assume W0 is an AI model not trained with the DP mechanism, and assume W1, W2, ..., Wn are n AI models trained with the DP mechanism and the corresponding added noises are R1, R2, ..., Rn; S22. Construct a parameter difference sequence and approximate the noise sequence by the variance of the parameter difference sequence.

[0021] The parameter difference sequence constructed in step S22 is specifically: {W1 - W0, W2 - W0, ..., Wn - W0}; the parameter sequence is specifically: {R1, R2, ..., Rn}. Step S22 specifically includes the following sub-steps: When the noise Ri tends to 0, the noise Ri has little effect on the differential privacy artificial intelligence model Wi, that is, Wi tends to W0.

[0022] By manipulating the model training hyperparameters, we quantify the impact of DP and propose a general method to extract the noise of DP models. Due to the direct effect of the DP mechanism on the parameters, there are differences in the model parameters when using and not using the DP mechanism. This deviation manifests as a change in the distribution of model parameters at the same location. It is necessary to quantify the impact of the DP mechanism and extract its effect on the parameter distribution. This part of the effect is attributed to the noise added to the parameters by the DP mechanism. The most intuitive operation to quantify the noise added by the DP mechanism to the model parameter distribution is to compare the deviation of the model parameter distribution under DP and non-DP training conditions. This operation is performed under the same hyperparameter and model architecture settings.

[0023] The training process of the DP AI model can be represented by a function h. Function h has three inputs, including the dataset X, the parameters θ of the training process, and the noise R of the DP mechanism. That is, W=h(X,θ,R), where W is the DP AI model obtained through the training procedure. Let f(R│ϵ) be the probability density function of the noise R, where ϵ is the privacy budget. Because W=h(X,θ,R) is a function of the noise R, W obeys f(R│ϵ).

[0024] Assume that W_0 is an AI model trained without DP mechanism. Let W1, W2, ..., Wn be n AI models trained with DP mechanism, and the corresponding noise added is R1, R2, ..., Rn. Use the variance of parameter difference sequence {W1 - W0, W2 - W0, ..., Wn - W0} to approximate the noise sequence {R1, R2, ..., Rn}. There are two reasons for doing this. First, the two variables obey the same probability distribution, that is, f(R│ϵ). Second, the changes of the two variables are interrelated. Specifically, when the noise R i When it approaches 0, the noise R i For DP AI model W i The influence of W is small, that is, i tends to W0, and vice versa. i - W0∝ R i and the two variables follow the same distribution, and use the maximum likelihood estimate of the variance of the distribution on {W1 - W0, W2 - W0, ..., Wn - W0} to approximate {R1, R2, ⋯, R n}. The connection between the privacy budget and the statistical parameter estimation method can be more effective than other audit methods. This audit method benefits from the parameter estimation method and requires less samples.

[0025] In step S3, the model noise is used as a parameter to estimate the true privacy budget through Rényi differential privacy if and only if the distribution of any two adjacent input data D and data D' satisfies D ∞ (f(D)||f(D')) ≤ ϵ; where ϵ is the privacy budget, f(D) and f(D') represent the mechanism of training using data D or D', respectively, and D ∞ () represents the Rayleigh divergence with parameter α = ∞.

[0026] In order to audit the privacy budget of the DP AI model, the audit method needs to establish a connection between the parameters of the DP AI model and its privacy budget. The parameters of the DP AI model are samples from a certain distribution. The privacy budget is a parameter of this distribution, which means that the distribution is affected by the privacy budget. The DP mechanism adds noise to the model according to the privacy budget. The intensity of the noise affects the consumption of the privacy budget. Therefore, the DP model noise obtained by maximum likelihood estimation is related to the privacy budget of the model. The privacy budget is calculated using the noise variance of the DP model. To measure the privacy budget, Rényi differential privacy provides a method based on the Rényi difference between the mechanism output distributions. Specifically, the privacy budget of the DP mechanism is quantified by sampling the differential privacy of the Rényi sampled Gaussian mechanism with model noise as a parameter.

[0027] The present invention directly uses model parameters for auditing, and utilizes parameter estimation methods in statistics to achieve accurate auditing of the privacy budget of the DP AI model. Model parameters contain rich knowledge and are affected by many factors, such as training data sets, parameters, model architecture, etc. In particular, the parameters of the DP AI model are affected by the noise introduced by the DP mechanism, pruning operations, and other actions during model training. Therefore, model parameters are the most direct source for auditing privacy budgets. This means that this privacy auditing method is more accurate.

[0028] Based on the Bayesian perspective, this paper proposes a more general DP AI model privacy budget audit method. In particular, the DP AI model is regarded as a sample in a distribution with the privacy budget as a parameter. This audit method can audit various DP AI models in the same way, including neural network models, logistic regression models, etc., and has a wider versatility than existing audit methods.

[0029] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art do not deviate from the spirit and scope of the present invention, and should be within the scope of protection of the claims attached to the present invention.

Claims

1. A privacy audit method for artificial intelligence decision models, characterized in that: The following steps are involved: S1. Construct a training set, and use the training set to build a non-differential privacy AI model and multiple differential privacy AI models; S2. Establish the connection between differential privacy AI model parameters and privacy budget; S3. Estimate the true privacy budget and complete the audit.

2. The privacy audit method for an artificial intelligence decision model according to claim 1, characterized in that: Specifically, the S1 step samples the model from a Bayesian perspective, and trains multiple neural network models optimized using differential privacy stochastic gradient descent and a single neural network model optimized using stochastic gradient descent.

3. The privacy audit method for an artificial intelligence decision model according to claim 1, characterized in that: The S2 step specifically includes the following sub-steps: S21. Let W0 be the AI ​​model without DP mechanism training, let W1, W2, ..., W n There are n AI models trained using the DP mechanism and the corresponding noise added is R1, R2, ..., R n ; S22. Construct a parameter difference sequence and approximate the noise sequence by the variance of the parameter difference sequence.

4. The privacy audit method for an artificial intelligence decision model according to claim 3, characterized in that: The parameter difference sequence constructed in step S22 is specifically: {W1 - W0, W2 - W0, ..., W n - W0}; the parameter sequence is: {R1, R2, ..., R n }.

5. The privacy audit method for an artificial intelligence decision model according to claim 4, characterized in that: The step S22 specifically includes the following sub-steps: i When the noise R i For the differential privacy artificial intelligence model W i The influence of W is small, that is, i Tends to W0.

6. The privacy audit method for artificial intelligence decision model according to claim 1, characterized in that: Specifically, in step S3, the model noise is used as a parameter to estimate the true privacy budget through Rényi differential privacy if and only if the distribution on any two adjacent input data D and data D' satisfies D ∞ (f(D)||f(D')) ≤ ϵ; where ϵ is the privacy budget, f(D) and f(D') represent the mechanism of training using data D or D', respectively, and D ∞ () represents the Rayleigh divergence with parameter α = ∞.

Images