Black box differential privacy security measurement method and device, equipment and storage medium

Abstract

The invention discloses a black box differential privacy security measurement method and device, equipment and a storage medium, and relates to the field of data encryption. Extracting two noise-added data sample sets from the training samples, scoring and sorting the first sample set by using a classifier, and constructing an attack set according to a scoring and sorting result; estimating an event formed by each data pair by using the second sample set, and estimating a lower bound value of a privacy budget confidence interval according to each event to determine a maximum event; adding noise to the maximum event to generate a test set, and performing kernel density estimation by using a kernel density function; and constructing a privacy loss function, calculating a maximum loss value and a loss maximum point corresponding to the maximum loss value in the input space, calculating the confidence level of the black box differential privacy algorithm, and calculating the lower bound value of the privacy budget. Compared with a black box differential privacy algorithm, the method has the advantages that potential violation events are pre-screened, so that the requirement of traversing all events is avoided, the operation time of an epsilon value estimation program is shortened, and the working efficiency is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data encryption, and in particular to a black-box differential privacy security measurement method, apparatus, device and storage medium. Background Art

[0002] With the widespread application of data mining technology, some manufacturers encourage researchers to further explore the intrinsic value of data by publishing user data sets. However, data sets usually contain a lot of personal privacy data. In the process of data set release, there are security risks, which may lead to the leakage of user privacy. For example, medical diagnosis records, personal consumption habits and usage preferences, etc., this information will be leaked due to the release of data sets.

[0003] In order to protect the privacy of users, the widely used mechanism is to use differential privacy to inject carefully designed noise into the original data, so as to achieve the purpose of hiding the original data. Differential privacy is a new privacy definition proposed by Dwork in 2006 for the privacy leakage problem of statistical databases. Under this definition, the calculation results of the database are insensitive to the changes of a specific record. The presence or absence of a single record in the data set has little effect on the calculation results. Therefore, the privacy leakage risk of a record due to its addition to the data set is controlled within a very small and acceptable range, and the attacker cannot obtain accurate individual information by observing the calculation results. From the definition of differential privacy protection, it can be seen that the privacy budget ε is used to control the probability ratio of the noise mechanism M to obtain the same output on the adjacent data sets, reflecting the privacy protection level of the noise mechanism M. The smaller ε, the higher the privacy protection level. In extreme cases, when ε is 0, it means that the probability distribution of the output of the noise mechanism M for the data sets D and D' is exactly the same. Since D and D' are adjacent data sets, it can be clearly concluded according to mathematical induction that when ε = 0, the output result of the noise mechanism M cannot reflect any useful information about the data set. Therefore, on the other hand, the value of ε also reflects the availability of data. Under the same circumstances, the smaller ε is, the lower the data availability is.

[0004] At present, differential privacy has become a mainstream technical means commonly used for data release, and privacy budget has also become a standard parameter for evaluating information leakage during data dissemination, which is used to quantify the difficulty of inferring parameters (i.e. user information) from algorithm releases. However, in real life, for security and intellectual property reasons, companies will not disclose their specific privacy protection algorithms, which brings challenges to information security evaluation.

[0005] Related technologies propose to use a large sample set to train the DP-Sniper model and make predictions, or use the Maximum Privacy Loss (MPL) algorithm to traverse all events and calculate the ε value. However, the computational efficiency of these two methods is relatively low, and for third-party security evaluation agencies, it is impossible to accurately measure and determine whether the desensitization performance of the black box program meets the claimed differential privacy requirements. Summary of the invention

[0006] The embodiments of the present application provide a black-box differential privacy security measurement method, apparatus, device and storage medium to solve the problem that the privacy budget of a black-box differential privacy program cannot be accurately measured.

[0007] On the one hand, the present application provides a black box differential privacy security measurement method, the method comprising:

[0008] A data set is formed based on different noise mechanisms of a black box differential privacy algorithm, and all data pairs in the data set are noised using the black box noise mechanism to generate training samples; the training samples are used to train the classifier, and all data pairs in the data set constitute an input space;

[0009] Extracting two noisy data sample sets from the training samples, scoring and sorting the first sample set using the trained classifier, and constructing an attack set according to the scoring and sorting results;

[0010] Based on the constructed attack set, the second sample set is used to estimate the events composed of each data pair, and the maximum event is determined according to the lower bound value of the confidence interval of the estimated privacy budget of each event;

[0011] The maximum event is noised using a black box noise mechanism to generate a test set, a kernel density function is used to perform kernel density estimation on the test set, a privacy loss function is constructed, and the maximum loss value and the corresponding maximum loss point in the input space are calculated;

[0012] The confidence level of the black-box differential privacy algorithm is calculated based on the maximum loss value and the maximum loss point to calculate the lower bound of the privacy budget.

[0013] Specifically, the data set D is represented as follows:

[0014] D={(a i ,a′ i )|i=1,2,...,k}

[0015] Among them (a i ,a′ i ) represents the i-th data pair, K represents the number of data pairs;

[0016] The training sample Dnoisy generated by adding noise is expressed as follows:

[0017] Dnoisy={(M n (a i ),M n (a′ i ))∣n=1,2,...,N;i=1,2,...,k}

[0018] Among them, M n represents the nth noise mechanism, N represents the number of noise mechanisms contained in the test black box, and M n (a i ) and M n (a′ i ) are respectively a i and a′ i The noisy data.

[0019] Specifically, the step of scoring and sorting the first sample set using the trained classifier and constructing the attack set according to the scoring and sorting results includes:

[0020] Extract N1 noise-added data M(a′) to form the first sample set B={b i ′|i=1,2,...,N1}, use the classifier to calculate each sample data b i ′’s score p(a| b i′);

[0021] Arrange the N1 score values ​​in descending order and take the first 0.01N1 sample data to construct the attack set S.

[0022] Specifically, determining the maximum event based on the lower bound value of the confidence interval of the estimated privacy budget of each event includes:

[0023] Extract N2 noise-added data and form a second sample set. Each sample data in the second sample set forms an event E. i (a i ,a′ i ), where i=1,2,...,N2;

[0024] According to the constructed attack set S, use N2 events to estimate the lower bound of the confidence interval of the privacy budget ε, and determine the largest event E with the largest lower bound of ε max .

[0025] Specifically, the maximum event is noised using a black box noise mechanism to generate a test set, and a kernel density function is used to perform kernel density estimation on the test set, including:

[0026] Extract the maximum event to act on different noise mechanisms M of the black box n , generate N3 pairs of noise data test set Q, expressed as follows:

[0027] Q={(M n (a*),M n (a′*))|n=1,2,...,N3}

[0028] Where a* and a′* represent the data pair (a*, a′*) of the maximum event;

[0029] According to the noise mechanism contained in the black-box differential privacy algorithm, the kernel density function is selected, and the kernel density estimation is performed on the maximum event at each point t in the output space C to obtain the kernel density estimation values ​​kde(t|a*) and kde(t|a′*).

[0030] Specifically, the step of constructing a privacy loss function and calculating a maximum loss value and a corresponding maximum loss point in the input space includes:

[0031] Construct the privacy loss function loss(t), which is expressed as follows:

[0032] loss(t)=|ln(kde(t|a*))-ln(kde(t|a′*)|

[0033] Search the entire output space C to find the point where loss(t) reaches the maximum value Corresponding maximum loss value L max ;

[0034] The output space C includes a space discrete type and a space continuous type; when the output space C is a continuous type, the interval is discretized and a set step size is used to perform a spatial search and determine.

[0035] Specifically, the confidence level of the black box differential privacy algorithm based on the maximum loss value and the maximum loss point is used to calculate the lower bound of the privacy budget, including:

[0036] Calculate the variance based on the kernel density estimate of the maximum event at point t in the output space and constant c;

[0037] When the output space C is continuous, the calculation is as follows:

[0038]

[0039] When the output space C is discrete, the calculation is as follows:

[0040]

[0041] The lower bound calculation formula for defining the privacy budget ε is as follows:

[0042]

[0043] ε LBrepresents the lower bound of the privacy budget ε, Φ -1 is the quantile function of the standard normal distribution, α is the confidence level parameter; K is the kernel function of the kernel density estimation.

[0044] On the other hand, the present application provides a black box differential privacy security measurement device, the device comprising:

[0045] A sample generation module is used to form a data set based on different noise mechanisms of the black box differential privacy algorithm, and to generate training samples by adding noise to all data pairs in the data set using the black box noise mechanism; the training samples are used to train the classifier, and all data pairs in the data set constitute the input space;

[0046] A sorting module, used to extract two noisy data sample sets from the training samples, score and sort the first sample set using the trained classifier, and construct an attack set according to the scoring and sorting results;

[0047] A first determination module is used to estimate the events composed of each data pair based on the constructed attack set using the second sample set, and determine the maximum event according to the lower bound value of the confidence interval of the estimated privacy budget of each event;

[0048] The second determination module is used to generate a test set by adding noise to the maximum event using a black box noise mechanism, perform kernel density estimation on the test set using a kernel density function, and determine the maximum loss value of the kernel density function and the corresponding maximum loss point in the input space;

[0049] The calculation module is used to construct the privacy loss function of the classifier based on the maximum loss value and the maximum loss point, and calculate the lower bound of the privacy budget according to the confidence level of the black-box differential privacy algorithm.

[0050] On the other hand, the present application provides a computer device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the black-box differential privacy security measurement method described in the above aspects.

[0051] On the other hand, the present application provides a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the black-box differential privacy security measurement method described in the above aspects.

[0052] The beneficial effects of the technical solution provided by the embodiment of the present application include at least: the solution first tries to use a small sample set to train the classifier model, and for multiple pairs of data in the input space, uses the trained classifier model to select events, screens out the event that is most likely to violate differential privacy, that is, the event with the largest ε value, and then uses the MPL algorithm to calculate its specific ε value. Compared with the existing black-box differential privacy algorithm, this solution avoids the need to traverse all events by pre-screening potential violation events, shortens the program running time for estimating the ε value, and improves work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flowchart of a black box differential privacy security measurement method provided by an embodiment of the present application;

[0054] Figure 2 A processing logic diagram of a black box differential privacy security measurement method is shown;

[0055] Figure 3 An algorithm flow chart of the black box differential privacy security measurement method is shown;

[0056] Figure 4 The structure block diagram of the black box differential privacy security measurement device provided by the embodiment of the present application is shown;

[0057] Figure 5 A structural block diagram of a computer device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0058] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0059] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0060] Figure 1 : is a flowchart of a black box differential privacy security measurement method provided by an embodiment of the present application, comprising the following steps:

[0061] S1, build a data set based on different noise mechanisms of the black box differential privacy algorithm, and add noise to all data pairs in the data set using the black box noise mechanism to generate training samples;

[0062] For data encryption providers, considering the protection of intellectual property and trade secrets, they generally do not disclose specific encryption algorithms and logics to the public. The confidence interval of the privacy budget ε they declare is for reference only. The public and third-party testing agencies only face a public black box program, so it is necessary to evaluate the confidence interval of the privacy budget in order to assess the level of privacy protection of the black box program for private data.

[0063] For the black-box differential privacy algorithm to be tested, it contains many noise mechanisms M, that is, interference noise acting on the privacy data. This step first selects appropriate data pairs (a, a′), that is, neighboring data sets, for different mechanisms M. These data pairs constitute the input space C of the mechanism. The lengths of the data pairs vary. In some embodiments, 16 pairs of data sets with a length of 5 can be generated and constitute its input space, also known as 16 events. Each set of data pairs in the data set constitutes an event.

[0064] Define the noise mechanism M(·): Assume there is a deterministic or random function M(a) that accepts the original data point a and returns a noisy version a′. For each pair of data (a, a′), in the set, apply the M function N times independently to a and a′ to generate training samples. For each event / data pair (a, a′), after applying the noise mechanism M, we can get the noisy data pair (M(a), M(a′)), that is, the noise data M(a) corresponds to a, and M(a′) corresponds to a′. Training samples can be obtained by applying different noise mechanisms to all data pairs.

[0065] S2, extract two noisy data sample sets from the training samples, use the trained classifier to score and sort the first sample set, and construct an attack set based on the scoring and sorting results;

[0066] The purpose of forming training samples is to train the classifier, that is, to build the classifier used by the black box differential privacy security measurement method, by setting the label set and the training set for multiple rounds of iterative testing. Iterative training can use a Logistic Regression classifier or a Multi-Layer Perceptron (MLP) classifier, and use samples to train the classifier to obtain the classifier parameters. Each sample pair is labeled as category 0 (representing a) or category 1 (representing a′) for evaluation and classification.

[0067] The classifier that has completed iterative training is verified using training samples, and the noisy data sample set is continued to be sampled from the training samples. In this step, the first sample set is extracted and input into the classifier for scoring and sorting. The score value represents the inverse differential privacy identification result, which is understood as a probabilistic event that can reversely infer the source privacy data. For such events that are most likely to violate differential privacy, this application selects and constructs an attack set according to the score, conducts a targeted attack to crack the attack set, and infers the gap between the confidence interval of the actual black box algorithm privacy budget and the theoretical claim.

[0068] S3, based on the constructed attack set, use the second sample set to estimate the events composed of each data pair, and determine the maximum event according to the lower bound value of the confidence interval of the estimated privacy budget of each event;

[0069] After constructing the attack set, the second sample set is extracted from the training sample again (or randomly extracted in advance). Each (noised) data pair in the second sample set is mapped to an event. According to the attack set S, the extracted events are used to estimate the lower bound of the confidence interval mapped to ε, and the event with the largest lower bound is used as the subsequent analysis object. The lower bound value determines that the maximum event is the most ideal and privacy-preserving event obtained after processing the sample using the black box differential privacy algorithm. The subsequent kernel density estimation is performed on the maximum event.

[0070] Figure 2 The processing logic diagram of the black-box differential privacy security measurement method is shown. For the mechanism to be tested (black-box program), the input space and events are generated based on the noise mechanism and then sampled. Then, time selection is performed, and the event with the largest ε lower bound is selected. Kernel density estimation is performed, and finally the actual privacy budget lower bound is calculated based on the kernel density estimate.

[0071] S4, add noise to the maximum event using the black box noise mechanism to generate a test set, use the kernel density function to estimate the kernel density of the test set, construct the privacy loss function and calculate the maximum loss value and the corresponding maximum loss point in the input space;

[0072] Kernel density estimation is a nonparametric statistical method used to estimate the distribution density function of data. In differential privacy analysis, kernel density estimation can be used to analyze the distribution characteristics of data and identify high-density and low-density areas, which is very helpful for understanding the overall structure and potential patterns of data. In this way, differential privacy technology can be applied more accurately to protect personal privacy while ensuring the accuracy of data analysis. In this scheme, the size of the kernel density estimate can be used to determine the distribution of events in the output space, the relative density between the two noisy data in the largest event, the sensitivity of noise to data changes, etc.

[0073] In this step, for the maximum event E that has been determined max , the data pair (a*, a′*) is used to apply various noise mechanisms of the black box program to form a test set, and then the kernel density function is used to estimate the kernel density of the test set of the maximum event. The kernel density estimation can be obtained by using a specific calculation formula, specifically calculating the kernel density of the sampled test set at each point t in the output space.

[0074] The purpose of constructing the privacy loss function is to calculate the maximum loss value and the corresponding point with the maximum loss in the input space. This is understood as finding the noisy data pair with the largest difference and the corresponding spatial point t in the output space C. This point represents the position point that can best distinguish the noisy data pair.

[0075] S5, calculates the lower bound of the privacy budget by calculating the confidence level of the black box differential privacy algorithm based on the maximum loss value and the maximum loss point;

[0076] This step uses all possible location points that can distinguish the noise-added data pairs, the confidence level parameter value given by the black box program, and the quantile function of the standard normal distribution to calculate the actual lower bound of ε.

[0077] Furthermore, because the black-box program will give a theoretical ε confidence interval, when the actual ε lower bound value calculated by this scheme is different from the lower bound value of the theoretical ε confidence interval and exceeds a certain threshold range, it means that the privacy protection level of the black-box program is insufficient. Otherwise, it means that the differential privacy algorithm has the claimed privacy protection capability.

[0078] In summary, the technical solution provided by this application first tries to use a small sample set to train the classifier model. For multiple pairs of data in the input space, the trained classifier model is used to select events, and the event that is most likely to violate differential privacy, that is, the event with the largest ε value, is screened out, and then the MPL algorithm is used to calculate its specific ε value. Compared with the existing black-box differential privacy algorithm, this solution avoids the need to traverse all events by pre-screening potential violation events, shortens the program running time for estimating the ε value, and improves work efficiency.

[0079] In the differential privacy algorithm, a privacy mechanism is applied to each possible output of the black box program, usually by adding Laplace noise to meet the requirements of differential privacy. This step ensures that any query result of a single count or frequency will not directly disclose personal information. Secondly, find the maximum value among all the noised results. Because each original count has been processed for privacy protection, even the most frequently occurring items have been "fuzzified" and their true counts are hidden under the noise. Finally, the mechanism reports this maximum value after adding noise as the query result. In this way, the outside world can only obtain the most likely option after privacy protection, but cannot accurately know the true frequency of each option, thereby protecting the privacy of individual records.

[0080] In some embodiments, the process of forming a data set in S1 can adopt the Report Noisy Max mechanism and choose to add Laplace noise. Assuming there are n candidate answers, the original maximum count value is c max , the goal is to find and report this maximum value while maintaining differential privacy. i Add Laplace noise, the formula is:

[0081]

[0082] Among them, c i ′ is the count value after adding noise, Lap(b) represents a Laplace distributed random variable, whose scale parameter is b, that is, the probability density function of generating the random number is expressed as follows:

[0083]

[0084] Here Δf is the sensitivity of the query. For count queries, if adding or subtracting an element changes at most one count, then Δf = 1. ε is the privacy budget, which controls the strength of privacy protection. A smaller ε means stronger privacy protection but may sacrifice the accuracy of the results.

[0085] After calculating all the noise-added values, select the largest ci′ as the reported result, that is:

[0086]

[0087] The above gives the original maximum count value This value can be used to construct a dataset.

[0088] For the training of the classifier, the training samples obtained by adding noise are classified according to the two labels of category 0 (representing a) or category 1 (representing a′), and the training set X = {x j}、label set Y={y j}, where j = 1, 2, ..., 2Nk. Based on the Logistic regression model, the classifier is designed, and the model probability p is expressed as follows:

[0089]

[0090] Where θ is the model parameter, and θ is trained and updated using the gradient descent method, as shown below:

[0091]

[0092] The loss function L(θ) of θ is expressed as follows:

[0093]

[0094] Where η is the learning rate, represents the gradient difference of the loss function, θ new and θ old Represent the gradient values ​​before and after the update respectively.

[0095] In some embodiments, the step of constructing the attack set S in S2 may include the following:

[0096] 1) Extract N1 noise-added data M(a′) to form the first sample set B={b i ′|i=1,2,...,N1}, use the classifier to calculate each sample data b i ′’s score p(a|b i ′);

[0097] Here b i ′ actually represents the M(a′) data corresponding to a. Because N1 noise data are sampled, N1 score values ​​p are obtained.

[0098] 2) Arrange the N1 score values ​​in descending order and take the first 0.01N1 sample data to construct the attack set S.

[0099] Figure 3 The algorithm flow chart of the black-box differential privacy security measurement method is shown. For the classifier, the training samples are sampled three times, and each sampling is based on the previous sampling and data processing results. The first sampling of N data pairs is used to train the classifier, the second sampling of N1 data is used to score and sort and calculate the attack set S, and the third sampling of N2 is used to determine the event E with the largest lower bound of ε. After determining the largest event, the kernel density estimation is continued.

[0100] In some embodiments, the maximum event E is determined in S3. max The specific steps may include:

[0101] 1) Extract N2 noise-added data and form a second sample set. Each sample data in the second sample set forms an event E i (a i ,a′ i ), where i=1,2,...,N2;

[0102] 2) According to the constructed attack set S, use N2 events to estimate the lower bound of the confidence interval of the privacy budget ε, and determine the largest event E with the largest lower bound of ε max .

[0103] In some embodiments, performing kernel density estimation on the test set using a kernel density function in S4 specifically includes the following steps:

[0104] 1) Extract the maximum event and act on different noise mechanisms M of the black box respectively n , generate N3 pairs of noise data test set Q, expressed as follows:

[0105] Q={(M n (a*),M n (a′*))|n=1,2,...,N3}

[0106] Where a* and a′* represent the data pair (a*, a′*) of the maximum event;

[0107] 2) According to the noise mechanism contained in the black-box differential privacy algorithm, a kernel density function is selected, and the kernel density estimation is performed on each point t of the maximum event in the output space C to obtain the kernel density estimation values ​​kde(t|a*) and kde(t|a′*).

[0108] In particular, the output space C includes space discrete type and space continuous type; when the output space C is continuous, the interval is discretized and the space search is determined using the set step size. If discr = 0, the return value of the noise mechanism M is a continuous value. If discr = 1, the return value of the noise mechanism M is a discrete value. For example, if C is continuous [0,1], a step size of 0.001 is used to search from 0 to 1, for a total of 1001 points.

[0109] For the output space C constructed in this application, the kernel density estimate of the data pair (a*, a′*) at point t in the continuous output space is expressed as:

[0110]

[0111] In the above formula, N3 represents the number of noise data pairs of sampling and noise mechanism, and K represents the kernel function of kernel density estimation.

[0112] For discrete types, kde(t|a*) and kde(t|a′*) are calculated as follows: n (a*) is compared with the corresponding data point t and M is calculated n (a*) is equal to the number of times t is recorded as n, so kde(t|a*) is expressed as follows:

[0113]

[0114] Similarly, M n (a′*) is compared with the corresponding data point t and M is calculated n (a′*) is equal to the degree of t, recorded as n′, so kde(t|a′*) is expressed as follows:

[0115]

[0116] Construct the privacy loss function loss(t), which is expressed as follows:

[0117] loss(t)=|ln(kde(t|a*))-ln(kde(t|a′*)|

[0118] Search the entire output space C to find the point where loss(t) reaches the maximum value Corresponding maximum loss value L max .

[0119] This involves a weighted summation of all sample points, where the weights are determined by the kernel function based on the distance t and the sample value. For each point t on the output space C, the logarithmic difference between kde(t|a*) and kde(t|a′*) is calculated. This metric reflects the relative density of a* compared to a′* at point t, and the logarithmic transformation enhances the sensitivity of the change in discrimination. Search the entire output space C to find the point that maximizes loss(t), that is, At the same time, record the maximum loss value L max , this point represents the position in the model output that can best distinguish a* and a′*.

[0120] In some embodiments, calculating the lower bound of the privacy budget for the output space may include the following steps:

[0121] When the output space C is continuous, the calculation is as follows:

[0122]

[0123] When the output space C is discrete, the calculation is as follows:

[0124]

[0125] The lower bound calculation formula for defining the privacy budget ε is as follows:

[0126]

[0127] ε LB represents the lower bound of the privacy budget ε, Φ -1 is the quantile function of the standard normal distribution, α is the confidence level parameter; K is the kernel function of the kernel density estimation.

[0128] After the above steps, the actual lower limit of the privacy budget of the black box program can be calculated. When this value is inconsistent with the theoretical value announced to the outside world and the difference is large, it means that its privacy protection level does not meet the standard. Otherwise, it can provide sufficient privacy encryption level.

[0129] In summary, this application selects the most representative event pairs in the input space of a large data set, that is, the neighboring data set that can best reflect the privacy protection characteristics of the algorithm. After identifying the best performing event pair, the MPL algorithm is used to accurately estimate the ε value corresponding to this event pair. The advantage of this solution is that it can greatly reduce the required computing resources and time while ensuring the calculation accuracy, which is especially important for processing large-scale data sets. Compared with existing means of evaluating black-box differential privacy algorithms, the introduction of the MPL algorithm significantly improves the evaluation efficiency and reduces the time cost required to estimate the privacy budget, thereby making the evaluation of privacy protection more efficient and feasible.

[0130] Figure 4 The structure block diagram of the black box differential privacy security measurement device provided by the embodiment of the present application is shown, and the device includes:

[0131] A sample generation module 410 is used to form a data set based on different noise mechanisms of a black box differential privacy algorithm, and to generate training samples by adding noise to all data pairs in the data set using the noise mechanism of the black box; the training samples are used to train the classifier, and all data pairs in the data set constitute an input space;

[0132] A sorting module 420 is used to extract two noisy data sample sets from the training samples, score and sort the first sample set using the trained classifier, and construct an attack set according to the scoring and sorting results;

[0133] A first determination module 430 is used to estimate the events composed of each data pair based on the constructed attack set using the second sample set, and determine the maximum event according to the lower bound value of the confidence interval of the estimated privacy budget of each event;

[0134] The second determination module 440 is used to generate a test set by adding noise to the maximum event using a black box noise mechanism, perform kernel density estimation on the test set using a kernel density function, and determine the maximum loss value of the kernel density function and the corresponding maximum loss point in the input space;

[0135] The calculation module 450 is used to construct a privacy loss function of the classifier based on the maximum loss value and the maximum loss point, and calculate the lower bound of the privacy budget according to the confidence level of the black box differential privacy algorithm.

[0136] The black-box differential privacy security measurement device provided in the embodiment of the present application can be applied to the black-box differential privacy security measurement method provided in the above embodiment. For relevant details, refer to the above method embodiment. The implementation principle and technical effect are similar and will not be repeated here.

[0137] It should be noted that the black-box differential privacy security measurement device provided in the embodiments of the present application is only illustrated by the division of the above-mentioned functional modules / functional units. In practical applications, the above-mentioned functions can be assigned to different functional modules / functional units as needed, that is, the internal structure of the black-box differential privacy security measurement device is divided into different functional modules / functional units to complete all or part of the functions described above. In addition, the implementation method of the black-box differential privacy security measurement method provided in the above-mentioned method embodiment and the implementation method of the black-box differential privacy security measurement device provided in this embodiment belong to the same concept. The specific implementation process of the black-box differential privacy security measurement device provided in this embodiment is detailed in the above-mentioned method embodiment, which will not be repeated here.

[0138] Figure 5The block diagram of a computer device provided by an exemplary embodiment of the present application is shown. It is a computer device such as a desktop computer, a laptop computer, a PDA, and a cloud server. The computer device may include, but is not limited to, a processor and a memory. Among them, the processor and the memory may be connected by a bus or in other ways. Among them, the processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, graphics processors (GPU), embedded neural network processors (NPU) or other dedicated deep learning coprocessors, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned various chips.

[0139] The processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content that needs to be displayed on the display screen. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0140] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the methods in the above-mentioned embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the method in the above-mentioned method implementation is realized. The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0141] In some embodiments, the computer device may further optionally include: a peripheral device interface and at least one peripheral device. The processor, the memory and the peripheral device interface may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit, a display screen and a keyboard.

[0142] The peripheral device interface can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor and the memory. In some embodiments, the processor, the memory, and the peripheral device interface are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor, the memory, and the peripheral device interface can be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0143] The display screen is used to display the UI (User Interface). The UI may include graphics, text, icons, videos and any combination thereof. When the display screen is a touch screen, the display screen also has the ability to collect touch signals on the surface or above the surface of the display screen. The touch signal can be input to the processor as a control signal for processing. At this time, the display screen can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen can be one, arranged on the front panel of the computer device; in other embodiments, the display screen can be at least two, respectively arranged on different surfaces of the computer device or in a folding design; in other embodiments, the display screen can be a flexible display screen, arranged on a curved surface or a folding surface of the computer device. Even, the display screen can be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode) and the like.

[0144] The power supply is used to power various components in the computer device. The power supply can be AC, DC, a disposable battery, or a rechargeable battery. When the power supply includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0145] Those skilled in the art will appreciate that the structure shown in this embodiment does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0146] The embodiment of the present application also discloses a computer-readable storage medium. Specifically, the computer-readable storage medium is used to store a computer program, and when the computer program is executed by the processor, the method in the above-mentioned method implementation is implemented. Those skilled in the art can understand that the implementation of all or part of the process in the above-mentioned implementation method of the present application can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the process of the implementation of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated as: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0147] This specific embodiment is merely an explanation of the present invention and is not a limitation of the present invention. After reading this specification, those skilled in the art may make non-creative modifications to the present embodiment as needed. However, as long as they are within the scope of the claims of the present invention, they are protected by the patent law.

Claims

1. A black box differential privacy security measurement method, characterized in that: The method comprises: A data set is formed based on different noise mechanisms of a black box differential privacy algorithm, and all data pairs in the data set are noised using the black box noise mechanism to generate training samples; the training samples are used to train the classifier, and all data pairs in the data set constitute an input space; Extracting two noisy data sample sets from the training samples, scoring and sorting the first sample set using the trained classifier, and constructing an attack set according to the scoring and sorting results; Based on the constructed attack set, the second sample set is used to estimate the events composed of each data pair, and the maximum event is determined according to the lower bound value of the confidence interval of the estimated privacy budget of each event; The maximum event is noised using a black box noise mechanism to generate a test set, a kernel density function is used to perform kernel density estimation on the test set, a privacy loss function is constructed, and the maximum loss value and the corresponding maximum loss point in the input space are calculated; The confidence level of the black-box differential privacy algorithm is calculated based on the maximum loss value and the maximum loss point to calculate the lower bound of the privacy budget.

2. The method according to claim 1, characterized in that The dataset D is represented as follows: D={(a i ,a′ i )∣i=1,2,...,k} Among them (a i ,a′ i ) represents the i-th data pair, K represents the number of data pairs; The training sample Dnoisy generated by adding noise is expressed as follows: Dnoisy={(M n (a i ),M n (a′ i ))∣n=1,2,...,N;i=1,2,...,k} Among them, M n represents the nth noise mechanism, N represents the number of noise mechanisms contained in the test black box, and M n (a i ) and M n (a′ i ) are respectively a i and a′ i The noisy data.

3. The method according to claim 2, characterized in that The step of scoring and sorting the first sample set using the trained classifier and constructing an attack set according to the scoring and sorting results includes: Extract N1 noise-added data M(a′) to form the first sample set B={b i ′|i=1,2,...,N1}, use the classifier to calculate each sample data b i ′’s score p(a|b i ′); Arrange the N1 score values ​​in descending order and take the first 0.01N1 sample data to construct the attack set S.

4. The method according to claim 3, characterized in that The maximum event is determined based on the lower bound value of the confidence interval of the estimated privacy budget for each event, including: N2 noise-added data are extracted and formed into a second sample set. Each sample data in the second sample set forms an event E. i (a i ,a′ i ), where i=1,2,...,N2; According to the constructed attack set S, use N2 events to estimate the lower bound of the confidence interval of the privacy budget ε, and determine the largest event E with the largest lower bound of ε max .

5. The method according to any one of claims 1 to 4, characterized in that: The step of adding noise to the maximum event using a black box noise mechanism to generate a test set, and performing kernel density estimation on the test set using a kernel density function, comprises: Extract the maximum event to act on different noise mechanisms M of the black box n , generate N3 pairs of noise data test set Q, expressed as follows: Q={(M n (a*),M n (a′*))∣n=1,2,...,N3} Where a* and a′* represent the data pair (a*, a′*) of the maximum event; According to the noise mechanism contained in the black-box differential privacy algorithm, the kernel density function is selected, and the kernel density estimation is performed on the maximum event at each point t in the output space C to obtain the kernel density estimation values ​​kde(t|a*) and kde(t|a′*).

6. The method according to claim 5, characterized in that The step of constructing a privacy loss function and calculating a maximum loss value and a corresponding maximum loss point in the input space includes: Construct the privacy loss function loss(t), which is expressed as follows: loss(t)=|ln(kde(t|a*))-ln(kde(t|a′*)| Search the entire output space C to find the point where loss(t) reaches the maximum value Corresponding maximum loss value L max ; The output space C includes a space discrete type and a space continuous type; when the output space C is a continuous type, the interval is discretized and a set step size is used to perform a spatial search and determine.

7. The method according to claim 6, characterized in that The confidence level of the black box differential privacy algorithm based on the maximum loss value and the maximum loss point is used to calculate the lower bound of the privacy budget, including: Calculate the variance based on the kernel density estimate of the maximum event at point t in the output space and constant c; When the output space C is continuous, the calculation is as follows: When the output space C is discrete, the calculation is as follows: The lower bound calculation formula for defining the privacy budget ε is as follows: ε LB represents the lower bound of the privacy budget ε, Φ -1 is the quantile function of the standard normal distribution, α is the confidence level parameter; K is the kernel function of the kernel density estimation.

8. A black box differential privacy security measurement device, characterized in that: The device comprises: A sample generation module is used to form a data set based on different noise mechanisms of the black box differential privacy algorithm, and to generate training samples by adding noise to all data pairs in the data set using the black box noise mechanism; the training samples are used to train the classifier, and all data pairs in the data set constitute the input space; A sorting module, used to extract two noisy data sample sets from the training samples, score and sort the first sample set using the trained classifier, and construct an attack set according to the scoring and sorting results; A first determination module is used to estimate the events composed of each data pair based on the constructed attack set using the second sample set, and determine the maximum event according to the lower bound value of the confidence interval of the estimated privacy budget of each event; The second determination module is used to generate a test set by adding noise to the maximum event using a black box noise mechanism, perform kernel density estimation on the test set using a kernel density function, and determine the maximum loss value of the kernel density function and the corresponding maximum loss point in the input space; The calculation module is used to construct the privacy loss function of the classifier based on the maximum loss value and the maximum loss point, and calculate the lower bound of the privacy budget according to the confidence level of the black-box differential privacy algorithm.

9. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the black-box differential privacy security measurement method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the black-box differential privacy security measurement method as described in any one of claims 1 to 7.

Images