Image classification model privacy protection method and device based on asymmetric differential privacy

Abstract

The present invention provides a privacy protection method and device for an image classification model based on asymmetric differential privacy. The method comprises: training a teacher classification model using a private image set; predicting an unlabeled public image set using the teacher classification model to obtain prediction records and category frequency distribution; extracting a neighborhood record set with prediction records under a preset insensitive category from the prediction record set; calculating asymmetric sensitivity using the prediction record and the neighborhood record set, and obtaining a noise distribution based on the asymmetric sensitivity; adding noise that conforms to the noise distribution to the frequency of the category frequency distribution; selecting the category with the largest frequency in the category frequency distribution after adding noise as a pseudo label for the public image; and training a student classification model using the public image set and the pseudo label. Also provided is a privacy protection device for an image classification model based on asymmetric differential privacy, a computer program product, and an electronic device. The present invention avoids bilateral errors and improves the utility of image classification models.

Description

Technical Field

[0001] The present invention relates to the field of privacy protection technology, and in particular to a privacy protection method and device for an image classification model based on asymmetric differential privacy. Background Art

[0002] The model training process of machine learning and deep learning relies on relevant data, and more data usually improves the performance of the model. However, there is relatively little public de-privatized data, and many model trainings rely on private data. In smart medical scenarios, various medical images are sensitive private data. If the medical image data is leaked, it will cause huge losses. Especially when it comes to medical image data and models that are migrated from the private field of medical institutions to the public field, once leaked, it will be irreversible.

[0003] Differential Privacy (DP) is a privacy protection method that aims to ensure that sensitive information related to individual identities is not leaked when analyzing data and publishing statistical results. In differential privacy, each individual in sensitive data is considered as an input, and the addition of noise makes the contribution of the analysis result to a single individual uncertain. This uncertainty makes it impossible for attackers to determine the contribution of a specific individual, thereby protecting the privacy of individuals.

[0004] In the training and prediction phase of deep learning, differential privacy methods that add noise to models have become a hot topic in recent years. The privacy of local data is guaranteed by perturbing the data or training process parameters, but perturbing the data will reduce the validity of the data while ensuring privacy. Therefore, how to add noise more subtly and achieve a balance between privacy and utility while ensuring differential privacy is the key point of the algorithm.

[0005] Papernot et al. proposed a teacher privacy aggregation model framework called PATE (Private Aggregation of Teacher Ensembles) in 2017. Figure 2 As shown in the figure. The framework uses the idea of ​​ensemble learning to vote on the prediction results of the teacher model and introduces noise in the voting results. Then, the prediction result with the highest number of votes is selected as the final prediction result of the teacher model. Next, the student classification model is supervised and trained using the teacher model and the public dataset. Finally, the student classification model is published as a public model. This framework provides a privacy-preserving teacher model aggregation method that prevents attackers from obtaining the original data through inversion attacks by adding noise.

[0006] The noise added by the existing teacher privacy aggregation model in the prediction label aggregation stage is symmetric noise, such as standard Laplace noise and Gaussian noise. However, using the symmetric noise mechanism to answer decision questions will produce bilateral errors, that is, there are bilateral errors in binary classification problems. For example, in the binary classification task of positive and negative COVID-19 images, if symmetric noise is added, the added noise may lead to false negative cases (that is, the patient itself is positive), reducing the effectiveness of the COVID-19 image classification model. Summary of the invention

[0007] The present invention aims to at least solve the technical problem that in the process of obtaining an image classification model based on the existing teacher privacy aggregation model framework, the noise added to the prediction label aggregation stage is symmetrical noise, which will produce bilateral errors when answering decision questions and reduce the effectiveness of the image classification model, and provide an image classification model privacy protection method and device based on asymmetric differential privacy.

[0008] In order to achieve the above-mentioned purpose of the present invention, according to the first aspect of the present invention, the present invention provides an image classification model privacy protection method based on asymmetric differential privacy, comprising: using a private image set to train multiple teacher classification models; using the multiple teacher classification models after training to simultaneously predict an unlabeled public image set; counting the prediction results of multiple teacher classification models for each public image to obtain a prediction record for each public image, and the prediction records of all public images constitute a prediction record set; performing frequency analysis on the prediction record of each public image to obtain the category frequency distribution of each public image; extracting from the prediction record set The prediction record of each public image is in a neighborhood record set under a preset insensitive category; the asymmetric sensitivity is calculated using the prediction record and the neighborhood record set of each public image according to the query function, and the noise distribution of each public image is obtained based on the asymmetric sensitivity; noise that conforms to the noise distribution of each public image is added to the frequency of the category frequency distribution of each public image; the category with the largest frequency in the category frequency distribution of the public image after adding noise is selected as the pseudo label of the public image; the student classification model is trained using the public image set and the pseudo labels of the public images, and the student classification model after the training is used as the image classification model.

[0009] In order to achieve the above-mentioned purpose of the present invention, according to the second aspect of the present invention, the present invention provides an image classification model privacy protection device based on asymmetric differential privacy, which is used to implement the image classification model privacy protection method based on asymmetric differential privacy described in the first aspect of the present invention, including: a teacher classification model training module, using a private image set to train multiple teacher classification models; a prediction module, using multiple teacher classification models after training to simultaneously predict an unlabeled public image set; a statistical module, counting the prediction results of multiple teacher classification models for each public image, obtaining a prediction record for each public image, and the prediction records of all public images constitute a prediction record set; performing frequency analysis on the prediction record of each public image to obtain the category frequency distribution of each public image; and a neighborhood A record acquisition module extracts a neighborhood record set of the prediction record of each public image under a preset insensitive category from the prediction record set; a noise distribution generation module calculates asymmetric sensitivity using the prediction record and neighborhood record set of each public image according to a query function, and obtains the noise distribution of each public image based on the asymmetric sensitivity; a noise addition module adds noise that conforms to the noise distribution of each public image to the frequency of the category frequency distribution of each public image; a pseudo-label generation module selects the category with the largest frequency as the pseudo-label of the public image in the category frequency distribution after adding noise to the public image; a student classification model training module trains the student classification model using the public image set and the pseudo-label of the public image, and uses the trained student classification model as the image classification model.

[0010] In order to achieve the above-mentioned purpose of the present invention, according to the third aspect of the present invention, the present invention provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the image classification model privacy protection method based on asymmetric differential privacy described in the first aspect of the present invention.

[0011] In order to achieve the above-mentioned purpose of the present invention, according to the fourth aspect of the present invention, the present invention provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image classification model privacy protection method based on asymmetric differential privacy as described in the first aspect of the present invention.

[0012] The present invention uses a private image set to train multiple teacher classification models, and then uses the trained multiple teacher classification models to predict the public image set, and counts the prediction results of multiple teacher classification models for each public image to obtain a prediction record of each public image. According to the privacy policy, the category output by the classification model is pre-divided into two categories: sensitive category and insensitive category. For each prediction record (one-to-one corresponding to a public image), a neighborhood record set of each prediction record under a preset insensitive category is extracted from the prediction record set. According to the query function, the prediction record and the neighborhood record set of each public image are used to calculate the asymmetric sensitivity. Based on the asymmetric sensitivity, the noise distribution of each public image is obtained, and the category frequency is denoised according to the noise distribution, so as to realize the addition of unilateral Laplace noise to the voting results (category frequencies) after the teacher classification model prediction category aggregation, which can avoid the generation of bilateral errors. At the same time, since the added noise value is relatively reduced, the utility of the image classification model is also improved.

[0013] The present invention is based on the teacher privacy aggregation model and combines asymmetric differential privacy protection through semi-supervised knowledge transfer, so that the data privacy of the teacher classification model learning process is guaranteed by (PP,∈)-ADP (i.e., differential privacy protection of the privacy policy function), and the degree of privacy protection is high;

[0014] In the training of the teacher classification model and the student classification model, the present invention uses a symmetric cross entropy loss function for asymmetric noise, which can balance the influence between different categories, thereby alleviating the problem of category imbalance;

[0015] In the present invention, the teacher classification model can effectively provide pseudo-label knowledge, and the student classification model can utilize this pseudo-label knowledge, allowing heterogeneity between classification teacher models and teacher classification models, and between teacher classification models and student classification models. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of a privacy protection method for an image classification model based on asymmetric differential privacy in a preferred embodiment of the present invention;

[0017] Figure 2 It is a structural diagram of the teacher privacy aggregation model framework in the prior art;

[0018] Figure 3 It is a schematic structural diagram of an electronic device in a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0019] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0020] In the description of the present invention, it is necessary to understand that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0021] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0022] The execution subject of the privacy protection method for the image classification model based on asymmetric differential privacy includes but is not limited to at least one of the electronic devices such as the server and the terminal that can be configured to execute the method provided in the embodiment of the present application. In other words, the privacy protection method for the image classification model based on asymmetric differential privacy can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0023] The privacy protection method for an image classification model based on asymmetric differential privacy provided by the present invention, in a preferred embodiment, is as follows: Figure 1 As shown, the method includes:

[0024] Step S1, train multiple teacher classification models using private image sets.

[0025] The private image set is a collection of unpublished images with privacy protection requirements. Private images are preferably, but not limited to, diagnostic images, such as CT images, MT images, ultrasound images, and angiography images. Each private image in the private image set has a real category label. The teacher classification model is preferably, but not limited to, a CNN neural network model. The teacher classification model is mainly used for private image classification. There are M categories, the first category K1, ..., the Mth category K M , M is a positive integer greater than or equal to 2, that is, M≥2. In some examples, the M categories include normal tissue and abnormal tissue; or, the M categories include tumor area size levels, such as normal, primary, secondary, tertiary, etc. In one example, the private image is a lung CT image, and the teacher classification model is used to determine whether the lung CT image belongs to a patient who is positive or negative for the new coronavirus, and each private lung CT image has a positive label or a negative label.

[0026] Step S2, using multiple trained teacher classification models to simultaneously predict the unlabeled public image set.

[0027] Each trained teacher classification model predicts the unlabeled public image set. Each public image in the unlabeled public image set is predicted by multiple teacher classification models respectively, and the prediction result is the category to which the public image belongs, that is, K1, ..., K M The public image is an image corresponding to the private image type, preferably but not limited to a diagnosis and treatment image, such as a CT image, a MT image, an ultrasound image, and an angiography image.

[0028] Step S3, counting the prediction results of multiple teacher classification models for each public image, obtaining the prediction record of each public image, and the prediction records of all public images constitute a prediction record set; performing frequency analysis on the prediction record of each public image to obtain the category frequency distribution of each public image.

[0029] Assume that the public image set is represented as G. For any public image g∈G, the prediction record of the public image g obtained by statistics is: r=(r1,r2,…r n ,…,r N ), where r n represents the predicted category of the nth teacher classification model for the public image g, n∈[1,N], r n ∈{K1,…,K M}, N represents the number of teacher classification models, and N is a positive integer greater than or equal to 2. The category frequency distribution of the public image g is expressed as: {n1(g),...,n m (g), ...n M (g)},n m(g) represents the frequency of the public image g being predicted as the mth category by N teacher classification models, 1≤m≤M. m (g): =|{n:n∈[N], F n (g) = K m}|,m∈[M],F n (g) represents the prediction result of the nth teacher classification model for the public image g.

[0030] Step S4, extracting a neighborhood record set of the prediction record of each public image under a preset insensitive category from the prediction record set.

[0031] Step S5, calculating the asymmetric sensitivity using the prediction record and the neighborhood record set of each public image according to the query function, and obtaining the noise distribution of each public image based on the asymmetric sensitivity.

[0032] Step S6: adding noise that conforms to the noise distribution of each public image to the frequency of the category frequency distribution of each public image.

[0033] Step SS7, select the category with the largest frequency in the category frequency distribution after adding noise to the public image as the pseudo label of the public image, and the pseudo label of the public image g is

[0034] Step S8, using the public image set and the pseudo labels of the public images to train the student classification model, and using the trained student classification model as the image classification model.

[0035] The student classification model is preferably, but not limited to, a CNN neural network model. The student classification model is trained using a semi-supervised method, the parameters of the student classification model after training are saved, and the CNN neural network model is deployed according to the saved student classification model parameters to obtain an image classification model. The image classification model is used to classify images to be tested, and the images to be tested are preferably, but not limited to, diagnostic images, such as CT images, MT images, ultrasound images, and angiography images.

[0036] In this embodiment, in step S1, the specific steps of using the private image set to train multiple teacher classification models include:

[0037] Step S11, divide the private image set into a training set and a test set for training the teacher classification model, and divide the training set (X, Y) into non-overlapping data subsets (X n , Y n ), n∈[N]={1,2,...,N}, n represents the data subset index; where X represents the private image and Y represents the corresponding true category label.

[0038] Step S12, constructing the network structure of the teacher classification model. The teacher classification model adopts a CNN convolutional neural network structure, including a plurality of convolution processing units and a fully connected layer connected in sequence.

[0039] Each convolution processing unit includes a convolution layer, a ReLU activation function layer, and a maximum pooling layer connected in sequence.

[0040] Among them, the convolution layer receives the input private image (b represents the batch size, 3 represents the number of image channels, and 32 represents the height and width of the image). The convolution layer uses a 3×3 convolution kernel. The number of output channels is 32, the stride is 1, and the padding is 1. The calculated output size is [b, 32, 32, 32]. The calculation formula of the convolution layer can be expressed as:

[0041] Y conv1 =X*K1;

[0042] The ReLU activation function layer performs a nonlinear transformation on the output of the convolutional layer. The calculation formula of the ReLU activation function layer is:

[0043] ReLU(fc input )=max(0,fc input );

[0044] The ReLU activation function layer compares the input value with zero. If the input value is greater than zero, the output is the input value itself; if the input value is less than or equal to zero, the output is zero. In short, the ReLU activation function does not make any changes when the input value is positive, and the output is the same as the input; when the input value is negative, the output value is set to zero.

[0045] The max pooling layer uses a 2×2 window for pooling with a stride of 2, halving the size and resulting in an output size of [b, 32, 16, 16].

[0046] The fully connected layer includes a linear fully connected layer. The output of the convolutional layer is Y conv1 Through the flattening operation, it becomes a one-dimensional vector fc input , as the input of the fully connected layer. The input dimension of the fully connected layer is 401408 (calculated according to the size of the convolutional layer output). The fully connected layer maps the input to M output nodes and obtains the predicted values ​​of M categories. In an example, the fully connected layer maps the input to 2 output nodes (2 categories) and uses linear transformation to calculate the final output, with a size of [b, 2].

[0047] Using the above CNN convolutional neural network, in the forward propagation process of the teacher classification model, the input private image X is processed by the convolution processing unit to obtain the convolution feature map conv1. Then, the convolution feature map is flattened into a one-dimensional vector fc input Finally, the flattened feature vector is linearly transformed through the fully connected layer to obtain the final output fc out .

[0048] Step S13, use each data subset to train a teacher classification model. In the training process of each teacher classification model, the Adam optimizer is used to minimize the loss function and adjust the weights and biases of the CNN convolutional neural network model. The update rule can be summarized as:

[0049] (1) Calculate the gradient g t For the current parameter θ:

[0050] m t =β1m t-1 +(1-β1)g t ;

[0051] (2) Calculate the first-order moment estimate m t and the second-order moment estimate v t :

[0052]

[0053] (3) Use the first-order moment estimate and the second-order moment estimate to update the parameter θ:

[0054]

[0055] Among them, β1 and β2 are hyperparameters that control the decay rate of the first-order moment and the second-order moment, η is the learning rate, ∈ is a very small constant used to ensure numerical stability, and t is the number of training times.

[0056] Step S14, using the test set to test the trained teacher classification model.

[0057] In this embodiment, the loss function may be an existing cross entropy loss function.

[0058] In this embodiment, in order to solve the overfitting problem caused by the teacher classification model using the cross entropy loss function as the loss function, the robustness of the inverted cross entropy loss function is flexibly explored, and the problem of biased prediction of categories with a large number of samples is alleviated. Further preferably, in step S1, each teacher classification model is trained based on a certain data subset of the private image set, and the goal of training is to minimize the first symmetric cross entropy loss function, and the first symmetric cross entropy loss function is:

[0059] lsl =al ce +bl rce ;

[0060] Where a represents the first hyperparameter, b represents the second hyperparameter; l ce represents the difference loss, l rce Indicates symmetric loss; a is used to solve the overfitting problem of the cross entropy loss function, while b is used to flexibly explore the robustness of the inverted cross entropy loss function. Both a and b are greater than 0. A smaller a value will lead to slower convergence but can alleviate overfitting, while a larger a value will lead to faster convergence but overfitting. Therefore, the value range of a is 0.05 to 0.3, preferably 0.1. For symmetric noise, b can be set to 1, and for asymmetric noise, b can be set to 0.1.

[0061] Difference loss ce The calculation formula is:

[0062] Symmetric loss rce The calculation formula is:

[0063] p(K m |x) indicates that the private image x is predicted as category K by the teacher classification model m The probability distribution of q(K m |x) indicates that the private image x is in category K m The true label probability distribution of m is a positive integer, and 1≤m≤M.

[0064] After training the teacher classification model with the first symmetric cross entropy loss function mentioned above, the overfitting problem of the teacher classification model can be effectively solved. By introducing a symmetric weight parameter a and b, the influence between different categories can be balanced, thereby alleviating the category imbalance problem.

[0065] In this embodiment, in step S8, the process of training the student classification model using the public image set and the pseudo-labels of the public images can refer to the training process of each teacher classification model mentioned above, which will not be repeated here. The cross-entropy loss function can also be used in the training process. In order to solve the overfitting problem caused by the cross-entropy loss function as the loss function of the student classification model, the robustness of the inverted cross-entropy loss function is flexibly explored, and the problem of the student classification model being biased towards predicting categories with a large number of samples is alleviated. Further preferably, in the step of training the student classification model using the public image set and the pseudo-labels of the public images, the student classification model is trained with the goal of minimizing the second symmetric cross-entropy loss function, and the second symmetric cross-entropy loss function is:

[0066] l2=a2lce2 +b2l rce2 ;

[0067] Wherein, a2 represents the third hyperparameter, b2 represents the fourth hyperparameter; l ce2 represents the difference loss of the student classification model, l rce2 Indicates the symmetric loss of the student classification model; a2 is used to solve the overfitting problem of the cross entropy loss function, and b2 is used to flexibly explore the robustness of the inverted cross entropy loss function. Both a2 and b2 are greater than 0. A smaller value will lead to slower convergence but can alleviate overfitting, and a larger a2 value will lead to faster convergence but overfitting. Therefore, the value range of a2 is 0.05 to 0.3, preferably 0.1. For symmetric noise, b2 can be set to 1, and for asymmetric noise, b2 can be set to 0.1.

[0068] Student classification model difference loss l ce2 The calculation formula is:

[0069]

[0070] Symmetric loss of student classification model l rce2 The calculation formula is:

[0071]

[0072] p2(K m |g) indicates that the public image g is predicted by the student classification model as the mth category K m The probability distribution of q2(K m |g) means the public image g is in the mth category K m The pseudo-label probability distribution of

[0073] In a preferred embodiment, in step S3, extracting a neighborhood record set of the prediction record of each public image under a preset insensitive category from the prediction record set includes:

[0074] Step a: Let M categories K1, ..., K M There are M′ preset insensitive categories in , M′<M;

[0075] Step b, determine the model subset composed of the teacher classification models corresponding to all prediction categories belonging to the insensitive category in the prediction record of the public image g. If the prediction category corresponding to the model subset in another prediction record in the prediction record set is exactly the same as the prediction category corresponding to the model subset in the prediction record of the public image g, then the other prediction record is considered to be a neighborhood record of the prediction record of the public image g, and all neighborhood records of the prediction record of the public image g constitute a neighborhood record set.

[0076] In this embodiment, a privacy policy function PP is defined i : K i →{0, 1}, where K i represents the class space, and the value of PP i (K m ) is 1 or 0, which is used to determine whether the input class K m is classified as a sensitive class or a non-sensitive class. The function PP i divides the class space K i into two different subsets: a sensitive class subset and a non-sensitive class subset. When the class K m is in the sensitive class subset, the value of PP i (K m ) is 1, indicating that the class K m is classified as sensitive; while when the class K m is in the non-sensitive subset, the value of PP i (K m ) is 0, indicating that the class K m is classified as non-sensitive.

[0077] Step b includes the definition of neighborhood records, and the definition of neighborhood records is as follows:

[0078] For the prediction record of the public image g: r=(r1, r2,... r n ,..., r N ), it is stipulated that if and only if PP i (r i ) = o and r i = r′ i , then r′=(r′1, r′2,... r′ N ) is the neighborhood record of r. PP i (r i ) = 0 indicates that the i-th class K i is a preset non-sensitive class, 1 ≤ i ≤ M, and the value of i may not be unique. In an example, if the prediction record of the public image g is: 0, 0, 1, 0, 1; and the class 0 is a preset non-sensitive class, then it can be seen that the prediction classes of the first, second, and third teacher classification models are non-sensitive classes. If there is a record in the prediction record set: 0, 0, 1, 0, 0; and the prediction classes of the first, second, and third teacher classification models of this record are also 0, then this record is considered to be the neighbor record of the prediction record of the public image g.

[0079] In step b, all neighborhood records of the prediction record of the public image g are obtained according to the above definition of neighborhood records and form a neighborhood record set N pp (D).

[0080] In a preferred embodiment, in step S5, the asymmetric sensitivity is calculated using the prediction record and the neighborhood record set of each public image according to the query function, including:

[0081] Step S51, querying and calculating the predicted records of the public image using the query function f(·) to obtain a first query result, which can be expressed as f(D). The query function f(·) is preferably, but not limited to, an average function, a counting function, or a sum function.

[0082] Step S52: Use the query function to perform query calculations on each neighborhood record in the neighborhood record set to obtain multiple neighborhood query results. Any neighborhood query result is represented by f(D′), D′∈N pp (D).

[0083] Step S53, calculating the absolute values ​​of the differences between the multiple neighborhood query results and the first query result to obtain multiple absolute values ​​of the differences, and selecting the absolute value of the difference with the largest value from the multiple absolute values ​​of the differences as the asymmetric sensitivity:

[0084] Represents a set of prediction records.

[0085] Further preferably, obtaining the noise distribution of each public image based on asymmetric sensitivity includes:

[0086] If the query function satisfies f(D′)-f(D)≥0, the positive noise distribution of the public image is:

[0087]

[0088] If the query function satisfies f(D′)-f(D)<0, the negative noise distribution of the public image is:

[0089]

[0090] Among them, the auxiliary parameters Δ af represents asymmetric sensitivity, ε represents privacy budget, λ represents the noise to be added, and λ + represents the positive noise distribution, λ - represents the negative noise distribution, f(D) represents the first query result obtained by querying the predicted record of the public image using the query function f(·), and f(D′) represents the neighborhood query result obtained by querying any neighborhood record D′ in the neighborhood record set using the query function f(·).

[0091] Further preferably, in the step of adding noise that conforms to the noise distribution of each public image to the frequency of the category frequency distribution of each public image, the noise is added according to the following formula:

[0092]

[0093] in, Represents the category K after adding noise to the public image g m The frequency of m (g) represents the category K of the public image g before adding noise m The frequency of; 1≤m≤M, M represents the number of categories.

[0094] The privacy protection method for image classification model based on asymmetric differential privacy provided by the present invention has the following beneficial technical effects:

[0095] High degree of privacy protection: Based on the teacher privacy aggregation model, the present invention combines asymmetric differential privacy protection through semi-supervised knowledge transfer, so that the data privacy of the deep learning process is guaranteed by (PP,∈)-ADP.

[0096] Avoid the impact of bilateral errors: The present invention takes into account the situation where bilateral errors need to be avoided in special scenarios, and adds asymmetric Laplace noise in the teacher voting aggregation stage, so that the model as a whole can achieve the prediction effect of unilateral errors, and the corresponding security issues are improved.

[0097] High utility: Furthermore, for asymmetric noise, the present invention can use a symmetric cross entropy loss function. In the case of class imbalance, the model may tend to predict the class with a larger number of samples. The symmetric cross entropy loss function introduces a symmetric weight parameter to balance the influence between different classes, thereby alleviating the class imbalance problem.

[0098] Generalization: Heterogeneity between teacher classification models and between teacher classification models and student classification models is allowed. The key is that the teacher classification model can effectively provide label knowledge, and the student classification model can utilize this label knowledge. In fact, the student classification model is not limited to deep learning algorithms, and ordinary machine learning algorithms can also be used as a choice for the teacher classification model.

[0099] The present invention also provides an image classification model privacy protection device based on asymmetric differential privacy, which is used to implement the above-mentioned image classification model privacy protection method based on asymmetric differential privacy, including:

[0100] The teacher classification model training module uses private image sets to train multiple teacher classification models;

[0101] The prediction module uses multiple trained teacher classification models to simultaneously predict the unlabeled public image set;

[0102] The statistical module counts the prediction results of multiple teacher classification models for each public image, obtains the prediction record of each public image, and the prediction records of all public images constitute a prediction record set; performs frequency analysis on the prediction record of each public image to obtain the category frequency distribution of each public image;

[0103] A neighborhood record acquisition module extracts a neighborhood record set of the prediction record of each public image under a preset insensitive category from the prediction record set;

[0104] A noise distribution generation module calculates asymmetric sensitivity using the prediction record and neighborhood record set of each public image according to the query function, and obtains the noise distribution of each public image based on the asymmetric sensitivity;

[0105] A noise adding module, adding noise that conforms to the noise distribution of each public image to the frequency of the category frequency distribution of each public image;

[0106] A pseudo-label generation module selects the category with the largest frequency in the category frequency distribution after adding noise to the public image as the pseudo-label of the public image;

[0107] The student classification model training module uses the public image set and pseudo labels of the public images to train the student classification model, and uses the trained student classification model as the image classification model.

[0108] In this embodiment, the teacher classification model training module, prediction module, statistics module, neighborhood record acquisition module, noise distribution generation module, noise addition module, pseudo label generation module and student classification model training module respectively correspond to steps S1, S2, S3, S4, S5, S6, S7 and S8 of the above-mentioned image classification model privacy protection method based on asymmetric differential privacy, and will not be repeated here.

[0109] The present invention also discloses a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-mentioned image classification model privacy protection method based on asymmetric differential privacy provided by the present invention. The computer program product should be understood as a software product that mainly implements its solution through a computer program, such as a program product integrated in the cloud or a software library.

[0110] The present invention also discloses an electronic device. In one embodiment, the electronic device includes at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the image classification model privacy protection method based on asymmetric differential privacy provided by the present invention.

[0111] like Figure 3 , which is a schematic diagram of the structure of an electronic device of an image classification model privacy protection method based on asymmetric differential privacy provided by an embodiment of the present invention. The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as an image classification model privacy protection method program based on asymmetric differential privacy.

[0112] Among them, the processor 10 can be composed of an integrated circuit in some embodiments, for example, it can be composed of a single packaged integrated circuit, or it can be composed of multiple integrated circuits with the same function or different functions, including one or more central processing units (CPU), microprocessors, digital processing chips, graphics processors and various control chips. The processor 10 is the control core (Control Unit) of the electronic device, which uses various interfaces and lines to connect the various components of the entire electronic device, and executes or executes the program or module stored in the memory 11 (for example, executes the privacy protection method of the image classification model based on asymmetric differential privacy, etc.), and calls the data stored in the memory 11 to execute various functions of the electronic device and process data.

[0113] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 11 may also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of the privacy protection method program of the image classification model based on asymmetric differential privacy, but also can be used to temporarily store data that has been output or is to be output.

[0114] The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0115] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0116] Figure 3 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 3 The structure shown does not constitute a limitation on the electronic device, and may include fewer or more components than shown, or combine certain components, or arrange the components differently. For example, although not shown, the electronic device may also include a power supply (such as a battery) for supplying power to each component. Preferably, the power supply may be logically connected to at least one processor 10 through a power management device, so that functions such as charging management, discharging management, and power consumption management are implemented through the power management device. The power supply may also include any components such as one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0117] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited by this structure.

[0118] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0119] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", "an implementation", "a preferred implementation" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0120] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. The privacy protection method of image classification model based on asymmetric differential privacy is characterized by: include: Train multiple teacher classification models using private image sets; Use multiple trained teacher classification models to make predictions on unlabeled public image sets simultaneously; Count the prediction results of multiple teacher classification models for each public image to obtain the prediction record of each public image, and the prediction records of all public images constitute a prediction record set; perform frequency analysis on the prediction record of each public image to obtain the category frequency distribution of each public image; Extracting a neighborhood record set of the prediction record of each public image under a preset insensitive category from the prediction record set; The asymmetric sensitivity is calculated using the prediction record and the neighborhood record set of each public image according to the query function, and the noise distribution of each public image is obtained based on the asymmetric sensitivity; Adding noise that matches the noise distribution of each public image to the frequency of the class frequency distribution of each public image; Select the category with the largest frequency in the category frequency distribution after adding noise to the public image as the pseudo label of the public image; Using the public image set and pseudo labels of the public images to train the student classification model, the trained student classification model is used as the image classification model; The step of extracting a neighborhood record set of the prediction record of each public image under a preset insensitive category from the prediction record set includes: Assume that the prediction record of the public image g is: r = (r1, r2, ... r n ,…,r N ), where r n represents the predicted category of the nth teacher classification model for the public image g, n∈[1,N], r n ∈{K1,…,K M }, K1, …, K M represents M different categories, M≥2; suppose there are M′ preset insensitive categories in M ​​categories, M′<M; N represents the number of teacher classification models; Determine a model subset consisting of teacher classification models corresponding to all prediction categories belonging to insensitive categories in the prediction record of the public image g. If the prediction category corresponding to the model subset in another prediction record in the prediction record set is exactly the same as the prediction category corresponding to the model subset in the prediction record of the public image g, then the other prediction record is considered to be a neighborhood record of the prediction record of the public image g, and all neighborhood records of the prediction record of the public image g constitute a neighborhood record set. The step of calculating the asymmetric sensitivity using the prediction record and the neighborhood record set of each public image according to the query function includes: Using the query function to perform query calculation on the prediction record of the public image to obtain a first query result; Utilize the query function to query and calculate the neighbor records in the neighbor record set respectively to obtain multiple neighbor query results; Calculate the absolute values ​​of differences between the plurality of neighborhood query results and the first query result, obtain a plurality of absolute values ​​of the differences, and select the absolute value of the difference with the largest value from the plurality of absolute values ​​of the differences as the asymmetric sensitivity; Wherein, obtaining the noise distribution of each public image based on asymmetric sensitivity includes: If the query function satisfies f(D′)-f(D)≥0, the positive noise distribution of the public image is: If the query function satisfies f(D′)-f(D)<0, the negative noise distribution of the public image is: Among them, the auxiliary parameters Δ af represents asymmetric sensitivity, ε represents the privacy budget, λ represents the noise to be added, f(D) represents the first query result obtained by querying the predicted record of the public image using the query function f(·), and f(D′) represents the neighborhood query result obtained by querying any neighborhood record D′ in the neighborhood record set using the query function f(·).

2. The privacy protection method for an image classification model based on asymmetric differential privacy as claimed in claim 1, characterized in that: In the step of adding noise that conforms to the noise distribution of each public image to the frequency of the category frequency distribution of each public image, the noise is added according to the following formula: in, Represents the category K after adding noise to the public image g m The frequency of m (g) represents the category K of the public image g before adding noise m The frequency of; 1≤m≤M, M represents the number of categories.

3. The privacy protection method for an image classification model based on asymmetric differential privacy as claimed in claim 1 or 2, characterized in that: In the step of training multiple teacher classification models using the private image set, each teacher classification model is trained based on the private image set, and the goal of the training is to minimize the first symmetric cross entropy loss function, and the first symmetric cross entropy loss function is: <h2 style=";text-align:left;direction:ltr">l<h2 style=";text-align:left;direction:ltr"> sl <h2 style=";text-align:left;direction:ltr"> (al)<h2 style=";text-align:left;direction:ltr"> ce <h2 style=";text-align:left;direction:ltr"> +bl<h2 style=";text-align:left;direction:ltr"> rce <h2 style=";text-align:left;direction:ltr"> ; Where a represents the first hyperparameter, b represents the second hyperparameter; l ce represents the difference loss, l rce represents symmetric loss; Difference loss ce The calculation formula is: Symmetric loss rce The calculation formula is: p(K m |x) indicates that the private image x is predicted by the teacher classification model as the mth category K m The probability distribution of q(K m |x) indicates that the private image x is in the mth category K m The true label probability distribution of m is a positive integer, and 1≤m≤M, where M represents the number of categories.

4. The privacy protection method for an image classification model based on asymmetric differential privacy as claimed in claim 1 or 2, characterized in that: In the step of training the student classification model using the public image set and the pseudo labels of the public images, the student classification model is trained with the goal of minimizing the second symmetric cross entropy loss function, and the second symmetric cross entropy loss function is: <h2 style=";text-align:left;direction:ltr">l2=a2l<h2 style=";text-align:left;direction:ltr"> ce2 <h2 style=";text-align:left;direction:ltr"> +b2l<h2 style=";text-align:left;direction:ltr"> rce2 <h2 style=";text-align:left;direction:ltr"> ; Wherein, a2 represents the third hyperparameter, b2 represents the fourth hyperparameter; l ce2 represents the difference loss of the student classification model, l rce2 represents the symmetric loss of the student classification model; Student classification model difference loss l ce2 The calculation formula is: Symmetric loss of student classification model l rce2 The calculation formula is: p2(K m |g) indicates that the public image g is predicted by the student classification model as the mth category K m The probability distribution of q2(K m |g) means the public image g is in the mth category K m The pseudo-label probability distribution of m is a positive integer, and 1≤m≤M, where M represents the number of categories.

5. An image classification model privacy protection device based on asymmetric differential privacy, used to implement the image classification model privacy protection method based on asymmetric differential privacy according to any one of claims 1 to 4, characterized in that: include: The teacher classification model training module uses private image sets to train multiple teacher classification models; The prediction module uses multiple trained teacher classification models to simultaneously predict the unlabeled public image set; The statistical module counts the prediction results of multiple teacher classification models for each public image, obtains the prediction record of each public image, and the prediction records of all public images constitute a prediction record set; performs frequency analysis on the prediction record of each public image to obtain the category frequency distribution of each public image; The neighborhood record acquisition module extracts the neighborhood record set of each public image's prediction record under the preset insensitive category from the prediction record set, including: Assume that the prediction record of the public image g is: r = (r1, r2, ... r n ,…,r N ), where r n represents the predicted category of the nth teacher classification model for the public image g, n∈[1,N], r n ∈{K1,…,K M }, K1, …, K M represents M different categories, M≥2; suppose there are M′ preset insensitive categories in M ​​categories, M′<M; N represents the number of teacher classification models; Determine a model subset consisting of teacher classification models corresponding to all prediction categories belonging to insensitive categories in the prediction record of the public image g. If the prediction category corresponding to the model subset in another prediction record in the prediction record set is exactly the same as the prediction category corresponding to the model subset in the prediction record of the public image g, then the other prediction record is considered to be a neighborhood record of the prediction record of the public image g, and all neighborhood records of the prediction record of the public image g constitute a neighborhood record set. A noise distribution generation module calculates asymmetric sensitivity using the prediction record and neighborhood record set of each public image according to a query function, and obtains the noise distribution of each public image based on the asymmetric sensitivity; wherein the calculation of asymmetric sensitivity using the prediction record and neighborhood record set of each public image according to the query function includes: Using the query function to perform query calculation on the prediction record of the public image to obtain a first query result; Utilize the query function to query and calculate the neighbor records in the neighbor record set respectively to obtain multiple neighbor query results; Calculate the absolute values ​​of differences between the plurality of neighborhood query results and the first query result, obtain a plurality of absolute values ​​of the differences, and select the absolute value of the difference with the largest value from the plurality of absolute values ​​of the differences as the asymmetric sensitivity; Wherein, obtaining the noise distribution of each public image based on asymmetric sensitivity includes: If the query function satisfies f(D′)-f(D)≥0, the positive noise distribution of the public image is: If the query function satisfies f(D′)-f(D)<0, the negative noise distribution of the public image is: Among them, the auxiliary parameters Δ af represents asymmetric sensitivity, ε represents privacy budget, λ represents noise to be added, f(D) represents the first query result obtained by querying the predicted record of the public image using the query function f(·), and f(D′) represents the neighborhood query result obtained by querying any neighborhood record D′ in the neighborhood record set using the query function f(·); A noise adding module, adding noise that conforms to the noise distribution of each public image to the frequency of the category frequency distribution of each public image; A pseudo-label generation module selects the category with the largest frequency in the category frequency distribution after adding noise to the public image as the pseudo-label of the public image; The student classification model training module uses the public image set and pseudo labels of the public images to train the student classification model, and uses the trained student classification model as the image classification model.

6. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, A memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the privacy protection method for the image classification model based on asymmetric differential privacy as described in any one of claims 1 to 4.

Images