Privacy protection image classification model training method, image classification method and device
Through teacher-student model training method and model reprogramming technology, the problem of poor results of existing privacy-protected image classification models is solved, significantly improving the training effect and efficiency of the model, and reducing the difficulty of application.
Patent Information
- Application Number
- CN202411728999.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-28
- Publication Date
- 2025-03-18
- Estimated Expiration
- 2044-11-28
AI Technical Summary
The existing privacy-protected image classification model is not effective during the training process, which affects the accuracy of the final classification. The use of pre-trained models will cause the model parameters to become larger and increase the application difficulty.
The teacher-student model training method is adopted to train the teacher model on the private data, and then vote on the non-private data. By disturbing the voting results, the pre-trained model is used to replace the original teacher model, transfer learning training is performed, and the teacher model is optimized through model reprogramming technology.
It significantly reduces the impact of privacy protection on model utility, improves the training effect of teacher models, reduces the amount of training parameters, reduces the training overhead, and improves transfer learning ability in cross-domain scenarios.
Smart Images

Figure CN119649121A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of data security and artificial intelligence, and in particular relates to a training method for a privacy-preserving image classification model, an image classification method and a device. Background Art
[0002] Deep learning models have been proven to have the risk of leaking training data when applied. Privacy-preserving deep learning models are one of the specific technologies in the field of privacy computing. They can be used to open models while protecting the privacy of the original data of deep learning model training. Privacy-preserving image classification models are widely used in medical image analysis, face recognition and other fields. Most of the existing privacy-preserving image classification models use conventional non-privacy model structures and add noise to the training update gradient to protect the privacy of the model, which greatly reduces the effect of the image classification model. At present, the transfer training model based on the pre-trained model is the best type of privacy image classification model. They use the model pre-trained on public non-privacy data as the basis, perform transfer learning on the privacy data set, and add noise to the training gradient to ensure the privacy of the model. However, the premise of using the transfer model is that the gap between public data and privacy data cannot be too large, otherwise the image classification effect of the privacy model will decrease. At the same time, the use of pre-trained models will cause the model parameters to become larger, which increases the difficulty of applying privacy models to a certain extent. Summary of the invention
[0003] The purpose of the present invention is to provide a privacy-preserving image classification model training method, an image classification method and a device, so as to solve the problem that the existing privacy-preserving image classification training model has poor effect and affects the classification accuracy of the final privacy-preserving image classification model.
[0004] The present invention is achieved by adopting the following technical solutions:
[0005] A method for training a privacy-preserving image classification model, wherein the privacy-preserving image classification model includes a teacher image classification sub-model and a student image classification sub-model, the method comprising:
[0006] Obtain a private image set for training the teacher image classification sub-model and the student image classification sub-model, wherein the private image set includes two parts: labeled images and category labels corresponding to the labeled images, and unlabeled images;
[0007] Get the pre-trained image classification model for the teacher image classification sub-model;
[0008] The pre-trained image classification model is reprogrammed and optimized to obtain a teacher image classification sub-model to be trained, and the labeled images and the category label parts corresponding to the labeled images in the private image set are evenly divided into N parts, which is the number of teacher models; the teacher image classification sub-model to be trained is independently trained on the private image dataset of the number of divided teacher models, and N trained teacher image classification sub-models are obtained;
[0009] The unlabeled images in the private image set are respectively input to N trained teacher image classification sub-models to obtain the label prediction of each teacher image classification sub-model for the unlabeled images; the N predicted labels of each unlabeled image are aggregated, and noise perturbation is added to the predicted label vote. If the result is less than the selection threshold, the prediction result of the unlabeled image is discarded. Otherwise, noise perturbation is added to the predicted label vote, and the label with the most votes is selected as the label of the image. The obtained label result is added to the second part of the private image set;
[0010] The second part of the privacy image set is input into the student image classification sub-model for training to obtain the privacy-preserving image classification model.
[0011] A further improvement of the present invention is that when obtaining a private image set for training a teacher image classification sub-model and a student image classification sub-model, the private image set is a target data set of a privacy-preserving image classification model, and includes two parts: a labeled private image data set and an unlabeled private image data set; the labeled private image data set contains private images and corresponding annotated labels, and the unlabeled private image data set only contains private images.
[0012] A further improvement of the present invention is that the pre-trained image classification model of the teacher image classification sub-model is obtained by using a public non-privacy data set to perform image classification training on the initialized image classification model, wherein the public non-privacy data set refers to an image classification data set that is publicly used and does not involve privacy issues, and the initialized image classification model refers to an untrained image classification model.
[0013] A further improvement of the present invention is that the model reprogramming optimization packages the pre-trained image classification model of the input teacher image classification sub-model, freezes all model parameters and then adds additional training parameters to obtain the teacher image classification sub-model to be trained, including:
[0014] Freeze input model parameters: Freeze all trainable parameters of the pre-trained image classification model of the input teacher image classification sub-model;
[0015] Add trainable noise parameters to the model input: Add trainable noise parameters around the input image to the model input;
[0016] Add a label classification mapping network to the output of the model: Add a label classification mapping network from the category probability of the pre-trained image classification model of the teacher image classification sub-model to the category probability of the private image set to the output of the model.
[0017] A further improvement of the present invention is that the teacher image classification sub-model to be trained is independently trained on the private image dataset divided into the number of teacher models, including:
[0018] The teacher image classification sub-model to be trained is copied N times, and independently trained on the corresponding N independently divided private image datasets. Different teacher image classification sub-models to be trained are independent of each other and do not cross-share any data information.
[0019] A further improvement of the present invention is that the second part of the private image set is input into the student image classification sub-model for training, including:
[0020] The student image classification sub-model is trained by the semi-supervised learning algorithm image classification model training algorithm and the input training data; the input training data is the second part of the private image set, which includes two parts: a labeled image dataset with labels obtained by voting and perturbation by the teacher model, and a remaining unlabeled image dataset that has not been voted by the teacher model or abandoned by the teacher vote; the labeled image dataset is used as the labeled data input in the semi-supervised image classification model training algorithm, and the unlabeled image classification dataset is used as the unlabeled data input in the semi-supervised image classification model training algorithm.
[0021] An image classification method, comprising:
[0022] Obtain image data to be classified;
[0023] The images to be classified are classified and identified by a privacy-preserving image classification model, and the classification of the images to be classified is obtained under the premise of privacy protection; the privacy-preserving image classification model is obtained by the training method of the privacy-preserving image classification model.
[0024] A privacy-preserving image classification model training device, the privacy-preserving image classification model includes a teacher image classification sub-model and a student image classification sub-model, the training device includes:
[0025] An acquisition unit is used to acquire an image set for training a privacy-preserving image classification model, wherein the image set includes labeled images, unlabeled images, and category labels corresponding to the labeled images; and acquire a pre-trained image classification model for a teacher image classification sub-model;
[0026] The teacher model training unit uses model reprogramming to modify the pre-trained image classification model as the teacher image classification sub-model to be trained, evenly divides the labeled images and the category labels corresponding to the labeled images in the image data set into N parts, and independently inputs them into N identical teacher image classification sub-models to be trained for training, thereby obtaining N teacher image classification sub-models;
[0027] The teacher prediction voting unit uses the N teacher image classification sub-models obtained by the teacher model unit to classify and identify the unlabeled images respectively, and obtain N classification prediction voting results for each unlabeled image; noise is added to the voting results of each unlabeled image, and the results that do not meet the selection threshold after adding noise are eliminated, and the remaining results are added with noise to obtain the predicted label results of some unlabeled images;
[0028] The student model training unit is trained by inputting the predicted label results of the unlabeled images and the partial unlabeled images into the initial student image classification model.
[0029] An image classification device, comprising:
[0030] A receiving unit, used to obtain image data to be classified,
[0031] The prediction unit is used to classify and identify the image to be classified through the privacy protection image classification model, and obtain the classification of the image to be classified under the premise of privacy protection; the privacy protection image classification model is obtained through the training method of the privacy protection image classification model.
[0032] An electronic device, comprising:
[0033] A memory for storing executable instructions;
[0034] The processor is used to implement the training method of the privacy-preserving image classification model when executing the executable instructions stored in the memory.
[0035] The present invention has at least the following beneficial technical effects:
[0036] The present invention provides a privacy-efficient image classification model training method, which utilizes a teacher-student model training method to train a teacher model on private data, and then votes on non-private data. By perturbing the voting results and selectively receiving them according to the concentration of the voting results and a set selection threshold, the impact of privacy protection on the model utility can be significantly reduced.
[0037] Furthermore, by replacing the original teacher model with a pre-trained model, during the teacher model training phase, the model pre-trained on public data is used to perform transfer learning training on private data. The knowledge learned on public data is used to significantly improve the training effect of the teacher model.
[0038] Furthermore, based on the model reprogramming technology, the pre-trained teacher model is modified, which greatly reduces the number of training parameters of the teacher model, thereby reducing the training overhead. In addition, the use of model reprogramming can significantly improve the transfer learning ability of the pre-trained model in cross-domain scenarios, thereby improving the effect of the teacher model. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 Schematic diagram of the training framework of the present invention.
[0040] Figure 2 Schematic diagram of model reprogramming of the present invention.
[0041] Figure 3 It is a flow chart of the disturbance voting method with a selection threshold of the present invention.
[0042] Figure 4 This is a comparison chart of the classification accuracy of the present invention and the comparison algorithm under different privacy budget conditions.
[0043] Figure 5 This is a comparison chart of the classification accuracy of the present invention and the comparison algorithm when there is a large gap between the pre-training data and the privacy data.
[0044] Figure 6 This is a structural block diagram of a privacy-preserving image classification model training device of the present invention.
[0045] Figure 7 The figure is a structural block diagram of an image classification device of the present invention. DETAILED DESCRIPTION
[0046] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to be able to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0047] The present invention provides a method for training a privacy-preserving image classification model, wherein the privacy-preserving image classification model includes a teacher image classification sub-model and a student image classification sub-model, and the method includes:
[0048] Obtain a private image set for training the teacher image classification sub-model and the student image classification sub-model, wherein the private image set includes two parts: labeled images and category labels corresponding to the labeled images, and unlabeled images;
[0049] Get the pre-trained image classification model for the teacher image classification sub-model;
[0050] The pre-trained image classification model is reprogrammed and optimized to obtain a teacher image classification sub-model to be trained, and the labeled images and the category label parts corresponding to the labeled images in the private image set are evenly divided into N parts, which is the number of teacher models; the teacher image classification sub-model to be trained is independently trained on the private image dataset of the number of divided teacher models, and N trained teacher image classification sub-models are obtained;
[0051] The unlabeled images in the private image set are respectively input to N trained teacher image classification sub-models to obtain the label prediction of each teacher image classification sub-model for the unlabeled images; the N predicted labels of each unlabeled image are aggregated, and noise perturbation is added to the predicted label vote. If the result is less than the selection threshold, the prediction result of the unlabeled image is discarded. Otherwise, noise perturbation is added to the predicted label vote, and the label with the most votes is selected as the label of the image. The obtained label result is added to the second part of the private image set;
[0052] The second part of the privacy image set is input into the student image classification sub-model for training to obtain the privacy-preserving image classification model.
[0053] When the acquisition is used to acquire a private image set for training the teacher image classification sub-model and the student image classification sub-model, the private image set is the target data set of the privacy-preserving image classification model, and includes two parts: a labeled private image data set and an unlabeled private image data set. The labeled private image data set contains private images and corresponding annotated labels, and the unlabeled private image data set only contains private images.
[0054] The pre-trained image classification model of the teacher image classification sub-model is obtained by performing image classification training on the initialized image classification model using a public non-privacy dataset; wherein the public non-privacy dataset refers to an image classification dataset that is publicly used and does not involve privacy issues, and the initialized image classification model refers to an untrained image classification model.
[0055] The model reprogramming optimization packages the pre-trained image classification model of the input teacher image classification sub-model, freezes all model parameters and then adds additional training parameters to obtain the teacher image classification sub-model to be trained, including:
[0056] Freeze input model parameters: Freeze all trainable parameters of the pre-trained image classification model of the input teacher image classification sub-model;
[0057] Add trainable noise parameters to the model input: Add trainable noise parameters around the input image to the model input;
[0058] Add a label classification mapping network to the output of the model: Add a label classification mapping network from the category probability of the pre-trained image classification model of the teacher image classification sub-model to the category probability of the private image set to the output of the model.
[0059] The teacher image classification sub-model to be trained is independently trained on the private image dataset divided into the number of teacher models, including:
[0060] The teacher image classification sub-model to be trained is copied N times, and independently trained on the corresponding N independently divided private image datasets. Different teacher image classification sub-models to be trained are independent of each other and do not cross-share any data information.
[0061] The step of inputting the second part of the private image set into the student image classification sub-model for training includes:
[0062] The student image classification sub-model is trained by the semi-supervised learning algorithm image classification model training algorithm and the input training data; the input training data is the second part of the private image set, which includes two parts: a labeled image dataset with labels obtained by voting and perturbation by the teacher model, and a remaining unlabeled image dataset that has not been voted by the teacher model or abandoned by the teacher vote; the labeled image dataset is used as the labeled data input in the semi-supervised image classification model training algorithm, and the unlabeled image classification dataset is used as the unlabeled data input in the semi-supervised image classification model training algorithm.
[0063] Example 1
[0064] Reference Figure 1 The present invention provides a privacy-efficient image classification model training method, which specifically includes the following steps:
[0065] Step 1: Teacher model training phase: The model training party confirms the private data D that can be used privacy , the number of teacher models N, and the private data D priacy The non-repetitive ones are equally divided into N training data sets {D1, D2, ... D N-1,D N}. For the pre-trained image classification model M source , using model reprogramming technology to modify M source And it is used as the teacher model, and N teacher models are repeatedly constructed, each teacher model is trained on its corresponding training data set.
[0066] Step 2: Teacher model voting and labeling phase, which includes the following steps:
[0067] Step 2.1: The model training party collects unlabeled non-private training data D public , and set the privacy check Gaussian noise standard deviation σ1, privacy check threshold T, voting perturbation Gaussian noise standard deviation σ2, query number N query .
[0068] Step 2.2: From non-private training data D public Number of extractions N query Unlabeled non-private samples
[0069] Step 2.3: For each sample x i Distribute to N trained teacher models {T1,T2,…T N-1 ,T N} Perform label prediction and get each sample x i The predicted label set
[0070] Step 2.4: Collect N query The predicted label set of unlabeled non-private samples For each predicted label L i Use the perturbation voting method with a selection threshold to perform private voting on the teacher's voting prediction labels to obtain samples Corresponding privacy labels
[0071] Step 2.5. From the Privacy tab Noise Select the samples with the corresponding label category of -1 and combine them with other unlabeled non-privacy samples to form an unlabeled dataset D unlabelled , and change the remaining privacy labels to L Noise and the corresponding samples form a labeled dataset D lablled .
[0072] Step 2.6: Calculate and generate the privacy label L Noise The privacy budget ε consumed.
[0073] Step 3: Student model training phase: The model training party obtains the training data D through step 3 public Part label Lnoise , D public According to whether the label is obtained, it is divided into D unlabelled and D labelled , using a semi-supervised learning algorithm, using D unlabelled and D labelled The data is used to train the student model and released.
[0074] refer to Figure 2 For the teacher model, the present invention uses the model reprogramming technology to modify the teacher model by freezing the pre-trained model M source training parameters, and in M source Before adding size H source ×W source -H target ×W target The trainable noise in M source The output Logits are added to the classification mapping network. The specific structure is as follows:
[0075] Trainable noise: Set the input size to H privacy ×W privacy Zoom in on the private image and zoom to H target ×W target . And add a size of H outside the scaled image source ×W source -H target ×W target The trainable noise is H source ×W source The picture is input into the pre-trained model M source .
[0076] Classification Mapping Network: M source The original output C source After the Logits, a trainable fully connected neural network is added to convert C source Logits are remapped to C privacy Logits and serve as the output of model reprogramming.
[0077] refer to Figure 3 , for the perturbation voting method with selection threshold, the following steps are performed:
[0078] Step 1: Get sample x i The predicted label L i gather
[0079] Step 2: Predict labels Statistics get C privacy Number of votes in each category And from V i Get the category with the most votes V votemax
[0080] Step 3: V votemax Plus size is Gaussian noise, and check Is it greater than or equal to the privacy check threshold T: If the check fails, return L i Privacy label If the current voting result does not meet the privacy check threshold T, no privacy label will be generated. If the check is successful, proceed to step 4.
[0081] Step 4: Target For each class in , add Gaussian noise from Select the category C with the largest number after adding noise votemax As a sample x i Tags, return
[0082] Example 2
[0083] In the description of the present invention, it is necessary to understand that "there is a large gap between public data and private data" means that there is a large gap in the data distribution between public data and private data, and "Logits" refers to the linear output of the model for each category in a multi-classification task, which has not yet been processed by activation functions such as softmax or sigmoid.
[0084] refer to Figure 4 In order to test the performance of this method under different privacy budgets and compare it with the three most advanced privacy image classification algorithms, this method uses ImageNet as the pre-training dataset and CIFAR-10 as the privacy dataset, and the evaluation indicator is classification accuracy. The results are shown in Figure 4 As shown in the figure, it can be seen that when the total privacy budget is 2, the classification accuracy of all algorithms is better than that when the privacy budget is 1, and the classification accuracy of this method is significantly better than the three comparison algorithms when the total privacy budget is 1 and 2, showing better performance.
[0085] refer to Figure 5 In order to test the impact of model reprogramming on the performance of this method, ImageNet is selected as the pre-training dataset, and SVHN, PathMNIST and BloodMNIST, three datasets with large gaps with the pre-training ImageNet dataset, are selected as privacy datasets. The privacy transfer learning training method is used as the comparison method, and the classification accuracy is used as the evaluation indicator. The results are shown in Figure 5As shown in the figure, it can be seen that under the same total privacy budget conditions, for three different data sets, the classification accuracy of this method is much higher than that of other methods, and it still has good performance when there is a large gap between the pre-training data set and the privacy data set.
[0086] The present invention provides an image classification method, comprising:
[0087] Obtain image data to be classified;
[0088] The images to be classified are classified and identified by a privacy-preserving image classification model, and the classification of the images to be classified is obtained under the premise of privacy protection; the privacy-preserving image classification model is obtained by the training method of the privacy-preserving image classification model.
[0089] like Figure 6 As shown, the present invention provides a privacy protection image classification model training device, the privacy protection image classification model includes a teacher image classification sub-model and a student image classification sub-model, and the training device includes:
[0090] An acquisition unit is used to acquire an image set for training a privacy-preserving image classification model, wherein the image set includes labeled images, unlabeled images, and category labels corresponding to the labeled images; and acquire a pre-trained image classification model for a teacher image classification sub-model;
[0091] The teacher model training unit uses model reprogramming to modify the pre-trained image classification model as the teacher image classification sub-model to be trained, evenly divides the labeled images and the category labels corresponding to the labeled images in the image data set into N parts, and independently inputs them into N identical teacher image classification sub-models to be trained for training, thereby obtaining N teacher image classification sub-models;
[0092] The teacher prediction voting unit uses the N teacher image classification sub-models obtained by the teacher model unit to classify and identify the unlabeled images respectively, and obtain N classification prediction voting results for each unlabeled image; noise is added to the voting results of each unlabeled image, and the results that do not meet the selection threshold after adding noise are eliminated, and the remaining results are added with noise to obtain the predicted label results of some unlabeled images;
[0093] The student model training unit is trained by inputting the predicted label results of the unlabeled images and the partial unlabeled images into the initial student image classification model.
[0094] like Figure 7 As shown, the present invention provides an image classification device, comprising:
[0095] A receiving unit, used to obtain image data to be classified,
[0096] A prediction unit is used to classify and identify the image to be classified through a privacy-preserving image classification model, and obtain the classification of the image to be classified under the premise of privacy protection; the privacy-preserving image classification model is obtained through the training method of the privacy-preserving image classification model described in any one of claims 1-6.
[0097] An electronic device provided by the present invention comprises: a memory for storing executable instructions; and a processor for implementing the training method of the privacy-preserving image classification model when executing the executable instructions stored in the memory.
[0098] An embodiment of the present invention further provides a storage medium storing executable instructions, wherein executable instructions are stored therein. When the executable instructions are executed by a processor, the processor will be caused to execute the artificial intelligence-based image classification method provided by an embodiment of the present invention.
[0099] All or part of the steps of the embodiment can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, random access memory (RAM), read-only memory (ROM), magnetic disk or optical disk, etc. Various media that can store program codes.
[0100] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, RAM, ROM, magnetic disks or optical disks.
[0101] Although the present invention has been described in detail above with general descriptions and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements may be made thereto based on the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all fall within the scope of protection claimed by the present invention.
Claims
1. A method for training a privacy-preserving image classification model, characterized in that: The privacy-preserving image classification model includes a teacher image classification sub-model and a student image classification sub-model. The method includes: Obtain a private image set for training the teacher image classification sub-model and the student image classification sub-model, wherein the private image set includes two parts: labeled images and category labels corresponding to the labeled images, and unlabeled images; Get the pre-trained image classification model for the teacher image classification sub-model; The pre-trained image classification model is reprogrammed and optimized to obtain a teacher image classification sub-model to be trained, and the labeled images and the category label parts corresponding to the labeled images in the private image set are evenly divided into N parts, which is the number of teacher models; the teacher image classification sub-model to be trained is independently trained on the private image dataset of the number of divided teacher models, and N trained teacher image classification sub-models are obtained; The unlabeled images in the private image set are respectively input to N trained teacher image classification sub-models to obtain the label prediction of each teacher image classification sub-model for the unlabeled images; the N predicted labels of each unlabeled image are aggregated, and noise perturbation is added to the predicted label vote. If the result is less than the selection threshold, the prediction result of the unlabeled image is discarded. Otherwise, noise perturbation is added to the predicted label vote, and the label with the most votes is selected as the label of the image. The obtained label result is added to the second part of the private image set; The second part of the privacy image set is input into the student image classification sub-model for training to obtain the privacy-preserving image classification model.
2. The method for training a privacy-preserving image classification model according to claim 1, characterized in that: When obtaining a private image set for training the teacher image classification sub-model and the student image classification sub-model, the private image set is a target dataset of the privacy-preserving image classification model, including two parts: a labeled private image dataset and an unlabeled private image dataset; the labeled private image dataset contains private images and corresponding annotated labels, and the unlabeled private image dataset only contains private images.
3. The method for training a privacy-preserving image classification model according to claim 1, characterized in that: The pre-trained image classification model of the teacher image classification sub-model is obtained by using a public non-privacy dataset to perform image classification training on the initialized image classification model, wherein the public non-privacy dataset refers to an image classification dataset that is publicly used and does not involve privacy issues, and the initialized image classification model refers to an untrained image classification model.
4. The method for training a privacy-preserving image classification model according to claim 1, wherein: The model reprogramming optimization packages the pre-trained image classification model of the input teacher image classification sub-model, freezes all model parameters and adds additional training parameters to obtain the teacher image classification sub-model to be trained, including: Freeze input model parameters: Freeze all trainable parameters of the pre-trained image classification model of the input teacher image classification sub-model; Add trainable noise parameters to the model input: Add trainable noise parameters around the input image to the model input; Add a label classification mapping network to the output of the model: Add a label classification mapping network from the category probability of the pre-trained image classification model of the teacher image classification sub-model to the category probability of the private image set to the output of the model.
5. The method for training a privacy-preserving image classification model according to claim 1, wherein: The teacher image classification sub-model to be trained is trained independently on the private image dataset divided into the number of teacher models, including: The teacher image classification sub-model to be trained is copied N times, and independently trained on the corresponding N independently divided private image datasets. Different teacher image classification sub-models to be trained are independent of each other and do not cross-share any data information.
6. The method for training a privacy-preserving image classification model according to claim 1, characterized in that: The second part of the private image set is input into the student image classification sub-model for training, including: The student image classification sub-model is trained by the semi-supervised learning algorithm image classification model training algorithm and the input training data; the input training data is the second part of the private image set, which includes two parts: a labeled image dataset with labels obtained by voting and perturbation by the teacher model, and a remaining unlabeled image dataset that has not been voted by the teacher model or abandoned by the teacher vote; the labeled image dataset is used as the labeled data input in the semi-supervised image classification model training algorithm, and the unlabeled image classification dataset is used as the unlabeled data input in the semi-supervised image classification model training algorithm.
7. An image classification method, characterized in that: include: Obtain image data to be classified; The images to be classified are classified and identified by a privacy-preserving image classification model, and the classification of the images to be classified is obtained under the premise of privacy protection; the privacy-preserving image classification model is obtained by the training method of the privacy-preserving image classification model described in any one of claims 1-6.
8. A privacy-preserving image classification model training device, characterized in that: The privacy-preserving image classification model includes a teacher image classification sub-model and a student image classification sub-model, and the training device includes: An acquisition unit is used to acquire an image set for training a privacy-preserving image classification model, wherein the image set includes labeled images, unlabeled images, and category labels corresponding to the labeled images; and acquire a pre-trained image classification model for a teacher image classification sub-model; The teacher model training unit uses model reprogramming to modify the pre-trained image classification model as the teacher image classification sub-model to be trained, evenly divides the labeled images and the category labels corresponding to the labeled images in the image data set into N parts, and independently inputs them into N identical teacher image classification sub-models to be trained for training, thereby obtaining N teacher image classification sub-models; The teacher prediction voting unit uses the N teacher image classification sub-models obtained by the teacher model unit to classify and identify the unlabeled images respectively, and obtain N classification prediction voting results for each unlabeled image; noise is added to the voting results of each unlabeled image, and the results that do not meet the selection threshold after adding noise are eliminated, and the remaining results are added with noise to obtain the predicted label results of some unlabeled images; The student model training unit is trained by inputting the predicted label results of the unlabeled images and the partial unlabeled images into the initial student image classification model.
9. An image classification device, characterized in that: include: A receiving unit, used to obtain image data to be classified, A prediction unit is used to classify and identify the image to be classified through a privacy-preserving image classification model, and obtain the classification of the image to be classified under the premise of privacy protection; the privacy-preserving image classification model is obtained through the training method of the privacy-preserving image classification model described in any one of claims 1-6.
10. An electronic device, characterized in that: include: A memory for storing executable instructions; A processor is used to implement the training method of the privacy-preserving image classification model described in any one of claims 1 to 6 when executing the executable instructions stored in the memory.
Citation Information
Patent Citations
Privacy model training method and device based on small amount of public data
CN112199717A
Semi-supervised learning image classification optimization method and system based on pseudo labels
CN113989549A
CoVID-19 chest X-ray image classification learning-oriented training data privacy protection method
CN115482435A
Near-Zero-Cost Differentially Private Deep Learning with Teacher Ensembles
US20210089882A1