A distributed privacy protection method and system based on semi-supervised transfer learning

Abstract

The present invention discloses a distributed privacy protection method and system based on semi-supervised transfer learning, which relates to the field of medical big data technology. The method comprises: constructing a distributed model; training a teacher model in the distributed model with an image data set labeled with disease categories as input to obtain a trained teacher model; inputting an image data set without annotating disease categories into the trained teacher model to obtain a disease category data set; encrypting disease category data sets output by multiple trained teacher models to determine a reference disease category data set; training a student model in the distributed model with an image data set without annotating disease categories as input and a reference disease category data set as output to obtain a trained student model; inputting image data of the disease category to be diagnosed into the trained student model to obtain the disease category of the disease to be diagnosed. The present invention can enhance the privacy protection performance of distributed technology.

Description

Technical Field

[0001] The present invention relates to the field of medical big data technology, and in particular to a distributed privacy protection method and system based on semi-supervised transfer learning. Background Art

[0002] The uneven distribution of medical resources and the difficulty of cross-regional medical treatment have always been the pain points of the development of the medical industry. Medical resources in hospitals of different regions and levels cannot be homogenized. Large hospitals are overcrowded and primary medical and health institutions are deserted. In the era of medical big data, the detection images of medical equipment can be transmitted to large hospitals through primary medical and health institutions. Large hospitals can diagnose diseases based on the detected image data to balance medical resources, but they face the following problems: First, the problem of data islands. Due to privacy issues, data between institutions will not be directly shared; second, a single computing node can no longer meet the needs; the sharing of medical resources by multiple large hospitals can more comprehensively diagnose diseases in the detection images and ensure the accuracy and comprehensiveness of the diagnosis. Therefore, distributed machine learning methods are usually used in actual data application scenarios. While using a large amount of data information to train models, how to effectively protect user data security and privacy needs to be paid attention to; third, there is also a problem of insufficient annotations under big data, which makes it impossible to effectively train models. Summary of the invention

[0003] The purpose of the present invention is to provide a distributed privacy protection method and system based on semi-supervised transfer learning, which can enhance the privacy protection performance of distributed technology.

[0004] To achieve the above object, the present invention provides the following solutions:

[0005] A distributed privacy protection method based on semi-supervised transfer learning, the method comprising:

[0006] Constructing a distributed model; the distributed model includes a student model and multiple teacher models;

[0007] Taking the image dataset with disease categories labeled as input, training the teacher model in the distributed model to obtain a trained teacher model; the output of the teacher model is the disease category;

[0008] Inputting an image dataset without disease category labels into the trained teacher model to obtain a disease category dataset;

[0009] Encrypting the disease category data sets output by the plurality of trained teacher models to determine a reference disease category data set;

[0010] Taking the image dataset without disease category labeling as input and the reference disease category dataset as output, training the student model in the distributed model to obtain a trained student model;

[0011] The image data of the disease category to be diagnosed is input into the trained student model to obtain the disease category of the disease to be diagnosed.

[0012] Optionally, resnet-50 is used as the teacher model in the distributed model; the number of categories in the classification layer of the teacher model is the number of disease categories in the image dataset with labeled disease categories.

[0013] Optionally, encrypting the disease category data sets output by the plurality of trained teacher models to determine a reference disease category data set specifically includes:

[0014] Noising the disease category data sets output by the multiple trained teacher models to obtain multiple predicted classification data sets;

[0015] A voting mechanism is applied to the multiple predicted classification data sets to determine a reference disease category data set.

[0016] Optionally, noise is added by a random response algorithm.

[0017] Optionally, the student model in the distributed model applies a loss function to denoise the reference disease category dataset; the loss function is a symmetric cross entropy function or a label smoothing regularization method.

[0018] A distributed privacy protection system based on semi-supervised transfer learning, the system comprising:

[0019] A construction module, used to construct a distributed model; the distributed model includes a student model and multiple teacher models;

[0020] The first training module is used to train the teacher model in the distributed model using the image dataset with the disease category labeled as input to obtain a trained teacher model; the output of the teacher model is the disease category;

[0021] A disease category data set determination module, used to input an image data set without disease category labeling into the trained teacher model to obtain a disease category data set;

[0022] A reference disease category data set determination module is used to encrypt the disease category data sets output by the plurality of trained teacher models to determine a reference disease category data set;

[0023] A second training module is used to train the student model in the distributed model using the image dataset without disease category labeling as input and the reference disease category dataset as output to obtain a trained student model;

[0024] The diagnosis result output module is used to input the image data of the disease category to be diagnosed into the trained student model to obtain the disease category of the disease to be diagnosed.

[0025] Optionally, resnet-50 is used as the teacher model in the distributed model; the number of categories of the classification layer of the teacher model is set to the number of disease categories in the image dataset with labeled disease categories.

[0026] Optionally, the reference disease category data set determination module includes:

[0027] A prediction classification data set determination submodule is used to add noise to the disease category data sets output by the multiple trained teacher models to obtain multiple prediction classification data sets;

[0028] The reference disease category data set determination submodule is used to apply a voting mechanism to the multiple prediction classification data sets to determine a reference disease category data set.

[0029] Optionally, noise is added by a random response algorithm.

[0030] Optionally, the student model in the distributed model applies a loss function to denoise the reference disease category dataset; the loss function is a symmetric cross entropy function or a label smoothing regularization method.

[0031] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0032] The present invention provides a distributed privacy protection method based on semi-supervised transfer learning, comprising: constructing a distributed model; the distributed model comprises a student end and multiple teacher ends; taking an image data set with labeled disease categories as input, training a teacher model in the distributed model to obtain a trained teacher model; the output of the teacher model is the disease category; inputting an image data set without labeled disease categories into the trained teacher model to obtain a disease category data set; encrypting the disease category data sets output by multiple trained teacher models to determine a reference disease category data set; taking an image data set without labeled disease categories as input and a reference disease category data set as output, training a student model in the distributed model to obtain a trained student model; inputting image data of the disease category to be diagnosed into the trained student model to obtain the disease category of the disease to be diagnosed. The present invention adopts a teacher-student model, sets multiple teacher ends, and each teacher end is trained by non-overlapping data. Multiple teacher ends can send migration knowledge to the student end at different times. After the neural network training of the student end is completed, if a new teacher adds knowledge, the student end can put the new teacher's knowledge into the original aggregated knowledge and re-aggregate it, so that attackers cannot obtain the private data of the teacher end by attacking the student end, thereby enhancing the privacy protection performance and allowing the teacher ends to be trained at different times. It has extremely high application value in real scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0034] Figure 1 A flow chart of the distributed privacy protection method based on semi-supervised transfer learning provided by the present invention;

[0035] Figure 2 A schematic diagram of the deep learning network structure provided by the present invention;

[0036] Figure 3 A schematic diagram of the structure of the teacher model and the student model provided by the present invention;

[0037] Figure 4 A schematic diagram of a distributed model based on semi-supervised transfer learning provided by the present invention;

[0038] Figure 5 Block diagram of the distributed privacy protection system based on semi-supervised transfer learning provided by the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] The purpose of the present invention is to provide a distributed privacy protection method and system based on semi-supervised transfer learning, which can enhance the privacy protection performance of distributed technology.

[0041] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] In the distributed model proposed by the present invention, there are multiple teacher terminals, each teacher model is trained with disjoint private data, and multiple teacher terminals can send migration knowledge to the student terminal at different times. After the student model training is completed, if a new teacher terminal adds new knowledge, the student terminal can put the knowledge added by the new teacher terminal into the original aggregated knowledge, re-aggregate, and train again to obtain a better distributed model. Specifically, the number of teacher terminals is generally an odd number.

[0043] like Figure 1 As shown, the present invention provides a distributed privacy protection method based on semi-supervised transfer learning, the method comprising:

[0044] Step S1: construct a distributed model; the distributed model includes a student model and multiple teacher models; specifically, Figure 2 As shown in Figure 2, the deep learning network uses resnet-50, including convolutional layers and classification layers; Figure 3 As shown, resnet-50 is used as the teacher model in the distributed model; the number of categories in the classification layer of the teacher model is the number of disease categories in the image data set with labeled disease categories. Further, resnet-50 is used as the student model in the distributed model; the number of categories in the classification layer of the student model is the number of disease categories in the image data set with labeled disease categories. In addition, the optimizer in the teacher model and the student model uses SGDM; the loss function in the teacher model is the cross entropy loss function; the loss function in the student model is the symmetric cross entropy loss function or the label smoothing regularization; that is, the loss function in the student model is selected from the two functions of the symmetric cross entropy loss function and the label smoothing regularization; and the present invention applies the pre-trained parameters of resnet-50 on ImageNet to reduce the difficulty of deep learning models.

[0045] Step S2: using the image dataset with labeled disease categories as input, training the teacher model in the distributed model to obtain a trained teacher model; the output of the teacher model is the disease category; the disease category is the label of the image dataset with labeled disease categories; specifically, when training the teacher model, selecting the number of teacher terminals participating in the training, that is, determining the number of teacher models participating in the training, training the teacher models participating in the training at the same time, and the data of the training set of each teacher terminal is disjoint.

[0046] Step S3: Input the image dataset without disease category labeling into the trained teacher model to obtain the disease category dataset; specifically, the student terminal sends the image dataset without disease category labeling to the trained teacher model. The image dataset without disease category labeling is an unlabeled migration dataset.

[0047] Step S4: encrypt the disease category data sets output by the multiple trained teacher models to determine a reference disease category data set.

[0048] S4 specifically includes:

[0049] Step S41: Noise the disease category data sets output by the multiple trained teacher models to obtain multiple predicted classification data sets; specifically, the noise is added by a random response algorithm. The noise is added to prevent privacy leakage. Multiple predicted classification data sets are sent to the student end.

[0050] Step S42: Apply a voting mechanism to the multiple prediction classification data sets to determine a reference disease category data set. Specifically, the student terminal aggregates the prediction classification data sets sent by multiple teacher terminals, and selects the prediction classification data set with the most votes as the reference disease category data set through a voting mechanism.

[0051] Step S5: Using the image dataset without disease category labeling as input and the reference disease category dataset as output, the student model in the distributed model is trained to obtain a trained student model. Specifically, since the data is denoised during teacher model training, noise reduction is required when the student model is applied to the data. The student model in the distributed model applies a loss function to denoise the reference disease category dataset; the loss function is a symmetric cross entropy function or a label smoothing regularization method.

[0052] Step S6: input the image data of the disease category to be diagnosed into the trained student model to obtain the disease category of the disease to be diagnosed.

[0053] The distributed model can better protect the privacy of the teacher model. After the student aggregates the migrated knowledge, external attackers cannot find the migration label of the specific teacher from the aggregated knowledge. As the number of teachers increases, the noise that each teacher can add also increases, which further strengthens the privacy protection of the teacher. In the worst case, the attacker obtains the knowledge migrated from a teacher from the student, which is just the one-hot label value after adding noise.

[0054] Figure 4 A schematic diagram of a distributed model based on semi-supervised transfer learning provided by the present invention, such as Figure 4 As shown, assuming n teacher terminals and one student terminal are learning:

[0055] ① The teacher side trains the teacher model based on the existing data as a training set, and stores the teacher model after the training is completed.

[0056] ②The student sends the unlabeled migration dataset to the teacher participating in the training.

[0057] ③ The teacher labels the training set and adds noise to the unlabeled migration data set to prevent privacy leakage. The noisy label value is called the predicted label and sent to the student end; the predicted label is also the predicted classification data set.

[0058] ④ The student side aggregates the prediction labels of multiple teacher sides, and selects the prediction label with the most votes as the final reference label value through a voting mechanism; the final reference label value is the reference disease category dataset.

[0059] ⑤The final reference label value is combined with the unlabeled migration set to form the training set of the student model and sent to the student end to train its own model.

[0060] The method of adding noise to the data set output by the teacher model on the teacher side specifically includes:

[0061] The present invention selects the local differential privacy algorithm - random response algorithm to protect the privacy of the teacher from being leaked. In the present invention, the combinatorial nature of differential privacy is not considered, and the classic privacy protection idea - random response (randomized response) is adopted to meet the concept of differential privacy. The random response algorithm focuses on the probability output after the disturbance of 0 and 1. For noise addition, the random response algorithm is selected in combination with asymmetric noise, and the label with the second largest probability value of the output layer is used as the flipped label (the probability of the flipped label after noise addition determines whether the correct label classification can be obtained in the prediction process).

[0062] The privacy parameter is ε. According to the random response definition, as shown in formula (1), ε is set as:

[0063]

[0064] Formula (1) is a monotonically increasing function. As p increases, ε also increases. Local differential privacy chooses p to be greater than one-half, that is, ε must be greater than 0. When ε is equal to 0, the query answer is completely random and the data availability is 0.

[0065]

[0066] As shown in formula (2), in the random response, the privacy parameter ε is greater than 0, and the positive probability p ranges from (0.5, 1), and the flip probability 1-p ranges from (0, 0.5). The correct label is output with probability p, and the flip label is output with probability 1-p.

[0067] η yk <1-η y (3)

[0068] In formula (3), η yk is the noise rate of the kth class, η y is the total noise rate, η y =1-p,

[0069] η yk <1-η y This means that the probability of the sample being mislabeled as any other class is less than its positive probability.

[0070] For n 1 is the total number of positive samples in the migration dataset, n 2 is the total number of negative samples. And the total noise rate 1-p is substituted into formula (3) to obtain:

[0071]

[0072] Simplified to the following formula

[0073]

[0074] Similarly, the negative sample noise rate and the total noise rate are substituted into:

[0075]

[0076] When equation (5) and equation (6) are satisfied at the same time, the method is robust.

[0077] Under the added noise in the present invention, the flip probability of the label is less than 50%. The random response returns a random probability flip. Under the condition of particularly strict privacy parameters, the single teacher model has a very small possibility that the flip probability of the label is greater than the positive probability. The multi-teacher model will reduce the very small probability index. Make the flip probability less than the positive probability, and avoid the wrong label of the image dominating.

[0078] In practical applications, the COVID medical data set is used as the training set of the teacher model. It is assumed that the positive CT image is 1 and the negative CT image is 0. Then, suppose a coin is used to give the answer, and this coin is a non-uniform coin, with a probability of p facing up and a probability of 1-p facing up. After tossing the coin, if the coin faces up, the true answer is given, otherwise, the wrong answer is given. Assuming that the proportion of positive images in the CT image is t, then, for the statistical proportion, the ratio of positive images to negative images is shown in formula (7):

[0079]

[0080] Of course, the real distribution is different from the noise statistical proportion, so the statistical results need to be corrected. The correction formula is shown in formula (8):

[0081]

[0082] Where n is the total number of samples in the data set, n 1 is the total number of positive samples.

[0083] Taking the logarithm of L, we get formula (9):

[0084] ln(L)=n 1 ln[t(2p-1)+(1-p)]+(nn 1 )ln[pt(2p-1)] (9)

[0085] Then, taking the derivative of t, we get formula (10):

[0086]

[0087] Simplifying formula (10), we get formula (11), which is the maximum likelihood estimate of t

[0088] Through Solving the mathematical expectation of , we can prove that the true estimate of t is As shown in formula (12):

[0089]

[0090] According to formula (13), the corrected estimated value N can be obtained:

[0091]

[0092] There is a certain difference between the estimated value N and the true value n, which shows that the addition of random response noise will have a certain impact on the distribution of the data set. The more evenly the labels of the data set are distributed, the smaller the impact on the distribution. The estimated value is equal to the true value if and only if the number of positive and negative samples in the data set is equal.

[0093] As a specific implementation, the loss function in the teacher model is cross entropy loss, specifically:

[0094] Cross entropy loss ce The calculation formula is shown in formula (14):

[0095]

[0096] The cross entropy loss is a function that strictly penalizes predictions that are close to the label.

[0097] The loss function in the student model uses the symmetric cross entropy loss function (Symmetric CrossEntropy). The symmetric loss function l rce It is obtained by reversing the annotation label and the predicted value, as shown in formula (15):

[0098]

[0099] The overall symmetric cross entropy loss function l sce Yes ce With l rce Add, as shown in formula (16):

[0100] l sce = l ce +l rce (16)

[0101] The loss function can also be fine-tuned by using the two hyperparameters a and b, as shown in formula (17), to obtain the extended symmetric loss function l sl :

[0102] l sl =al ce +bl rce (17)

[0103] This function uses the model's ability to correctly judge sample categories to weaken the strict penalty of cross entropy loss, thereby achieving the purpose of reducing the impact of noisy labels on model performance in the present invention.

[0104] Normally, the cross entropy function encourages predictions to be close to 1 and 0. When the model does not have an overfitting problem, the loss function of the student model uses a symmetric cross entropy loss function.

[0105] When the model has the problem of overfitting, the present invention targets the overfitting problem of the student model on the student side. In order to improve the problem of poor generalization ability, the loss function of the student model adopts the Smooth Label Critierion method to obtain better results.

[0106] In formula (18), K is the number of data set categories, and α is a hyperparameter, which is generally set to 0.1. This method can reduce the impact of noise data in the noisy data.

[0107]

[0108] The present invention adopts a teacher-student model, sets multiple teacher ends, and each teacher end is trained by non-overlapping data. Multiple teacher ends can send migration knowledge to the student end at different times. After the neural network training of the student end is completed, if a new teacher adds knowledge, the student end can put the new teacher's knowledge into the original aggregated knowledge and re-aggregate it, so that attackers cannot obtain the private data of the teacher end by attacking the student end, thereby enhancing the privacy protection performance and allowing the teacher ends to be trained at different times. It has extremely high application value in real scenarios.

[0109] like Figure 5 As shown, the present invention provides a distributed privacy protection system based on semi-supervised transfer learning, the system comprising:

[0110] Module 1 is constructed to construct a distributed model; the distributed model includes a student model and multiple teacher models; specifically, resnet-50 is used as the teacher model in the distributed model; the number of categories in the classification layer of the teacher model is set to the number of disease categories in the image data set with disease categories labeled. Furthermore, resnet-50 is used as the student model in the distributed model; the number of categories in the classification layer of the student model is set to the number of disease categories in the image data set with disease categories labeled.

[0111] The first training module 2 is used to train the teacher model in the distributed model with the image data set labeled with disease categories as input to obtain a trained teacher model; the output of the teacher model is the disease category.

[0112] The disease category data set determination module 3 is used to input the image data set without disease category labeling into the trained teacher model to obtain the disease category data set.

[0113] The reference disease category data set determination module 4 is used to encrypt the disease category data sets output by the multiple trained teacher models to determine the reference disease category data set.

[0114] The second training module 5 is used to train the student model in the distributed model with the image dataset without disease category labeling as input and the reference disease category dataset as output to obtain a trained student model.

[0115] The diagnosis result output module 6 is used to input the image data of the disease category to be diagnosed into the trained student model to obtain the disease category of the disease to be diagnosed.

[0116] Wherein, the reference disease category data set determination module 4 includes:

[0117] The predicted classification data set determination submodule is used to add noise to the disease category data sets output by the multiple trained teacher models to obtain multiple predicted classification data sets; specifically, the noise is added through a random response algorithm.

[0118] The reference disease category data set determination submodule is used to apply a voting mechanism to the multiple prediction classification data sets to determine a reference disease category data set.

[0119] In addition, the student model in the distributed model applies a loss function to denoise the reference disease category data set; the loss function is a symmetric cross entropy function or a label smoothing regularization method.

[0120] The distributed privacy protection method and system based on semi-supervised transfer learning provided by the present invention have the following technical effects:

[0121] 1. It improves the risk of privacy leakage in traditional distributed technology. Attackers cannot obtain private data on the teacher side by attacking the student side, thus enhancing privacy protection performance.

[0122] 2. A new distributed model is provided, and the model can be effectively aggregated.

[0123] 3. The heterogeneity of the teacher model and the student model is achieved, and the teacher model can be trained at different times, which has extremely high application value in real scenarios.

[0124] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0125] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A distributed privacy protection method based on semi-supervised transfer learning, characterized in that: The method comprises: Constructing a distributed model; the distributed model includes a student model and multiple teacher models; Taking the image dataset with disease categories labeled as input, training the teacher model in the distributed model to obtain a trained teacher model; the output of the teacher model is the disease category; Inputting an image dataset without disease category labels into the trained teacher model to obtain a disease category dataset; The disease category data sets output by the multiple trained teacher models are encrypted to determine a reference disease category data set; specifically including: adding noise to the disease category data sets output by the multiple trained teacher models to obtain multiple predicted classification data sets; applying a voting mechanism to the multiple predicted classification data sets to determine a reference disease category data set; wherein, adding noise to the disease category data sets output by the multiple trained teacher models to obtain multiple predicted classification data sets is specifically: selecting a random response algorithm combined with asymmetric noise, taking the label with the second largest probability value of the output layer as a flip label, outputting the correct label with probability p, and outputting the flip label with probability 1-p, respectively adding noise to the multiple disease category data sets to obtain multiple predicted classification data sets; the calculation formula of probability p is: e ε p=1-e ε Among them, ε represents the privacy parameter; Taking the image dataset without disease category labeling as input and the reference disease category dataset as output, training the student model in the distributed model to obtain a trained student model; Inputting the image data of the disease category to be diagnosed into the trained student model to obtain the disease category of the disease to be diagnosed; Among them, multiple teacher models can send migration knowledge to the student model at different times. After the student model training is completed, if a new teacher model adds new knowledge, the student model can put the knowledge added by the new teacher model into the original aggregated knowledge, re-aggregate, and train again to obtain a better distributed model.

2. The distributed privacy protection method based on semi-supervised transfer learning according to claim 1, characterized in that: Resnet-50 is used as the teacher model in the distributed model; the number of categories in the classification layer of the teacher model is the number of disease categories in the image data set with labeled disease categories.

3. The distributed privacy protection method based on semi-supervised transfer learning according to claim 1, characterized in that: The student model in the distributed model applies a loss function to denoise the reference disease category data set; the loss function is a symmetric cross entropy function or a label smoothing regularization method.

4. A distributed privacy protection system based on semi-supervised transfer learning, characterized in that: The system comprises: A construction module, used to construct a distributed model; the distributed model includes a student model and multiple teacher models; The first training module is used to train the teacher model in the distributed model using the image dataset with the disease category labeled as input to obtain a trained teacher model; the output of the teacher model is the disease category; A disease category data set determination module, used to input an image data set without disease category labeling into the trained teacher model to obtain a disease category data set; The reference disease category data set determination module is used to encrypt the disease category data sets output by the multiple trained teacher models to determine the reference disease category data set; specifically including: adding noise to the disease category data sets output by the multiple trained teacher models to obtain multiple predicted classification data sets; applying a voting mechanism to the multiple predicted classification data sets to determine the reference disease category data set; wherein, adding noise to the disease category data sets output by the multiple trained teacher models to obtain multiple predicted classification data sets is specifically: selecting a random response algorithm combined with asymmetric noise, taking the label with the second largest probability value of the output layer as the flip label, outputting the correct label with probability p, and outputting the flip label with probability 1-p to add noise to the multiple disease category data sets respectively to obtain multiple predicted classification data sets; the calculation formula of probability p is: Among them, ε represents the privacy parameter; A second training module is used to train the student model in the distributed model using the image dataset without disease category labeling as input and the reference disease category dataset as output to obtain a trained student model; A diagnosis result output module, used for inputting image data of the disease category to be diagnosed into the trained student model to obtain the disease category of the disease to be diagnosed; Among them, multiple teacher models can send migration knowledge to the student model at different times. After the student model training is completed, if a new teacher model adds new knowledge, the student model can put the knowledge added by the new teacher model into the original aggregated knowledge, re-aggregate, and train again to obtain a better distributed model.

5. The distributed privacy protection system based on semi-supervised transfer learning according to claim 4, characterized in that: Resnet-50 is used as the teacher model in the distributed model; the number of categories of the classification layer of the teacher model is set to the number of disease categories in the image data set with disease categories labeled.

6. The distributed privacy protection system based on semi-supervised transfer learning according to claim 4, characterized in that: The student model in the distributed model applies a loss function to denoise the reference disease category data set; the loss function is a symmetric cross entropy function or a label smoothing regularization method.

Images