Privacy protection method and system for federated learning of unlabeled public datasets

Abstract

This invention proposes a privacy protection method and system for federated learning of unlabeled public datasets, belonging to the technical field of federated learning privacy protection. The method comprises: a server issues federated learning tasks and unlabeled public datasets to multiple clients, and notifies auxiliary nodes to generate homomorphic encryption key pairs; clients are divided according to the amount of local private data reported by the clients; the clients perform inference on the unlabeled public dataset, encrypt the predicted values ​​with the public key, and upload them to the server; the server aggregates the data based on the client type, generates global prediction results and pseudo-labels, and distributes them to all clients; and the clients iteratively optimize their local models based on the obtained global prediction results and pseudo-labels. This invention can solve the problem of knowledge misleading caused by unlabeled public datasets in actual federated distillation learning.

Description

Technical Field

[0001] The present invention belongs to the technical field of federated learning privacy protection, and in particular to a federated learning privacy protection method and system for unlabeled public datasets. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] With the rapid development of artificial intelligence (AI) technology, federated learning, a distributed machine learning framework, has gained widespread application in sensitive fields such as healthcare and finance. By keeping data local to the device and exchanging only model parameters, it effectively reduces the risk of data privacy leaks. Federated distillation, a current technology, addresses the heterogeneity of participating models by introducing public datasets as a knowledge transfer medium. A typical solution, the FedMD framework, allows clients to exchange model predictions (logits) using public datasets, avoiding the need to directly share raw data or model parameters.

[0004] However, existing federated learning methods have difficult-to-solve technical problems in the context of unlabeled public datasets, such as:

[0005] (1) Knowledge misleading problem: When the amount of local data among participants differs significantly (e.g., some clients have very little data) or the public dataset lacks true labels, insufficiently trained clients will produce low-quality predictions on the public dataset. During the global aggregation process, these low-quality predictions will mask the correct knowledge of high-quality clients, leading to deviations in the global model aggregation results. This problem is particularly prominent in the early stages of training, not only reducing the performance of each participant's model, but also causing the final model to lose accuracy and slow down convergence.

[0006] (2) Privacy protection vulnerabilities: Existing technologies often directly process client prediction values ​​on the server side without encrypting the transmitted information, allowing malicious servers to infer the original data features through inference attacks.

[0007] (3) Data heterogeneity sensitivity: In non-IID scenarios, existing solutions lack a mechanism to screen the quality of predictions from participating parties. When client data distribution is highly heterogeneous, directly aggregating all predictions will amplify the negative impact of low-quality predictions and exacerbate the instability of the global model. Summary of the Invention

[0008] To overcome the above-mentioned deficiencies in the prior art, the present invention provides a privacy protection method and system for federated learning of unlabeled public datasets, which can solve the problem of knowledge misleading that may be caused by unlabeled public datasets in actual federated distillation learning.

[0009] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0010] A first aspect of the present invention provides a privacy protection method for federated learning of unlabeled public datasets.

[0011] Privacy-preserving methods for federated learning on unlabeled public datasets include:

[0012] The server sends federated learning tasks and unlabeled public datasets to multiple clients, notifies the auxiliary nodes to generate homomorphic encryption key pairs, and sends the public keys to each client.

[0013] Each client reports the amount of local private data to the server and retains the encryption key; the server divides the clients into large-volume clients and regular clients based on the amount of local private data;

[0014] The client performs inference on an unlabeled public dataset and uploads the resulting prediction value to the server after public key encryption. The server aggregates the data based on the client type, generates global prediction results and pseudo labels, and sends them to all clients. The client iteratively optimizes the local model based on the global prediction results and pseudo labels.

[0015] Furthermore, the server performs aggregation processing according to the client type, including: the server aggregates the prediction value ciphertexts of all clients and large-scale clients respectively to generate a global prediction result and pseudo-label prediction value ciphertext of the unlabeled public data set.

[0016] Furthermore, the server performs aggregation processing according to the client type, which also includes: the server sends the global prediction result and the pseudo-label prediction value ciphertext to the auxiliary node, and the auxiliary node uses the private key to decrypt and return the plaintext to the server; the server generates a pseudo-label based on the decrypted pseudo-label prediction value.

[0017] Furthermore, the server only performs aggregation processing on the encrypted ciphertext, that is, the entire aggregation processing is directly executed in the ciphertext state.

[0018] Furthermore, the client performs iterative optimization of the local model based on the obtained global prediction results and pseudo labels, including: when the sample data has no corresponding pseudo labels, iterative optimization is performed using the KL divergence loss function of the global soft label; when the sample data has corresponding pseudo labels, iterative optimization is performed using the weighted loss function of the pseudo labels.

[0019] Furthermore, when the maximum category probability in the pseudo-label prediction value of the sample data does not meet the pseudo-label threshold, it is considered that the sample data has no corresponding pseudo-label.

[0020] Furthermore, the server uses whether the amount of local private data exceeds a data amount threshold as a basis for classifying the client types; and the set data amount threshold is between the minimum and maximum values ​​of the local private data amounts of all clients.

[0021] A second aspect of the present invention provides a federated learning privacy protection system for unlabeled public datasets.

[0022] A privacy-preserving system for federated learning on unlabeled public datasets, including:

[0023] The server is configured to: during the initialization phase, issue federated learning tasks and unlabeled public datasets to multiple clients and notify auxiliary nodes to generate homomorphic encryption key pairs; classify clients into large-volume clients and regular clients based on the amount of local private data; and during the joint training phase, perform aggregation processing based on client type, generate global prediction results and pseudo-labels, and issue them to all clients.

[0024] Auxiliary nodes are configured to: generate homomorphic encryption keys and decrypt ciphertext;

[0025] Multiple clients are configured to: report the amount of local private data to the server and retain the encryption key; perform inference on unlabeled public datasets and upload the obtained prediction values ​​to the server after public key encryption; and iteratively optimize the local model based on the obtained global prediction results and pseudo labels.

[0026] Furthermore, the server includes a dual aggregation unit for performing aggregation processing of large data volumes of all clients in parallel.

[0027] Furthermore, the auxiliary node is independent of the server and the client, deletes the plaintext data immediately after decryption is completed, and the private key is not transmitted to the server and the client.

[0028] One or more of the above technical solutions have the following beneficial effects:

[0029] (1) The present invention divides clients into large-volume clients and regular clients. While aggregating the prediction values ​​of all clients to generate a global prediction result, the prediction values ​​of large-volume clients are separately aggregated and pseudo-labeled after threshold verification. Since large-volume clients usually have more sufficient local data support, their prediction quality is higher. The pseudo-labels generated in this way can effectively filter out the interference of low-quality predictions. At the same time, the client uses the KL divergence loss function and the weighted loss function to optimize the model according to whether there are pseudo-labels. This can reduce the masking effect of low-quality predictions from undertrained clients on the global aggregation, significantly alleviate the problem of knowledge misleading, and improve model accuracy and convergence speed.

[0030] (2) This invention introduces an auxiliary node to generate a homomorphic encryption key pair. The prediction values ​​uploaded by the client are encrypted using the public key. The server only aggregates the encrypted ciphertext, making it impossible to obtain the original prediction content of a single client. The auxiliary node immediately deletes the plaintext after decryption, and the private key is not transmitted to the server or client. This end-to-end encryption mechanism can effectively prevent malicious servers from inferring the original data features through inference attacks, significantly enhancing the privacy protection strength and security of the system.

[0031] (3) This invention classifies client types by data volume threshold, focusing on using the predictions of large-volume clients (whose data is more comprehensive and predictions are more reliable) to generate pseudo-labels, forming a screening mechanism for the prediction quality of participating parties. Compared with existing technologies, this method can reduce the negative impact of low-quality predictions in non-IID scenarios, even if the client data distribution is highly heterogeneous, ensure the convergence stability of the model under different levels of heterogeneity, and thus reduce the interference of data heterogeneity on the global model.

[0032] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0034] Figure 1 This is a flowchart of the privacy protection method for federated learning of unlabeled public datasets in Example 1 of the present invention.

[0035] Figure 2 This is a flowchart of client division in embodiment 1 of the present invention.

[0036] Figure 3 This is a result diagram of the test evaluation performed in the following scenario in Example 1 of the present invention.

[0037] Figure 4 This is a result diagram of the test evaluation conducted under scenario 2 in embodiment 1 of the present invention.

[0038] Figure 5 This is a structural diagram of the federated learning privacy protection system for unlabeled public datasets in Example 2 of the present invention. DETAILED DESCRIPTION

[0039] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0040] It should be noted that the terms used herein are for describing particular embodiments only and are not intended to limit the exemplary embodiments according to the present invention.

[0041] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0042] The overall concept proposed in this paper is to provide a privacy-preserving method for federated learning of unlabeled public datasets. This method is suitable for federated learning scenarios with large variations in data volume and computational performance, and supports the participation of heterogeneous models. During training, all information uploaded by clients is encrypted. The server cannot access the original content of any individual client and can only aggregate the encrypted ciphertext. This effectively prevents malicious servers from inferencing individual clients and improves the overall privacy protection and security of the system.

[0043] Example 1

[0044] This embodiment discloses a privacy protection method for federated learning of unlabeled public datasets.

[0045] like Figure 1 As shown in Figure 2, the privacy protection method for federated learning on unlabeled public datasets includes:

[0046] Step S1: The server sends federated learning tasks and unlabeled public datasets to multiple clients, notifies the auxiliary node to generate a homomorphic encryption key pair, and sends the public key to each client;

[0047] Step S2: Each client reports the amount of local private data to the server and retains the encryption key; the server divides the clients into large data volume clients and regular clients based on the amount of local private data;

[0048] In step S3, the client performs inference on the unlabeled public dataset and uploads the obtained prediction value to the server after public key encryption. The server performs aggregation processing based on the client type, generates global prediction results and pseudo labels, and sends them to all clients. The client performs iterative optimization of the local model based on the obtained global prediction results and pseudo labels.

[0049] Based on the above process, the present invention can solve the problem of knowledge misleading caused by unlabeled public datasets in actual federated distillation learning. To facilitate understanding of the technical solution of the present invention, the specific implementation method of the technical solution of the present invention is further explained and illustrated below.

[0050] The privacy protection method for federated learning of unlabeled public datasets provided by the present invention mainly includes two stages and three roles in its implementation; the two stages are the initialization stage and the joint training stage, and the three roles are the federated learning participants. ,server and auxiliary nodes Specifically, the initialization phase is implemented through steps S1 and S2, and the joint training phase is implemented through step S3.

[0051] In step S1, the server sends federated learning tasks and unlabeled public datasets to multiple clients, and notifies the auxiliary node to generate a homomorphic encryption key pair, and sends the public key to each client.

[0052] During this phase, the server and all participating parties jointly maintain a public dataset to assist in knowledge distillation and the transfer of model knowledge. This public dataset is unlabeled data. The server issues federated learning tasks, and the auxiliary nodes generate public and private keys for homomorphic encryption.

[0053] Specifically, the server Select this federated learning task Participating clients , , unlabeled public dataset , global training rounds , and sends relevant information to the clients participating in this federated learning. Indicates the Clients, Indicates the total number of clients. Notify the secondary node , generate homomorphic encryption public key and private key , send the public key to the client .

[0054] In step S2, after receiving messages from the server and auxiliary nodes, each client counts the amount of private data available for training in the task and reports it to the server, retaining the encryption key. The server sets a data volume threshold based on the local information uploaded by the participants, classifying them as large-volume clients and regular clients.

[0055] like Figure 2 As shown, each participant calculates and reports its local data volume to the server. The server then sets a data volume threshold based on this data volume and categorizes the participant into high-volume clients and regular clients. In practice, the server classifies clients based on whether the amount of local private data exceeds the data volume threshold. The data volume threshold is set between the minimum and maximum local private data volumes of all clients.

[0056] Specifically, each participant Receive federated learning tasks After that, the statistical task Local trainable data in the calculation of local data value , ; and send the data to the server Amount of local data sent , keep the public key Locally. Among them, Represents local data, Indicates the data index number of local data. Server Based on the amount of data uploaded by each participant Setting data volume thresholds If the client The amount of data held Satisfy the value greater than or equal to the set threshold , that is, satisfy , then the client that meets this condition is recorded as a large data volume client , , and the remaining clients are recorded as regular clients , ,and .in, Indicates the minimum amount of local private data of all clients. Indicates the maximum amount of local private data of all clients; Indicates the client ID of large data volume. Indicates the index number in the large data client. Indicates the total number of large data volume clients; Represents a general client identifier, Indicates the index number of the regular client. Indicates the total number of regular clients.

[0057] In step S3, the client infers the unlabeled public dataset and uploads the obtained prediction value to the server after public key encryption. The server aggregates the data based on the client type, generates global prediction results and pseudo labels, and sends them to all clients. The client iteratively optimizes the local model based on the obtained global prediction results and pseudo labels.

[0058] During the joint training phase, each participant trains a local model using private data. After updating the local model, all participants (i.e., clients) use the trained local model to perform inference on the unlabeled public dataset. The output predictions are encrypted using the public key and uploaded to the server. After receiving the ciphertext predictions from each client, the server first aggregates the ciphertext predictions from all participants to obtain a global prediction for the public dataset. Simultaneously, the server aggregates the ciphertext predictions from large-volume clients to generate ciphertext pseudo-label predictions. The server sends the aggregated ciphertext to the auxiliary node, which decrypts the ciphertext using its private key and returns the plaintext to the server. The server sets a pseudo-label threshold for the pseudo-label predictions, generates pseudo-labels, and distributes the global predictions and pseudo-labels for the public dataset to all participants. After receiving this information, each participant performs a local model update to further learn global knowledge and optimize the local model, completing one round of federated learning iteration. The joint training phase continues for multiple rounds until the preset training rounds or convergence criteria are reached, ultimately resulting in each participant obtaining an optimized federated learning model locally. In this embodiment, reaching the preset training round 150 is used as the iteration termination condition. This can be achieved by the following method:

[0059] Step S3-1: The client performs reasoning on an unlabeled public dataset.

[0060] Participants use local data to train private model updates, perform inference on unlabeled public datasets based on the trained private model (i.e., local model), and encrypt the output prediction value with a public key and upload it to the server.

[0061] Specifically, the participants Using private data Training a local model ,in, Represents the current global round. Get the locally updated private model Afterwards, we used the unlabeled public dataset Perform inference and obtain output prediction value , ; Among them, the output prediction value Local model for participants For all samples in the public dataset The raw logits output of a single sample data The output prediction value is:

[0062] ;

[0063] in, For the model Data The predicted probability of each category, The number of data categories. Use encryption key Encrypted output prediction value Get the local prediction value ciphertext , participants Then output the local prediction value ciphertext Upload Server .

[0064] Step S3-2: The server performs aggregation processing according to the client type.

[0065] 1) The server aggregates the output prediction value ciphertexts of all participants to generate a global soft label ciphertext (i.e., the global prediction result); at the same time, it aggregates the output prediction value ciphertexts of large-scale clients to generate a pseudo-label prediction ciphertext, which is sent to the auxiliary node for decryption.

[0066] Specifically, the server Received from all parties Encrypted local prediction value of the uploaded public dataset , aggregate to generate the global soft label ciphertext, namely: ;in, represents the global soft label ciphertext, Represents sample data Global soft label.

[0067] Sample data The global soft label ciphertext is:

[0068] ;

[0069] in, Represents sample data The global soft label ciphertext of Represents global aggregation, Represents the client For samples No. Class prediction probability value ciphertext.

[0070] The server will send all the large data volume clients The local prediction value ciphertext , aggregate to generate pseudo-label prediction value ciphertext, namely:

[0071] ;

[0072] in, represents the pseudo-label prediction value ciphertext, Represents sample data The server will global soft label ciphertext and pseudo-label prediction value ciphertext Send to secondary node Decryption.

[0073] 2) The auxiliary node uses the private key to decrypt the aggregate value and returns the decrypted result to the server.

[0074] Specifically, the auxiliary node Received global soft label ciphertext and pseudo-label prediction value ciphertext Then, use the private key Decrypt, obtain the global soft label and pseudo label prediction value, and return it to the server ,Right now:

[0075] ;

[0076] ;

[0077] in, Indicates decryption of the global soft label ciphertext, Represents the decrypted pseudo-label prediction value ciphertext; represents the global soft label prediction value, Represents the pseudo label prediction value.

[0078] 3) The server receives the decryption result, sets a threshold test on the pseudo-label prediction value, generates a pseudo-label, and sends the global prediction value and pseudo-label of the public dataset to the participants.

[0079] Specifically, the server Receive global soft labels and pseudo-label predictions , for data The pseudo-label prediction value of , calculate the maximum predicted category probability, that is:

[0080] ;

[0081] in, Represents sample data Predicted value The maximum value of the categories, Represents sample data No. Class prediction value.

[0082] server Setting the pseudo-label threshold , if the sample data The maximum predicted category probability is greater than the set pseudo label threshold , that is, satisfy , , then generate data The corresponding pseudo label is:

[0083] ;

[0084] in, Represents sample data Pseudo labels of Indicates a function that returns sample data The class with the highest probability among the predicted probability categories; Represents sample data No. Class prediction value.

[0085] server Set the global soft label of this round and some pseudo labels Composed label pairs, namely:

[0086] ;

[0087] in, Represents the label pair composed of Represents sample data Global soft label, Represents sample data Pseudo labels, Indicates the global round; represents the sample data, and , Represents a public dataset. If the sample data fails to pass the pseudo-label test to generate a pseudo-label, the pseudo-label is empty and the label pair only contains the global soft label. Send tag pair To all parties involved .

[0088] It should be noted that the server only aggregates the encrypted ciphertext, that is, the entire aggregation process is performed directly in the ciphertext state, which can effectively prevent malicious servers from launching inference attacks on individual clients and improve the overall privacy protection strength and security of the system.

[0089] Step S3-3: The client performs iterative optimization of the local model.

[0090] The client iteratively optimizes its local model based on the global prediction results and pseudo-labels. This includes: Each participant distills its private model using the received global updates; when the sample data does not have a corresponding pseudo-label, iterative optimization is performed using the KL divergence loss function of the global soft label; when the sample data has a corresponding pseudo-label, iterative optimization is performed using the weighted loss function of the pseudo-label. During the joint training phase, these steps are iteratively executed until the preset training rounds or convergence conditions are reached, completing the federated learning process.

[0091] Specifically, the participants Receive global soft labels and some pseudo labels After that, update the local model ;in, is the learning rate, is the local optimization objective function, represents the gradient of the loss function calculated on the local data with respect to the model parameters. The local update optimization objective functions for the global soft label and pseudo label are expressed as:

[0092] ;

[0093] ;

[0094] in, represents the local update optimization objective function of the global soft label, represents the local update optimization objective function of the pseudo-label; is the Kullback-Leibler divergence, is the cross entropy loss function. When the sample data has no corresponding pseudo label, the KL divergence loss function of the global soft label is used Perform iterative optimization; when the sample data has corresponding pseudo labels, the weighted loss function of the pseudo labels is used Perform iterative optimization; As an optional embodiment, when the maximum category probability in the pseudo-label prediction value of the sample data does not meet or exceed the pseudo-label threshold, it is considered that the sample data has no corresponding pseudo-label.

[0095] All parties involved For public datasets Perform multiple inference predictions and need to communicate with the server Multiple rounds of interaction are performed to upload and obtain the update process.

[0096] To further demonstrate the superiority of the method provided by this invention, this example evaluated the above solution using the ResNet18 convolutional neural network on the public dataset Cifar10. The participants' local data was partitioned using a Dirichlet distribution, with the local test data and training data sharing the same distribution. The public dataset was uniformly distributed.

[0097] This example sets up two federated distillation methods for comparative experiments, as Scenario 1, namely: FedMD federated model distillation directly aggregates the prediction results of all participants; DS-FL semi-supervised federated learning based on distillation adjusts the hyperparameter distillation temperature to reduce the entropy of the global aggregation result, ensure the correctness of the aggregation result, and reduce the problem of knowledge misleading. Figure 3 As shown in Figure 1, in FedMD, the prediction results of all participants are directly aggregated to generate global soft labels. However, when labels are missing or unevenly distributed, the aggregated results often fail to highlight key knowledge, making it difficult for participants to extract effective information. This leads to unstable model training and ultimately low accuracy. The DS-FL method introduces a distillation temperature adjustment mechanism. By increasing the temperature, it weakens the prediction entropy, thereby enhancing the salience of dominant knowledge and alleviating the non-IID problem. However, this method also weakens the uniqueness of the client's local model and fails to fully utilize the personalized advantages of federated learning. In contrast, this method introduces a pseudo-label generation mechanism to guide the learning of global knowledge, reduce the occurrence of knowledge drift, and effectively preserve the client's local feature expression, achieving superior results in terms of stability and performance. Therefore, the method of generating pseudo-labels by thresholding the aggregated results of a large number of clients can reduce the knowledge misleading problem of global aggregation results and improve the accuracy and stability of the federated learning model.

[0098] This embodiment also divides the unbalanced data for each client by setting the hyperparameters in the Dirichlet distribution, simulating the non-independent and identically distributed phenomenon of the data, as scenario 2, that is, setting three levels of data heterogeneity: high, medium and low to conduct experimental tests on the method of the present invention. Figure 4 As shown, as the value of α decreases, the data distribution differences between clients increase significantly. Although the model convergence speed decreases slightly in scenarios with highly heterogeneous data, the present invention can maintain a good convergence trend and stable performance under all settings. The final accuracy continues to improve with the training rounds, indicating that the proposed algorithm has good robustness and generalization capabilities when facing different degrees of data heterogeneity. Therefore, the method of the present invention can ensure the convergence of the model under different degrees of data heterogeneity, reduce the impact of low-quality clients on the global aggregation results, improve the model convergence speed, and reduce the impact of data heterogeneity problems on the accuracy of local models.

[0099] Example 2

[0100] This embodiment discloses a federated learning privacy protection system for unlabeled public datasets.

[0101] like Figure 5 As shown in Figure 1, the privacy protection system for federated learning of unlabeled public datasets includes:

[0102] The server is configured to: during the initialization phase, issue federated learning tasks and unlabeled public datasets to multiple clients and notify auxiliary nodes to generate homomorphic encryption key pairs; classify clients into large-volume clients and regular clients based on the amount of local private data; and during the joint training phase, perform aggregation processing based on client type, generate global prediction results and pseudo-labels, and issue them to all clients.

[0103] Auxiliary nodes are configured to: generate homomorphic encryption keys and decrypt ciphertext;

[0104] Multiple clients are configured to: report the amount of local private data to the server and retain the encryption key; perform inference on unlabeled public datasets and upload the obtained prediction values ​​to the server after public key encryption; and iteratively optimize the local model based on the obtained global prediction results and pseudo labels.

[0105] Furthermore, the server includes a dual aggregation unit for performing aggregation processing of large data volumes of all clients in parallel.

[0106] Furthermore, the auxiliary node is independent of the server and the client, deletes the plaintext data immediately after decryption is completed, and the private key is not transmitted to the server and the client.

[0107] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computer device. Alternatively, they can be implemented using program code executable by a computing device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0108] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A privacy-preserving method for federated learning of unlabeled public datasets, characterized by: include: The server sends federated learning tasks and unlabeled public datasets to multiple clients, notifies the auxiliary nodes to generate homomorphic encryption key pairs, and sends the public keys to each client. Each client reports the amount of local private data to the server and retains the encryption key; the server divides the clients into large-volume clients and regular clients based on the amount of local private data; The client performs inference on the unlabeled public dataset and uploads the obtained prediction value to the server after public key encryption; The server aggregates the data based on the client type, generates a global prediction result and a pseudo-label, and sends it to all clients. The server aggregates the prediction value ciphertexts of all clients and large-volume clients separately to generate a global prediction result and a pseudo-label prediction value ciphertext for the unlabeled public dataset. The client performs iterative optimization of the local model based on the obtained global prediction results and pseudo labels, including: when the sample data has no corresponding pseudo labels, iterative optimization is performed using the KL divergence loss function of the global prediction results; when the sample data has corresponding pseudo labels, iterative optimization is performed using the weighted loss function of the pseudo labels.

2. The method for privacy protection of federated learning for unlabeled public datasets according to claim 1, characterized in that: The server performs aggregation processing according to the client type, and also includes: the server sends the global prediction result and the pseudo-label prediction value ciphertext to the auxiliary node, and the auxiliary node uses the private key to decrypt and return the plaintext to the server; the server generates a pseudo-label based on the decrypted pseudo-label prediction value.

3. The privacy protection method for federated learning of unlabeled public datasets according to claim 1, characterized in that: The server only aggregates the encrypted ciphertext, that is, the entire aggregation process is performed directly in the ciphertext state.

4. The method for privacy protection of federated learning for unlabeled public datasets according to claim 1, wherein: When the maximum category probability in the pseudo-label prediction value of the sample data does not meet the pseudo-label threshold, it is considered that the sample data has no corresponding pseudo-label.

5. The method for privacy protection of federated learning for unlabeled public datasets according to claim 1, wherein: The server uses whether the amount of local private data exceeds a data amount threshold as a basis for classifying the client type; and the set data amount threshold is between the minimum and maximum values ​​of the local private data amounts of all clients.

6. A federated learning privacy protection system for unlabeled public datasets, employing the privacy protection method according to any one of claims 1 to 5, characterized in that: include: The server is configured to: during the initialization phase, issue federated learning tasks and unlabeled public datasets to multiple clients, and notify auxiliary nodes to generate homomorphic encryption key pairs; Clients are divided into large-volume clients and regular clients based on the amount of local private data; During the joint training phase, aggregation processing is performed based on the client type to generate global prediction results and pseudo labels, which are then distributed to all clients. Auxiliary nodes are configured to: generate homomorphic encryption keys and decrypt ciphertext; Multiple clients are configured to: report the amount of local private data to the server and retain the encryption key; Inference is performed on unlabeled public datasets, and the obtained prediction values ​​are encrypted with the public key and uploaded to the server; based on the obtained global prediction results and pseudo labels, the local model is iteratively optimized.

7. The privacy protection system for federated learning of unlabeled public datasets according to claim 6, characterized in that: The server includes dual aggregation units for performing aggregation processing of large amounts of data for all clients in parallel.

8. The privacy protection system for federated learning of unlabeled public datasets according to claim 6, characterized in that: The auxiliary node is independent of the server and the client, deletes the plaintext data immediately after decryption is completed, and the private key is not transmitted to the server and the client.

Images