Federal semi-supervised learning method and device

Abstract

The invention discloses a federal semi-supervised learning method and device, and belongs to the technical field of machine learning. The method comprises the following steps: executing M times of random sampling, and extracting K clients from clients participating in training in each sampling to form M subsets; sending the current global model to the clients in the M subsets to serve as current local models of the clients; receiving, from each client, a trained model completing the current round of training; aggregating the trained models of the clients in each subset to obtain M sub-consensus models; and updating the current global model according to the accuracy of each sub-consensus model on the test set. According to the method, differentiated cooperative training subsets are generated through multiple times of random sampling so as to relieve local label distribution deviation, double-layer optimization of local deviation correction-global preferential selection is achieved by combining the model distance and the sub-consensus model accuracy, noise interference of a deviation model can be restrained, and effective fusion of global knowledge can be enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning and relates to a federated semi-supervised learning method and device. Background Art

[0002] Federated learning, a distributed machine learning framework, uses local private data to train local models on multiple client devices, uploading only model parameters (not raw data) to edge servers for aggregation to generate a global model. This allows the global model to learn potential insights from distributed device data while ensuring that the data remains private. With the explosive growth of IoT devices and increasingly stringent privacy policies, federated learning has shown great potential in scenarios such as medical image analysis, smart city perception, and industrial automation. However, in practical applications, the high cost of data annotation and the scarcity of professional annotation resources have prompted researchers to turn to federated semi-supervised learning.

[0003] Currently, mainstream federated semi-supervised learning methods can be roughly divided into two categories. The first method assumes that each client contains partially labeled data, for example, each client holds both labeled and unlabeled data, or assumes that labeled and unlabeled data are isolated from each other but there is sample overlap between clients. The second method assumes that some clients have fully labeled data, while some clients only have unlabeled data. For example, FedIRM proposes learning inter-class relationships extracted from labeled clients and sharing them between labeled and unlabeled clients; Fed-Consist proposes a consistency-based method that applies multiple enhancements to unlabeled images and maximizes their predicted similarity.

[0004] Existing federated semi-supervised learning methods still have limitations when addressing challenges in practical application scenarios. On the one hand, the heterogeneity of data distribution among clients can lead to model aggregation bias. Especially in partially labeled client scenarios, the label distribution of unlabeled data may deviate seriously from the global distribution, exacerbating the divergence in the direction of local model updates. On the other hand, existing methods usually adopt static weight aggregation strategies (based on data volume or average weight distribution), ignoring the heterogeneous reliability of different client models during dynamic training due to the amount of labeled resources, differences in enhanced data strength, and different local convergence states, making it difficult to achieve efficient personalized knowledge distillation. Summary of the Invention

[0005] To address the imbalance in model reliability between labeled clients and unlabeled clients and the data heterogeneity problem in non-independent and identically distributed scenarios, the present invention provides a federated semi-supervised learning method and device.

[0006] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0007] In a first aspect, the present invention provides a federated semi-supervised learning method, which is applied to a central server and includes:

[0008] Perform M random samplings, each sampling extracts K clients from the clients participating in the training to form M subsets;

[0009] Send the current global model to the clients in the M subsets as the current local model of each client;

[0010] Receive trained models from each client;

[0011] Aggregate the trained models of K clients in each subset to obtain M sub-consensus models;

[0012] According to the accuracy of each sub-consensus model on the test set, the current global model is updated, and the updated global model is used as the current global model for the next round of training.

[0013] Furthermore, the trained models of the K clients in each subset are aggregated to obtain M sub-consensus models, including:

[0014] Get the average model of the subset;

[0015] assigning a distance weight to each client based on the distance between the trained model of the clients in the subset and the average model;

[0016] According to the distance weight of each client, the trained models of K clients in the subset are aggregated to obtain the sub-consensus model.

[0017] Furthermore, the average model of the subset is obtained, including:

[0018] ,

[0019] in, Representation subset The average model of ; Representation subset The amount of local data of the i-th client, ; Representation subset The trained model for the i-th client.

[0020] Furthermore, based on the distance between the trained model of the client in the subset and the average model, a distance weight is assigned to each client, including:

[0021] ,

[0022] ,

[0023] in, Representation subset The distance weight of the i-th client in; express and The gradient difference between norm;

[0024] Aggregate the trained models of the K clients in the subset to obtain a sub-consensus model, including:

[0025] ,

[0026] in, Representation subset Sub-consensus model.

[0027] Furthermore, the current global model is updated according to the accuracy of each sub-consensus model on the test set, including:

[0028] Calculate the accuracy weight of the sub-consensus model based on its accuracy on the test set;

[0029] Based on the accuracy weights of the M sub-consensus models, the current global model is updated.

[0030] Furthermore, based on the accuracy of the sub-consensus model on the test set, the accuracy weight of the sub-consensus model is calculated, including:

[0031] ,

[0032] in, Represents the sub-consensus model Accuracy on the test set; represents the temperature parameter; Represents the sub-consensus model The accuracy weight of

[0033] Based on the accuracy weights of the M sub-consensus models, the current global model is updated, including:

[0034] ,

[0035] in, Indicates the current training round; Representation subset The sub-consensus model in the current training round; Represents the updated current global model.

[0036] In a second aspect, the present invention further provides a federated semi-supervised learning method, which is applied to a client and includes:

[0037] Receive the current local model from the central server;

[0038] If the client's local data is labeled data, the local data is used to train the current local model based on the cross-entropy loss to obtain the trained model;

[0039] If the client's local data is unlabeled data, it includes:

[0040] Based on the mean teacher consistency regularization framework, the received current local model is used as the student model, and the student model is trained by the teacher model, wherein when the client is selected by the central server for the first time, the initial global model is received from the central server as the teacher model; the trained student model is used as the trained model;

[0041] Send the trained model to the central server.

[0042] Furthermore, based on the mean teacher consistency regularization framework, the received current local model is used as the student model, and the teacher model guides the student model for training, including:

[0043] Inputting local data into the student model and the teacher model respectively to generate student predictions and teacher predictions;

[0044] The distribution of teacher predictions is sharpened by a temperature parameter, including:

[0045] ,

[0046] in, represents the sharpened teacher prediction distribution; represents the teacher prediction distribution before sharpening; Indicates temperature adjustment parameters;

[0047] Based on the student prediction and the sharpened teacher prediction, the student model parameters are updated using a mean square error loss function;

[0048] The teacher model obtains the parameters of the student model by performing an exponential moving average after each iteration.

[0049] In a third aspect, the present invention further provides a federated semi-supervised learning device, which is applied to a central server and includes:

[0050] The random sampling module is used to perform M random samplings. Each sampling selects K clients from the clients participating in the training to form M subsets.

[0051] The current local model distribution module is used to send the current global model to the clients in the M subsets as the current local model of each client;

[0052] The trained model receiving module is used to receive the trained models of each client;

[0053] The sub-consensus model acquisition module is used to aggregate the trained models of K clients in each subset to obtain M sub-consensus models;

[0054] The current global model update module is used to update the current global model according to the accuracy of each sub-consensus model on the test set, and the updated global model is used as the current global model for the next round of training.

[0055] In a fourth aspect, the present invention further provides a federated semi-supervised learning device, which is applied to a client and includes:

[0056] The current local model receiving module is used to receive the current local model from the central server;

[0057] The labeled client training module is used to train the current local model using the local data based on the cross entropy loss if the local data of the client is labeled data, to obtain a trained model;

[0058] The unlabeled client training module is used if the client's local data is unlabeled data, and includes:

[0059] Based on the mean teacher consistency regularization framework, the received current local model is used as the student model, and the student model is trained by the teacher model, wherein when the client is selected by the central server for the first time, the initial global model is received from the central server as the teacher model; the trained student model is used as the trained model;

[0060] The trained model sending module is used to send the trained model to the central server.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] The federated semi-supervised learning method provided by the present invention aims at the model aggregation deviation phenomenon caused by the heterogeneity of client data distribution, and designs a multiple random subsampling mechanism. By generating differentiated collaborative training subsets through independent multiple rounds of random sampling, it can effectively alleviate the deviation problem between the local label distribution of unlabeled data and the global center; the present invention proposes a dual dynamic weight aggregation strategy combining distance weighting and accuracy weighting. In order to address the defect of uneven model reliability caused by the static weight strategy, this method adaptively calibrates the aggregation weight according to the model distance within the subset, which can suppress the noise interference of the deviation model, and increase the weight of the high-performance sub-consensus model at the global level, forming a "local correction-global optimization" double-layer optimization mechanism. The method of the present invention can effectively improve the accuracy of the model, and all evaluation indicators show excellent performance when the proportion of unlabeled clients changes. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A flowchart of a federated semi-supervised learning method on a central server side provided by an embodiment of the present invention;

[0064] Figure 2 A schematic diagram of the process flow of the client-side federated semi-supervised learning method provided by an embodiment of the present invention;

[0065] Figure 3 A schematic diagram of the structure of a federated semi-supervised learning device on the central server side provided by an embodiment of the present invention;

[0066] Figure 4 A schematic diagram of the structure of a federated semi-supervised learning device on the client side provided by an embodiment of the present invention;

[0067] Figure 5 A schematic diagram of the structure of a federated semi-supervised learning system provided by an embodiment of the present invention;

[0068] Figure 6 This is a diagram of the internal structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. The same reference numerals in the drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. The embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations of the technical solution of the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0070] The term "and / or" in this document simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0071] In the following embodiments, clients are divided into two categories based on the labeling status of their local data sets: clients whose local data is unlabeled are called unlabeled clients, and clients whose local data is labeled are called labeled clients.

[0072] Example 1:

[0073] An embodiment of the present invention provides a federated semi-supervised learning method for training a global model by a central server and multiple clients, wherein the method is applied on the central server side. Figure 1 is a flowchart of the federated semi-supervised learning method, Figure 1 The flowchart shown in FIG is a processing flow for a training cycle in the federated semi-supervised learning process. In addition, this flowchart only shows the logical sequence of the method described in this embodiment. Under the premise of no conflict, in other possible embodiments of the present invention, different methods can be used. Figure 1 The steps shown or described are accomplished in the order shown.

[0074] The federated semi-supervised learning method provided in this embodiment can be applied to a terminal and can be executed by the federated semi-supervised learning device of Example 3, which can be implemented by software and / or hardware and can be integrated into a terminal.

[0075] See also Figure 1 The method of the embodiment of the present invention specifically includes the following steps:

[0076] Step 11: Perform M random samplings. Each sampling step selects K clients from the clients participating in the training to form M subsets.

[0077] If a client is selected multiple times in M random samplings, the central server only needs to send the current global model for the first time, and subsequent repeated calls will be directly trained based on the existing model parameters.

[0078] Multiple subsampling essentially constructs an integrated learning space for model parameters, which helps alleviate the risk of bias that may be introduced by single sampling. It adopts the "first acceptance + continued training" strategy to effectively reduce the communication load in multi-round sampling scenarios.

[0079] Step 12: Send the current global model to the clients in the M subsets as the current local model of each client.

[0080] Step 13: Receive the trained models from each client.

[0081] Step 14: Aggregate the trained models of the K clients in each subset to obtain M sub-consensus models.

[0082] The purpose of this step is to dynamically increase the weight of trained models that are closer to the average model and reduce the weight of models that are farther away. Therefore, for the K clients in each subset, a weight distribution strategy is designed based on the distance between the trained models for aggregation.

[0083] Subset Take this as an example to explain how to obtain a sub-consensus model. The specific steps are as follows:

[0084] Step 141: Get a subset The average model includes:

[0085] ,

[0086] in, Representation subset The average model of ; Representation subset The amount of local data of the i-th client, ; Representation subset The trained model for the i-th client.

[0087] Step 142: Based on Subset The distance between the trained model of the client and the average model is calculated, and a distance weight is assigned to each client.

[0088] Different from simply averaging the clients, the distance-weighted aggregation of the present invention dynamically scales the weight of each client. The specific formula is:

[0089] ,

[0090] ,

[0091] in, Representation subset The distance weight of the i-th client in; express and The gradient difference between norm.

[0092] By dividing the model distance by the amount of local data , which can reduce the impact of local iteration on model drift. Then the distance weights in the subset are Normalize to the [0,1] interval to get the distance weight , using dynamic distance weights It can reduce the impact of noise interference caused by the client-trained model with large parameter deviation, thereby improving the robustness of the model.

[0093] Step 143: Based on the distance weight of each client, the trained models of the K clients in the subset are aggregated to obtain a sub-consensus model, including:

[0094] ,

[0095] in, Representation subset Sub-consensus model.

[0096] Step 15: Update the current global model based on the accuracy of each sub-consensus model on the test set, and use the updated global model as the current global model for the next round of training.

[0097] After distance-weighted dynamic aggregation, a set of sub-consensus models are obtained. To improve the performance of the global model, this paper proposes dynamically adjusting the weight of each sub-consensus model based on its accuracy on the test set, rather than aggregating the sub-consensus models with equal weights. Specifically, the accuracy of each sub-consensus model is calculated using the test set, and then the accuracy weight of the corresponding sub-consensus model is generated through a normalized exponential function (Softmax).

[0098] Specifically, based on the accuracy of the sub-consensus model on the test set, the accuracy weight of the sub-consensus model is calculated, including:

[0099] ,

[0100] in, Represents the sub-consensus model Accuracy on the test set; Represents the temperature parameter, which is used to control the sharpness of the weight distribution; Represents the sub-consensus model The accuracy weight of .

[0101] Based on the accuracy weights of the M sub-consensus models, the high-precision sub-consensus model is given a higher weight, thereby alleviating the local optimal problem under non-independent and identically distributed data. Finally, the current global model is updated to , the calculation formula is:

[0102] ,

[0103] in, Indicates the current training round; Representation subset The sub-consensus model in the current training epoch.

[0104] Example 2:

[0105] An embodiment of the present invention provides a federated semi-supervised learning method for training a global model using a central server and multiple clients. The method is applied to a client side sampled and selected by the central server. Figure 2 is a flowchart of the federated semi-supervised learning method, Figure 2 The flowchart shown in FIG is a processing flow for a training cycle in the federated semi-supervised learning process. In addition, this flowchart only shows the logical sequence of the method described in this embodiment. Under the premise of no conflict, in other possible embodiments of the present invention, different methods can be used. Figure 2 The steps shown or described are accomplished in the order shown.

[0106] The federated semi-supervised learning method provided in this embodiment can be applied to a terminal and can be executed by the federated semi-supervised learning device of Example 4, which can be implemented by software and / or hardware and can be integrated into a terminal.

[0107] See also Figure 2 The method of the embodiment of the present invention specifically includes the following steps:

[0108] Step 21: Receive the current local model from the central server.

[0109] Step 22: If the local data of the client is labeled data, the local data is used to train the current local model based on the cross entropy loss to obtain a trained model.

[0110] For the annotation client, the local dataset (image) is subjected to random cropping, rotation and other enhancement operations to increase data diversity. Then, the local model is trained using the cross-entropy loss. The cross-entropy loss is expressed as:

[0111] ,

[0112] in, is the true label of the client’s local data, is the prediction of the local data label by the current local model.

[0113] Step 23: If the local data of the client is unlabeled data, the following steps are included:

[0114] Based on the consistency regularization framework of the mean teacher, the received current local model is used as the student model, and the student model is trained by guiding the teacher model, wherein, when the client is selected by the central server for the first time, the initial global model is received from the central server as the teacher model; the student model that has completed the training is used as the trained model.

[0115] The specific steps of using the teacher model to guide the student model for training include:

[0116] Step 231: Input local data into the student model and teacher model respectively to generate student predictions and teacher predictions .

[0117] In this embodiment, data enhancement processing is required before local data is input into the student model and the teacher model.

[0118] Step 232: Sharpening the distribution of teacher predictions by a temperature parameter, including:

[0119] ,

[0120] in, represents the sharpened teacher prediction distribution; represents the teacher prediction distribution before sharpening; Indicates the temperature adjustment parameter.

[0121] The present invention uses temperature parameters A sharpening operation is performed on the distribution of teacher predictions to reduce the interference of low-confidence predictions. By doing this, Adjusted to be sharper , which moves the sample prediction results away from the decision boundary, thereby generating higher quality consistency alignment targets.

[0122] Step 233: Based on the student prediction and the sharpened teacher prediction, use the mean square error loss function , updating the student model parameters, including:

[0123] .

[0124] Step 234: The teacher model obtains the parameters of the student model by performing an exponential moving average after each iteration, including:

[0125] ,

[0126] in, is the momentum coefficient, not marked on the client Finally, the updated student model is returned to the central server as its trained model.

[0127] Step 24: Send the trained model to the central server.

[0128] Example 3:

[0129] Based on the same inventive concept as Example 1, this embodiment of the present invention further provides a federated semi-supervised learning device for implementing the federated semi-supervised learning method of Example 1. The implementation solution provided by this device is similar to the implementation solution described in the method of Example 1. Therefore, the specific limitations in the federated semi-supervised learning device embodiment provided below can be referred to the limitations of the federated semi-supervised learning method in Example 1 and will not be repeated here.

[0130] like Figure 3 As shown, an embodiment of the present invention provides a federated semi-supervised learning device, which is applied to a central server side and includes:

[0131] A random sampling module, configured to perform M random samplings, wherein each sampling step extracts K clients from the plurality of clients to form M subsets;

[0132] The current local model distribution module is used to send the current global model to the clients in the M subsets as the current local model of each client;

[0133] The trained model receiving module is used to receive the trained models of each client;

[0134] The sub-consensus model acquisition module is used to aggregate the trained models of K clients in each subset to obtain M sub-consensus models;

[0135] The current global model update module is used to update the current global model according to the accuracy of each sub-consensus model on the test set, and the updated global model is used as the current global model for the next round of training.

[0136] Example 4:

[0137] Based on the same inventive concept as Example 2, this embodiment of the present invention also provides a federated semi-supervised learning device for implementing the federated semi-supervised learning method of Example 2. The implementation solution provided by this device is similar to the implementation solution described in the method of Example 2. Therefore, the specific limitations in the federated semi-supervised learning device embodiment provided below can be referred to the limitations of the federated semi-supervised learning method in Example 2 and will not be repeated here.

[0138] like Figure 4 As shown, an embodiment of the present invention provides a federated semi-supervised learning device, which is applied to a client side sampled and selected by a central server, and includes:

[0139] The current local model receiving module is used to receive the current local model from the central server;

[0140] The labeled client training module is used to train the current local model using the local data based on the cross entropy loss if the local data of the client is labeled data, to obtain a trained model;

[0141] The unlabeled client training module is used if the client's local data is unlabeled data, and includes:

[0142] Based on the mean teacher consistency regularization framework, the received current local model is used as the student model, and the student model is trained by the teacher model, wherein when the client is selected by the central server for the first time, the initial global model is received from the central server as the teacher model; the trained student model is used as the trained model;

[0143] The trained model sending module is used to send the trained model to the central server.

[0144] Example 5:

[0145] like Figure 5 As shown, the embodiment of the present invention also provides a federated semi-supervised learning system, including: a client layer, a sampling layer and an edge layer. The client layer consists of The edge layer and the sampling layer include a server, and the server includes the federated semi-supervised learning device described in the aforementioned embodiment 3. The division of labor between the edge layer and the sampling layer is as follows: Figure 5 The specific limitations in the federated semi-supervised learning system embodiments provided below can be found in the limitations on the federated semi-supervised learning device in Examples 3 and 4, and will not be repeated here.

[0146] In order to verify the effect of the federated semi-supervised learning system of the present invention, a simulation experiment was designed in this embodiment. The simulation experiment was evaluated on two natural image classification datasets SVHN and CIFAR-100. All datasets were divided into 80% training sets and 20% test sets. The images were upsampled from 32×32 to 40×40, and then 32×32 areas were randomly cropped and normalized. This embodiment uses a lightweight convolutional neural network containing two 5×5 convolutional layers, a maximum pooling layer, and a fully connected layer as the global model. The stochastic gradient descent optimization algorithm is used, and the learning rate is set to 0.03 (labeled client) and 0.021 (unlabeled client). The number of communication rounds is set to 200 rounds, the local training cycle is 1 round, the number of random subsampling times M is 3, the number of clients in the subset K is 5, and the scaling factor is 0. The federated learning scenario is set according to the Dirichlet distribution Generate non-independent and identically distributed data partitions to achieve partial label heterogeneity of clients. After this Non-IID data partitioning strategy, the number of classes and samples of each client is different from each other, so not all clients contain samples from all classes.

[0147] First, the experiment set up 10 clients, including 1 labeled client and 9 unlabeled clients. The present invention was compared with the most advanced existing federated semi-supervised learning methods, including: (1) the FedIRM method, which calculates the inter-class relationship based on the labeled clients and uses this relationship as additional supervision information for the unlabeled clients; (2) the Fed-Consist method, which calculates the consistency loss for the multiple augmented input predictions of unlabeled data in the mean teacher framework. In addition, as shown in Table 1, the experiment also used the federated averaging algorithm (Fedavg) results trained on all 10 labeled clients as the upper limit of the classification results, and the FedAvg results trained on a single labeled client as the lower limit of the classification results for comparison. The effects of different methods were compared using four widely recognized evaluation metrics, namely accuracy, area under the receiver operating characteristic (ROC) curve (AUC), precision, and recall.

[0148] Table 1 Test results of different modes and methods on the SVHN and CIFAR-100 datasets

[0149]

[0150] As shown in Table 1, quantitative comparisons of four evaluation metrics between our proposed method and existing state-of-the-art methods on two benchmark datasets show that the results of the two compared federated semi-supervised learning methods (FedIRM and Fed-Consist) and our proposed method (Ours) are between the FedAvg algorithm's performance using all 10 annotated clients (upper bound) and that of the FedAvg algorithm using only one annotated client (lower bound). Experiments demonstrate that our proposed federated semi-supervised learning system, incorporating dynamic weighted consensus with random sampling, outperforms existing methods on all metrics across both datasets. Compared to Fed-Consist, the proposed system significantly improves performance on the SVHN dataset, with accuracy increasing by 3.16% to 67.20%, AUC by 2.5% to 94.89%, precision by 2.83% to 70.96%, and recall by 1.21% to 65.46%. This demonstrates that our proposed method can more efficiently utilize unlabeled data to improve performance and validates the effectiveness of the dual dynamic weight aggregation strategy under the Dirichlet distribution. Compared with FedIRM, the accuracy of the proposed method on the CIFAR-100 dataset is improved from 14.29% to 15.16%, the AUC is improved from 79.48% to 80.06%, the precision is improved from 14.20% to 15.89%, and the recall is improved from 14.56% to 15.16%. These advantages are due to the fact that the assumption of fixed inter-class relationships in the FedIRM method fails in the scenario where the data of each client are not independent and identically distributed.

[0151] Secondly, to evaluate the performance of the methods under different unlabeled client scales, a comparative experiment was conducted using the SVHN dataset in a federated semi-supervised learning scenario with a fixed number of labeled clients at 1, testing the performance of each method when the total number of clients varied from 5 to 10. The experimental results are shown in Table 2. As the number of unlabeled clients increases from 4 to 9, the improvement in accuracy and AUC indicators of the present invention compared to Fed-Consist increases from 1.92% to 3.16% and from 1.52% to 2.50%, respectively, showing advantages in both overall accuracy and inter-category discrimination capabilities. This is because when unlabeled clients are added, the random subsampling strategy can effectively select subsets with greater differences, reducing the impact of noise; at the same time, the dual dynamic weight aggregation simultaneously performs gradient similarity measurement and local update amplitude evaluation, achieving personalized parameter fusion to address the challenges of heterogeneous data distribution - by suppressing the weights of clients with severe category distribution deviations and enhancing the contributions of clients with category complementarity, the model can still maintain excellent generalization performance when the number of unlabeled clients increases.

[0152] Table 2 Sensitivity analysis results of unlabeled client ratio

[0153]

[0154] As shown in Table 3, the test results of all solutions improved with an increase in the proportion of annotated clients (2 annotated clients, 8 unannotated clients), but the proposed method outperformed the comparison methods in all four metrics. Compared to FedIRM and Fed-Consist, the proposed method achieved 16.22% and 1.17% higher accuracy and 9.12% and 1.03% higher AUC on the SVHN dataset, respectively. Traditional methods such as FedIRM, due to their strong assumption of a shared inter-class correlation matrix, are unable to accommodate local data distribution variations among multiple annotating clients as the number of annotated clients increases. This is particularly true for class bias variations among different annotating clients in heterogeneous scenarios, leading to global models becoming trapped in local optima. While Fed-Consist improves model generalization through consistency regularization, its static equal-weight strategy fails to leverage the supervisory signal gain from the increase in annotated clients. As the number of labeled data increases, the performance gap between models with reliable clients (annotated clients) and models with low confidence (unannotated clients) widens. Equal-weight aggregation dilutes key parameters of high-precision models with low-quality updates, resulting in a recall of only 70.83%. This method can maximize the propagation efficiency of high-value supervisory signals in the scenario of incremental labeled data, and achieve the coordinated optimization of efficient utilization of heterogeneous resources and unbalanced knowledge expression.

[0155] Table 3 Test results when the ratio of labeled clients to unlabeled clients is 2:8

[0156]

[0157] Example 6:

[0158] The embodiment of the present invention further provides a computer device, which may be a server, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements the federated semi-supervised learning method in the aforementioned embodiment 1 or embodiment 2.

[0159] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0160] Example 7:

[0161] An embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements the steps of the federated semi-supervised learning method in the aforementioned embodiment 1 or the steps of the federated semi-supervised learning method in the aforementioned embodiment 2.

[0162] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0163] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0164] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0165] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0166] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which are all protected by the present invention.

Claims

1. A federated semi-supervised learning method, characterized in that The method is applied to a central server and includes: Perform M random samplings, each sampling extracts K clients from the clients participating in the training to form M subsets; Send the current global model to the clients in the M subsets as the current local model of each client; Receive trained models from each client; Aggregate the trained models of K clients in each subset to obtain M sub-consensus models; According to the accuracy of each sub-consensus model on the test set, the current global model is updated, and the updated global model is used as the current global model for the next round of training.

2. The federated semi-supervised learning method according to claim 1, characterized in that Aggregate the trained models of the K clients in each subset to obtain M sub-consensus models, including: Get the average model of the subset; assigning a distance weight to each client based on the distance between the trained model of the clients in the subset and the average model; According to the distance weight of each client, the trained models of K clients in the subset are aggregated to obtain the sub-consensus model.

3. The federated semi-supervised learning method according to claim 2, characterized in that Get the average model for a subset, including: ， in, Representation subset The average model of ; Representation subset The amount of local data of the i-th client, ; Representation subset The trained model for the i-th client.

4. The federated semi-supervised learning method according to claim 3, wherein: Assigning a distance weight to each client based on the distance between the trained model of the client in the subset and the average model, including: ， ， in, Representation subset The distance weight of the i-th client in; express and The gradient difference between norm; Aggregate the trained models of the K clients in the subset to obtain a sub-consensus model, including: ， in, Representation subset Sub-consensus model.

5. The federated semi-supervised learning method according to claim 1, wherein: Based on the accuracy of each sub-consensus model on the test set, the current global model is updated, including: Calculate the accuracy weight of the sub-consensus model based on its accuracy on the test set; Based on the accuracy weights of the M sub-consensus models, the current global model is updated.

6. The federated semi-supervised learning method according to claim 5, characterized in that According to the accuracy of the sub-consensus model on the test set, the accuracy weight of the sub-consensus model is calculated, including: ， in, Represents the sub-consensus model Accuracy on the test set; represents the temperature parameter; Represents the sub-consensus model The accuracy weight of Based on the accuracy weights of the M sub-consensus models, the current global model is updated, including: ， in, Indicates the current training round; Representation subset The sub-consensus model in the current training round; Represents the updated current global model.

7. A federated semi-supervised learning method, characterized in that The method is applied to a client, and includes: Receive the current local model from the central server; If the client's local data is labeled data, the local data is used to train the current local model based on the cross-entropy loss to obtain the trained model; If the client's local data is unlabeled data, it includes: Based on the mean teacher consistency regularization framework, the received current local model is used as the student model, and the student model is trained by the teacher model, wherein when the client is selected by the central server for the first time, the initial global model is received from the central server as the teacher model; the trained student model is used as the trained model; Send the trained model to the central server.

8. The federated semi-supervised learning method according to claim 7, wherein: Based on the mean teacher consistency regularization framework, the received current local model is used as the student model, and the teacher model guides the student model for training, including: Inputting local data into the student model and the teacher model respectively to generate student predictions and teacher predictions; The distribution of teacher predictions is sharpened by a temperature parameter, including: ， in, represents the sharpened teacher prediction distribution; represents the teacher prediction distribution before sharpening; Indicates temperature adjustment parameters; Based on the student prediction and the sharpened teacher prediction, the student model parameters are updated using a mean square error loss function; The teacher model obtains the parameters of the student model by performing an exponential moving average after each iteration.

9. A federated semi-supervised learning device, characterized in that The device is applied to a central server and includes: The random sampling module is used to perform M random samplings. Each sampling selects K clients from the clients participating in the training to form M subsets. The current local model distribution module is used to send the current global model to the clients in the M subsets as the current local model of each client; The trained model receiving module is used to receive the trained models of each client; The sub-consensus model acquisition module is used to aggregate the trained models of K clients in each subset to obtain M sub-consensus models; The current global model update module is used to update the current global model according to the accuracy of each sub-consensus model on the test set, and the updated global model is used as the current global model for the next round of training.

10. A federated semi-supervised learning device, characterized in that The device is applied to a client, and includes: The current local model receiving module is used to receive the current local model from the central server; The labeled client training module is used to train the current local model using the local data based on the cross entropy loss if the local data of the client is labeled data, to obtain a trained model; The unlabeled client training module is used if the client's local data is unlabeled data, and includes: Based on the mean teacher consistency regularization framework, the received current local model is used as the student model, and the student model is trained by the teacher model, wherein when the client is selected by the central server for the first time, the initial global model is received from the central server as the teacher model; the trained student model is used as the trained model; The trained model sending module is used to send the trained model to the central server.

Images