A federated construction method and classification method of a medical data classification model

Abstract

The application provides a federated construction method of a medical data classification model, which is used for jointly constructing a medical data classification model by using multiple labeled clients and multiple unlabeled clients. The labeled client indicates that a data set on the client includes multiple medical data and corresponding disease labels. The unlabeled client indicates that a data set on the client only includes multiple medical data. The method comprises the following steps: a federated client clustering step is used for clustering each client with similar data distribution into a sub-federated set; a federated semi-supervised self-training step is used for sorting the clients in each sub-federated set, and performing a label processing on the unlabeled clients according to the sorting result so that all medical data of each unlabeled client is assigned a target pseudo label; and a federated client training step is used for performing multiple rounds of federated training based on all labeled clients and all unlabeled clients assigned with the target pseudo label to obtain the medical data classification model.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, specifically, to medical data classification technology in the field of artificial intelligence, and more specifically, to medical data classification technology based on semi-supervised federated learning, that is, a federated construction method and classification method of a medical data classification model. Background Art

[0002] With increasing emphasis on data security, collecting all the data needed for modeling is becoming increasingly difficult. Federated learning is a distributed model training framework that fully protects data privacy. It allows clients to train locally using their own data while sharing model parameters with a central server. After multiple rounds of communication between the server and client, a better global model is ultimately constructed. Traditional federated learning methods require clients to have complete data and label information. However, obtaining data labels is often difficult, and some even require professional annotation. Taking medical data as an example, doctors can annotate some data during a patient's hospitalization, but data from other time periods cannot be annotated without the doctor's expertise. Large amounts of unlabeled data provide diverse data information, while labeled data provides valuable supervisory information. In the federated modeling process, most institutions participating in federated learning often have the data annotation capabilities and willingness to perform data annotation, while individuals with diverse data are unable to participate in the federated model construction due to the difficulty of annotation. Typical federated learning scenarios have abundant labeled data, while real-world scenarios often have sparsely distributed labeled data, resulting in serious limitations for existing federated learning methods.

[0003] To solve the above-mentioned defects, some researchers propose that a semi-supervised federated learning (SSFL) method based on self-training can be used to label the unlabeled data, so as to make full use of the information of the unlabeled data to optimize the model. Specifically, first, a labeled client is used for federated training to build an initial model; then, the initial model is used to assign pseudo-labels to the data in the unlabeled client, and the data with pseudo-labels is added to the federated training; finally, a data classification model is built based on all the data. Although the semi-supervised federated learning method based on self-training can assign pseudo-labels to the data in the unlabeled client, simply combining semi-supervised learning with federated learning is not enough, because there is a difference in the data distribution between the labeled client and the unlabeled client, and the initial model obtained by federated modeling using the labeled client will have a high error when assigning pseudo-labels to the data of the unlabeled client, resulting in poor pseudo-label accuracy. At the same time, when the data distribution of the client has similarity, the semi-supervised federated learning method based on self-training will use the obtained initial model to assign pseudo-labels to the data of each unlabeled client in a random selection manner, which ignores the pseudo-label assignment order of the unlabeled client and also leads to poor pseudo-label accuracy.

[0004] In summary, the existing technology still has two problems: on the one hand, when there is a difference in the data distribution between the labeled client and the unlabeled client, the generalization of the initial model trained by the labeled client is insufficient, resulting in poor accuracy of pseudo-label assignment; on the other hand, when the data distribution of the labeled client and the unlabeled client is similar, the pseudo-label assignment is performed in a random selection manner, which ignores the pseudo-label assignment order of the unlabeled client and also leads to poor pseudo-label accuracy.

[0005] It should be noted that the background art is only used to introduce the related information of the present application, so as to facilitate the understanding of the technical solutions of the present application, but does not mean that the related information must be prior art. In the absence of evidence that the related information has been disclosed before the filing date of the present application, the related information should not be regarded as prior art. SUMMARY

[0006] Therefore, the purpose of the present application is to overcome the defects of the prior art and provide a federated construction method of a medical data classification model and a medical data classification method.

[0007] The purpose of the present application is achieved by the following technical solutions.

[0008] According to a first aspect of the present application, a method for federated construction of a medical data classification model is provided, which is used for jointly constructing a medical data classification model by a plurality of labeled clients and a plurality of unlabeled clients, wherein the labeled client indicates that the data set on the client includes a plurality of medical data and corresponding disease labels, and the unlabeled client indicates that the data set on the client only includes a plurality of medical data. The method comprises: a federated client clustering step: obtaining a first global classification model based on a plurality of rounds of federated training of all labeled clients, and performing a first round of label processing on each unlabeled client using the first global classification model to assign initial pseudo labels to all medical data on each unlabeled client; clustering the unlabeled clients and the labeled clients according to a preset clustering rule based on all labeled clients and all unlabeled clients assigned with initial pseudo labels to obtain a plurality of sub-federated sets, wherein each sub-federated set contains a plurality of clients, and each sub-federated set contains at most one labeled client; a federated semi-supervised self-training step: sorting all clients in each sub-federated set containing a labeled client according to a distance degree according to a preset sorting rule, and performing a second round of label processing on each unlabeled client in each sub-federated set containing a labeled client according to a preset manner according to the distance degree sorting result, so that all medical data of each unlabeled client in each sub-federated set containing a labeled client are assigned with target pseudo labels; performing a plurality of rounds of federated training based on all labeled clients and unlabeled clients assigned with target pseudo labels to obtain a second global classification model, and performing a third round of label processing on each unlabeled client in each sub-federated set containing only unlabeled clients using the second global classification model, so that all medical data of each unlabeled client in each sub-federated set containing only unlabeled clients are assigned with target pseudo labels; a federated client training step: performing a plurality of rounds of federated training based on all labeled clients and all unlabeled clients assigned with target pseudo labels according to a preset training rule to obtain a medical data classification model.

[0009] In some embodiments of the present application, the preset clustering rule is: performing a plurality of rounds of local training based on all labeled clients and unlabeled clients assigned with initial pseudo labels to obtain a first local model corresponding to each client, and clustering the unlabeled clients and the labeled clients based on the parameters of the first local model corresponding to each client according to a preset clustering algorithm to obtain a plurality of sub-federated sets, wherein the first local model corresponding to each client includes a plurality of network layers.

[0010] In some embodiments of the present application, the preset clustering algorithm is:

[0011]

[0012] wherein, Pearson(C m , C n ) represents a correlation coefficient between the first local model corresponding to the mth client and the first local model corresponding to the nth client, C m represents the first local model corresponding to the mth client, C n represents the first local model corresponding to the nth client, Pear m represents a set of last layer network parameters of the first local model corresponding to the mth client, represents an average value of the set of last layer network parameters of the first local model corresponding to the mth client, Pear n represents a set of last layer network parameters of the first local model corresponding to the nth client, represents an average value of the set of last layer network parameters of the first local model corresponding to the nth client.

[0013] In some embodiments of the present application, the preset ranking rule is to calculate a distance degree between each unlabeled client and a labeled client in each sub-federated set containing the labeled client according to a Mahalanobis distance algorithm, and to sort the distance degrees to obtain a distance degree ranking result of the clients in each sub-federated set.

[0014] In some embodiments of the present application, the preset manner is to perform multiple labeling processes in a left-to-right or right-to-left order with the labeled client as an end point when the labeled client is located at both ends of the distance degree ranking result, wherein each labeling process includes: performing multiple rounds of federated training based on the labeled client and all unlabeled clients that have been assigned a target pseudo-label in the sub-federated set to obtain a current intermediate global classification model, and labeling the unlabeled client adjacent to the unlabeled client of the last labeling process with the current intermediate global classification model.

[0015] In some embodiments of the present application, the preset manner is to perform multiple labeling processes in a left-to-right or right-to-left order with the labeled client as an end point when the labeled client is located at both ends of the distance degree ranking result, wherein each labeling process includes: performing multiple rounds of federated training based on the labeled client and all unlabeled clients that have been assigned a target pseudo-label in the sub-federated set to obtain a current intermediate global classification model, and labeling the unlabeled client adjacent to the unlabeled client of the last labeling process with the current intermediate global classification model.

[0016] In some embodiments of the present application, the preset training rule is: determining the order of each unlabeled client to join the federal training according to the processing order of each unlabeled client in each sub-federal set being assigned a target pseudo label; based on all labeled clients performing multiple rounds of federal training, adding each unlabeled client assigned a target pseudo label to the federal training process in stages according to the order of each unlabeled client joining the federal training until all unlabeled clients participate in the federal training process; and based on the labeled clients and all unlabeled clients assigned a target pseudo label jointly performing multiple rounds of federal training to obtain a medical data classification model.

[0017] According to a second aspect of the present application, a medical data classification method is provided, which comprises: step T1, obtaining medical data to be processed; and step T2, classifying the medical data to be processed using the medical data classification model obtained by the method of the first aspect of the present application.

[0018] Compared with the prior art, the present application has the following advantages: (1) by clustering clients with similar data distribution into a sub-federal set, the problem of poor pseudo label assignment accuracy caused by different data distribution of clients is reduced; (2) the clients in each sub-federal set are sorted according to similarity, and the pseudo label assignment for unlabeled clients is sorted according to the similarity sorting result, further improving the pseudo label assignment accuracy; (3) in the construction process of the medical data classification model, higher confidence data information is learned first, avoiding the noise problem caused by false pseudo labels, and thus improving the classification accuracy of the medical data classification model. BRIEF DESCRIPTION OF DRAWINGS

[0019] The embodiments of the present application are further described below with reference to the accompanying drawings, in which:

[0020] Figure 1 FIG. 1 is a flowchart of a federal construction method of a medical data classification model according to an embodiment of the present application;

[0021] Figure 2 FIG. 2 is an example schematic diagram of a federal client clustering step according to an embodiment of the present application;

[0022] Figure 3 FIG. 3 is an example schematic diagram of a client correlation coefficient color matrix according to an embodiment of the present application;

[0023] Figure 4 FIG. 4 is an example schematic diagram of a federal semi-supervised self-training step according to an embodiment of the present application;

[0024] Figure 5 FIG. 5 is an example schematic diagram of a federal client training step according to an embodiment of the present application;

[0025] Figure 6 A medical data classification method flowchart according to an embodiment of the present application;

[0026] Figure 7 An experimental test set data distribution example schematic diagram. DETAILED DESCRIPTION

[0027] For the purpose of the present application, the technical solutions and advantages are more clear and explicit, the following will be further described in detail by specific embodiments combined with the drawings. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0028] As mentioned in the background section, there are two problems in the prior art: on the one hand, when there is a difference in the data distribution of the labeled client and the unlabeled client, the generalization of the initial model trained by the labeled client is insufficient, resulting in poor accuracy of the pseudo-label assignment; on the other hand, when the data distribution of the labeled client and the unlabeled client is similar, the pseudo-label assignment is performed in a random manner, which ignores the pseudo-label assignment order of the unlabeled client and also leads to poor pseudo-label accuracy.

[0029] In order to solve the defects of the prior art, the inventors analyzed the prior art and found that, due to the difference in the data distribution of the labeled client and the unlabeled client, the initial model constructed by the labeled client will have a high error when performing pseudo-label assignment on the unlabeled client, and the incorrect pseudo-label will bring more noise information, which is not conducive to the construction of the medical data classification model. Therefore, the inventors cluster the clients with obvious difference in data distribution, so that the clients with similar data distribution are divided into a sub-federal set, and then perform pseudo-label assignment, in order to improve the accuracy of pseudo-label assignment. Further, in the case of similar data distribution of the clients, the prior art usually adopts a random manner to perform pseudo-label assignment on each client, which ignores the label assignment order between the clients, which leads to poor pseudo-label accuracy. In order to improve the accuracy of pseudo-label assignment, the inventors sort the clients with similar data distribution, and perform pseudo-label assignment based on the sorting results of the clients, in order to improve the accuracy of pseudo-label assignment. Based on the analysis of the prior art, the inventors propose a federal construction scheme based on client clustering and similarity sorting to train a medical data classification model. In this scheme, first, the clients with similar data distribution are clustered into a sub-federal set by client clustering, in order to reduce the problem of poor pseudo-label assignment accuracy caused by different data distribution of the clients; then, the clients in each sub-federal set are sorted according to the similarity, and the pseudo-label assignment is performed for the unlabeled clients according to the similarity sorting results, in order to further improve the pseudo-label assignment accuracy; finally, the unlabeled clients after pseudo-label assignment and the labeled clients are used to construct a medical image classification model.

[0030] In probability, as shown in the formula (1), the present application provides a federated construction method of a medical data classification model, which is used for jointly constructing a medical data classification model by a plurality of labeled clients and a plurality of unlabeled clients, wherein the labeled client indicates that the data set on the client includes a plurality of medical data and corresponding disease labels, and the unlabeled client indicates that the data set on the client only includes a plurality of medical data, and the method comprises the following steps. Figure 1

[0031] In order to better understand the present application, each step will be described in detail below in conjunction with specific examples.

[0032] I. Federated client clustering step

[0033] The federated client clustering step is used to cluster each client with similar data distribution into a sub-federated set, wherein each sub-federated set includes a plurality of clients and at most one labeled client. It should be noted that the client clustering operation can improve the accuracy of subsequent target pseudo-label assignment and reduce noise interference. The following is an example of clustering operation. Figure 2 ​The displayed federal client clustering step example introduces the specific execution process.

[0034] As shown in Figure 2 , first, the clients are divided into labeled clients and unlabeled clients.

[0035] Then, based on all labeled clients, 1-5 rounds of federal training are performed to obtain a first global classification model, wherein the FedAvg algorithm can be used for federal aggregation during the federal training process. It should be noted that the number of rounds of federal training is not limited to 1-5 times, and the number of training rounds can be determined according to actual needs, and the first global classification model trained at this time is only used for basic classification, and the classification accuracy of the model does not need to be guaranteed.

[0036] Then, the first global classification model is used to perform the first round of labeling processing for each unlabeled client to assign initial pseudo-labels to all medical data on each unlabeled client. It should be noted that when the first round of labeling processing is performed, the initial pseudo-labels assigned to the medical data do not have high confidence but may have large classification errors. Although there is a certain classification error, the client can still be clustered according to the underlying data distribution of each client after being assigned an initial pseudo-label, and the purpose of assigning an initial pseudo-label is to cluster the client according to data similarity, and the classification accuracy does not affect the clustering result.

[0037] Finally, based on all labeled clients and all unlabeled clients assigned with initial pseudo-labels, the unlabeled clients and the labeled clients are clustered according to a preset clustering rule to obtain a plurality of sub-federal sets, wherein each sub-federal set includes a plurality of clients, and each sub-federal set includes at most one labeled client. According to an embodiment of the present application, the preset clustering rule is that a plurality of rounds of local training are performed based on all labeled clients and unlabeled clients assigned with initial pseudo-labels to obtain a first local model corresponding to each client, and the unlabeled clients and the labeled clients are clustered based on the parameters of the first local model corresponding to each client according to a preset clustering algorithm to obtain a plurality of sub-federal sets, wherein the first local model corresponding to each client includes a plurality of network layers.

[0038] According to an embodiment of the present application, the preset clustering algorithm is:

[0039]

[0040] wherein Pearson(C m ,C n ) represents the correlation coefficient between the first local model corresponding to the mth client and the first local model corresponding to the nth client, C mdenotes the first local model corresponding to the mth client, C n denotes the first local model corresponding to the nth client, Pear m denotes the set of the last layer network parameters of the first local model corresponding to the mth client, denotes the average value of the last layer network parameters of the first local model corresponding to the mth client, Pear n denotes the set of the last layer network parameters of the first local model corresponding to the nth client, denotes the average value of the last layer network parameters of the first local model corresponding to the nth client. It should be noted that the parameters of the last layer network of the first local model corresponding to each client (Pear Figure 2 ) represent the distribution of the underlying data on the respective client, so the correlation coefficient can be solved based on the parameters of the last layer network, and several clients with larger correlation coefficients are allocated to the same sub-federal set (this process is abstractly represented as the clustering operation in Figure 2 , and after clustering, the clients are abstractly represented as points of different colors, where the green points represent multiple clients aggregated into a sub-federal set, the blue points represent multiple clients aggregated into a sub-federal set, and the orange points represent multiple clients aggregated into a sub-federal set).

[0041] ​To better understand how to cluster clients according to the correlation coefficients, the following takes the correlation coefficients between the 9 clients shown in Table 1 as an example, where 0-9 represent the labels of the clients. In Table 1, each row represents the correlation coefficients between the corresponding client and other clients, and the correlation coefficient of each client with itself is 1 (the maximum correlation). Among them, the first row represents the correlation coefficients between client 0 and all clients. In this row, the correlation coefficients between client 0 and clients 5, 6, and 7 are positive, indicating that client 0 has certain similarity with clients 5, 6, and 7, and these clients can be considered to be classified into a sub-federation set. The second row represents the correlation coefficients between client 1 and all clients. In this row, the correlation coefficients between client 1 and clients 2, 6, and 9 are positive, indicating that client 1 has certain similarity with clients 2, 6, and 9, and these clients can be considered to be classified into a sub-federation set. The meanings represented by other rows are consistent with the foregoing description, and will not be described in detail here. As can be seen from Table 1, when the correlation coefficient is positive, the two clients corresponding to the correlation coefficient have certain similarity. However, considering that many clients have certain similarity, the client with a larger correlation coefficient is selected to be assigned to a sub-federation set. For example, the correlation coefficient between client 0 and client 6 is 0.11, the correlation coefficient between client 1 and client 6 is 0.94, the correlation coefficient between client 2 and client 6 is 0.47, the correlation coefficient between client 7 and client 6 is 0.14, and the correlation coefficient between client 9 and client 6 is 0.48. At this time, client 1 and client 6 can be selected to be classified into a sub-federation set, and client 0, 2, 7, and 9 have higher correlation coefficients with other clients and are not classified into the same sub-federation set. The correlation coefficients can also be combined with the color depth to cluster the clients, as shown in Table 2, which shows the color matrix corresponding to the correlation coefficients between the 9 clients in Table 1. As can be seen from Table 2, the two clients with a larger correlation coefficient have a darker color, and the two clients with a smaller correlation coefficient have a lighter color. Figure 3 Figure 3

[0042] Table 1

[0043] 0 1 -0.21 -0.82 -0.41 -0.81 0.4 0.11 1 -0.17 -0.82 1 -0.21 1 0.73 -0.8 -0.41 -0.97 0.94 -0.18 -0.92 0.73 2 -0.82 0.73 1 -0.18 0.32 -0.85 0.47 -0.8 -0.43 1 3 -0.41 -0.8 -0.18 1 0.87 0.66 -0.95 -0.44 0.96 -0.19 4 -0.81 -0.41 0.32 0.87 1 0.21 -0.68 -0.82 0.17 0.32 5 0.4 -0.97 -0.85 0.66 0.21 1 -0.85 0.36 0.83 -0.85 6 0.11 0.94 0.47 -0.95 -0.68 -0.85 1 0.14 -0.99 0.48 7 1 -0.18 -0.8 -0.44 -0.82 0.36 0.14 1 -0.2 -0.8 8 -0.17 -0.92 -0.43 0.96 0.71 0.83 -0.99 -0.2 1 -0.43 9 -0.82 0.73 1 -0.19 0.32 -0.85 0.48 -0.8 -0.43 1 Label 0 1 2 3 4 5 6 7 8 9

[0044] II. Federation semi-supervised self-training step

[0045] ​​The federal semi-supervised self-training step is used to sort the clients in each sub-federal set, and the sorted results are used to label the unlabeled clients so that all medical data of each unlabeled client is assigned a target pseudo label. It should be noted that the federal semi-supervised self-training step assigns multiple target pseudo labels according to the distance degree sorting results, so that the overall distribution of clients in the sub-federal set is more similar, further improving the pseudo label assignment accuracy, which is beneficial to subsequent medical data classification model construction. The following introduces the specific execution process with the example of the federal semi-supervised self-training step shown. Figure 4 The example of the federal semi-supervised self-training step shown is used to introduce the specific execution process.

[0046] First, according to a preset sorting rule, sort all clients in each sub-federal set containing labeled clients according to the distance degree. According to an embodiment of the present application, the preset sorting rule is to calculate the distance degree between each unlabeled client and the labeled client in each sub-federal set containing labeled clients according to the Mahalanobis distance algorithm, and sort according to the distance degree to obtain the distance degree sorting result of the clients in each sub-federal set. It should be noted that in the distance degree calculation process, the mean of each client data in each sub-federal set is first solved and used as the baseline data of each client, and then the Mahalanobis distance algorithm is used to calculate the distance degree between the baseline data of each unlabeled client in each sub-federal set and the baseline data of the labeled client and sort. When sorting, take the labeled client as the reference, sort to the left of the labeled client when the distance degree value is negative, and sort to the right of the labeled client when the distance degree value is positive. Take Figure 4 The sorting result of the sub-federal set shown is taken as an example. The sorting result of sub-federal set 1 is: client 1 < client 2 < client 3; the sorting result of sub-federal set 2 is: client 4 < client 5 < client 6, and no sorting is performed in sub-federal set 3 which does not contain labeled clients.

[0047] Then, according to the distance degree sorting result, a second labeling process is performed on each unlabeled client in each sub-federal set containing labeled clients in a preset manner, so that all medical data of each unlabeled client in each sub-federal set containing labeled clients is assigned a target pseudo label.

[0048] According to one embodiment of the present invention, the preset method is that when a labeled client is located at both ends of the distance sorting result, multiple labeling processes are performed in the order of passing to the left or to the right with the labeled client as the endpoint, wherein each labeling process includes: performing multiple rounds of federated training based on the labeled clients in the sub-federation set and all unlabeled clients that have been assigned target pseudo-labels to obtain the current intermediate global classification model, and using the current intermediate global classification model to perform labeling on the unlabeled clients adjacent to the unlabeled clients processed in the previous labeling process.

[0049] In order to better understand the labeling process when the labeled client is at both ends of the distance sorting result, Figure 4 The labeling process of sub-federation set 1 is shown as an example. Figure 4 In the distance ranking result of the clients in the sub-federation set 1, the result is client 1 < client 2 < client 3, where client 1 is a labeled client. The specific process of labeling the unlabeled client 2 and unlabeled client 3 based on the ranking result of the sub-federation set 1 is as follows: First, multiple rounds of federation training are performed based on client 1 to obtain the current intermediate global classification model ( Figure 4 The model represented by blue in the middle will be continuously updated); the intermediate global classification model is then used to classify the medical data in client 2, and the classification results with a classification probability greater than or equal to 0.9 are selected from the classification results output by the model as pseudo labels to label the medical data in client 2; the medical data of client 2 that has been assigned the target pseudo label is then added to the federated training process and the medical data of client 1 are updated together with the current intermediate global classification model ( Figure 4 The updated current intermediate global classification model is used to label the remaining medical data in client 2. The classification results with a classification probability greater than or equal to 0.9 are still selected from the classification results output by the model as pseudo labels to continue labeling the medical data in client 2. This process is repeated until all medical data in client 2 are assigned target pseudo labels, completing the pseudo label assignment for client 2. Then, the current intermediate global classification model ( Figure 4 The model represented by the blue middle image classifies the medical data of client 3 and selects the classification results with a classification probability greater than or equal to 0.9 from the classification results output by the model as pseudo labels to label the medical data in client 3; the medical data of client 3 that has been assigned the target pseudo label is then added to the federated training process and the medical data of client 1 and client 2 are updated together with the current intermediate global classification model ( Figure 4the model represented by the medium purple color, which is continuously updated), and the updated current intermediate global classification model ( Figure 4 The model represented by the medium purple color labels the remaining medical data in the client 3, and still selects the classification result with a classification probability greater than or equal to 0.9 from the classification results output by the model as the pseudo label to label the medical data in the client 3. The foregoing process is repeatedly repeated until all medical data in the client 3 is assigned a target pseudo label, and the pseudo label assignment to the client 3 is completed. It should be noted that in the labeling processing mode when the labeled client is located at both ends of the distance degree sorting result, one labeling processing operation refers to labeling processing for all medical data in a client.

[0050] According to an embodiment of the present application, the preset mode is that when the labeled client is located in the middle of the distance degree sorting result, multiple labeling processing is performed in turn from left to right with the labeled client as the end point, and each labeling processing includes: based on the labeled client in the sub-federal set and all unlabeled clients that have been assigned a target pseudo label in the direction of the labeling processing, performing multiple rounds of federal training to obtain a current intermediate global classification model, and using the current intermediate global classification model to label the unlabeled clients adjacent to the unlabeled clients of the last labeling processing in the direction of the labeling processing.

[0051] In order to better understand the labeling processing mode when the labeled client is located in the middle of the distance degree sorting result, the following is described: Figure 4 The labeling process of the sub-federal set 2 shown is described by way of example. In the Figure 4 The distance degree sorting result in the sub-federal set 2 is client 4 < client 5 < client 6, and the client 5 is the labeled client. The specific process of labeling processing for the unlabeled client 4 and the unlabeled client 6 based on the distance degree sorting result of the sub-federal set 2 is as follows: first, based on the current intermediate global classification model ( Figure 4 represented by the medium green color, which is continuously updated); then, using the model to simultaneously classify the medical data in the client 4 and the client 6, and selecting the classification result with a classification probability greater than or equal to 0.9 from the classification results output by the model as the pseudo label to label the medical data in the client 4 and the client 6; and then adding the medical data in the client 4 and the client 6 that have been assigned a target pseudo label to the federal training process together with the medical data in the client 5 to update the current intermediate global classification model ( Figure 4The updated current intermediate global classification model is used to classify the remaining medical data in the client 4 and the client 6, and the classification result with a classification probability greater than or equal to 0.9 is still selected from the classification results output by the model as a pseudo-label to label the medical data in the client 4 and the client 6; the foregoing operation is repeatedly performed until all the medical data in the client 4 and the client 6 are assigned the target pseudo-label. Figure 4 The client distance ranking result shown in the above table is relatively simple, and when there is a more complex distance ranking result, for example, when the client C is a labeled client, and the distance ranking result is client A < client B < client C < client E < client F, at this time, the client B and the client E are labeled based on the client C at the same time according to the foregoing processing process, and then the client A and the client F are labeled based on the client B, the client C and the client E according to the foregoing process. The labeling processing process corresponding to other distance ranking results is similar, and will not be described here. It should be noted that in the labeling processing mode when the labeled client is in the middle of the distance ranking result, one labeling processing operation refers to labeling all the medical data of the unlabeled clients adjacent to the unlabeled client of the last labeling processing in the direction of the labeling processing at the same time. Still taking client A < client B < client C < client E < client F as an example, one labeling processing is to label the client B and the client E based on the client C at the same time, or to label the client A and the client F based on the client C, the client B and the client E at the same time.

[0052] Finally, a second global classification model is obtained by performing multiple rounds of federated training based on all the labeled clients and the unlabeled clients assigned with the target pseudo-label, and a third round of labeling processing is performed on each unlabeled client in each sub-federated set containing only the unlabeled clients by using the second global classification model, so that all the medical data of each unlabeled client in each sub-federated set containing only the unlabeled clients are assigned with the target pseudo-label.

[0053] In order to better understand the labeling processing process of the sub-federated set containing only the unlabeled clients, the labeling processing process of the sub-federated set 3 in the above table is taken as an example for description. The sub-federated set 3 contains the unlabeled client 7 and the unlabeled client 8, and the labeling processing process of the sub-federated set is as follows: first, the second global classification model is obtained by performing multiple rounds of federated training based on the client 1 and the client 2 and the client 2, the client 3, the client 4 and the client 6 assigned with the target pseudo-label, Figure 4 Figure 4 ​the model output, the classification result with a classification probability greater than or equal to 0.9 is selected as a pseudo label to label the medical data in the client 7 and the client 8; then, the medical data in the client 7 and the client 8 to which the target pseudo label is assigned are added to the federated training process and the medical data in the clients 1-6 to update the second global classification model Figure 4 the model output, the classification result with a classification probability greater than or equal to 0.9 is selected as a pseudo label to label the medical data in the client 7 and the client 8; then, the medical data in the client 7 and the client 8 to which the target pseudo label is assigned are added to the federated training process and the medical data in the clients 1-6 to update the second global classification model

[0054] It should be noted that in the federated semi-supervised self-training step, no matter how the medical data in the unlabeled client is labeled, there may be a case where the classification probability of a certain medical data is always less than 0.9. At this time, the medical data can be discarded.

[0055] III. Federated Client Training Step

[0056] The federated client training step is used to perform multiple rounds of federated training based on all labeled clients and all unlabeled clients to which the target pseudo label is assigned according to a preset training rule to obtain a medical data classification model. According to an embodiment of the present application, the preset training rule is: the order of each unlabeled client joining the federated training is determined according to the processing order of each unlabeled client in each sub-federated set being assigned to the target pseudo label; after performing multiple rounds of federated training based on all labeled clients, each unlabeled client to which the target pseudo label is assigned is added to the federated training process in stages according to the order of each unlabeled client joining the federated training until all unlabeled clients participate in the federated training process; and the medical data classification model is obtained based on the labeled clients and all unlabeled clients to which the target pseudo label is assigned performing multiple rounds of federated training. It should be noted that the clients with high label quality are preferentially added to the federated training, and then the unlabeled clients to which the target pseudo label is assigned are gradually added, so that the medical data classification model learns more confident data information in the construction process, avoids the noise problem caused by false pseudo labels, and further improves the classification accuracy of the medical data classification model.

[0057] To better understand the federated client training steps, Figure 5 The federated client training steps shown are used to illustrate the specific execution process, where Figure 5 The clients involved are Figure 4 Shown are individual clients after the federated semi-supervised self-training step. Figure 5 The steps of the federated client training are as follows: First, multiple rounds of federated training are performed based on the labeled client 1 and client 5 to obtain the medical data classification model of stage 1 ( Figure 5 Then, the unlabeled clients 2, 4, and 6, which are given target pseudo labels, are added to the federated training process and multiple rounds of federated training are performed together with the labeled clients 1 and 5 to obtain the medical data classification model of stage 2 ( Figure 5 Then, the unlabeled client 3, which is assigned the target pseudo-label, is added to the federated training process and performs multiple rounds of federated training together with the unlabeled client 2, unlabeled client 4, unlabeled client 6, labeled client 1 and client 5 to obtain the medical data classification model of stage 3 ( Figure 5 Then, unlabeled client 7 and unlabeled client 8 are added to the federated training process and multiple rounds of federated training are performed together with clients 1-6 to obtain the medical data classification model of stage 4 ( Figure 5 The model is represented by red in the figure); finally, multiple rounds of federated training are performed based on clients 1-8 to obtain the final medical data classification model. It should be noted that during the federated semi-supervised self-training process, client 1 in sub-federated set 1 first assigns a target pseudo-label to client 2, and client 5 in sub-federated set 2 simultaneously assigns target pseudo-labels to clients 4 and 6. Therefore, clients 2, 4, and 6 are the first to join the federated training (sub-federated set 1 and sub-federated set 2 perform pseudo-label assignment in parallel); clients 1 and 2 in sub-federated set 1 then assign a target pseudo-label to client 3, so client 3 is the second to join the federated training; clients 7 and 8 in sub-federated set 3 assign target pseudo-labels based on clients 1-6, so clients 7-8 are the third to join the federated training.

[0058] During the federated client training process, the communication rounds of participating clients are controlled, allowing the model to prioritize fitting clean data (labeled clients) rather than noisy data (pseudo-labeled data), thereby ensuring that the final trained medical data classification model has better data classification capabilities. It should also be noted that during the federated client training process, FedProx is used for federated aggregation, as shown below:

[0059]

[0060] wherein, denotes the federated aggregation optimization objective, F k (w) denotes the cross-entropy loss of the kth local model, w denotes the local medical data classification model parameter, w t denotes the parameter of the global medical data classification model, μ denotes the hyperparameter.

[0061] The medical data classification model trained based on the foregoing embodiments can be used for medical data classification. Based on this, as shown in Figure 6 the present application proposes a medical data classification method, which comprises the following steps: step T1, obtaining to-be-processed medical data; step T2, using the medical data classification model obtained by the method of the foregoing embodiments to classify and process the to-be-processed medical data. It should be noted that the to-be-processed medical data can be medical image data or medical text data, wherein the medical image data includes X-ray images, computed tomography images, ultrasonic images, nuclear magnetic resonance images, and nuclear medicine images, etc.; and the medical text data includes medical history, clinical examination, diagnosis scheme, and prescription, etc.

[0062] In order to better understand the difference between the present application and the prior art, the following experiments are performed on different data sets to compare the difference between the prior art and the present application. It should be noted that the same experimental configuration is used in the experimental process, and ResNet18 network is used as the backbone network of the classification network.

[0063] Before the experiment, the existing CIFAR-10 and PAD-UFES-20 data sets are selected as the experimental data sets. The CIFAR-10 data set contains 60,000 32x32 color images, which are divided into 10 categories in total, with 6,000 images in each category; the PAD-UFES-20 data set contains 2,298 skin disease images of 1,373 patients taken by various smartphones, involving 6 categories, of which 3 categories are skin cancer and the other 3 categories are skin diseases.

[0064] In the process of performing the experiment based on the CIFAR-10 data set, first, 5,000 images are randomly selected from the CIFAR-10 data set as a validation set and 5,000 images are randomly selected as a test set, and the remaining images are used as a test set; then, the test set is distributed to 10 clients in a non-independent and identically distributed manner according to the mixed distribution mode of the Dirichlet distribution and the Gaussian distribution, and the distribution result is as shown in Figure 7Then, respectively set the situation including one labeled client, two labeled clients and three labeled clients to perform the federated client clustering step, the federated semi-supervised self-training step and the federated client training step described in the above embodiment to obtain the corresponding medical data classification model, wherein the client with less data is selected as the labeled client, for example, when only one labeled client is included, Figure 7 Client 4 in the example is used as a labeled client. The selection of labeled clients in other cases is similar and will not be elaborated on in detail. Then, the corresponding medical data classification model is trained in the cases of one labeled client, two labeled clients, and three labeled clients according to the existing federated learning method (baseline), semi-supervised federated learning method (semi-federated), and centralized semi-supervised learning method (semi). Finally, the classification accuracy of the corresponding medical data classification model trained by the present invention and the prior art under the same circumstances is compared using the test set and validation set. The experimental results are shown in Table 2. NC in Table 2 l Indicates the number of clients with labels, NC u represents the number of unlabeled clients, Model-Acc refers to the accuracy of the medical data classification model on the test set, and PL-Acc refers to the accuracy of the target pseudo-labels assigned during the experiment. As shown in Table 2, the present invention achieves higher classification accuracy and labeling accuracy than the existing federated learning method (baseline) and semi-supervised federated learning method (semi-federated). The centralized semi-supervised learning method is not included in the experimental comparison and indicates the upper limit of the classification accuracy and labeling accuracy achievable by the existing technology.

[0065] Table 2

[0066]

[0067] In the process of the experiment on the PAD-UFES-20 dataset, first, the training set, the validation set and the test set are divided according to the ratio of 7:3:1; then, the test set is distributed to the 6 clients in a non-independent and identically distributed manner according to the mixed distribution mode of the Dirichlet distribution and the Gaussian distribution; then, the federated client clustering step, the federated semi-supervised self-training step and the federated client training step described in the foregoing embodiments are executed respectively in the case of containing one labeled client and two labeled clients, to obtain the corresponding medical data classification model, wherein the labeled client selection mode is consistent with the foregoing experiment, and details are not repeated here; then, the corresponding medical data classification model is trained in the case of containing one labeled client and two labeled clients according to the existing federated learning method (baseline), the semi-supervised federated learning method (semi-federated) and the centralized semi-supervised learning method (semi); finally, the classification accuracy of the corresponding medical data classification model trained by the present application and the prior art under the same condition is compared by using the test set and the validation set, and the experimental results are shown in Table 3, and it can be known from Table 3 that the present application has higher classification accuracy than the federated learning method (baseline) and the semi-supervised federated learning method (semi-federated) of the prior art.

[0068] Table 3

[0069]

[0070] Compared with the prior art, the present application has the following beneficial effects: (1) the data distribution similar clients are clustered into a sub-federal set by the client clustering mode, so as to reduce the problem of poor pseudo-label assignment accuracy caused by different data distribution of the clients; (2) the clients in each sub-federal set are sorted according to the similarity, and the pseudo-label assignment for the unlabeled clients is performed according to the similarity sorting result, so as to further improve the pseudo-label assignment accuracy; (3) the medical data classification model learns the data information with higher confidence first in the construction process, so as to avoid the noise problem caused by the false pseudo-label, and further improve the classification accuracy of the medical data classification model.

[0071] It should be noted that although the foregoing steps are described in a specific order, it does not mean that the steps must be performed in the above specific order, and in fact, some of the steps can be executed concurrently or even in a changed order, as long as the required functions can be achieved.

[0072] The present application can be a system, a method and / or a computer program product. The computer program product can include a computer readable storage medium on which is loaded a computer readable program instruction for enabling a processor to implement various aspects of the present application.

[0073] A computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves.

[0074] Embodiments of the application have been described above, with the understanding that these embodiments are exemplary only, and are not restrictive, in terms of the scope of the embodiments disclosed. Many modifications and variations of the described embodiments are possible, in light of the above teachings, without departing from the scope and spirit of the described embodiments. The choice of words in this document is intended to best explain the principles of the embodiments, practical application, or technical improvement in the art, or to enable others skilled in the art to utilize the embodiments disclosed herein.

Claims

1. A method for constructing a federated medical data classification model, which is used to construct a medical data classification model by combining multiple labeled clients and multiple unlabeled clients, wherein: The labeled client indicates that the data set on the client includes multiple medical data and their corresponding disease labels, and the unlabeled client indicates that the data set on the client only includes multiple medical data. The method includes: Federated client clustering steps: A first global classification model is obtained by performing multiple rounds of federated training on all labeled clients, and the first global classification model is used to perform a first round of labeling processing on each unlabeled client so that all medical data on each unlabeled client are assigned initial pseudo labels; Based on all labeled clients and all unlabeled clients assigned initial pseudo-labels, cluster the unlabeled clients and labeled clients according to a preset clustering rule to obtain multiple sub-federated sets, where each sub-federated set contains multiple clients and each sub-federated set contains at most one labeled client; Federated semi-supervised self-training steps: According to the preset sorting rules, all clients in each sub-federation set containing labeled clients are sorted by distance, and a second round of labeling is performed on each unlabeled client in each sub-federation set containing labeled clients according to the preset method based on the distance sorting results, so that all medical data of each unlabeled client in each sub-federation set containing labeled clients are assigned the target pseudo-label; Perform multiple rounds of federated training based on all labeled clients and unlabeled clients assigned target pseudo labels to obtain a second global classification model, and use the second global classification model to perform a third round of labeling processing on each unlabeled client in each sub-federated set that only contains unlabeled clients, so that all medical data of each unlabeled client in each sub-federated set that only contains unlabeled clients are assigned target pseudo labels; Federated client training steps: Based on all labeled clients and all unlabeled clients assigned target pseudo-labels, multiple rounds of federated training are performed according to preset training rules to obtain a medical data classification model.

2. The method according to claim 1, characterized in that The preset clustering rules are: Based on all labeled clients and unlabeled clients assigned initial pseudo-labels, multiple rounds of local training are performed respectively to obtain the first local model corresponding to each client, and based on the parameters of the first local model corresponding to each client, the unlabeled clients and labeled clients are clustered according to a preset clustering algorithm to obtain multiple sub-federal sets, wherein the first local model corresponding to each client includes multiple network layers.

3. The method according to claim 2, characterized in that The preset clustering algorithm is: Among them, Pearson (C m ,C n ) represents the correlation coefficient between the first local model corresponding to the mth client and the first local model corresponding to the nth client, C m Indicates the first local model corresponding to the mth client, C n Indicates the first local model corresponding to the nth client, Pear m Represents the set of the last network layer parameters of the first local model corresponding to the mth client, represents the average value of the last network layer parameters of the first local model corresponding to the mth client, Pear n Represents the set of the last network layer parameters of the first local model corresponding to the nth client, Represents the average value of the last network layer parameters of the first local model corresponding to the nth client.

4. The method according to claim 3, characterized in that The preset sorting rule is to calculate the distance between each unlabeled client and the labeled client in each sub-federation set containing the labeled client according to the Mahalanobis distance algorithm, and sort them according to the size of the distance to obtain the distance sorting result of the clients in each sub-federation set.

5. The method according to claim 4, characterized in that The preset method is that when the tagged client is located at both ends of the distance sorting result, multiple labeling processes are performed in the order of passing to the left or to the right with the tagged client as the endpoint, wherein each labeling process includes: Based on the labeled clients in the sub-federation set and all the unlabeled clients that have been assigned target pseudo-labels, multiple rounds of federated training are performed to obtain the current intermediate global classification model, and the current intermediate global classification model is used to label the unlabeled clients adjacent to the unlabeled clients processed in the last labeling process.

6. The method according to claim 5, characterized in that The preset method is that when the tagged client is located in the middle of the distance sorting result, multiple labeling processes are performed with the tagged client as the endpoint in the order of passing to the left and passing to the right, wherein each labeling process includes: Based on the labeled clients in the sub-federation set and all the unlabeled clients that have been assigned target pseudo-labels in the direction of the current labeling process, multiple rounds of federated training are performed to obtain the current intermediate global classification model, and the current intermediate global classification model is used to label the unlabeled clients adjacent to the unlabeled clients of the previous labeling process in the direction of the current labeling process.

7. The method according to claim 6, characterized in that The default training rules are: The order in which each unlabeled client joins the federation training is determined by the order in which each unlabeled client in each sub-federation set is assigned a target pseudo-label. After executing multiple rounds of federated training on all labeled clients, each unlabeled client that is assigned a target pseudo-label is added to the federated training process in stages according to the order in which each unlabeled client joins the federated training until all unlabeled clients participate in the federated training process; A medical data classification model is obtained by jointly performing multiple rounds of federated training on labeled clients and all unlabeled clients assigned target pseudo labels.

8. A medical data classification method, characterized in that: The method comprises: Step T1: Obtaining medical data to be processed; Step T2: Classify the medical data to be processed using the medical data classification model obtained by the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method according to any one of claims 1 to 8.

10. An electronic device, characterized in that: include: one or more processors; as well as a memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1 to 8 by executing the executable instructions.

Images