Federal construction method and classification method of medical data classification model

Abstract

The invention provides a federal construction method of a medical data classification model, which is used for combining a plurality of labeled clients and a plurality of unlabeled clients to construct the medical data classification model, and is characterized in that the labeled clients indicate that a data set on the clients comprises a plurality of medical data and corresponding disease labels; a label-free client indicates that a data set on the client only comprises a plurality of medical data. The method comprises the following steps: a federated client clustering step is used for clustering clients with similar data distribution into a sub-federated set; the federal semi-supervised self-training step is used for sorting the clients in each sub-federal set, and labeling the non-label clients according to a sorting result so as to enable all medical data of each non-label client to be endowed with target false labels; and the federated client training step is used for executing multiple rounds of federated training based on all the labeled clients and all the label-free clients endowed with the target pseudo labels to obtain a medical data classification model.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and more specifically, to a medical data classification technology in the field of artificial intelligence, and more specifically, to a medical data classification technology based on semi-supervised federated learning, that is, a federated construction method and classification method of a medical data classification model. Background Art

[0002] As the emphasis on data security increases, it becomes increasingly difficult to collect all data for modeling. Federated Learning is a distributed model training framework that can fully protect data privacy, allowing clients to train locally with their own data while sharing model parameters with a central server. After multiple rounds of communication between the server and the client, a better global model is eventually built. Traditional federated learning methods require the client to have complete data and label information. However, it is often difficult to obtain data labels, and some even require professional annotation. Taking medical data as an example, doctors can annotate some data during the patient's hospitalization, but data in other time periods cannot be annotated without the doctor's expertise. A large amount of unlabeled data provides a variety of different data information, while labeled data provides valuable supervision information. In the process of federated modeling, most institutions participating in federated learning often have the ability to annotate data and are willing to annotate data, while individuals with various data cannot participate in the construction of federated models due to the difficulty of annotation. Typical federated learning scenarios have rich labeled data, while real-world scenarios have sparsely distributed labeled data, which leads to serious limitations in existing federated learning methods.

[0003] In order to solve the above defects, some researchers have proposed that a semi-supervised federated learning (SSFL) method based on self-training can be used to label unlabeled data so that the information of unlabeled data can be fully utilized to optimize the model. Specifically, it is to first use the labeled client for federated training to build an initial model; then, use the initial model to assign pseudo labels to the data in the unlabeled client, and add the data with pseudo labels to the federated training; finally, build a data classification model based on all the data. Although the semi-supervised federated learning method based on self-training can assign pseudo labels to the data in the unlabeled client, it is not feasible to simply combine semi-supervised learning with federated learning. This is because there are differences in the data distribution of labeled clients and unlabeled clients. The initial model obtained by using labeled clients for federated modeling will have a high error when assigning pseudo labels to the data of unlabeled clients, resulting in poor pseudo-label accuracy. At the same time, when the data distribution of clients is similar, the semi-supervised federated learning method based on self-training will use the obtained initial model to assign pseudo labels to the data of each unlabeled client by randomly selecting clients. This ignores the order of pseudo-label assignment of unlabeled clients and also leads to poor pseudo-label accuracy.

[0004] In summary, there are still two problems in the prior art: on the one hand, when there is a difference in data distribution between labeled clients and unlabeled clients, the generalization of the initial model trained on the labeled clients is insufficient, resulting in poor accuracy of pseudo-label assignment; on the other hand, when the data distribution of labeled clients and unlabeled clients is similar, pseudo-label assignment is performed by randomly selecting clients. This assignment method ignores the order of pseudo-label assignment of unlabeled clients, which also leads to poor accuracy of pseudo-labels.

[0005] It should be noted that this background technology is only used to introduce the relevant information of the present invention to help understand the technical solution of the present invention, but it does not mean that the relevant information is necessarily the prior art. If there is no evidence that the relevant information has been disclosed before the application date of the present invention, the relevant information shall not be regarded as the prior art. Summary of the invention

[0006] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a federated construction method of a medical data classification model and a medical data classification method.

[0007] The objectives of the present invention are achieved through the following technical solutions.

[0008] According to a first aspect of the present invention, a method for federally constructing a medical data classification model is provided, which is used to jointly construct a medical data classification model with multiple labeled clients and multiple unlabeled clients, wherein a labeled client indicates that the data set on the client includes multiple medical data and their corresponding disease labels, and an unlabeled client indicates that the data set on the client only includes multiple medical data. The method comprises: a federal client clustering step: performing multiple rounds of federal training based on all labeled clients to obtain a first global classification model, using the first global classification model to perform a first round of labeling processing for each unlabeled client so that all medical data on each unlabeled client are assigned an initial pseudo-label; based on all labeled clients and all unlabeled clients assigned initial pseudo-labels, clustering the unlabeled clients and labeled clients according to preset clustering rules to obtain multiple sub-federated sets, wherein each sub-federated set contains multiple clients, and each sub-federated set contains at most one labeled client; a federal semi-supervised self-training step: performing clustering according to the pre-set clustering rules; According to the preset sorting rules, all clients in each sub-federated set containing labeled clients are distance-sorted, and a second round of labeling is performed for each unlabeled client in each sub-federated set containing labeled clients in a preset manner according to the distance sorting result, so that all medical data of each unlabeled client in each sub-federated set containing labeled clients are assigned a target pseudo-label; multiple rounds of federated training are performed based on all labeled clients and unlabeled clients assigned the target pseudo-label to obtain a second global classification model, and the second global classification model is used to perform a third round of labeling for each unlabeled client in each sub-federated set containing only unlabeled clients, so that all medical data of each unlabeled client in each sub-federated set containing only unlabeled clients are assigned a target pseudo-label; federated client training step: based on all labeled clients and all unlabeled clients assigned the target pseudo-label, multiple rounds of federated training are performed according to preset training rules to obtain a medical data classification model.

[0009] In some embodiments of the present invention, the preset clustering rule is: performing multiple rounds of local training on all labeled clients and unlabeled clients assigned initial pseudo-labels to obtain a first local model corresponding to each client, and clustering the unlabeled clients and labeled clients according to a preset clustering algorithm based on the parameters of the first local model corresponding to each client to obtain multiple sub-federation sets, wherein the first local model corresponding to each client includes multiple network layers.

[0010] In some embodiments of the present invention, the preset clustering algorithm is:

[0011]

[0012] Among them, Pearson (C m ,C n ) represents the correlation coefficient between the first local model corresponding to the mth client and the first local model corresponding to the nth client, C m represents the first local model corresponding to the mth client, C n Indicates the first local model corresponding to the nth client, Pear m represents the set of the last network layer parameters of the first local model corresponding to the mth client, represents the average value of the last network layer parameters of the first local model corresponding to the mth client, Pear n Represents the set of the last network layer parameters of the first local model corresponding to the nth client, Represents the average value of the last network layer parameters of the first local model corresponding to the nth client.

[0013] In some embodiments of the present invention, the preset sorting rule is to calculate the distance between each unlabeled client and the labeled client in each sub-federation set containing labeled clients according to the Mahalanobis distance algorithm, and sort them according to the size of the distance to obtain the distance sorting result of the clients in each sub-federation set.

[0014] In some embodiments of the present invention, the preset method is that when a labeled client is located at both ends of the distance sorting result, multiple labeling processes are performed in the order of passing to the left or to the right with the labeled client as the endpoint, wherein each labeling process includes: performing multiple rounds of federated training based on the labeled clients in the sub-federation set and all unlabeled clients that have been assigned target pseudo-labels to obtain the current intermediate global classification model, and using the current intermediate global classification model to perform labeling processing on the unlabeled clients adjacent to the unlabeled clients processed in the previous labeling process.

[0015] In some embodiments of the present invention, the preset method is that when a labeled client is located in the middle of the distance sorting result, multiple labeling processes are performed with the labeled client as the endpoint in the order of passing to the left and passing to the right, wherein each labeling process includes: performing multiple rounds of federated training based on the labeled clients in the sub-federation set and all unlabeled clients that have been assigned target pseudo-labels in the direction of the current labeling process to obtain the current intermediate global classification model, and using the current intermediate global classification model to perform labeling on the unlabeled clients adjacent to the unlabeled clients of the previous labeling process in the direction of the current labeling process.

[0016] In some embodiments of the present invention, the preset training rules are: determining the order in which each unlabeled client joins the federated training according to the processing order in which the target pseudo-labels are assigned to each unlabeled client in each sub-federated set; after performing multiple rounds of federated training based on all labeled clients, adding each unlabeled client assigned the target pseudo-label to the federated training process in stages according to the order in which each unlabeled client joins the federated training until all unlabeled clients participate in the federated training process; and obtaining a medical data classification model by jointly performing multiple rounds of federated training based on labeled clients and all unlabeled clients assigned the target pseudo-labels.

[0017] According to the second aspect of the present invention, a medical data classification method is provided, the method comprising: step T1, obtaining medical data to be processed; step T2, using the medical data classification model obtained by the method described in the first aspect of the present invention to classify the medical data to be processed.

[0018] Compared with the prior art, the advantages of the present invention are: (1) by clustering clients, clients with similar data distribution are clustered into a sub-federation set to reduce the problem of poor pseudo-label assignment accuracy caused by different data distributions of clients; (2) the clients in each sub-federation set are sorted by similarity, and pseudo-labels are assigned to unlabeled clients according to the similarity sorting results, further improving the accuracy of pseudo-label assignment; (3) during the construction process of the medical data classification model, data information with higher confidence is learned first, avoiding the noise problem caused by erroneous pseudo-labels, thereby improving the classification accuracy of the medical data classification model. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0020] Figure 1 A schematic diagram of a process flow of a method for constructing a federation of a medical data classification model according to an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of an example of a federated client clustering step according to an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of an example of a client correlation coefficient color matrix according to an embodiment of the present invention;

[0023] Figure 4 Schematic diagram of an example of a federated semi-supervised self-training step according to an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of an example of a federated client training step according to an embodiment of the present invention;

[0025] Figure 6 A schematic diagram of a flow chart of a medical data classification method according to an embodiment of the present invention;

[0026] Figure 7 This is a diagram showing an example of the experimental test set data distribution. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0028] As mentioned in the background technology section, there are still two problems in the prior art: on the one hand, when there is a difference in data distribution between labeled clients and unlabeled clients, the initial model trained on the labeled clients is not general enough, resulting in poor accuracy of pseudo-label assignment; on the other hand, when the data distribution of labeled clients and unlabeled clients is similar, pseudo-label assignment is performed randomly. This assignment method ignores the order of pseudo-label assignment of unlabeled clients, which also leads to poor accuracy of pseudo-labels.

[0029] In order to solve the defects of the prior art, the inventor analyzed the prior art and found that due to the difference in data distribution between labeled clients and unlabeled clients, the initial model constructed using labeled clients will have a high error when assigning pseudo labels to unlabeled clients, and the wrong pseudo labels will bring more noise information, which is not conducive to the construction of medical data classification models. Therefore, the inventor clustered the clients with obvious differences in data distribution, so that the clients with similar data distribution were divided into a sub-federation set and then assigned pseudo labels to improve the accuracy of pseudo label assignment. Furthermore, in the case of similar data distribution of clients, the prior art usually adopts a random method to assign pseudo labels to each client, while ignoring the order of label assignment between clients, which will lead to poor pseudo label accuracy. In order to improve the accuracy of pseudo label assignment, the inventor sorted the clients with similar data distribution, and assigned pseudo labels to the sorted results by client to improve the accuracy of pseudo label assignment. Based on the analysis of the prior art, the inventors proposed a federated construction scheme based on client clustering and similarity sorting to train a medical data classification model. In this scheme, first, clients with similar data distribution are clustered into a sub-federated set by client clustering to reduce the problem of poor pseudo-label assignment accuracy caused by different data distributions of clients; then, the clients in each sub-federated set are sorted by similarity, and pseudo-labels are assigned to unlabeled clients according to the similarity sorting results to further improve the accuracy of pseudo-label assignment; finally, the unlabeled clients and labeled clients after pseudo-label assignment are used to jointly construct a medical image classification model.

[0030] In terms of probability, Figure 1 As shown, the present invention provides a federal construction method for a medical data classification model, which is used to jointly construct a medical data classification model with multiple labeled clients and multiple unlabeled clients, wherein a labeled client indicates that the data set on the client includes multiple medical data and their corresponding disease labels, and an unlabeled client indicates that the data set on the client only includes multiple medical data. The method includes: a federal client clustering step: based on all labeled clients, multiple rounds of federal training are performed to obtain a first global classification model, and the first global classification model is used to perform a first round of labeling processing for each unlabeled client so that all medical data on each unlabeled client are assigned an initial pseudo-label; based on all labeled clients and all unlabeled clients assigned initial pseudo-labels, the unlabeled clients and labeled clients are clustered according to preset clustering rules to obtain multiple sub-federal sets, wherein each sub-federal set contains multiple clients, and each sub-federal set contains at most one labeled client; a federal semi-supervised self-training step: according to the preset A sorting rule is set to perform distance sorting on all clients in each sub-federated set containing labeled clients, and a second round of labeling is performed for each unlabeled client in each sub-federated set containing labeled clients in a preset manner according to the distance sorting result, so that all medical data of each unlabeled client in each sub-federated set containing labeled clients are assigned a target pseudo-label; multiple rounds of federated training are performed based on all labeled clients and unlabeled clients assigned the target pseudo-label to obtain a second global classification model, and the second global classification model is used to perform a third round of labeling for each unlabeled client in each sub-federated set containing only unlabeled clients, so that all medical data of each unlabeled client in each sub-federated set containing only unlabeled clients are assigned the target pseudo-label; federated client training step: based on all labeled clients and all unlabeled clients assigned the target pseudo-label, multiple rounds of federated training are performed according to preset training rules to obtain a medical data classification model.

[0031] In order to better understand the present invention, each step is described in detail below in conjunction with specific embodiments.

[0032] 1. Federated Client Clustering Steps

[0033] The federated client clustering step is used to cluster clients with similar data distribution into a sub-federated set, where a sub-federated set includes multiple clients and a sub-federated set contains at most one labeled client. It should be noted that the client clustering operation can improve the accuracy of subsequent target pseudo-label assignment and reduce noise interference. Figure 2The federated client clustering steps are shown as an example to introduce the specific implementation process.

[0034] like Figure 2 As shown, first, the clients are divided into labeled clients and unlabeled clients.

[0035] Then, 1-5 rounds of federated training are performed based on all labeled clients to obtain the first global classification model, wherein the FedAvg algorithm can be used for federated aggregation during the federated training process. It should be noted that the number of rounds of federated training is not limited to 1-5 times, and the number of training rounds can be determined according to actual needs. At this time, the first global classification model trained is only used for basic classification, and the classification accuracy of the model does not need to be guaranteed.

[0036] Then, the first global classification model is used to perform the first round of labeling for each unlabeled client so that all medical data on each unlabeled client are assigned initial pseudo labels. It should be noted that during the first round of labeling, the initial pseudo labels assigned to the medical data do not have a high confidence level but may have a large classification error. Although there is a certain classification error, the clients can still be clustered according to the underlying data distribution of each client after being assigned the initial pseudo labels. The purpose of assigning the initial pseudo labels is to cluster the clients according to data similarity, and the classification accuracy does not affect the clustering results.

[0037] Finally, based on all labeled clients and all unlabeled clients assigned initial pseudo-labels, clustering is performed on the unlabeled clients and labeled clients according to preset clustering rules to obtain multiple sub-federated sets, wherein each sub-federated set contains multiple clients, and each sub-federated set contains at most one labeled client. According to one embodiment of the present invention, the preset clustering rule is: based on all labeled clients and unlabeled clients assigned initial pseudo-labels, multiple rounds of local training are performed to obtain the first local model corresponding to each client, and based on the parameters of the first local model corresponding to each client, the unlabeled clients and labeled clients are clustered according to a preset clustering algorithm to obtain multiple sub-federated sets, wherein the first local model corresponding to each client includes multiple network layers.

[0038] According to one embodiment of the present invention, the preset clustering algorithm is:

[0039]

[0040] Among them, Pearson (C m ,C n ) represents the correlation coefficient between the first local model corresponding to the mth client and the first local model corresponding to the nth client, C mrepresents the first local model corresponding to the mth client, C n Indicates the first local model corresponding to the nth client, Pear m represents the set of the last network layer parameters of the first local model corresponding to the mth client, represents the average value of the last network layer parameters of the first local model corresponding to the mth client, Pear n Represents the set of the last network layer parameters of the first local model corresponding to the nth client, represents the average value of the last network layer parameter of the first local model corresponding to the nth client. It should be noted that since the last network layer parameter of the first local model corresponding to each client ( Figure 2 In ) represents the distribution of the underlying data on each client, so the correlation coefficient can be solved based on the parameters of the last network layer, and several clients with larger correlation coefficients are assigned to the same sub-federation set (this process is abstractly represented as Figure 2 After the clustering operation, each client is abstractly represented as a point of different colors, where the green points represent multiple clients aggregated into a sub-federation set, the blue points represent multiple clients aggregated into a sub-federation set, and the orange points represent multiple clients aggregated into a sub-federation set).

[0041] In order to better understand how to cluster clients based on correlation coefficients, the following is an example of the correlation coefficients between the 9 clients shown in Table 1. 0-9 in Table 1 represent the client numbers. In Table 1, each row represents the correlation coefficient between the corresponding client and other clients, and the correlation coefficient between each client and itself is 1 (maximum correlation). Among them, the first row represents the correlation coefficient between client 0 and all clients. In this row, the correlation coefficient between client 0 and clients 5, 6, and 7 is positive, indicating that client 0 has certain similarities with clients 5, 6, and 7, and these clients can be considered to be classified into a sub-federation set; the second row represents the correlation coefficient between client 1 and all clients. In this row, the correlation coefficient between client 1 and clients 2, 6, and 9 is positive, indicating that client 1 has certain similarities with clients 2, 6, and 9, and these clients can be considered to be classified into a sub-federation set; the meanings represented by other rows are consistent with the above description, so I will not go into details here. As can be seen from Table 1, when the correlation coefficient is positive, there is a certain similarity between the two corresponding clients. However, considering that many clients have a certain similarity, the clients with larger correlation coefficients are allocated to a sub-federation set. For example, the correlation coefficient between client 0 and client 6 is 0.11, the correlation coefficient between client 1 and client 6 is 0.94, the correlation coefficient between client 2 and client 6 is 0.47, the correlation coefficient between client 7 and client 6 is 0.14, and the correlation coefficient between client 9 and client 6 is 0.48. At this time, client 1 and client 6 can be divided into a sub-federation set. Clients 0, 2, 7, and 9 have higher correlation coefficients with other clients, so they are not divided into the same sub-federation set. The correlation coefficient can also be combined with the depth of color to cluster clients, such as Figure 3 As shown, it shows the color matrix corresponding to the correlation coefficients between the 9 clients corresponding to Table 1. Figure 3 It can be seen that the colors of the two clients with larger correlation coefficients are darker, and the colors of the two clients with smaller correlation coefficients are lighter.

[0042] Table 1

[0043] 0 1 -0.21 -0.82 -0.41 -0.81 0.4 0.11 1 -0.17 -0.82 1 -0.21 1 0.73 -0.8 -0.41 -0.97 0.94 -0.18 -0.92 0.73 2 -0.82 0.73 1 -0.18 0.32 -0.85 0.47 -0.8 -0.43 1 3 -0.41 -0.8 -0.18 1 0.87 0.66 -0.95 -0.44 0.96 -0.19 4 -0.81 -0.41 0.32 0.87 1 0.21 -0.68 -0.82 0.17 0.32 5 0.4 -0.97 -0.85 0.66 0.21 1 -0.85 0.36 0.83 -0.85 6 0.11 0.94 0.47 -0.95 -0.68 -0.85 1 0.14 -0.99 0.48 7 1 -0.18 -0.8 -0.44 -0.82 0.36 0.14 1 -0.2 -0.8 8 -0.17 -0.92 -0.43 0.96 0.71 0.83 -0.99 -0.2 1 -0.43 9 -0.82 0.73 1 -0.19 0.32 -0.85 0.48 -0.8 -0.43 1 Label 0 1 2 3 4 5 6 7 8 9

[0044] 2. Federated Semi-Supervised Self-Training Steps

[0045] The federated semi-supervised self-training step is used to sort the clients in each sub-federated set, and label the unlabeled clients with the sorting results so that all medical data of each unlabeled client is assigned a target pseudo-label. It should be noted that in the federated semi-supervised self-training step, multiple target pseudo-label assignments are performed according to the distance sorting results, so that the overall distribution of clients in the sub-federated set is more similar, further improving the accuracy of pseudo-label assignment, which is beneficial to the subsequent construction of medical data classification models. Figure 4 The example of federated semi-supervised self-training steps is shown to introduce the specific implementation process.

[0046] First, all clients in each sub-federation set that includes labeled clients are distance-sorted according to a preset sorting rule. According to one embodiment of the present invention, the preset sorting rule is to calculate the distance between each unlabeled client and the labeled client in each sub-federation set that includes labeled clients according to the Mahalanobis distance algorithm, and to obtain the distance sorting result of the clients in each sub-federation set by sorting according to the size of the distance. It should be noted that during the distance calculation process, the mean of the data of each client in each sub-federation set is first solved and used as the baseline data of each client, and then the Mahalanobis distance algorithm is used to calculate the distance between the baseline data of each unlabeled client in each sub-federation set and the baseline data of the labeled client and sort them. When sorting, the labeled client is used as the benchmark. When the distance value is negative, it is sorted on the left side of the labeled client, and when the distance value is positive, it is sorted on the right side of the labeled client. Figure 4 Taking the sub-federation sorting results shown as an example, the sorting result of sub-federation set 1 is: client 1 < client 2 < client 3; the sorting result of sub-federation set 2 is: client 4 < client 5 < client 6. Sub-federation set 3 does not contain any labeled clients and is not sorted.

[0047] Then, according to the distance sorting results, a second round of labeling is performed for each unlabeled client in each sub-federation set containing labeled clients in a preset manner, so that all medical data of each unlabeled client in each sub-federation set containing labeled clients are assigned target pseudo-labels.

[0048] According to one embodiment of the present invention, the preset method is that when a labeled client is located at both ends of the distance sorting result, multiple labeling processes are performed in the order of passing to the left or to the right with the labeled client as the endpoint, wherein each labeling process includes: performing multiple rounds of federation training based on the labeled clients in the sub-federation set and all unlabeled clients that have been assigned target pseudo-labels to obtain the current intermediate global classification model, and using the current intermediate global classification model to perform labeling processing on the unlabeled clients adjacent to the unlabeled clients processed in the previous labeling process.

[0049] In order to better understand the labeling process when the labeled client is at both ends of the distance sorting result, Figure 4 The labeling process of sub-federation set 1 is shown as an example. Figure 4 In the example, the distance ranking result of the clients in the sub-federation set 1 is client 1 < client 2 < client 3, where client 1 is a labeled client. The specific process of labeling unlabeled client 2 and unlabeled client 3 based on the ranking result of sub-federation set 1 is as follows: First, multiple rounds of federation training are performed based on client 1 to obtain the current intermediate global classification model ( Figure 4 The model represented by blue in the middle will be continuously updated); the intermediate global classification model is then used to classify the medical data in client 2, and the classification results with a classification probability greater than or equal to 0.9 are selected from the classification results output by the model as pseudo labels to label the medical data in client 2; the medical data of client 2 that has been assigned the target pseudo label is then added to the federated training process and the medical data of client 1 is updated together with the current intermediate global classification model ( Figure 4 The updated current intermediate global classification model is used to label the remaining medical data in Client 2. The classification results with a classification probability greater than or equal to 0.9 are still selected from the classification results output by the model as pseudo labels to continue labeling the medical data in Client 2. This process is repeated until all medical data in Client 2 are assigned the target pseudo labels, completing the pseudo label assignment for Client 2. Then, multiple rounds of federated training are performed based on all the medical data of Client 2 and the medical data of Client 1 that are assigned pseudo labels to obtain the current intermediate global classification model ( Figure 4 The model represented by blue in the middle) classifies the medical data of client 3, and still selects the classification results with a classification probability greater than or equal to 0.9 from the classification results output by the model as pseudo labels to label the medical data in client 3; then the medical data of client 3 that has been assigned the target pseudo label is added to the federated training process and the medical data of client 1 and client 2 are updated together with the current intermediate global classification model ( Figure 4The model represented by purple in the figure is continuously updated) and the updated current intermediate global classification model ( Figure 4 The model represented by purple in the figure is used to label the remaining medical data in client 3, and the classification results with a classification probability greater than or equal to 0.9 are still selected from the classification results output by the model as pseudo labels to label the medical data in client 3. The above process is repeated until all medical data in client 3 are assigned the target pseudo labels, and the pseudo labels for client 3 are assigned. It should be noted that in the labeling processing method when the labeled clients are located at both ends of the distance sorting result, one labeling processing operation refers to labeling all medical data in one client.

[0050] According to one embodiment of the present invention, the preset method is that when a labeled client is located in the middle of the distance sorting result, multiple labeling processes are performed with the labeled client as the endpoint in the order of passing to the left and passing to the right, wherein each labeling process includes: performing multiple rounds of federated training based on the labeled clients in the sub-federation set and all unlabeled clients that have been assigned target pseudo-labels in the direction of the current labeling process to obtain the current intermediate global classification model, and using the current intermediate global classification model to perform labeling on the unlabeled clients adjacent to the unlabeled clients of the previous labeling process in the direction of the current labeling process.

[0051] In order to better understand the labeling process when a tagged client is in the middle of the distance sorting result, Figure 4 The labeling process of sub-federation set 2 is shown as an example. Figure 4 In the example, the distance ranking result in sub-federation set 2 is client 4 < client 5 < client 6, where client 5 is a labeled client. The specific process of labeling the unlabeled client 4 and the unlabeled client 6 based on the distance ranking result of sub-federation set 2 is as follows: First, the current intermediate global classification model ( Figure 4 The model represented by green in the figure will be continuously updated); the model is then used to classify the medical data in client 4 and client 6 at the same time, and the classification results with a classification probability greater than or equal to 0.9 are selected from the classification results output by the model as pseudo labels to label the medical data in client 4 and client 6; the medical data in client 4 and client 6 that have been assigned the target pseudo labels are then added to the federated training process and the medical data in client 5 are used to update the current intermediate global classification model ( Figure 4The model represented by green in the middle), and the updated current intermediate global classification model is used to classify the remaining medical data in Client 4 and Client 6. The classification results with a classification probability greater than or equal to 0.9 are still selected from the classification results output by the model as pseudo labels to label the medical data in Client 4 and Client 6; the aforementioned operations are repeated until all medical data in Client 4 and Client 6 are assigned target pseudo labels. Figure 4 The client distance sorting results shown in are relatively simple. When there are more complex distance sorting results, for example, when client C is a labeled client and the distance sorting result is client A<client B<client C<client E<client F, at this time, firstly, based on client C, client B and client E are labeled at the same time according to the aforementioned processing process, and then based on client B, client C and client E, client A and client F are labeled at the same time according to the aforementioned process. The labeling processes corresponding to other distance sorting results are similar, but they will not be repeated here. It should also be noted that in the labeling processing method when the labeled client is located in the middle of the distance sorting result, one labeling processing operation refers to labeling all medical data of unlabeled clients adjacent to the unlabeled client processed by the last labeling in the direction of the labeling processing. Still taking client A<client B<client C<client E<client F as an example, one labeling processing at this time is labeling client B and client E based on client C, or labeling client A and client F based on client C, client B and client E.

[0052] Finally, multiple rounds of federated training are performed based on all labeled clients and unlabeled clients assigned target pseudo-labels to obtain a second global classification model, and the second global classification model is used to perform a third round of labeling processing for each unlabeled client in each sub-federated set that only contains unlabeled clients, so that all medical data of each unlabeled client in each sub-federated set that only contains unlabeled clients are assigned target pseudo-labels.

[0053] To better understand the labeling process for a sub-federation containing only unlabeled clients, Figure 4 The labeling process shown in the sub-federation set 3 in the example is used for explanation. The sub-federation set 3 includes unlabeled client 7 and unlabeled client 8. The labeling process of this sub-federation set is as follows: first, multiple rounds of federation training are performed based on client 1 and client 2 and client 2, client 3, client 4 and client 6 assigned with target pseudo labels to obtain the second global classification model ( Figure 4The model represented by brown in the middle), and the second global classification model is used to classify the unlabeled client 7 and the unlabeled client 8 at the same time, and the classification results with a classification probability greater than or equal to 0.9 are selected from the classification results output by the model as pseudo labels to label the medical data in client 7 and client 8; then, the medical data in client 7 and client 8 that have been assigned the target pseudo labels are added to the federated training process and the medical data in clients 1-6 are updated together with the second global classification model ( Figure 4 The model indicated in red) is used, and the remaining medical data in Client 7 and Client 8 are classified using the updated second global classification model, and the classification results with a classification probability greater than or equal to 0.9 are still selected from the classification results output by the model as pseudo labels to label the medical data in Client 7 and Client 8; the aforementioned operations are repeated until all medical data in Client 7 and Client 8 are assigned the target pseudo labels. It should be noted that in the process of labeling the sub-federation set containing only unlabeled clients, all sub-federation sets containing only unlabeled clients will be combined into one sub-federation set for labeling at the same time.

[0054] It should be noted that in the federated semi-supervised self-training step, no matter what method is used to label the medical data in the unlabeled client, there may be a situation where the classification probability of some medical data is always less than 0.9. At this time, you can choose to discard the medical data.

[0055] 3. Federated Client Training Steps

[0056] The federated client training step is used to perform multiple rounds of federated training based on all labeled clients and all unlabeled clients assigned target pseudo-labels according to preset training rules to obtain a medical data classification model. According to one embodiment of the present invention, the preset training rules are: determine the order in which each unlabeled client joins the federated training according to the processing order in which each unlabeled client in each sub-federated set is assigned a target pseudo-label; after performing multiple rounds of federated training based on all labeled clients, each unlabeled client assigned a target pseudo-label is added to the federated training process in stages according to the order in which each unlabeled client joins the federated training until all unlabeled clients participate in the federated training process; perform multiple rounds of federated training based on labeled clients and all unlabeled clients assigned target pseudo-labels to obtain a medical data classification model. It should be noted that clients with high label quality are given priority to join the federated training, and then unlabeled clients assigned target pseudo-labels are gradually added, so that the medical data classification model can learn data information with higher confidence first during the construction process, avoid the noise problem caused by erroneous pseudo-labels, and thus improve the classification accuracy of the medical data classification model.

[0057] To better understand the federated client training steps, Figure 5 The federated client training steps shown are used to introduce the specific execution process, where: Figure 5 The clients involved are Figure 4 Individual clients shown after the federated semi-supervised self-training step. Figure 5 The steps of federated client training are as follows: First, multiple rounds of federated training are performed based on labeled client 1 and client 5 to obtain the medical data classification model of stage 1 ( Figure 5 Then, the unlabeled clients 2, 4, and 6, which are given the target pseudo-label, are added to the federated training process and multiple rounds of federated training are performed together with the labeled clients 1 and 5 to obtain the medical data classification model of stage 2 ( Figure 5 Then, the unlabeled client 3 assigned with the target pseudo-label is added to the federated training process and multiple rounds of federated training are performed together with the unlabeled client 2, unlabeled client 4, unlabeled client 6, labeled client 1 and client 5 to obtain the medical data classification model of stage 3 ( Figure 5 Then, unlabeled client 7 and unlabeled client 8 are added to the federated training process and multiple rounds of federated training are performed together with clients 1-6 to obtain the medical data classification model of stage 4 ( Figure 5 The model represented by red in the figure); finally, multiple rounds of federated training are performed based on clients 1-8 to obtain the final medical data classification model. It should be noted that during the federated semi-supervised self-training process, client 1 in sub-federated set 1 first assigns a target pseudo-label to client 2, and client 5 in sub-federated set 2 assigns a target pseudo-label to client 4 and client 6 at the same time. Therefore, clients 2, 4, and 6 are the first to join the federated training (sub-federated set 1 and sub-federated set 2 perform pseudo-label assignment processing in parallel); clients 1 and 2 in sub-federated set 1 assign target pseudo-labels to client 3, so client 3 joins the federated training as the second priority; clients 7 and 8 in sub-federated set 3 assign target pseudo-labels based on clients 1-6, so clients 7-8 join the federated training as the third priority.

[0058] During the training of the federated client, the communication rounds of the clients participating in the federated training are controlled, so that the model can give priority to fitting clean data (labeled clients) rather than noisy data (pseudo-labeled data), thereby making the final trained medical data classification model have better data classification capabilities. It should also be noted that during the training of the federated client, FedProx is used for federated aggregation, which is specifically expressed as follows:

[0059]

[0060] in, represents the federated aggregation optimization objective, F k (w) represents the cross entropy loss of the kth local model, w represents the parameters of the local medical data classification model, and w t represents the parameters of the global medical data classification model, and μ represents the hyperparameter.

[0061] The medical data classification model trained based on the above-mentioned embodiment can be used for medical data classification. Figure 6 As shown, the present invention proposes a medical data classification method, the method comprising: step T1, obtaining the medical data to be processed; step T2, using the medical data classification model obtained by the method described in the above embodiment to classify the medical data to be processed. It should be noted that the medical data to be processed can be medical image data or medical text data, wherein the medical image data includes X-ray images, computer tomography images, ultrasound images, magnetic resonance images, nuclear medicine images and other data; the medical text data includes medical history, clinical examination, diagnosis plan and prescription data.

[0062] In order to better understand the difference between the present invention and the prior art, the following experiments are conducted using different data sets to compare the difference between the prior art and the present invention. It should be noted that the same experimental configuration is used in the experimental process, and the ResNet18 network is used as the backbone network of the classification network.

[0063] Before conducting the experiment, the existing CIFAR-10 and PAD-UFES-20 datasets were selected as experimental datasets. The CIFAR-10 dataset contains 60,000 32x32 color images, which are divided into 10 categories, with 6,000 images in each category; the PAD-UFES-20 dataset contains 2,298 skin disease images of 1,373 patients taken by various smartphones, involving 6 categories, of which 3 categories are skin cancer and the other 3 categories are skin diseases.

[0064] In the experiment based on the CIFAR-10 dataset, 5000 images were randomly selected from the CIFAR-10 dataset as the validation set and 5000 images as the test set, and the remaining images were used as the test set. Then, the test set was distributed to 10 clients in a non-independent and identically distributed manner according to a hybrid distribution method that combines Dirichlet distribution and Gaussian distribution. The distribution results are shown in Figure 2. Figure 7As shown; then, respectively set the scenarios including one labeled client, two labeled clients and three labeled clients to perform the federated client clustering step, the federated semi-supervised self-training step and the federated client training step described in the above embodiment to obtain the corresponding medical data classification model, wherein the client with less data is selected as the labeled client, for example, when only one labeled client is included, Figure 7 Client 4 in the example is used as a labeled client. The selection of labeled clients in other cases is similar and will not be elaborated in detail. Then, the corresponding medical data classification model is trained in the cases of one labeled client, two labeled clients and three labeled clients according to the existing federated learning method (baseline), semi-supervised federated learning method (semi-federated) and centralized semi-supervised learning method (semi). Finally, the classification accuracy of the corresponding medical data classification model trained by the present invention and the prior art under the same situation is compared by using the test set and the validation set. The experimental results are shown in Table 2. NC in Table 2 l Indicates the number of clients with labels, NC u It indicates the number of unlabeled clients, Model-Acc refers to the accuracy of the medical data classification model on the test set, and PL-Acc refers to the accuracy of the target pseudo-label assigned during the experiment. As shown in Table 2, the present invention has higher classification accuracy and labeling accuracy than the federated learning method (baseline) and semi-supervised federated learning method (semi-federated) in the prior art. The centralized semi-supervised learning method does not participate in the experimental comparison, which indicates the upper limit of the classification accuracy and labeling accuracy that can be achieved under the prior art.

[0065] Table 2

[0066]

[0067] In the process of conducting experiments on the PAD-UFES-20 dataset, the training set, validation set, and test set are first divided into training set, validation set, and test set in a ratio of 7:3:1; then, the test set is distributed to 6 clients in a non-independent and identically distributed manner according to a mixed distribution method that combines Dirichlet distribution and Gaussian distribution; then, the federated client clustering step, the federated semi-supervised self-training step, and the federated client training step described in the aforementioned embodiment are respectively set up to perform the scenarios including one labeled client and two labeled clients to obtain the corresponding medical data classification model, wherein the labeled client selection method is consistent with the aforementioned experiment and will not be repeated here; then, according to the existing federated learning method The method (baseline), semi-supervised federated learning method (semi-federated) and centralized semi-supervised learning method (semi) are used to train corresponding medical data classification models in the case of one labeled client and two labeled clients; finally, the classification accuracy of the corresponding medical data classification models trained by the present invention and the prior art under the same situation are compared using the test set and the validation set. The experimental results are shown in Table 3. It can be seen from Table 3 that the present invention has higher classification accuracy than the federated learning method (baseline) and semi-supervised federated learning method (semi-federated) under the prior art.

[0068] Table 3

[0069]

[0070] Compared with the prior art, the beneficial effects of the present invention are: (1) by clustering clients, clients with similar data distribution are clustered into a sub-federation set to reduce the problem of poor pseudo-label assignment accuracy caused by different data distributions of clients; (2) the clients in each sub-federation set are sorted by similarity, and pseudo-labels are assigned to unlabeled clients according to the similarity sorting results, further improving the accuracy of pseudo-label assignment; (3) during the construction process of the medical data classification model, data information with higher confidence is learned first, avoiding the noise problem caused by erroneous pseudo-labels, thereby improving the classification accuracy of the medical data classification model.

[0071] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0072] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0073] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0074] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for constructing a medical data classification model by federation, which is used to construct a medical data classification model by combining multiple labeled clients and multiple unlabeled clients, wherein: The labeled client indicates that the data set on the client includes multiple medical data and their corresponding disease labels, and the unlabeled client indicates that the data set on the client only includes multiple medical data. The method includes: Federated client clustering steps: A first global classification model is obtained by performing multiple rounds of federated training on all labeled clients, and the first global classification model is used to perform a first round of labeling processing on each unlabeled client so that all medical data on each unlabeled client are assigned initial pseudo labels; Based on all labeled clients and all unlabeled clients assigned with initial pseudo labels, cluster the unlabeled clients and labeled clients according to a preset clustering rule to obtain multiple sub-federation sets, where each sub-federation set contains multiple clients, and each sub-federation set contains at most one labeled client; Federated semi-supervised self-training steps: According to the preset sorting rule, all clients in each sub-federation set containing labeled clients are sorted by distance, and according to the distance sorting result, a second round of labeling is performed for each unlabeled client in each sub-federation set containing labeled clients in a preset manner, so that all medical data of each unlabeled client in each sub-federation set containing labeled clients are assigned a target pseudo-label; Perform multiple rounds of federated training based on all labeled clients and unlabeled clients assigned target pseudo labels to obtain a second global classification model, and use the second global classification model to perform a third round of labeling processing for each unlabeled client in each sub-federated set that only includes unlabeled clients, so that all medical data of each unlabeled client in each sub-federated set that only includes unlabeled clients are assigned target pseudo labels; Federated client training steps: Based on all labeled clients and all unlabeled clients assigned target pseudo-labels, multiple rounds of federated training are performed according to preset training rules to obtain a medical data classification model.

2. The method according to claim 1, characterized in that The preset clustering rule is: Based on all labeled clients and unlabeled clients assigned initial pseudo-labels, multiple rounds of local training are performed respectively to obtain the first local model corresponding to each client, and based on the parameters of the first local model corresponding to each client, the unlabeled clients and labeled clients are clustered according to a preset clustering algorithm to obtain multiple sub-federal sets, wherein the first local model corresponding to each client includes multiple network layers.

3. The method according to claim 2, characterized in that The preset clustering algorithm is: Among them, Pearson (C m ,C n ) represents the correlation coefficient between the first local model corresponding to the mth client and the first local model corresponding to the nth client, C m represents the first local model corresponding to the mth client, C n Indicates the first local model corresponding to the nth client, Pear m represents the set of the last network layer parameters of the first local model corresponding to the mth client, represents the average value of the last network layer parameters of the first local model corresponding to the mth client, Pear n Represents the set of the last network layer parameters of the first local model corresponding to the nth client, Represents the average value of the last network layer parameters of the first local model corresponding to the nth client.

4. The method according to claim 3, characterized in that The preset sorting rule is to calculate the distance between each unlabeled client and the labeled client in each sub-federation set containing the labeled client according to the Mahalanobis distance algorithm, and sort them according to the distance to obtain the distance sorting result of the clients in each sub-federation set.

5. The method according to claim 4, characterized in that The preset method is that when the tagged client is located at both ends of the distance sorting result, multiple labeling processes are performed in the order of passing to the left or to the right with the tagged client as the endpoint, wherein each labeling process includes: Based on the labeled clients in the sub-federation set and all the unlabeled clients that have been assigned the target pseudo-label, multiple rounds of federated training are performed to obtain the current intermediate global classification model, and the current intermediate global classification model is used to label the unlabeled clients adjacent to the unlabeled clients that were previously labeled.

6. The method according to claim 5, characterized in that The preset method is that when the tagged client is located in the middle of the distance sorting result, the tagged client is used as an endpoint to perform multiple labeling processes in the order of passing to the left and passing to the right, wherein each labeling process includes: Based on the labeled clients in the sub-federation set and all the unlabeled clients that have been assigned target pseudo-labels in the direction of the current labeling process, multiple rounds of federated training are performed to obtain the current intermediate global classification model, and the current intermediate global classification model is used to label the unlabeled clients adjacent to the unlabeled clients of the previous labeling process in the direction of the current labeling process.

7. The method according to claim 6, characterized in that The default training rules are: The order in which each unlabeled client joins the federation training is determined according to the processing order in which each unlabeled client in each sub-federation set is assigned a target pseudo-label; After executing multiple rounds of federated training based on all labeled clients, each unlabeled client assigned with the target pseudo-label is added to the federated training process in stages according to the order in which each unlabeled client joins the federated training until all unlabeled clients participate in the federated training process; A medical data classification model is obtained by jointly performing multiple rounds of federated training on labeled clients and all unlabeled clients assigned target pseudo-labels.

8. A medical data classification method, characterized in that: The method comprises: Step T1, obtaining medical data to be processed; Step T2: classify the medical data to be processed using the medical data classification model obtained by the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of the method according to any one of claims 1 to 8.

10. An electronic device, characterized in that: include: one or more processors; as well as A memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1 to 8 by executing the executable instructions.

Images