A Federated Semi-Supervised Medical Image Diagnostic Method Based on Structural Alignment and Pseudo-Label Self-Correction

Abstract

This invention provides a federated semi-supervised medical image diagnosis method based on structural alignment and pseudo-label self-correction, comprising the following steps: First, medical images from labeled clients are input into a deep network for training a fixed number of times to obtain network model parameters and virtual class centers, which are then uploaded to a server as global initialization settings. Next, the server distributes the global model parameters to both labeled and unlabeled clients, and distributes the global virtual class centers to the unlabeled clients. The virtual class centers are trained and updated on the labeled clients. On the unlabeled clients, a prediction consistency loss between the student and teacher networks is established, a structural matching loss is established to align the global virtual class centers, and a class-aware contrastive loss is established to learn intra-class compactness and inter-class separability. Further, all client model parameters are uploaded and the global model parameters are updated. This invention can improve the accuracy and generalization of federated semi-supervised image detection while ensuring privacy.

Description

Technical Field

[0001] This invention relates to a federal semi-supervised medical image diagnostic method based on structural alignment and pseudo-label self-correction, belonging to the field of medical image classification. Background Technology

[0002] With the rapid development of artificial intelligence technology, deep learning methods have been widely applied in fields such as medical imaging, segmentation, and diagnosis. Although deep learning methods have made significant progress in medical image diagnosis, they typically require a large amount of labeled data for network model training. On the one hand, labeling medical image data often faces high costs and difficulties; on the other hand, uploading data from various medical institutions to a server poses a risk of leaking patient privacy. These problems hinder the application of deep learning in actual clinical medical diagnosis. Federated semi-supervised learning, as a new distributed machine learning paradigm, enables data collaboration among multiple clients (medical institutions) using both labeled and unlabeled data simultaneously, avoiding direct data transfer between clients and effectively protecting patient privacy. However, existing federated semi-supervised methods typically suffer from the following two problems: 1) Due to differences in patient populations, acquisition equipment, or environments, medical image data distributed across various medical institutions exhibits significant distributional differences, leading to insufficient generalization ability of the global model; 2) Due to the lack of supervision information, the use of a large amount of unlabeled data results in poor reliability of local client models, thus affecting the accuracy of the global model. How to effectively and securely collaborate on the large amounts of unlabeled medical image data among medical institutions while ensuring the privacy and security of both doctors and patients has become a key research focus in federated semi-supervised learning. To address these issues, this invention proposes a federated semi-supervised medical image diagnosis method based on structural alignment and pseudo-label self-correction. It designs a cross-client structural information matching strategy and a pseudo-label self-correction strategy, enabling the construction of a federated model using unlabeled client-side medical image segmentation auxiliary models, thereby improving the generalization and reliability of the federated semi-supervised model. Summary of the Invention

[0003] This invention provides a federated semi-supervised medical image diagnosis method based on structural alignment and pseudo-label self-correction. Based on a semi-supervised federated learning paradigm, a cross-client structural information matching strategy based on virtual class centers is designed to achieve consistency of structural information across multiple clients, solving the client drift problem. Furthermore, a pseudo-label self-correction strategy based on class-aware contrastive learning is proposed to address the performance degradation of the network model caused by incorrect predictions from unlabeled clients. The proposed method improves the accuracy of federated semi-supervised medical image diagnosis while protecting user privacy.

[0004] To solve the above problems, the present invention adopts the following technical solution:

[0005] A federally supervised semi-supervised medical image diagnostic method based on structural alignment and pseudo-label self-correction includes the following steps:

[0006] Step 1: m labeled data clients {S1, S2, L, S...} m Each client (S1) collects its own private labeled medical image data, where the l-th client contains N... l Data and label pairs This represents the i-th sample. Represents the label of the i-th sample; n unlabeled data clients {S m+1 S m+2 L, S m+n Each client collects its own private, label-free medical image data, where the u-th client S... u Contains N u Data

[0007] Step 2: Input the medical images of labeled clients into the deep network for a fixed number of training iterations to obtain the network model parameters of the labeled clients. At the same time, use the learned deep features to calculate the virtual class centers of specific classes, and upload the network model parameters of the labeled clients and the virtual class centers to the central server.

[0008] Step 3: The central server uses the network model parameters and virtual class centers uploaded by the tagged clients as the initialization parameters for the global model and the global virtual class centers. The server then distributes the global model parameters to both tagged and untagged clients, and simultaneously distributes the global virtual class centers to the untagged clients.

[0009] Step 4: The labeled client initializes and trains the model using global parameters, calculates virtual class centers for a specific class using the learned deep features, and uploads the updated network model parameters and virtual class centers to the central server.

[0010] Step 5: The unmarked client receives the global model parameters and global virtual class center sent by the server, and uses the global model parameters to initialize the local model;

[0011] Step 6: The labelless client performs random data augmentation on the labelless medical image data, repeating the process twice to obtain two sets of augmented data, which are then input into the student network and teacher network of the average teacher model, respectively. The student network uses the learned deep features to calculate the virtual class center of a specific class, calculates the structural matching loss between each virtual class center and the global virtual class center of the corresponding class, and calculates the prediction consistency loss between the student network and the teacher network. At the same time, the pseudo-label self-correction loss is calculated using the standardized features output by the student network and the teacher network.

[0012] Step 7: The structure matching loss, consistency loss, and pseudo-label self-correction weighted sum calculated in Step 6 are used to optimize the parameters of the unlabeled client model.

[0013] Step 8: Send the unmarked client model parameters to the server. The server uses the marked client model parameters and the unmarked client model parameters to update the global model parameters, and uses the virtual class center of the marked client as the global virtual class center.

[0014] Step 9: Repeat steps 3-8 above until the global model converges.

[0015] Preferably, the virtual class centers of the client are extracted from local data by the client's local network model using deep features. The virtual class centers of each class are extracted by averaging the deep features of a specific class. For the c-th class virtual class center of the labeled data client... Specifically, this can be expressed as:

[0016]

[0017] For the c-th virtual class center of unmarked data clients Specifically, this can be expressed as:

[0018]

[0019] in, For dataset D l The number of samples in class c, 1 [·] It is a conditional filtering function. For network models from samples Deep features of learning in the middle. It is a network model for unlabeled data Probability prediction.

[0020] Preferably, the cross-client structural information matching method involves a central server collecting virtual class centers from all labeled data clients and averaging them to obtain a global virtual class center for each labeled client. The central server then distributes the global virtual class centers to each unlabeled data client, which matches its local virtual class center with the global virtual class center. Specifically, this can be represented as follows:

[0021]

[0022] Where C is the number of categories, C is the global virtual class center of class c.

[0023] Preferably, the pseudo-label self-correction strategy calculates a contrastive loss based on the standardized features output by the unlabeled client student network and the teacher network, which can be specifically expressed as:

[0024]

[0025]

[0026]

[0027] in, and , respectively, are the standardized feature outputs of the student network and the teacher network, Q is a dynamic queue of size C×J, J is the number of samples from each pseudo-class, P(i) represents the set index of all positive samples, and τ is the temperature hyperparameter.

[0028] Beneficial effects:

[0029] 1. This invention introduces a cross-client structure matching strategy. By aligning the virtual class centers of labeled clients with those of unlabeled clients, it achieves consistency of structural information between labeled and unlabeled clients, solves the client drift problem, and can utilize unlabeled clients to construct a federated model of medical image segmentation auxiliary model, thereby improving the generalization of the federated semi-supervised model.

[0030] 2. This invention introduces a pseudo-label self-correction strategy. Based on the similarity between similar data features of unlabeled clients and the heterogeneity between data features of different classes, it corrects erroneous pseudo-labels automatically generated by the network, improving the quality of pseudo-labels and enhancing the robustness of the unlabeled client model. In the proposed semi-supervised federated learning method, each client only needs to pass model parameters and virtual class centers to the server, effectively preventing client privacy leaks. Attached Figure Description

[0031] Figure 1 This is a network framework diagram of a federal semi-supervised medical image diagnosis method based on structural alignment and pseudo-label self-correction in this invention. Detailed Implementation

[0032] The invention will be further explained below with reference to examples.

[0033] The main implementation process of this invention is as follows; see relevant procedures. Figure 1 .

[0034] Step 1: m labeled data clients {S1, S2, L, S...} m Each client collects its own private labeled medical image data, where the l-th client S... l Contains N lData and label pairs This represents the i-th sample. Represents the label of the i-th sample; n unlabeled data clients {S m+1 S m+2 L, S m+n Each client collects its own private, label-free medical image data, where the u-th client S... u Contains N u Data

[0035] Step 2: Input the labeled client's medical images into the deep network for a fixed number of training iterations to obtain the network model parameters for the labeled clients. Simultaneously, use the learned deep features to calculate virtual class centers for specific classes, and upload the labeled client's network model parameters and virtual class centers to the central server.

[0036] 1) The supervised loss is calculated using the probability output predicted by the deep network and the true label to train the network, which can be specifically expressed as:

[0037]

[0038] Where, θ l This indicates the client's network parameters. Indicates network to Probability prediction;

[0039] 2) For the c-th virtual class center of the tag data client Specifically, this can be expressed as:

[0040]

[0041] Virtual class center point The update method is as follows:

[0042]

[0043] in, For dataset D l The number of samples in class c, 1 [·] It is a conditional filtering function. For network models from samples Deep features of learning in the middle. This represents the mean center point of the t-th iteration. Let β represent the center point of the c-th virtual class of the l-th client in the (t+1)-th iteration, and β be the balance parameter, which is 0.002 in this invention.

[0044] Step 3: The central server uses the network model parameters and virtual class centers uploaded by the tagged clients as the initialization parameters for the global model and the global virtual class centers. The server then distributes the global model parameters to both tagged and untagged clients, and simultaneously distributes the global virtual class centers to the untagged clients.

[0045] Step 4: The labeled client initializes and trains the model using global parameters, calculates the virtual class centers of a specific class using the learned deep features, and uploads the updated network model parameters and virtual class centers to the central server; the calculation of network model parameters and virtual class centers is similar to that in Step 2.

[0046] Step 5: The unmarked client receives the global model parameters and global virtual class center sent by the server, and uses the global model parameters to initialize the local model;

[0047] Step 6: The label-free client performs random data augmentation on the label-free medical image data, repeating this process twice to obtain two sets of augmented data. These two sets are then input into the student network and teacher network of the average teacher model, respectively. The student network uses the learned deep features to calculate virtual class centers for specific classes, calculates the structural matching loss between each virtual class center and the global virtual class center of the corresponding class, and calculates the prediction consistency loss between the student network and the teacher network. Simultaneously, the pseudo-label self-correction loss is calculated using the standardized features output by the student network and the teacher network.

[0048] 1) Add data perturbation to unlabeled medical image data by performing random rotation and affine transformation. The image is horizontally flipped with a probability of 0.5, with random rotation angles ranging from -10° to 10°, corresponding to translation interval parameters of (0.02, 0.02) in both length and width dimensions. Repeat this process twice to obtain two sets of training data, which are then input into the student network and teacher network of the average teacher model, respectively.

[0049] 2) The student network uses the learned deep features to calculate the virtual class center of a specific class, which can be specifically represented as:

[0050]

[0051] in, For dataset D u The number of samples in class c, 1 [·] It is a conditional filtering function. For network models from samples Deep features of learning in the middle. It is a network model for unlabeled data Probability prediction.

[0052] 3) Calculate the structural matching loss between each virtual class center and the global virtual class center of the corresponding category, which can be expressed as:

[0053]

[0054] Where C is the number of categories, C is the global virtual class center of class c.

[0055] 4) Calculate the prediction consistency loss between the student network and the teacher network, which can be specifically expressed as:

[0056]

[0057] Here, η and η′ represent different perturbations. θ u and θ′ u This represents the network parameters of the student and teacher networks.

[0058] 5) Calculate the pseudo-label self-correction loss using the standardized features output by the student network and the teacher network, which can be specifically expressed as:

[0059]

[0060]

[0061]

[0062] in, Know , respectively, are the standardized feature outputs of the student network and the teacher network, Q is a dynamic queue of size C×J, J is the number of samples from each pseudo-class, P(i) represents the set index of all positive samples, and τ is the temperature hyperparameter.

[0063] Step 7: The structure matching loss, consistency loss, and pseudo-label self-correction weighted sum calculated in Step 6 are used to optimize the unlabeled client model parameters by summing them together. The specific calculation is as follows:

[0064] L u =λ1·L cs +λ2·L inter +λ3·L intra (16)

[0065] Step 8: Send the unmarked client model parameters to the server. The server will average the marked client model parameters and the unmarked client model parameters to update the global model parameters, and average the virtual class centers of the marked clients to update the global virtual class centers.

[0066] Step 9: Repeat steps 3-8 above until the global model converges.

[0067] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural changes made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A federated semi-supervised medical image diagnosis method based on structural alignment and pseudo-label self-correction, comprising the following steps: Step 1: m labeled data clients {S1, S2, ..., S...} m Each client collects its own private labeled medical image data, where the l-th client S... l Contains N l Data and label pairs Let i represent the i-th sample. Represents the label of the i-th sample; n unlabeled data clients {S m+1 ,S m+2 S m+n Each client collects its own private, label-free medical image data, where the u-th client S... u Contains N u Data Step 2: Input the medical images of labeled clients into the deep network for a fixed number of training iterations to obtain the network model parameters of the labeled clients. At the same time, use the learned deep features to calculate the virtual class centers of specific classes, and upload the network model parameters of the labeled clients and the virtual class centers to the central server. Step 3: The central server uses the network model parameters and virtual class centers uploaded by the tagged clients as the initial global model parameters and global virtual class centers. The server then distributes the global model parameters to both tagged and untagged clients, and simultaneously distributes the global virtual class centers to the untagged clients. Step 4: The labeled client initializes and trains the model using global parameters, calculates virtual class centers for a specific class using the learned deep features, and uploads the updated network model parameters and virtual class centers to the central server. Step 5: The unmarked client receives the global model parameters and global virtual class center sent by the server, and uses the global model parameters to initialize the local model; Step 6: The labelless client performs random data augmentation on the labelless medical image data, repeating the process twice to obtain two sets of augmented data, which are then input into the student network and teacher network of the average teacher model, respectively. The student network uses the learned deep features to calculate the virtual class center of a specific class, calculates the structural matching loss between each virtual class center and the global virtual class center of the corresponding class, and calculates the prediction consistency loss between the student network and the teacher network. At the same time, the pseudo-label self-correction loss is calculated using the standardized features output by the student network and the teacher network. Step 7: The structure matching loss, consistency loss, and pseudo-label self-correction weighted sum calculated in Step 6 are used to optimize the parameters of the unlabeled client model by summing them together. Step 8: Send the unmarked client model parameters to the server. The server uses the marked client model parameters and the unmarked client model parameters to update the global model parameters, and uses the virtual class center of the marked client as the global virtual class center. Step 9: Repeat steps 3-8 above until the global model converges.

2. The federally supervised semi-supervised medical image diagnosis method based on structural alignment and pseudo-label self-correction as described in claim 1, characterized in that, In steps 2 and 3, the virtual class centers of the client are specifically derived from the deep features learned by each client's local network model. The virtual class centers of each class are extracted by averaging the deep features of a specific class. For the labeled data client's virtual class centers in step 2... Specifically, it is expressed as follows: Virtual class center point The update method is as follows: in, For dataset D l The number of samples in class c, 1 [·] It is a conditional filtering function. For network models from samples Deep features of learning in the middle This represents the mean center point of the t-th iteration. Let represent the virtual class center point of the c-th class of the l-th client in the (t+1)-th iteration, and β be the balance parameter, which is set to 0.002 in this invention; define the virtual class center of the unlabeled data client in step 3. Specifically, it is expressed as follows: in, For dataset The number of samples in class c. For network models from samples Deep features of learning in the middle It is a network model for unlabeled data Probability prediction.

3. The federally supervised semi-supervised medical image diagnosis method based on structural alignment and pseudo-label self-correction as described in claim 1, characterized in that, A cross-client structure information matching strategy is introduced. The central server collects virtual class centers from all labeled data clients, averages them to obtain the global virtual class center, and sends the global virtual class center to each unlabeled data client. The unlabeled data client aligns its local virtual class center with the global virtual class center, specifically as follows: Where C is the number of categories, It serves as the global virtual class center for class c.

4. The federally supervised semi-supervised medical image diagnosis method based on structural alignment and pseudo-label self-correction as described in claim 1, characterized in that, A pseudo-label self-correction strategy is introduced, and the contrastive loss is calculated using the standardized features output by the student network and the teacher network, specifically expressed as follows: in, and , respectively, are the standardized feature outputs of the student network and the teacher network, Q is a dynamic queue of size C×J, J is the number of samples from each pseudo-class, P(i) represents the set index of all positive samples, and τ is the temperature hyperparameter.

5. The federally supervised semi-supervised medical image diagnosis method based on structural alignment and pseudo-label self-correction as described in claim 1, characterized in that, The structure matching loss, consistency loss, and pseudo-label self-correction weighted sum calculated in step 6 are used to optimize the parameters of the unlabeled client model, specifically expressed as follows: Wherein, λ1, λ2, and λ3 are the weighting coefficients of consistency loss, structure matching loss, and pseudo-standard self-correction loss, respectively.

Images