Self-supervised pre-training method based on federal learning

Abstract

The invention provides a self-supervised pre-training method based on federal learning, so as to improve the liver cancer image segmentation effect. Medical image segmentation aims at accurately extracting a region of interest to assist diagnosis and treatment, but a traditional method depends on a large amount of labeled data and is difficult to obtain in the medical field. According to the method, a ViT encoder is used for carrying out feature learning on an unmarked liver cancer masking image, masking pixel blocks are reconstructed through a ViT decoder, and then embedded features are transmitted to a downstream task; a current local model and a previous global model are combined, the training process is optimized, the feature representation distance between the current model and the global model is reduced, meanwhile, the feature representation distance between the current model and the local model is increased, a client model gradually approaches the global model through comparison, and a better global model is obtained to segment a liver cancer image. According to the method, performance reduction caused by data isomerism is effectively relieved, the generalization ability of the model is improved, and the problem of insufficient data labels is solved, so that application in the field of artificial intelligence assisted medical treatment is promoted.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence, specifically a self-supervised pre-training method based on federated learning, focusing on medical image segmentation, especially for the detection and diagnosis of liver cancer. Background Art

[0002] Medical image segmentation is a key task in the field of computer vision, which aims to accurately extract regions of interest such as tumors and organs from medical images to assist clinical diagnosis and treatment decisions. For liver cancer image segmentation, early detection and treatment planning are crucial, which can significantly improve the survival rate of patients. However, traditional medical image segmentation methods usually rely on supervised learning and require a large amount of labeled data from multiple medical centers, which is often difficult to achieve in medical practice, especially in the context of increasing attention to data privacy and ethical issues.

[0003] To address this challenge, self-supervised learning and federated learning technologies have gradually attracted attention. Self-supervised learning uses a large amount of unlabeled data for feature self-learning, which enhances the generalization ability of the model and adapts to different data distributions. Federated learning allows model training on distributed data from multiple medical centers, which not only protects patient privacy but also improves the accuracy of the model on diverse data. Combining these two advanced technologies, self-supervised pre-training of liver cancer images can be efficiently carried out in a multi-center environment, significantly improving the performance of the model in downstream tasks, thereby promoting the advancement of medical image analysis and promoting the development of precision medicine.

[0004] The advantage of segmenting medical images based on the federated learning framework is that it can be jointly trained on data from multiple centers, which not only solves the problem of data annotation shortage, but also protects patient privacy. Common federation methods include FedAvg, FedProx, FedBN, and MOON. FedAvg combines data privacy protection and model training by decomposing the task into local model training and global model aggregation. Each node generates weights through local training and updates the global model through average aggregation. Its advantage is that it efficiently uses distributed data for training, while reducing data storage requirements and improving model generalization capabilities. FedProx introduces regularization terms based on FedAvg to alleviate the heterogeneity problem between clients, improve model consistency and system robustness, and can more effectively converge to a better global model, especially when the data distribution difference is significant. FedBN solves the problem of inconsistent statistics caused by data heterogeneity and reduces training instability by adding batch normalization layers (BN) to local models. Finally, MOON exploits the similarities between model representations for contrastive learning, corrects local training biases, improves the model's feature extraction capabilities in non-IID data scenarios, controls drift, and bridges the gap between local and global models.

[0005] Although federated learning can protect data privacy and jointly train in multiple centers, its training process still relies on a large number of fully annotated images. However, in practical applications, especially in multi-center scenarios, there is often a situation where data is abundant but annotations are scarce. To this end, self-supervised methods can be combined to conduct self-supervised training on medical data from different centers and learn semantic features extensively. This not only increases the number of participating clients and improves the generalization ability of the model, but also reduces the burden of data processing in hospitals. Currently, in the self-supervised task of medical data, SelfMedMAE is a self-supervised pre-training framework based on reconstruction learning. It cuts a large amount of unlabeled data into blocks of equal volume and treats it as a sequence, randomly masks some blocks as reconstruction targets, and in the process of restoring the complete image, the ViT-based encoder can learn rich semantic knowledge without annotations, thereby achieving more accurate segmentation in subsequent tasks. Summary of the invention

[0006] In view of the shortcomings in the prior art, the purpose of the present invention is to provide a self-supervised pre-training method based on federated learning to address the problems of lack of data annotation and patient privacy protection in a multi-center environment. This method aims to increase the generalization ability of the model and improve the accuracy of lesion segmentation in medical images, thereby providing more reliable support for the early detection and diagnosis of major diseases such as liver cancer. At the same time, by utilizing the framework of federated learning, each participating hospital center can conduct joint training without sharing the original patient data, effectively ensuring the privacy and security of patient information. This innovative method not only promotes the efficient use of data, but also lays a solid foundation for the realization of precision medicine, provides clinicians with more accurate image analysis tools, and helps improve the overall quality of medical services.

[0007] In order to solve the problems existing in the prior art, the present invention provides a self-supervised pre-training method based on federated learning, which uses a masked autoencoder to perform medical image analysis, and includes the following steps:

[0008] 1) First, 75% of the input medical image of each client is randomly masked. The pre-trained model uses the ViT encoder to aggregate the contextual information of the unmasked area to infer the content of the masked part, and uses the MSE function as the reconstruction loss to reconstruct the complete image of the masked part;

[0009] 2) The client converts the ViT encoder embedding vector obtained in step 1) into a higher-dimensional feature representation through the MLP multi-layer perceptron layer, uploads the embedding vector to the central server for comparative learning, and updates the client parameters, and then aggregates to obtain the global model and distributes the updated parameters to each client;

[0010] 3) The client uses these parameters for local training and optimization, repeatedly uploading updated parameters to further improve the model until the model reaches the expected performance level.

[0011] As a preferred solution of the present invention, the model framework for client training includes a ViT encoder for learning medical image features, a Transformer decoder for image reconstruction, and an MLP for converting embedded vectors into higher-dimensional and more complex feature representations.

[0012] As a preferred solution of the present invention, the ViT encoder converts medical image data into a high-dimensional feature representation, specifically by dividing the randomly masked image into small blocks and linearly embedding them into a vector space of fixed dimensionality, combining a self-attention mechanism to capture the relationship between the small blocks, and extracting the global features of the image.

[0013] As a preferred solution of the present invention, in step 2), the client transmits the high-dimensional features after passing through the MLP to the central server, and updates the client using the contrast loss obtained by self-supervised contrastive learning.

[0014] As a preferred solution of the present invention, reconstruction loss, self-supervised contrast loss and cross entropy loss are added according to a certain parameter ratio to obtain a mixed loss, and the mixed loss is used to localize client model parameters.

[0015] As a preferred solution of the present invention, for the current t-th round of communication training, the following steps are included:

[0016] Step 1: The central server sends the global model of the current round t to each client, and the client processes the current global model and trains the local model of the current round;

[0017] Step 2: The client saves the local model that needs to be trained, the global model sent back from the central server in the previous round, and the local model obtained from the previous round of training. The client performs the reconstruction task on the local masked image. The reconstruction loss function is shown in formula (1):

[0018]

[0019] in, n Indicates the number of clients participating in the image data, y i is the actual target value of the i-th image data, is the model's predicted value for the i-th image data;

[0020] At the same time, the high-dimensional features after the ViT encoder passes through the MLP are trained using a hybrid loss function of self-supervised contrast loss and cross entropy loss. The self-supervised contrast loss function is shown in formula (2):

[0021]

[0022] Among them, sim() represents the cosine similarity function, τ is the temperature parameter, which is used to control the loss value and ensure the convergence of the loss. represents the feature representation learned by the local model in round t, represents the feature representation learned by the local model in round t-1, Represents the feature representation learned by the global model in round t-1;

[0023] The three loss functions are added together according to a certain parameter ratio to obtain the training mixed loss function formula as shown in (3):

[0024]

[0025] in, s is the local dataset sample for current training, is the local model to be trained in the current round t, is the local model of the t-1th round, is the global model of the t-1th round, μ is the hyperparameter used to control the weight of the self-supervised contrast loss;

[0026] Step 3: After local training is completed, each client sends the trained local model to the central server for model aggregation;

[0027] Step 4: After collecting the local models of all clients, the central server aggregates the models using weighted average to obtain the global model of the tth round. At this point, the tth round of communication training is completed; it returns to step 1 for the next round of communication training; the overall optimization goal is shown in formula (4):

[0028]

[0029] in, n Represents the total amount of data for all clients, n k represents the number of data sets from the kth client, is the mixed loss of the kth client, w glob is the global model to be trained. Therefore, the overall optimization goal of the model is to use a mixed loss for each client. After the training is completed, the weighted average is used for aggregation to finally obtain the optimal global model w glob

[0030] According to the ideas provided by the present invention, see the specific implementation technical solution Figure 1, mainly includes two parts: first, self-supervised pre-training is performed based on the method of reconstructing images, and second, the self-supervised pre-training method is carried out under the federated learning framework, and comparative learning is performed on the feature embedding after ViT encoding to update the parameters, thereby narrowing the encoding gap between different clients.

[0031] The self-supervised pre-training method uses the MAE masked autoencoder for image analysis, masks a portion of the volume block in the input image as a sequence input, and then trains the model to reconstruct the complete image from the partially masked image. During the reconstruction process, a ViT-based encoder and decoder are used, which can effectively aggregate the contextual information of the unmasked part from the perspective of the sequence, thereby inferring the content of the masked part. This method not only enables the model to better understand the overall structure of the image, but also improves the prediction accuracy in the absence of information, bringing significant performance improvements to image analysis tasks.

[0032] At the same time, the system's self-supervised pre-training is based on a federated learning framework. The embedding vectors encoded by the encoder of each client in the previous step are uploaded to the central server for comparative learning. The contrast loss and reconstruction loss are combined to update the parameters of each client, and then the global model is aggregated on the central server. The updated model parameters are sent to each client. Each client then uses these updated parameters for local training and optimization. After completion, it repeatedly uploads the updated embedding to continue to improve the model. This process is repeated until the model reaches the expected performance level, thus completing the self-supervised pre-training under multiple centers.

[0033] As described above, the self-supervised pre-training method based on federated learning of the present invention has the following beneficial effects: through joint self-supervised pre-training under multi-center data, the accuracy of the lesion segmentation model in downstream tasks is significantly improved; the self-supervised framework allows hospital centers that do not have labeled medical images to participate in training, enhancing the generalization ability of the model and the diversity of data; under the federated learning framework, each hospital center does not need to share the original data, but only needs to upload the processed model embedding, ensuring the privacy and security of patient data. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Flowchart of the self-supervised pre-training method based on federated learning in the present invention.

[0035] Figure 2 Schematic diagram of the network structure of the self-supervised pre-training method based on federated learning in the present invention.

[0036] Figure 3 Schematic diagram of updating privacy parameters of federated learning of the present invention.

[0037] Figure 4Epoch and Loss Score relationship diagram of the self-supervised pre-training method based on federated learning in the present invention. DETAILED DESCRIPTION

[0038] 1. Federated Learning

[0039] Federated learning is a distributed machine learning technology that aims to achieve collaborative training among collaborating institutions (such as hospitals) by keeping data locally. In this process, institutions only exchange model parameters without sharing local data, thereby achieving cross-institutional model training while ensuring data privacy.

[0040] The training process of federated learning is mainly divided into the following steps:

[0041] (1) Initialize the global model: Initialize a global model including initial parameters on the central server;

[0042] (2) Model sending: Figure 3 As shown, the central server transmits the current global model to each client; the current global model is distributed to each client as a local model for local training. Let the local model of the kth (k=1,2,...,K) local client be

[0043] (3) Perform local training: Client S k The model is trained using local data. After receiving the model from the central server, the client uses the cross entropy loss function and the stochastic gradient descent method for training. Let L CE Indicates client S k The loss function of the model can be expressed as formula (1), where η represents the learning rate during training:

[0044]

[0045] (4) Uploading local models: After training is completed, each client uploads its updated local model to the central server;

[0046] (5) Aggregate local models: The central server is responsible for aggregating the local models uploaded by each client, usually using a weighted average. The weights are usually based on the amount of data from each client, and the global model The aggregation can be expressed as the formula:

[0047]

[0048] Among them, |D| represents the total number of samples in each client data set, |D k| indicates that it comes from client S k The number of samples of local data.

[0049] (6) Obtaining the final global model: The main goal of federated learning is to learn a global model from each client through a central server without exchanging data. w , therefore, the overall optimization objective of the model is as shown in formula (3):

[0050]

[0051] Among them, L k (w) represents the client S k The cross entropy loss when training the model is shown in (4):

[0052]

[0053] In formula (4), D k Indicates client S k The corresponding data set, E indicates that the expectation is for the data set D k The expectation of all samples (x, y) in D, (x, y) ~ D k Indicates data D k A data sample (x,y) in , where x represents the sample in the data and y represents the label in the data.

[0054] 2. Self-supervised pre-training based on contrastive learning

[0055] The present invention mainly studies the self-supervised pre-training framework based on contrastive learning, which is a derivative technology of self-supervised learning. It improves the understanding of semantic context features of different client models from the perspective of reconstructing images and the common constraints of contrastive learning. Its main purpose is to allow the encoder to learn the feature embedding of a large number of unlabeled samples, so that the encoder can more fully understand the semantic context features of the samples when encoding in the segmentation task in the downstream task. The introduction of contrastive learning allows the feature embeddings encoded by encoders between different clients to have similar representations for the same feature embeddings in a certain feature space, while different feature embeddings have far-away representations. This method combining self-supervised pre-training and contrastive learning is particularly suitable for multi-center feature learning in the absence of labeled data.

[0056] In the encoding stage, 75% of the input medical image of each client is randomly masked. The pre-trained model uses ViT Encoder to aggregate the contextual information of the unmasked area to infer the content of the masked part, and uses the MSE function as the reconstruction loss to reconstruct the complete image of the masked part. i , cut it inton The size is A sub-volume block of which 75% will be masked as a masked block, and the other 25% of the visible part will be used as a visual block, and the position information is added to these visual blocks in the form of a sequence as the input of the ViT encoder. The ViT encoder used in the present invention stacks 12 Transformer blocks with 12 heads and an embedding dimension of 384, and uses the position embedding form of sincos. After the encoding is completed, the liver cancer CT image data will be mapped to a deeper feature embedding as a deep expression of semantic context information.

[0057] set up Represents image x i , for image x i Generate feature embedding vector z i , whose positive and negative feature embedding vectors are and The definition of its self-supervised contrast loss function is as shown in formula (5):

[0058]

[0059] in Indicates z i The corresponding feature embedding vector of Indicates z i The negative feature embedding vector of τ Represents the temperature parameter.

[0060] The goal of the decoding stage is to embed the above features into the vector z iAfter the restoration of the Transformer block, the original complete input sequence is reconstructed, that is, the masked block is used as the learning target. Then, the obtained ViT encoder embedding vector is converted into a higher-dimensional feature representation using the MLP multi-layer perceptron layer, and the embedding vector is uploaded to the central server for comparative learning to update the client parameters, and then aggregated to obtain the global model and distribute the updated parameters to each client. Since the decoder needs to reconstruct the masked pixels, the input contains two parts of information: the feature vector generated by the encoder for the unmasked pixel block and the masked pixel block information. The masked pixel block information is all represented by the same learnable feature vector. The decoder here is essentially an architecture containing the Transformer block, so the input vector contains the position code. Like the encoder, the Transformer cannot learn the position information, so the position information needs to be added to indicate the position of a certain pixel block in the original image. The decoder is only used in the pre-training stage. When migrating to downstream tasks, the decoder can be replaced with any architecture according to the actual needs of the user. The last layer of the decoder is a linear layer. Assuming the block size is 16×16, the output dimension of the linear layer is 256, and then reshape it to 16×16 to get the reconstructed pixel block. Finally, the MSE function is used as the loss function to measure the similarity between the reconstructed image and the real image. The formula is shown in formula (6):

[0061]

[0062] in, n Indicates the number of clients participating in the image data, y i is the actual target value of the i-th image data, is the model's predicted value for the i-th image data;

[0063] The final loss of the self-supervised pre-training process can be defined as:

[0064] L mix =L MSE +μL SC (7)

[0065] Example:

[0066] like Figure 2As shown, a self-supervised pre-training method based on federated learning is shown. The model framework of client training includes a ViT encoder for learning medical image features, a Transformer decoder for image reconstruction, and an MLP for converting embedded vectors into higher-dimensional and more complex feature representations. The ViT encoder converts medical image data into high-dimensional feature representations, specifically by dividing the randomly masked image into small blocks and linearly embedding them into a vector space of fixed dimensionality, combining a self-attention mechanism to capture the relationship between small blocks, and extracting the global features of the image. The present invention is based on the self-supervised mask pre-training method for improvement, and aims to learn the features of unlabeled images. By using the ViT encoder and ViT decoder to reconstruct the mask image, the learning ability of the self-supervised model in unlabeled images is improved. It can solve the problem of insufficient medical image data and improve the accuracy of lesion segmentation.

[0067] The communication training for the current round t includes the following steps:

[0068] Step 1: The central server sends the global model of the current round t to each client, and the client processes the current global model and trains the local model of the current round;

[0069] Step 2: The client saves the local model that needs to be trained, the global model sent back from the central server in the previous round, and the local model obtained from the previous round of training. The client performs the reconstruction task on the local masked image. The reconstruction loss function is shown in formula (8):

[0070]

[0071] in, n Indicates the number of clients participating in the image data, y i is the actual target value of the i-th image data, is the model's predicted value for the i-th image data;

[0072] At the same time, the high-dimensional features after the ViT Encoder passes through the MLP are trained using a mixed loss function of self-supervised contrast loss and cross entropy loss. The self-supervised contrast loss function is shown in formula (9):

[0073]

[0074] Among them, sim() represents the cosine similarity function, τ is the temperature parameter, which is used to control the loss value and ensure the convergence of the loss. represents the feature representation learned by the local model in round t, represents the feature representation learned by the local model in round t-1, Represents the feature representation learned by the global model in round t-1;

[0075] The three loss functions are added together according to a certain parameter ratio to obtain the training mixed loss function formula as shown in (10):

[0076]

[0077] in, s is the local dataset sample for current training, is the local model to be trained in the current round t, is the local model of the t-1th round, is the global model of the t-1th round, μ is the hyperparameter used to control the weight of the self-supervised contrast loss;

[0078] Step 3: After local training is completed, each client sends the trained local model to the central server for model aggregation;

[0079] Step 4: After collecting the local models of all clients, the central server aggregates the models using weighted average to obtain the global model of the tth round. At this point, the tth round of communication training is completed; it returns to step 1 for the next round of communication training; the overall optimization goal is shown in formula (11):

[0080]

[0081] in, n Represents the total amount of data for all clients, n k represents the number of data sets from the kth client, is the mixed loss of the kth client, w glob is the global model to be trained. Therefore, the overall optimization goal of the model is to use a mixed loss for each client. After the training is completed, the weighted average is used for aggregation to finally obtain the optimal global model w glob .

[0082] Summarizing the above steps, the pseudo code of the algorithm structure of a self-supervised pre-training method based on federated learning proposed in the present invention is as follows:

[0083]

[0084]

[0085] As described above, the self-supervised pre-training method based on federated learning of the present invention has the following beneficial effects: through self-supervised learning and joint training of multi-center data, the accuracy of the lesion segmentation model is significantly improved; hospital centers that do not have labeled medical images are allowed to participate in training, which enhances the generalization ability of the model and the diversity of data; under the federated learning framework, each hospital center does not need to share the original data, but only needs to upload the processed model embedding, ensuring the privacy and security of patient data.

[0086] Comparative experiment:

[0087] 1. Experimental Preparation

[0088] In order to demonstrate the superiority of the present invention for multi-center medical image lesion segmentation, experiments were conducted on a private liver cancer dataset. The private dataset of the present invention comes from Guangdong Provincial People's Hospital and contains 247 liver cancer computed tomography (CT) images. These images were annotated by two imaging medical staff with more than ten years of work experience. The loss of the reconstructed image was calculated using the annotated images to learn its features.

[0089] In order to simulate the experimental setting of federated learning, the data set needs to be divided to simulate each local client. Before training, the present invention divides the data set images and their label information of each client into a training set and a test set at a ratio of about 4:1, and their numbers are 197:50 respectively. Then, the images in the training set are divided into a data set that conforms to the federated learning framework under three centers at a ratio of 65:65:67.

[0090] 2. Comparative Experiment

[0091] The present invention not only compares the experimental results analysis with centralized learning Central, but also compares with some methods in the field of federated learning. This includes the FedAvg algorithm, which is a commonly used federated method that updates the global model by performing local training on each participant and averaging the model weights. In addition, it is also compared with the MOON algorithm that solves the problem of client data heterogeneity to comprehensively evaluate the performance advantages and applicability of the present invention. By comparing with the methods of other researchers, it is proved that the present invention can effectively segment lesions in liver cancer CT medical images, and the segmentation accuracy effect is better than other methods.

[0092] Table 1 Comparative analysis results of the present invention and other network models on private data sets (Test Dice)

[0093]

[0094] From the data analysis in Table 1, it can be seen that the FedAvg method and the MOON method perform relatively well on private datasets and can achieve good results. However, although these two methods have shown their potential under the federated learning framework, in most cases, their performance still fails to surpass the performance of the traditional centralized method, namely the Central method. This phenomenon shows that there are still certain performance bottlenecks in the practical application of federated learning, especially in complex tasks involving multi-center data integration.

[0095] However, it is worth noting that, compared with the FedAvg and MOON methods, the federated comparative self-supervised pre-training method of the present invention has shown significant advantages in medical image segmentation tasks. Specifically, when dealing with lesion segmentation tasks, this method not only improves the accuracy of the segmentation model, but also shows a strong model generalization ability. This performance improvement may be due to the enhanced ability of the self-supervised pre-training mechanism to represent features in the case of limited labeled data, and the better adaptability to the distribution of data from different centers in the federated learning framework. This result shows that the method of the present invention can effectively improve the accuracy of lesion segmentation in a federated learning environment, and provides a powerful solution for multi-center medical image analysis tasks.

[0096] from Figure 4 It can be seen that during the training process, the loss function value shows a significant rapid downward trend in the early epochs. This phenomenon shows that the model parameters effectively optimize the training error in the initial stage, showing strong learning ability and rapid convergence. As the training continues, the rate of decline of the loss function gradually slows down and eventually stabilizes. This stable downward curve usually indicates that the learning rate of the model is properly configured and the training process is efficient and stable. In addition, the final stable value of the loss function is low, which further shows that the model has reached a good local optimal solution on the training data and can effectively capture the patterns in the data. These observations show that the model method proposed in the present invention performs well in the training process, can quickly adapt to the data and stably optimize its prediction performance.

[0097] In summary, the present invention provides a self-supervised pre-training method based on federated learning, which aims to improve the effect of medical image analysis. By designing a self-supervised mask pre-training process, the method improves the utilization rate of data, so that those hospital centers that do not have annotated medical images can participate in training. This not only enhances the generalization ability of the model, but also enriches the diversity of data and promotes the understanding of the characteristics of different patient groups. The method combines privacy-preserving federated learning with self-supervised learning technology, aiming to improve the accuracy and robustness of liver cancer image segmentation. By jointly training on medical data from multiple medical centers, the image segmentation model is optimized to achieve more efficient and accurate liver cancer image analysis, thereby better assisting clinical decision-making and treatment planning. Combined with the advantages of federated learning, each hospital center does not need to share the original data, but only needs to upload the processed model embedding, thereby effectively protecting the privacy and security of patient data. This design is particularly important in the current medical environment where data protection and ethics are increasingly valued. In addition, on this basis, contrastive learning is used to correct the local model of the client through the similarity between model representations, thereby further improving the accuracy and stability of the segmentation model in practical applications. This method provides new ideas for promoting the advancement of medical image segmentation technology and helps to achieve more efficient auxiliary decision-making in clinical diagnosis.

[0098] Therefore, the present invention effectively overcomes various shortcomings in the prior art and has high industrial utilization value.

[0099] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A self-supervised pre-training method based on federated learning, characterized in that: The masked autoencoder is used for medical image analysis, and the method comprises the following steps: 1) First, 75% of the input medical image of each client is randomly masked. The pre-trained model uses the ViT encoder to aggregate the contextual information of the unmasked area to infer the content of the masked part, and uses the MSE function as the reconstruction loss to reconstruct the complete image of the masked part; 2) The client converts the ViT encoder embedding vector obtained in step 1) into a higher-dimensional feature representation through the MLP multi-layer perceptron layer, uploads the embedding vector to the central server for comparative learning, and updates the client parameters, and then aggregates to obtain the global model and distributes the updated parameters to each client; 3) The client uses these parameters for local training and optimization, repeatedly uploading updated parameters to further improve the model until the model reaches the expected performance level.

2. The self-supervised pre-training method based on federated learning according to claim 1, characterized in that: The model framework trained by the client includes a ViT encoder for learning medical image features, a Transformer decoder for image reconstruction, and an MLP for converting embedded vectors into higher-dimensional and more complex feature representations.

3. The self-supervised pre-training method based on federated learning according to claim 1 or 2, characterized in that: The ViT encoder converts medical image data into high-dimensional feature representation by dividing the randomly masked image into small blocks and linearly embedding them into a vector space of fixed dimensionality, combining a self-attention mechanism to capture the relationship between the small blocks and extract the global features of the image.

4. The self-supervised pre-training method based on federated learning according to claim 3, characterized in that: In step 2), the client transmits the high-dimensional features after passing through the MLP to the central server, and the client is updated using the contrastive loss obtained by self-supervised contrastive learning.

5. The self-supervised pre-training method based on federated learning according to claim 4, characterized in that: The reconstruction loss, self-supervised contrast loss and cross entropy loss are added according to a certain parameter ratio to get the mixed loss, and the mixed loss is used to localize the client model parameters.

6. The self-supervised pre-training method based on federated learning according to claim 1, characterized in that: The communication training for the current round t includes the following steps: Step 1: The central server sends the global model of the current round t to each client, and the client processes the current global model and trains the local model of the current round; Step 2: The client saves the local model that needs to be trained, the global model sent back from the central server in the previous round, and the local model obtained from the previous round of training. The client performs the reconstruction task on the local masked image. The reconstruction loss function is shown in formula (1): in, n Indicates the number of clients participating in the image data, y i is the actual target value of the i-th image data, is the model's predicted value for the i-th image data; At the same time, the high-dimensional features after the ViT encoder passes through the MLP are trained using a hybrid loss function of self-supervised contrast loss and cross entropy loss. The self-supervised contrast loss function is shown in formula (2): Among them, sim() represents the cosine similarity function, τ is the temperature parameter, which is used to control the loss value and ensure the convergence of the loss. represents the feature representation learned by the local model in round t, represents the feature representation learned by the local model in round t-1, Represents the feature representation learned by the global model in round t-1; The three loss functions are added together according to a certain parameter ratio to obtain the training mixed loss function formula as shown in (3): in, s is the local dataset sample for current training, is the local model to be trained in the current round t, is the local model of the t-1th round, is the global model of the t-1th round, μ is the hyperparameter used to control the weight of the self-supervised contrast loss; Step 3: After local training is completed, each client sends the trained local model to the central server for model aggregation; Step 4: After collecting the local models of all clients, the central server aggregates the models using weighted average to obtain the global model of the tth round. At this point, the tth round of communication training is completed; it returns to step 1 for the next round of communication training; the overall optimization goal is shown in formula (4): Among them, n represents the total amount of data of all clients, n k represents the number of data sets from the kth client, is the mixed loss of the kth client, w glob is the global model to be trained. Therefore, the overall optimization goal of the model is to use a mixed loss for each client. After the training is completed, the weighted average is used for aggregation to finally obtain the optimal global model w glob .

Images