Heterogeneous federal learning method and system based on personalized collaborative generation
By employing a heterogeneous federated learning method that generates personalized collaborative data, a unified latent space is constructed using a conditional variational autoencoder to address the heterogeneity of data and models, and to generate targeted synthetic samples. This solves the problems of data and model heterogeneity and insufficient privacy protection in federated learning, and improves the recognition performance and communication efficiency of undersampled classes.
Patent Information
- Application Number
- CN202511472189.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-15
- Publication Date
- 2026-02-17
AI Technical Summary
Existing federated learning methods cannot effectively address the problem of data and model heterogeneity, especially in scenarios where data is not independent and identically distributed and model architectures are heterogeneous. Undersampling-based learning has poor performance, insufficient privacy protection, and high communication overhead.
We employ a heterogeneous federated learning approach that utilizes personalized collaborative generation. By capturing local data distribution through a conditional variational autoencoder, we construct a unified latent space, enabling cross-model knowledge transfer and generating targeted synthetic samples. Combined with a collaborative optimization mechanism, we address the coexistence of data and model heterogeneity, enhance undersampling learning, and protect privacy.
It enables heterogeneous federated learning without the need for public datasets, reduces communication overhead, improves the recognition performance of undersampled classes, enhances privacy protection, adapts to dynamic and heterogeneous scenarios, and improves the personalized adaptability of models.
Smart Images
Figure CN121543668A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of cross-technology of federated learning, generative artificial intelligence and personalized machine learning, and specifically relates to a personalized collaborative generation method suitable for a heterogeneous federated learning environment, which can be applied to edge device (client) collaborative training, personalized model construction in privacy-sensitive fields (medical image analysis), and is particularly suitable for scenarios where data are non-independent and identically distributed (Non-IID) and model architectures are heterogeneous. BACKGROUND
[0002] With the growth of edge computing and privacy protection needs, federated learning (FL) has become the core paradigm of distributed training, which can realize multi-client collaborative training without aggregating raw data, and is widely used in medical, financial and other fields. However, federated learning in real scenarios faces the challenge of double heterogeneity:
[0003] Data heterogeneity: the local data of clients presents Non-IID characteristics, such as class distribution offset (label skew, different clients identify and classify different images), feature distribution difference, and uneven data volume, which makes it difficult for traditional global models to adapt to local needs;
[0004] Model heterogeneity: clients use different architectures (e.g. ResNet8 / ResNet50) due to hardware capabilities or task requirements (real-time detection / high-precision classification), and direct parameter sharing or gradient aggregation is not possible.
[0005] Further analysis shows that existing technologies for data heterogeneity (such as FedProx, PerFedAvg) only adapt to local data through regularization or meta-learning, and cannot handle model architecture differences, thus solving single heterogeneity; methods for model heterogeneity (such as FedMD, FedRep) rely on knowledge distillation or partial layer sharing, but require high-quality public data sets (which are often not available in practice), and it is difficult to balance global knowledge and local personalization; underclass learning is weak: under extreme data skew, the recognition performance of clients for "long-tail classes" (extremely small samples) drops sharply, and existing generative methods (such as FedGen) lack targetedness and cannot accurately supplement underclass samples; privacy and efficiency imbalance: some methods transmit complete model parameters or generate samples, resulting in high communication overhead or risk of raw data privacy leakage (such as model reverse attacks).
[0006] Therefore, there is an urgent need for a personalized federated learning method that does not require public data sets, can simultaneously solve data and model heterogeneity, and can specifically enhance underclass learning. SUMMARY
[0007] Purpose of the invention: To address the shortcomings of existing technologies in meeting the requirements of single heterogeneity, reliance on public data, poor performance of undersampled classes, and insufficient privacy protection, this invention provides a heterogeneous federated learning method and system based on personalized collaborative generation (denoted as: pFedCoGen). Through conditional variational autoencoders, it captures local data distribution and unifies the latent space, enabling cross-model knowledge transfer and collaborative generation of targeted synthetic samples. This achieves four objectives: no public data, dual heterogeneity adaptation, undersampled class enhancement, and privacy protection.
[0008] Technical Solution: A heterogeneous federated learning method based on personalized collaborative generation, suitable for co-training on edge devices. By integrating conditional variational autoencoders and collaborative optimization mechanisms, it achieves heterogeneous federated learning without a common dataset in two stages, addressing the problem of coexisting data and model heterogeneity. This includes:
[0009] Distribution feature learning stage: Each client trains a variational autoencoder locally to capture the local data distribution, protects data privacy through privacy-enhanced prior distribution offset, and shares it between the server and the client through a secure channel;
[0010] Collaborative generative learning stage: By selecting a reference encoder and combining it with a latent space similarity map to construct a unified feature representation space, and combining collaborative knowledge update, class recognition capability-driven auxiliary generator pairing, and adaptive synthetic sample generation, cross-model knowledge transfer and undersampling class-specific enhancement are achieved. Finally, a hybrid loss is used to optimize the local model.
[0011] Furthermore, the method includes the following implementation steps:
[0012] (1) Client k deploys and trains a CVAE model to prepare for knowledge transfer; the training objective of the CVAE model is to maximize the lower bound of conditional evidence, and then replace the prior distribution from the standard normal distribution with a biased distribution. The biased vector is shared between the server and the client through a secure channel. After training, the client only uploads the decoder parameters and local label distribution to the server, and then the server constructs a globally unified latent representation space to realize cross-model knowledge transfer.
[0013] (2) The server calculates the KL divergence between the label distribution of each client and the uniform distribution, and then selects the client k with the most uniform distribution and its corresponding reference encoder based on minimizing the KL divergence. The reference encoder is used as the global latent extractor.
[0014] Each client extracts the latent representation of the local sample based on the selected reference encoder, calculates the model output, and uploads the calculation results to the server. The server constructs a latent space similarity graph based on the cosine similarity of the received latent representations of the local samples, and finds J nearest neighbors for each sample to form a neighborhood set.
[0015] (3) The server and client collaborate to achieve knowledge updates and targeted data enhancement, and optimize the local model, specifically including:
[0016] Collaborative knowledge updates, with servers performing calculations based on neighbor sets formed by nearest neighbors. This is used to guide the client to build a local knowledge base after receiving the data, and to incorporate KL divergence loss into the training of real samples in order to balance the local original sample data with the globally shared knowledge.
[0017] The auxiliary generator is paired, and the client calculates the accuracy of various image recognition methods. The data is uploaded to the server, which then calculates the matching score and uses it to find the optimal auxiliary generator for client k.
[0018] Adaptive sample generation: The client k generates several synthetic samples based on the auxiliary generator to construct an augmented dataset. Then, a hybrid loss is used for training. Real samples are combined with cross-entropy and knowledge distillation to ensure global knowledge integration. The weights of the two are balanced through hyperparameters. The server calculates the generation distribution.
[0019] Hybrid training involves the client training with hybrid loss based on the constructed augmentation set. This process is repeated for a set number of communication rounds to output a personalized model for each client.
[0020] In step (1) of the above method, a CVAE model is constructed for the variational autoencoder, denoted as... The CVAE model mentioned above includes a conditional encoder model. Conditional decoder model The condition encoder model Original sample Labels corresponding to the original samples Mapped to a multivariate Gaussian distribution; conditional decoder model Reconstructing the original samples using transposed convolution and the ResBlock architecture ;
[0021] Conditional encoder model The corresponding multivariate Gaussian distribution is mathematically represented as:
[0022] ;
[0023] The objective expression for the training loss function of the CVAE model is as follows:
[0024] ,
[0025] In the formula, The latent distribution of the encoder output. Let p(z) be the reconstruction distribution of the decoder, and p(z) be the prior distribution. Represents the original sample. These are the labels corresponding to the original samples. Represents the latent vector. These represent the parameters of the encoder and decoder, respectively. This represents the number of local samples on client k. Indicates the first One sample, This represents the symbol for calculating the KL divergence for the conditional decoder.
[0026] Wherein, the offset distribution is expressed as , It is an offset vector. It represents the standard identity matrix, which is independent in each dimension and has a variance of 1.
[0027] Furthermore, in step (3), the server calculates based on the neighbor set formed by nearest neighbors. The mathematical expression for this calculation is:
[0028] ,
[0029] In the formula, This represents the guidance corresponding to the i-th sample of client k. , This represents the client index corresponding to the j-th neighbor in the neighbor set. This represents the sample index corresponding to the j-th neighbor in the neighbor set. It is a neighborhood group. Indicates the corresponding neighbor of the j-th neighbor ;
[0030] The client calculates the accuracy of various image recognition methods using the following formula:
[0031] ,
[0032] in This represents the recognition accuracy of client k in the c-th category;
[0033] The server calculates the matching score using the following expression:
[0034] ,
[0035] In the formula, This represents the class distribution of the generator g;
[0036] The formula for calculating the generated fraction is:
[0037] ,
[0038] The proportion of recommendations generated by client k in category c. Let represent the recognition accuracy of client k in the j-th neighbor set.
[0039] Furthermore, the hybrid training described in step (3) includes the following mathematical calculation process:
[0040] The enhanced set constructed by the client is represented as ,in This represents the actual sample data of client k. This represents the synthetic sample data of client k;
[0041] The loss function for hybrid training is as follows:
[0042] ,
[0043] ,
[0044] ,
[0045] For the cross-entropy loss of the synthetic samples, The loss is for the real samples. The cross-entropy loss is the value of the real samples. Indicates by temperature parameter For guidance The probability distribution obtained after softening. Indicates using the same temperature parameters For the current output of the model The probability distribution obtained after softening. All of these are hyperparameters.
[0046] This method is applicable to heterogeneous scenarios with both image-aware data and models, and enables collaborative visual model training.
[0047] Based on the implementation and application of the above method, this invention provides a federated learning system for multi-center medical image collaborative diagnosis. This system uses the heterogeneous federated learning method based on personalized collaborative generation to perform federated learning of the model. The system will establish personalized disease diagnosis models under the premise of protecting the privacy of medical image data from different medical institutions. Different medical institutions correspond to the clients in the method and train their own conditional variational autoencoders based on their local private medical image data. The server receives and stores decoder parameters and label distributions from all medical institutions. Different medical institutions only need to upload the decoder parameters of the conditional variational autoencoder and the local label distribution to the server. The original medical image data is always kept locally in the medical institutions.
[0048] Beneficial effects: This invention integrates Conditional Variational Autoencoder (CVAE) and collaborative optimization: It captures local distributions and generates privacy-preserving synthetic samples without requiring a public dataset, while simultaneously achieving knowledge transfer from heterogeneous models through a unified latent space; Undersampling class targeting enhancement: Based on class recognition capability analysis and auxiliary generator pairing, generated samples accurately supplement weak classes in the client, solving the "long-tail class" learning problem; Dynamic adaptive strategy: Prior distribution shift protects privacy, generated distribution (… It adapts to client requirements, balances real / synthetic samples with mixed loss, and is suitable for dynamic heterogeneous scenarios; it has low communication overhead: it only uploads decoder parameters, latent representations, and logits, reducing communication volume by more than 60% compared to traditional methods (transmitting the complete model). Attached Figure Description
[0049] Figure 1 This is an implementation framework diagram of the method described in this invention;
[0050] Figure 2 This is a comparison chart of accuracy across multiple datasets;
[0051] Figure 3 This is an accuracy graph for undersampling.
[0052] Figure 4 This is a diagram showing the influence of the proportion of synthetic samples. Detailed Implementation
[0053] Combination Figure 1 As shown, the method of this invention includes dividing the framework into two stages: distributed feature learning and collaborative generation learning, combining the CVAE generation model and the collaborative optimization mechanism, as detailed below:
[0054] Distribution Feature Learning Phase: This phase deploys a Conditional Variational Autoencoder (CVAE) model for local training on each client, laying the foundation for knowledge transfer. A CVAE model is deployed on each client k. CVAE model By condition encoder composition:
[0055] Conditional encoder Input the original sample and their corresponding tags Output the multivariate Gaussian distribution of the latent space (vectors in the latent space). The conditional distribution of samples and labels is captured through a conditional encoder;
[0056] decoder Input latent vector z and label Reconstruct the original sample To ensure the authenticity of the generated samples;
[0057] CVAE model The training objective is to maximize the lower bound of conditional evidence, and the objective loss function is: Its mathematical representation is:
[0058] , It is the identifier of the condition encoder. It is the decoder's identifier. This is the identifier for divergence calculation. The above formula uses mathematical form to balance reconstruction accuracy and latent distribution regularization.
[0059] Privacy-enhanced prior distribution shift: To prevent attackers from reconstructing the original data through decoder parameters, the standard normal prior of traditional CVAE is shifted. Replace with , Shared between server and client via an AES-256 encrypted channel. Only decoder parameters are uploaded to the server, ensuring privacy protection.
[0060] The collaborative generative learning phase mainly involves collaborative training between the server and the client to achieve knowledge transfer and targeted enhancement, including the following steps:
[0061] Step 1: Construct a unified latent space to address model heterogeneity; the server calculates the label distribution for each client. With uniform distribution The reference encoder is selected from the client with the most even distribution, and its corresponding CVAE encoder. To ensure high category coverage for the global latent extractor, each client extracts local sample latent representations using its selected CVAE encoder. ,calculate The data is then uploaded, and the server constructs a similarity graph based on the cosine similarity of the local sample latent representation, matching J=5 nearest neighbors for each sample to establish cross-client semantic associations.
[0062] Step 2: Collaborative Knowledge Update (Supplementing Global Knowledge) - The server calculates guidance based on the sample nearest neighbor set. This value is used to characterize the predicted distribution of similar samples, thereby guiding the client to build a local knowledge base based on this value. This step involves incorporating knowledge distillation (KL divergence loss) into training on real samples to balance local data features with globally shared knowledge.
[0063] Step 3: The class recognition capability-driven auxiliary generator is paired to specifically enhance undersampled classes, and the client calculates the recognition accuracy of each class. This value reflects the undersampling class. The larger the value, the weaker the matching score for the corresponding category c in image recognition; the server calculates the matching score. , For generator Its category generation capability allows it to match the optimal auxiliary generator for client k. k This ensures that the generator's strong classes accurately complement the client's weak classes.
[0064] Step 4: Adaptive Sample Generation and Hybrid Training Server Calculates Generation Distribution More resources are allocated to weaker classes, and client-side generation is enabled. ) synthetic samples;
[0065] Building an enhanced training set Using mixed loss Training: Real samples are distilled using cross-entropy to ensure global knowledge integration; synthetic samples are enhanced using cross-entropy for targeted enhancement; hyperparameters... Balance the weights of the two.
[0066] To illustrate the technical solution of this invention in detail, please refer to the accompanying drawings. Figures 1-4 The experimental data will be used to elaborate on this.
[0067] The personalized federated learning system implemented in this invention has the following core architecture: Figure 1 As shown, the implementation includes a client-side module (local training, parameter upload, loss feedback) and a server-side module (graph construction, weight allocation, GAT optimization). It processes 5146 frames of samples from the MNIST, FashionMNIST, and CIFAR-10 datasets, and the specific implementation is as follows:
[0068] Datasets: MNIST (10 classes, 60k training / 10k test), FashionMNIST (10 classes, 60k training / 10k test), CIFAR-10 (10 classes, 50k training / 10k test); training data are partitioned according to Dirichlet distribution. It exhibits moderate heterogeneity. To ensure high heterogeneity, the test set is uniformly divided.
[0069] Model and Hyperparameters: The CVAE encoder is ResNet-based (3 ResBlocks), and the decoder is a transposed convolution; the client model is ResNet8 / 14 / 20 cyclically distributed; the number of communication rounds is 100, the number of local training rounds is 3, the SGD optimizer is used (learning rate 0.01), and the batch size is 8; the hyperparameters in this embodiment are set as follows:
[0070] Baseline method: Compare 10 heterogeneous federated learning methods (FD, FML, LG-FedAvg, FedGen, FedProto, FedKD, FedGH, FedTGP, FedKTL, FedMRL) with metrics including average accuracy, undersampled class accuracy, and communication cost.
[0071] The implementation steps are as follows:
[0072] Phase 1: Distribution Feature Learning (1-4 rounds of communication) The client initializes CVAE and trains it for 3 rounds using local data (e.g., MNIST client 1 with 5000 samples of classes 0-1), optimizing parameters according to ELBO. After training, the decoder parameters and label distribution are uploaded to the server, which then sends them via a secure channel. The upload time is less than 1.5 seconds, and the compressed CIFAR-10 decoder parameters are 21MB.
[0073] Phase 2: Collaborative Generative Learning (5-100 rounds of communication) The server calculates the KL divergence between the label distribution of each client and the uniform distribution, selecting the client with the most uniform distribution (Client 5) as a reference, and its encoder as the global latent extractor. Clients download this encoder, extract sample latent representations and logits, and upload them. The server matches 5 nearest neighbors for each sample based on the cosine similarity of the latent representations, calculates guiding logits, and sends them to the clients. Clients build a knowledge base and calculate class recognition accuracy (e.g., for class 5). The server matches the optimal auxiliary generator (the dominant class of generator 3 is 5), and the client generates 450 synthetic samples of class 5 according to the generation distribution. After constructing the augmentation set, it is trained for 3 rounds with mixed loss, and the updated logits and accuracy are uploaded.
[0074] Implementation results: Figure 2 This presentation shows a comparison of accuracy across multiple datasets, demonstrating the accuracy of pFedCoGen against 10 baselines under varying heterogeneity. pFedCoGen consistently delivers the best results. Figure 3 This demonstrates how pFedCoGen improves the accuracy of undersampled classes in the highly heterogeneous FashionMNIST dataset.
[0075] Overall performance: Under high heterogeneity, MNIST accuracy was 91.46% (+3.50% compared to FedKD), FashionMNIST accuracy was 80.32% (+5.68% compared to FedMRL), and CIFAR-10 accuracy was 41.27% (+2.50% compared to FedGH).
[0076] Undersampling enhancement: FashionMNIST client 6-class 5 accuracy improved from 22.00% (FedGH) to 82.00%;
[0077] Communication and Privacy: Achieving 35% CIFAR-10 accuracy requires a transmission of 0.82GB, lower than FML (3.83GB) and FedKD (8.79GB), and the encoder encryption ensures no privacy leaks.
[0078] Based on the above embodiments and descriptions, this invention provides a federated learning system for collaborative diagnosis of multi-center medical images. It assumes at least two medical institutions, Client A and Client B, respectively. Client A's data primarily focuses on tumor diagnosis, with a very high proportion (70%) of malignant tumor samples in its local dataset, while benign lesions and normal samples are relatively scarce. Especially for a certain "rare subtype of tumor," the sample size is sufficient. On Client A, we build a local model using ResNet20 to identify this type of medical image data. Client B corresponds to a primary care comprehensive medical institution, which has less data on tumor diagnosis, especially for "rare subtypes of tumor," and is limited by computational resources. Therefore, a lightweight model, MobileNetV2, is deployed for rapid initial screening. Our application addresses the problem of knowledge transfer between Client A and Client B while protecting privacy, ultimately enabling both Client A and Client B to demonstrate good diagnostic capabilities across various types of cases.
[0079] Local Training: Client A and Client B each train their own Conditional Variational Autoencoder (CVAE) locally using their private medical image data. Client A's CVAE learns to encode and reconstruct tumor-related image features with high quality; Client B's CVAE excels at processing images of normal and benign lesions. After training, both clients only upload the CVAE decoder parameters and local label distribution to the server. The original medical image data remains within the hospital.
[0080] The server receives and stores decoder parameters and label distributions from all clients, then analyzes the label distributions of all hospitals, selects the CVAE encoder of the hospital with the most uniform distribution (e.g., a medium-sized hospital) as the reference encoder, and distributes it to all hospitals (including client A and client B).
[0081] Clients A and B use the reference encoder to extract latent representations for all their local samples, compute the output of their local models for these samples, and then upload the latent representation logits to the server. The cloud server integrates all the information to construct a global latent space similarity graph. In this graph, although medical images from different hospitals and processed by different models, those with semantically similar meanings (such as both displaying "malignant tumor") will have latent representations that are close to each other.
[0082] Client B evaluates its model's diagnostic accuracy across all categories and uploads it to the server. The server clearly identifies that Hospital B's ability to identify malignant tumors and "rare subtypes of tumors" is extremely weak. The server then calculates a matching score and finds that Client A's decoder has the greatest advantage in generating images of malignant tumors and "rare subtypes of tumors." Therefore, the server designates Client A's decoder as an auxiliary generator for Client B. Based on Client B's weaknesses, the server recommends a personalized generation distribution, instructing it to generate a large number of synthetic images for malignant tumors and "rare subtypes of tumors." Client B uses the paired decoder from Client A to generate the specified synthetic images, which, together with local real images, form an enhanced training set. Subsequently, training is performed using a hybrid loss function. The hybrid training process involves using cross-entropy loss and knowledge distillation loss based on guided logits on real samples to fuse global knowledge. On synthetic samples, cross-entropy loss is used to directly strengthen the weak categories. After training, Client B uploads the updated model logits and the new category recognition accuracy to begin the next round of collaboration.
[0083] Through multiple iterations of the above process, the lightweight MobileNetV2 model of client B (a primary care hospital) significantly improved its sensitivity and recognition accuracy for "malignant tumors" and "rare subtypes of tumors," which were previously its blind spots, while retaining its diagnostic capabilities for common diseases. Meanwhile, client A (an oncology hospital) also improved its accuracy in recognizing normal images and benign lesions, achieving true, safe, and targeted collaborative optimization. While protecting the data privacy of each medical institution, the system overcomes the dual challenges of heterogeneous data distribution and model architecture to build a federated learning system. This system enables each participating hospital to obtain a personalized disease diagnosis model that not only maintains high accuracy for common diseases in their local area but also significantly improves their diagnostic capabilities for rare and weakly diagnosed diseases. Through server coordination and using generative models (CVAEs) as a medium, the diagnostic experience of data-rich hospitals in specific diseases is safely and efficiently transferred to data-scarce hospitals, achieving targeted knowledge complementarity and enhancement across models and data distributions.
Claims
1. A heterogeneous federated learning method based on personalized collaborative generation, characterized in that, This method is applicable to co-training on edge devices. By integrating conditional variational autoencoders and collaborative optimization mechanisms, it achieves heterogeneous federated learning without a common dataset in two stages, addressing the problem of coexisting data and model heterogeneity. This includes: Distribution feature learning stage: Each client trains a variational autoencoder locally to capture the local data distribution, and protects data privacy through privacy-enhanced prior distribution offset, sharing only the parameters of the conditional encoder between the server and the client through a secure channel; Collaborative generative learning stage: By selecting a reference encoder and combining it with a latent space similarity map to construct a unified feature representation space, and combining collaborative knowledge update, class recognition capability-driven auxiliary generator pairing, and adaptive synthetic sample generation, cross-model knowledge transfer and undersampling class-specific enhancement are achieved. Finally, a hybrid loss is used to optimize the local model.
2. The heterogeneous federated learning method based on personalized collaborative generation according to claim 1, characterized in that, The method includes the following implementation steps: (1) The client deploys and trains a CVAE model to prepare for knowledge transfer. The training objective of the CVAE model is to maximize the lower bound of conditional evidence and then replace the prior distribution with a biased distribution from the standard normal distribution. The biased vector is shared between the server and the client through a secure channel. After training, the client only uploads the decoder parameters and local label distribution to the server. Then the server constructs a globally unified latent representation space to realize cross-model knowledge transfer. (2) The server calculates the KL divergence between the label distribution of each client and the uniform distribution, and then selects the client with the most uniform distribution and its corresponding reference encoder based on minimizing the KL divergence. The reference encoder is used as the global latent extractor. Each client extracts the latent representation of the local sample based on the selected reference encoder, calculates the model output, and uploads the calculation results to the server. The server constructs a latent space similarity graph based on the cosine similarity of the received latent representations of the local samples, and finds the nearest neighbors for each sample to form a neighborhood set. (3) The server and client collaborate to achieve knowledge updates and targeted data enhancement, and optimize the local model, specifically including: Collaborative knowledge updates, with servers performing calculations based on neighbor sets formed by nearest neighbors. This is used to guide the client to build a local knowledge base after receiving the data, and to incorporate KL divergence loss into the training of real samples in order to balance the local original sample data with the globally shared knowledge. The auxiliary generator pairing process involves the client calculating the accuracy of various image recognition methods and uploading the results to the server. The server then calculates the matching score and matches the optimal auxiliary generator for the client k. Adaptive sample generation: The client generates several synthetic samples based on the auxiliary generator to build an augmented dataset. Then, a hybrid loss is used for training. Real samples are combined with cross-entropy and knowledge distillation to ensure global knowledge integration. The weights of the two are balanced through hyperparameters. The server calculates the generation distribution. Hybrid training involves the client training with hybrid loss based on the constructed augmentation set. This process is repeated for a set number of communication rounds to output a personalized model for each client.
3. The heterogeneous federated learning method based on personalized collaborative generation according to claim 2, characterized in that, In step (1), client k constructs a CVAE model for local training based on a variational autoencoder, denoted as The CVAE model mentioned above includes a conditional encoder model. Conditional decoder model The condition encoder model Original sample Labels corresponding to the original samples Mapped to a multivariate Gaussian distribution; conditional decoder model Reconstructing the original samples using transposed convolution and the ResBlock architecture ; Conditional encoder model The corresponding multivariate Gaussian distribution is mathematically represented as: ； The objective expression for the training loss function of the CVAE model is as follows: ， In the formula, The latent distribution of the encoder output. For the reconstruction distribution of the decoder, It is the standard identity matrix, and p(z) is the prior distribution; Represents the original sample. These are the labels corresponding to the original samples. Represents the latent vector. These represent the parameters of the encoder and decoder, respectively. This represents the number of local samples on client k. Indicates the first One sample, This represents the symbol for calculating the KL divergence for the conditional decoder.
4. The heterogeneous federated learning method based on personalized collaborative generation according to claim 3, characterized in that, The offset distribution is represented as , It is an offset vector. It represents the standard identity matrix, which is independent in each dimension and has a variance of 1.
5. The heterogeneous federated learning method based on personalized collaborative generation according to claim 2, characterized in that, In step (3), the server calculates based on the neighbor set formed by nearest neighbors. The mathematical expression for this calculation is: ， In the formula, This indicates the i-th sample corresponding to client k. value, This represents the client index corresponding to the j-th neighbor in the neighbor set. This represents the sample index corresponding to the j-th neighbor in the neighbor set. It is a neighborhood group. Indicates the corresponding neighbor of the j-th neighbor ; The client calculates the accuracy of various image recognition methods using the following formula: ， in This represents the recognition accuracy of client k in the c-th category; The server calculates the matching score using the following expression: ， In the formula, This represents the class distribution of the generator g; The formula for calculating the generated fraction is: ， The proportion of recommendations generated by client k in category c. Let represent the recognition accuracy of client k in the j-th neighbor set.
6. The heterogeneous federated learning method based on personalized collaborative generation according to claim 2, characterized in that, The hybrid training described in step (3) includes the following mathematical calculation process: The enhanced set constructed by the client is represented as ,in This represents the actual sample data of client k. This represents the synthetic sample data of client k; The loss function for hybrid training is as follows: ， ， ， For the cross-entropy loss of the synthetic samples, The loss is for the real samples. The cross-entropy loss is the value of the real samples. Indicates by temperature parameter For guidance The probability distribution obtained after softening. Indicates using the same temperature parameters For the current output of the model The probability distribution obtained after softening. All of these are hyperparameters.
7. The heterogeneous federated learning method based on personalized collaborative generation according to any one of claims 1-6, characterized in that, This method is applicable to heterogeneous scenarios with both image-aware data and models, and enables collaborative visual model training.
8. A federated learning system for multi-center collaborative diagnosis of medical images, characterized in that, The system performs federated learning of the model using the heterogeneous federated learning method based on personalized collaborative generation as described in any one of claims 1-6. The system establishes personalized disease diagnosis models under the premise of protecting the privacy of medical image data from different medical institutions. Different medical institutions correspond to the clients in the method and train their own conditional variational autoencoders based on their local private medical image data. The server receives and stores decoder parameters and label distributions from all medical institutions. Different medical institutions only need to upload the decoder parameters of the conditional variational autoencoder and the local label distribution to the server. The original medical image data is always kept locally in the medical institution.