Personalized federal learning-based leucorrhea conventional microscopic examination image classification method
By introducing a knowledge distillation mechanism in the federal learning of medical images, the problems of non-independent and homogeneous distribution of medical image data and privacy protection are solved, the personalized optimization and global consistency of the model are achieved, and the generalization ability and communication efficiency are improved.
Patent Information
- Application Number
- CN202411881255.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-19
- Publication Date
- 2025-04-15
AI Technical Summary
Because medical image data is distributed in different medical institutions, it leads to the non-independent and homogeneous distribution of the data, making it difficult to build a highly generalized model. At the same time, traditional federated learning methods have shortcomings in privacy protection and communication efficiency.
Using a personalized federated learning method based on knowledge distillation, the aggregation mechanism of relevant consistency losses and global average logit outputs is achieved to achieve personalized optimization and global consistency of the client model, while reducing communication overhead.
It improves the generalization ability and personalization performance of the model, ensures data privacy protection, reduces communication overhead, and achieves a balance between personalization and global optimization when processing multi-source heterogeneous data.
Smart Images
Figure CN119832302A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical image-assisted diagnosis, and in particular to a method for classifying leucorrhea routine microscopic examination images based on personalized federated learning. Background Art
[0002] In the modern medical field, computer-aided diagnosis (CAD) has gradually become an important tool to assist doctors in efficiently identifying diseases. Especially in recent years, with the rapid development of artificial intelligence technology, CAD systems have been increasingly used in medical practice. Traditional deep learning methods usually rely on centralized data training. However, since medical data is distributed in different medical institutions, it is restricted by factors such as privacy and information protection laws and cannot be centrally managed and processed, thus forming a "data island" problem. To solve this dilemma, Google introduced a technical framework called Federated Machine Learning (FL). Through federated learning, users can only upload encrypted model parameters to the central server without disclosing privacy data, achieving training results that are better than local models.
[0003] Although federated learning has shown great potential advantages, it still faces many challenges in its development, especially the non-independent and identically distributed (Non-IID) characteristics of client data. The Non-IID characteristics of medical data are particularly prominent, mainly because the databases of different hospitals have significant differences in data types and scales. For example, medical data in different regions may have similarities in disease types, but there are large differences in data magnitude and distribution. This imbalance makes it difficult for a unified medical model to fully adapt to each client, thereby limiting the application scope of traditional machine learning models.
[0004] Compared with traditional federated learning, personalized federated learning (PFL) proposes an effective improvement solution that can better alleviate the impact of the Non-IID problem. Current research has proposed a variety of strategies to achieve personalization, such as fine-tuning the global model or developing personalized models based on the global model. However, these methods still have deficiencies in generalization ability and cannot meet the needs of all clients. In response to the communication cost problem, some scholars have proposed personalized federated learning methods based on knowledge distillation, such as FedMD, FedProto, FedPAC, etc. Although these methods have achieved certain results in reducing communication overhead and improving model performance, there are still problems such as low communication efficiency and unsatisfactory model generalization ability.
[0005] Routine vaginal discharge is an important medical examination item to judge the health level of the female urinary and reproductive system. Microscopic examination is the most commonly used method for routine vaginal discharge examination. At present, the classification of routine vaginal discharge microscopic examination images in hospitals mainly relies on manual microscopic examination, but this method is time-consuming and labor-intensive, and is easily affected by the experience and subjectivity of medical staff, and the accuracy and consistency of detection are difficult to guarantee. In recent years, artificial intelligence technology has shown great potential in routine vaginal discharge microscopic examination analysis, but there are still problems with privacy protection, algorithm bias and supervision. Summary of the invention
[0006] The purpose of the present invention is to provide a method for classifying leucorrhea routine microscopic images based on personalized federated learning to solve the problems raised in the above background technology.
[0007] To achieve the above object, the present invention provides the following technical solutions:
[0008] The present invention provides a personalized federated learning method based on knowledge distillation, which is defined as FedCC (hereinafter referred to as “FedCC”), and the specific steps are as follows:
[0009] S1, system initialization;
[0010] S1-1, the server initializes the global category logit output, sets the number of participating clients C, defines the learning rate η, the cross entropy loss weight parameter α, and the distillation loss weight β;
[0011] S1-2, each client i initializes the corresponding local model parameters W i , initialize the local category logit output And counter The local model is a model stored locally on the client;
[0012] S2. Client local training:
[0013] S2-1, local data set S from client i i Draw a sample of batch B from
[0014] S2-2, traverse each sample (X b ,y b ), where b∈B, the computational model output F i (X b ), calculate the loss L = αL CE +(1-α)L KD +βL CC ,in, is the cross entropy loss, |γ| is the number of categories, y b,i is the label of sample b belonging to category i, is the distillation loss, τ is the relaxation parameter used to soften the global logit output, is the local model logit value z l The probability distribution calculated by the softmax function is: is the global average logit value z g The probability distribution obtained by the softmax function, KL is the KL divergence, is the correlation consistency loss, i and j represent the i-th sample and j-th sample in batch B respectively. and Represents the output of the local model in the middle layer of the network, and represents the corresponding global output, Represents x in the embedding space i and x j The correlation between them is then updated using the gradient descent method to update the model parameters and the category y b logit output and counter;
[0015] S2-3. For each category 1=1, 2, ..., |γ|, calculate the average logit output and Send to the server;
[0016] Step S3: Server-side aggregation and update: for each client, the average logit output of each category is accumulated. Then calculate the global average logit output for each category The differentiated logit output is then calculated for each class of each client Then Send to client i;
[0017] Step S4: Each client i receives the After that, update the local model;
[0018] Step S5, repeating steps S2 to S4 until the predetermined training conditions are met.
[0019] In the embodiment of the present invention, the relevant consistency knowledge distillation in step S2-2 constructs a new loss function L by calculating the correlation difference between the student model (client model) and the teacher model (global logit output) in the feature space, so that the client model can effectively absorb global knowledge while retaining the local data features. This mechanism not only helps the client model to better adapt to the respective local data distribution, but also promotes the consistency between different client models and improves the generalization ability of the overall model. In addition, the relevant consistency knowledge distillation is carefully integrated into the training process in the present invention, and the accuracy and stability of feature correlation are ensured through the calculation and normalization of the Gram matrix. This method avoids the privacy risks brought by the direct transmission of global model parameters, and at the same time, through the knowledge distillation method, efficient information sharing and knowledge transfer are achieved, further enhancing the performance and robustness of the model. Finally, the introduction of relevant consistency knowledge distillation enables the federated learning method of the present invention to better coordinate the knowledge differences between different data sources when processing multi-source data, and achieve a balance between personalization and global optimization. Through this innovative knowledge distillation mechanism, the present invention has significant technical advantages and application potential in the field of personalized federated learning, and is particularly suitable for scenarios with high requirements for data privacy and model personalization, such as medical image analysis.
[0020] Based on the above algorithm FedCC, the present invention also provides a classification model for routine microscopic examination of leucorrhea based on personalized federated learning, and the specific steps are as follows:
[0021] Step 1: Data collection. As a specific embodiment of the present invention, experiments are conducted on two data sets, which are the public data set CIFAR-10 and the routine microscopic examination data set of leucorrhea collected by laboratory professionals.
[0022] Step 2: Data cleaning. Data cleaning of public data is to ensure the integrity of the data set and ensure data consistency. For data collected in the laboratory, data cleaning can improve data quality and reduce noise infection.
[0023] Step 3: Data standardization and normalization. Standardization and normalization are common methods for adjusting data size. In image processing, normalization is to scale pixel values from [0,255] to [0,1] to ensure consistency between different images.
[0024] Step 4: Data partitioning, using the Dirichlet distribution (β) method to partition the data set to simulate data heterogeneity scenarios. Specifically, the degree of heterogeneity of the data distribution is controlled by adjusting the parameter β of the Dirichlet distribution. When β is small, the data distribution differences between clients are large; when β is large, the data distribution is closer to independent and identically distributed. This mechanism can flexibly simulate real scenarios in federated learning and evaluate the performance of the algorithm under different data distribution conditions. The present invention uses two β values (β1=1.0 and β2=0.1) for testing.
[0025] Step 5: Model training. Train all client models and select the deep learning network ResNet18 as the basic model. This network performs well in image recognition tasks, has strong feature extraction capabilities and high computing efficiency. At the same time, its network structure is relatively simple, which is convenient for rapid deployment and training, helping to achieve experimental goals and ensure the performance and generalization ability of the model.
[0026] Step 6: Result recognition, obtain the recognition result according to the model in step 5.
[0027] Compared with the prior art, the present invention has the following beneficial effects:
[0028] (1) In the processing of multi-source routine microscopic image data of leucorrhea, the data generated by different medical institutions or equipment have obvious heterogeneity and non-independent and identically distributed (non-IID) characteristics. Traditional centralized learning methods often have difficulty in building a model with strong generalization when dealing with these differences due to inconsistent data distribution, while sharing raw data will lead to privacy issues. The present invention uses a federated learning framework to centralize data processing and model training on the local client, without sharing raw data, thus ensuring data privacy.
[0029] (2) The present invention constructs a unified global knowledge representation by aggregating the category logit outputs calculated by each client, thereby effectively integrating multi-source heterogeneous information without directly contacting the data, significantly improving the model's adaptability to different data sources, and providing an efficient and secure solution for cross-institutional collaboration.
[0030] (3) The present invention introduces a relevant consistency knowledge distillation mechanism in federated learning, which significantly improves the personalized performance and global consistency of the model.
[0031] (4) The method proposed in this paper introduces relevant consistency knowledge distillation in federated learning, innovatively combines global category logit output, realizes refined knowledge sharing and personalized optimization, and breaks through the limitations of traditional federated learning in processing non-IID data. By only transmitting the average logit output, the method reduces communication overhead and improves efficiency. At the same time, the relevant consistency loss calculation and logit output aggregation mechanism ensure the stability and convergence of training. It is efficient, simple, and easy to deploy, and is suitable for practical scenarios.
[0032] (5) The multi-source leucorrhea routine microscopic classification method proposed in the present invention helps to assist clinical physicians in the accurate diagnosis and treatment of precancerous lesions of the lower reproductive tract and reduces the cost of the medical care system. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 A flowchart of a personalized federated learning framework for introducing relevant consistent knowledge distillation for the present invention;
[0034] Figure 2 The figure is a schematic diagram of the system flow of the implementation process of the present invention. DETAILED DESCRIPTION
[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0036] The main abbreviations and key terms involved in the present invention are explained as follows:
[0037] CAD: Computer aided diagnosis, is a technology that uses imaging, medical image processing technology and other possible physiological and biochemical methods combined with computer analysis and calculation to assist in the discovery of lesions and improve the accuracy of diagnosis;
[0038] FL: Federated machining learning / Federated learning, a machine learning framework that can effectively help multiple institutions use data and perform machine learning modeling while meeting the requirements of user privacy protection, data security, and government regulations;
[0039] PFL: Personalized Federated Learning, personalized federated learning;
[0040] Non-IID: Non-independent and identically distributed, non-independent and identically distributed.
[0041] Example 1
[0042] A personalized federated learning method based on knowledge distillation
[0043] This paper proposes a personalized federated learning method based on knowledge distillation, such as Figure 1 As shown, the present invention first performs local training on each local client, and then all clients upload average logits and labels. These logits are then integrated into a global average logit, and these global average logits and labels are distributed to local clients. The clients then calculate the relevant consistency distillation loss based on the local logit and the global average logit, and finally obtain a better model.
[0044] In detail, a personalized federated learning method based on knowledge distillation of the present invention is defined as FedCC (hereinafter referred to as "FedCC"), which includes the following steps:
[0045] Step S1: System initialization
[0046] S1-1: The server initializes the global category logit output, sets the number of participating clients C (for the subsequent CIFAR10 dataset C=10, microscopy data C=3), defines the learning rate η=0.01, the cross entropy loss weight parameter α=0.5, and the distillation loss weight β=0.1.
[0047] S1-2: Each client i initializes the local model parameters W i , initialize the local category logit output And counter
[0048] Step S2: Client local training
[0049] S2-1: local dataset S from client i i A batch of B=16 samples is drawn from
[0050] S2-2: Iterate over each sample (X b ,y b ), where b∈B, the computational model output F i (X b ), calculate the loss L = αL CE +(1-α)L KD +βL CC ,in, is the cross entropy loss, |γ| is the number of categories, y b,i is the label of sample b belonging to category i, is the distillation loss, τ is the relaxation parameter used to soften the global logit output, τ = 1.0 in the embodiment, is the local model logit value z l The probability distribution calculated by the softmax function is: is the global average logit value z g The probability distribution obtained by the softmax function; KL is the KL divergence, is the correlation consistency loss, i and j represent the i-th sample and j-th sample in batch B respectively. and Represents the output of the local model in the middle layer of the network, and represents the corresponding global output, Represents x in the embedding space i and x j The correlation between them; then the model parameters are updated using the gradient descent method, and the category y is updated b logit output and counter;
[0051] S2-3: For each category l = 1, 2, ..., |γ|, calculate the average logit output and Sent to the server.
[0052] Step S3: Server-side aggregation and update
[0053] For each client, sum the average logit output for each category Then calculate the global average logit output for each category The differentiated logit output is then calculated for each class of each client Then Sent to client i.
[0054] Step S4: The client receives the updated logit output: Each client i receives the logit output sent by the server. Then update the local model.
[0055] Step S5, repeating steps S2 to S4 until the predetermined training conditions are met.
[0056] Example 2
[0057] In an embodiment of the present invention, in order to meet the needs of efficient computing, the present invention preferably uses a high-performance GPU server at the hardware level, which can significantly improve the computing efficiency of model training and reasoning. At the software level, the PyTorch framework widely used in the field of deep learning is preferred, and related dependent libraries are integrated to build an efficient and stable experimental platform, which lays a solid foundation for the development of research work and ensures the smooth development of experiments.
[0058] A classification model for routine microscopic examination images of leucorrhea based on personalized federated learning, the specific steps are as follows:
[0059] Step 1: Data collection. This embodiment conducts experiments on two data sets, namely the public data set CIFAR-10 and the routine microscopic examination data set of leucorrhea collected by laboratory professionals. Specifically, CIFAR10 has 60,000 images, including 10 categories of airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships and trucks, 6,000 images in each category. The routine microscopic examination of leucorrhea comes from the collection of laboratory professionals, and the Nugent score has 314 images, including 110 normal flora, 96 intermediate flora, and 108 bacterial vaginosis.
[0060] Step 2: Data cleaning. During data cleaning, the integrity of the data set and data consistency must be ensured for public data. For data collected in the laboratory, data cleaning can improve data quality and reduce noise infection.
[0061] Step 3: Data standardization and normalization. Standardization and normalization are common methods for adjusting data size. In image processing, normalization is to scale pixel values from [0,255] to [0,1] to ensure consistency across different images.
[0062] Step 4: Data partitioning, using the Dirichlet distribution (β) method to partition the data set to simulate data heterogeneity scenarios. Specifically, the degree of heterogeneity of the data distribution is controlled by adjusting the parameter β of the Dirichlet distribution. When β is small, the data distribution differences between clients are large; when β is large, the data distribution is closer to independent and identically distributed. This mechanism can flexibly simulate real scenarios in federated learning and evaluate the performance of the algorithm under different data distribution conditions. The present invention uses two β values (β1=1.0 and β2=0.1) for testing.
[0063] Step 5: Model training. Train all client models and select the deep learning network ResNet18 as the basic model. This network performs well in image recognition tasks, has strong feature extraction capabilities and high computing efficiency. At the same time, its network structure is relatively simple, which is convenient for rapid deployment and training, helping to achieve experimental goals and ensure the performance and generalization ability of the model.
[0064] Step 6: Result recognition, obtain the recognition result according to the model in step 5.
[0065] Example 3
[0066] Model evaluation and validation
[0067] In the embodiment of the present invention, in order to verify the effectiveness of the method FedCC provided in Embodiment 1 of the present invention in dealing with data heterogeneity, the performance difference between it and other federated learning algorithms is compared. The specific steps of model evaluation and verification are as follows:
[0068] (1) Different federated learning algorithms are used to build models. The classic FedAvg algorithm and the personalized federated learning algorithm FedPAC based on knowledge distillation are used to build models respectively, and the corresponding performance indicators are obtained from different data sets, see Table 1.
[0069] (2) Model performance comparison: the performance parameters of each algorithm model obtained in step (1) are compared with the performance of the algorithm model FedCC provided by the present invention. The specific comparison method is: when β is large (β1=1.0), the traditional federated learning algorithm (FedAvg) and the personalized federated learning algorithms (FedCC and FedPAC) are close in performance; when β is small (β2=0.1), that is, when the data heterogeneity is strong, the performance of the traditional federated learning algorithm is lower.
[0070] In an embodiment of the present invention, FedAvg adopts a weighted average method to aggregate model parameters. The server side calculates the average value of all client model parameters and returns the result to the client. FedPAC explicitly aligns local and global features by utilizing global semantic knowledge to learn more accurate shared and personalized representations. At the same time, by quantifying the model contribution of each client classifier combination, it proposes and solves an optimization problem of optimal weight allocation, thereby improving the performance and adaptability of the personalized model.
[0071] Table 1 Comparison of indicators of different algorithms under different heterogeneity
[0072]
[0073] As can be seen from Table 1, the performance of the algorithm model FedCC provided by the present invention is optimal in different data sets, and it has the ability to solve the problem of data heterogeneity.
[0074] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0075] It should be noted that the above content only illustrates the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications all fall within the protection scope of the claims of the present invention.
Claims
1. A classification method for routine leucorrhea microscopic examination images based on personalized federated learning, characterized in that , including the following steps: S1, system initialization; S1-1, the server initializes the global category logit output, sets the number of participating clients C, defines the learning rate η, the cross entropy loss weight parameter α, and the distillation loss weight β; S1-2, each client i initializes the corresponding local model parameters W i , initialize the local category logit output And counter S2. Client local training: S2-1, local data set S from client i i Draw a sample of batch B from S2-2, traverse each sample (X b ,y b ), where b∈B, the calculation model outputs Fi(X b ), calculate the loss L = αL CE +(1-α)L KD +βL CC ,in, is the cross entropy loss, |γ| is the number of categories, y b，i is the label of sample b belonging to category i, is the distillation loss, τ is the relaxation parameter used to soften the global logit output, is the local model logit value z l The probability distribution calculated by the softmax function is: is the global average logit value z g The probability distribution obtained by the softmax function, KL is the KL divergence, is the correlation consistency loss, i and j represent the i-th sample and j-th sample in batch B respectively. and Represents the output of the local model in the middle layer of the network, and represents the corresponding global output, represents the embedding space χ i and χ j The correlation between them is then updated using the gradient descent method to update the model parameters and the category y b logit output and counter; S2-3. For each category l=1, 2, ..., |γ|, calculate the average logit output and Send to the server; Step S3: Server-side aggregation and update: for each client, the average logit output of each category is accumulated. Then calculate the global average logit output for each category The differentiated logit output is then calculated for each class of each client Then Send to client i; Step S4: Each client i receives the After that, update the local model; Step S5, repeating steps S2 to S4 until the predetermined training conditions are met.
2. According to the personalized federated learning-based leucorrhea routine microscopic image classification method of claim 1, it is characterized in that: The personalized federated learning combines relevant consistent knowledge distillation.
3. Application of the leucorrhea routine microscopic examination image classification method based on personalized federated learning as described in any one of claims 1-2 in the leucorrhea routine microscopic examination image classification model.