Federated Learning Method for Image Analysis Models Based on Noisy Heterogeneous CXR Images

Abstract

The application discloses a kind of based on noisy heterogeneous CXR image image analysis model federated learning method, through the federated learning framework integrated participant selection module and payment module realized by the method, participant selection module is designed to distinguish the participant with low noise, high representative distribution of data, and payment module is according to the contribution incentive of each participant, in the method described in the application, the evaluation of contribution is based on Shapley value method widely used in coalition game. Compared with the prior art, the method described in the application does not need to repeatedly train the model, greatly reduces the consumption of time and computing resources, while obtaining good global model performance.

Description

Technical Field

[0001] This invention relates to the field of model prediction and federated learning, specifically to a federated learning method for image analysis models based on noisy heterogeneous CXR images, and particularly to a performance contribution value evaluation model and a reward payment module in model construction. Background Technology

[0002] In federated learning, multiple parties collaborate to train a high-performance global model. Each participant's data is stored locally, rather than uploaded to a central server. Only the model training parameters are uploaded to update the global model parameters. The benefits of federated learning include the following:

[0003] (1) Data privacy can be preserved;

[0004] (2) The bandwidth requirements during communication can be greatly reduced.

[0005] Due to these two characteristics, federated learning has been applied in many fields.

[0006] Existing technologies, when used for medical image analysis or related machine-aided diagnostic applications, such as using federated learning for COVID-19 diagnosis, suffer from unsatisfactory diagnostic performance (e.g., accuracy) due to the following three issues.

[0007] Question 1: Heterogeneous data.

[0008] First, the number of patients in the hospitals varied; second, the proportion of positive patients in the hospitals differed. Based on this experimental design and existing data, it is evident that CXR images from different participants are heterogeneous.

[0009] Question 2: Noise data.

[0010] Due to differences in diagnostic knowledge and techniques regarding COVID-19 among doctors in different countries, the data contains noise. Noisy data can be categorized into many types, such as closed-set noise and open-set noise. In this paper, we primarily study closed-set noise, where "COVID-19" is incorrectly labeled as another known category, "non-COVID-19". Experimental results show that during federated learning training, the greater the noise of the participating nodes, the lower the accuracy of the global model.

[0011] Question 3: Low willingness to participate.

[0012] Participating in the federated learning process consumes bandwidth and computing resources, and may also leak some information contained in the transmitted local model, so most participants may not be very willing to participate.

[0013] To diagnose COVID-19, existing federated learning methods primarily aim to improve the performance of local models. For example, some sources indicate that if participants determine the global COVID-19 diagnostic model does not offer performance benefits when its parameters are sent, they will not participate in federated learning. Existing technologies have also designed decentralized federated learning frameworks for COVID-19 diagnosis to provide local models with their own data in a safer manner. The technical approach provided in this invention differs from these existing works in that it achieves a better global model, making it more applicable to a broader COVID-19 population, rather than solely benefiting local models.

[0014] Many existing works use Shapley values ​​as the basis for measuring the contributions of federated learning participants, and the mainstream solution involves repeated training using different combinations of federated learning participants. Given the large size of local hospital datasets, directly applying this solution to COVID-19 diagnosis scenarios would predictably lead to even greater computational and communication costs on top of the existing computational costs. Furthermore, repeatedly combining different participants to calculate Shapley values ​​could make it easier for malicious nodes to infer the local model parameters of each participant node, potentially leading to the leakage of medical privacy.

[0015] As an improvement, to reduce the exponentially large computational complexity associated with the number of participants, some researchers specify a combination of one or more participants, typically specifying the target participant's joining order as last. They then estimate the Shapley value using the marginal contribution of each federated learning participant in the given training sequence of the federated learning model. The marginal contribution of later participants to the performance of the federated learning model may be smaller, potentially leading to incorrect convergence of the Shapley value estimate. Summary of the Invention

[0016] Purpose of the invention: To address the shortcomings of existing technologies in federated learning models, especially the insufficient federated learning and training of models when using federated learning for COVID-19 diagnosis, this invention provides a federated learning method for image analysis models based on noisy heterogeneous CXR images. This method is applicable to hospital impact model federated learning, including COVID-19 machine vision image processing, and related modeling.

[0017] Technical Solution: A federated learning method for image analysis models based on noisy heterogeneous CXR images. This method determines the participation order of nodes based on their contributions and performs computation on all participating nodes in a single federated learning process. The method includes the following steps:

[0018] (1) Establish a medical image analysis model based on CXR images and calculate the performance contribution value PCV of P n , where the value model υ(.) of the image analysis model is determined, and the expression of the value model is as follows:

[0019] υ(·) = α * Sensi + (1 - α) * Speci.

[0020] Among them, Sensi represents sensitivity, Speci represents specificity, α is a constant coefficient within [0, 1], and υ(·) represents the value brought by P i N in a federated learning round and a permutation;

[0021] (2) When measuring the marginal Sensi contribution to the medical image analysis model, the definition of MissedDR is as follows:

[0022] MissedDR n = 1 - Sensi n ,

[0023]

[0024] Among them, TP represents the number of disease patients correctly analyzed, and FN represents the number of disease patients misanalyzed;

[0025] (3) For the value model, define υ(·) = MissedDR n , then the marginal MissedDR contribution of P n N to the global model is calculated as follows:

[0026] υ(z ∪ n) - υ(z) = MissedDR z∪n - MissedDR z

[0027] = (1 - Sensi z∪n ) - (1 - Sensi z )

[0028] = Sensi z - Sensi z∪n ;

[0029] (4) Measure the marginal Speci contribution to the medical image analysis model, where the definition of MisDR is as follows:

[0030] MisDR n = 1 - Speci n ,

[0031]

[0032] Where TN represents the number of healthy individuals who were correctly diagnosed, FP represents the number of healthy individuals who were incorrectly diagnosed, and υ(z∪n) represents PN. n The sensitivity value, therefore υ(z∪n)-v(z) is a PN n The marginal specific contribution of PN to the global model increases with the magnitude of the difference. n The greater the contribution to the Speci improvement of the global model;

[0033] (5) Define υ(·)=MisDR for the value model n Then it means PN n The marginal MisDR contribution to the global model is calculated as follows:

[0034] v(z∪n)-υ(z)=MisDR z∪n -MisDR z

[0035] =(1-Speci z∪n )-(1-Speci z )

[0036] =Speci z -Speci z∪n ,

[0037] If PN n This causes MisDR to decrease, which in turn carries PN. n Speci, i.e., Speci z∪n If the difference is high, then the difference in the above formula is negative, and the larger the absolute value of the difference, the higher the contribution of Speci.

[0038] (6) Define the performance contribution value (PCV) of the medical image analysis model as follows:

[0039] PCV n =λ*PCV n +μ*S n (υ),

[0040] in, In WN r In the middle, S n (v) represents a Shapley value similar to the following formula. The Shapley value is an indicator for evaluating contribution. WN r This indicates the node selected in the r-th round;

[0041] (7) Definition Combination The Shapley value is defined as follows:

[0042]

[0043] in, Let υ(z)∪i represent the set of PNs, υ(z)∪i represent the game value containing PNi, and υ(z) represent the game value not containing PNs. i Game value;

[0044] The above formula is similar to PNi through averaging The expected marginal contribution of all possible permutations to the game;

[0045] |z|! represents the number of possible permutations in z∪i with i as the last element, and the probability that i will eventually join the alliance z∪i.

[0046] (8) Define the reward calculation scheme, specifically, the node set is denoted as WN, and when running r-th rounds of federated learning, the total budget is B, WN i The compensation received is as follows:

[0047]

[0048] exist

[0049] set up This represents the performance improvement that PNi brings to the global model in round r. It is the sum of the model performance improvements brought to the global model by all PNs in the r-th round;

[0050] (9) In the r-th federated learning round, the central node will assign the global model weight parameters ω r Send to all PNs, then PNs use their private local data S n Update the model by running SGD to obtain local parameters. That is, PN n The loss function l based on the descent gradient will be calculated. n (·) and learning rate η;

[0051] The medical image analysis model has been updated as follows:

[0052]

[0053] (10) PNs will Send to the central node, the central node only uses PN n In the validation dataset D v The parameters sent are used to calculate the PCV for each PN, which is expressed as PCV. n ;

[0054] Finally, the set of nodes selected to participate in the next round is denoted as Winner Nodes (WNs). Then, from WNs to each i, the central node aggregates the model parameters as follows, with the expression as follows:

[0055]

[0056] In the method described, in the case u(·)=MissedDR n In, if PN n This causes MissedDR to decrease, which in turn carries PN. n Sensi, i.e., Sensi z∪n If the difference is high, then the difference is negative; the larger the absolute value of the difference, the higher the contribution of Sensi. In the case υ(·)=MisDR n In, if PN n This causes MisDR to decrease, which in turn carries PN. n Speci, i.e., Speci z∪n The difference in this formula is negative.

[0057] Furthermore, the method calculates P n In the PCV performance contribution process, α = 1 is set, and the larger the absolute value of the difference, the higher the contribution of Speciali.

[0058] Furthermore, in the method, the degree of bias towards the MissedDR diagnostic model can be adjusted by setting the value of α;

[0059] When selecting a low-MissedDR diagnostic model, set α to a relatively high value; when selecting a low-MisDR diagnostic model, set α to a lower value.

[0060] Furthermore, the process by which the method trains the medical image analysis model is as follows:

[0061] S1. The central server selects participating nodes based on their accumulated data contributions in previous rounds and sends the global model to each selected node.

[0062] S2. Select nodes to train local models based on the global model and local data;

[0063] S3. Transfer the updated local model to the central server;

[0064] S4. The central server aggregates the received local models based on the data values ​​of each participant to obtain the final model performance, and uses the aggregated model as the new global model. At the same time, the central server makes payments based on the cumulative contribution of each participant node.

[0065] Beneficial effects: The method described in this invention is based on the Shapley value method, which is widely used in coalition games. Compared with the prior art, the technical solution provided by this invention solves the three problems listed in the background art: data heterogeneity, noise, and low participation willingness. Furthermore, the modeling method of this invention does not require repeated model training, which greatly reduces the consumption of time and computing resources, while achieving good global model performance. Finally, it can be widely used in machine-aided analysis of hospital images and related image processing models. Attached Figure Description

[0066] Figure 1 This is a flowchart of the method described in this invention;

[0067] Figure 2 The performance contribution value of the present invention varies under noisy and non-independent identically distributed conditions;

[0068] Figures 3-5 show a comparison of the accuracy and sensitivity of the present invention across different participant data types.

[0069] Among them: Figure 3(a) and Figure 3(b) are the comparison of accuracy and sensitivity under the experimental setup with noisy and independent and identically distributed data; Figure 4(a) and Figure 4(b) are the comparison of accuracy and sensitivity under the experimental setup with no noise and non-independent and identically distributed data; Figure 5(a) and Figure 5(b) are the comparison of accuracy and sensitivity under the experimental setup with noisy and non-independent and identically distributed data. Detailed Implementation

[0070] To illustrate the technical solutions disclosed in this invention in detail, the invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0071] This invention provides a federated learning method for image analysis models based on noisy heterogeneous CXR images. This method can be used to implement a novel federated learning framework for medical image analysis, thereby enabling the construction of machine vision analysis models for medical images. The method integrates a participant selection module and a payment module. The participant selection module is designed to distinguish participants with data possessing low-noise, highly representative distributions, while the payment module is designed based on incentives for each participant's contribution.

[0072] It is well known that deep learning requires a significant amount of training data to develop better COVID-19 diagnostic models. However, due to medical data privacy concerns, many hospitals may be unwilling to contribute data to train these models. Distributed machine learning frameworks, specifically federated learning, offer a promising solution to this problem.

[0073] The following is a detailed description of the application of the method described in this invention in federated learning and model building for COVID-19 diagnosis.

[0074] The entire process of the method described in this invention is as follows: Figure 1 As shown. The main idea is a federated learning process for COVID-19 based on noisy heterogeneous CXR images, and the implementation flow of the method is as follows:

[0075] First, the central server selects participating nodes based on their accumulated data contributions in previous rounds and sends the global model to each selected node. Second, the selected nodes train local models using the global model and their local data. Next, they transmit the updated local models to the central server. Finally, the central server aggregates the received local models based on each participant's data values ​​to obtain the final model performance, and uses the aggregated model as the new global model. Simultaneously, the central server distributes payments based on each participating node's accumulated contribution.

[0076] The contribution of each participating node is positively correlated with its contribution to model performance. In addition to accuracy, this invention also considers the false positive and false negative rates of COVID-19 diagnosis. To calculate the contribution of a particular participating node, this invention removes that node and recalculates the model performance. When considering the order of participation, this invention uses the Shapley value. The number of retraining iterations increases exponentially when calculating the contribution of a single individual. Unlike existing work, the method described in this invention only requires one federated learning iteration to calculate the order-related contributions of all participating nodes. Furthermore, the federated learning process considers the different contributions of participants to achieve better global model performance.

[0077] Specifically, the method described in this invention includes the following calculations:

[0078] (1) Calculate P n PCV is the performance contribution value.

[0079] First, establish a value model υ(·) designed for diagnosing COVID-19.

[0080] υ(·)=α*Sensi+(1-α)*Speci.

[0081] Sensi represents sensitivity, and Speci represents specificity. These two metrics can also be replaced with others. α is a constant coefficient within [0, 1]. υ(·) represents PN. i The value brought about in a federal learning cycle, in a permutation.

[0082] (2) When the aim is to measure the marginal Sensi contribution to a COVID-19 diagnostic model, MissedDR is defined as follows:

[0083] MissedDR n =1-Sensi n ,

[0084]

[0085] In this context, TP represents the number of COVID-19 patients who were correctly diagnosed as positive. FN represents the number of COVID-19 patients who were incorrectly diagnosed as negative. Generally, Sensi represents the proportion of patients diagnosed as positive who are actually positive, also known as the true positive rate or recall rate.

[0086] It should be noted that in this embodiment, a positive result is referred to as a disease patient in the technical solution, and a negative result is referred to as a healthy person in the above technical solution.

[0087] (3) At this point, define υ(·) = MissedDR n Then the following formula represents PN n Marginal MissedDR contribution to the global model:

[0088] υ(z∪n)-υ(z)=MissedDR z∪n -MissedDR z

[0089] =(1-Sensi z∪n )-(1-Sensi z )

[0090] =Sensi z -Sensi z∪n ,

[0091] In this case, α = 1. If PN n This causes MissedDR to decrease, which in turn carries PN. n Sensi, i.e., Sensi z∪n If the difference is high, then the difference in the above formula is negative. The larger the absolute value of the difference, the higher the contribution of Sensi.

[0092] (4) When the aim is to measure the marginal Speci contribution to a COVID-19 diagnostic model, MisDR is defined as follows:

[0093] MisDR n =1-Speci n ,

[0094]

[0095] Where TN represents the number of healthy individuals correctly diagnosed as negative cases, FP represents the number of healthy individuals incorrectly diagnosed as positive cases, and v(z∪n) represents PN. n The sensitivity value, therefore υ(z∪n)-υ(z) is the PN n The marginal specific contribution of PN to the global model increases with the magnitude of the difference. n The greater the contribution to the Speci improvement of the global model, the better.

[0096] (5) At this point, define υ(·)=MisDR n Then the following formula represents PN n Marginal MisDR contribution to the global model:

[0097] υ(z∪n)-u(z)=MisDR z∪n -MisDR z

[0098] =(1-Speci z∪n )-(1-Speci z )

[0099] =Speci z -Speci z∪n ,

[0100] If PN n This causes MisDR to decrease, which in turn carries PN. n Speci, i.e., Speci z∪n If the difference is high, then the difference in the above equation is negative. In this case, α = 0. The larger the absolute value of the difference, the higher the contribution of Speci.

[0101] (6) Define the performance contribution value, as shown in the following expression:

[0102] PCV n =λ*PCV n +μ*S n (υ),

[0103] in, In WN r In the middle, S n (v) represents a Shapley value similar to the following formula. WN r This indicates the node selected in the r-th round.

[0104] (7) Definition The Shapley score is generally considered a metric for evaluating contribution. In general, the Shapley score is a method that approximates the improved contribution of each node by calculating the marginal contribution of each possible combination and then averaging them. The arrangement of PNs is denoted by z. The Shapley value is defined as:

[0105]

[0106] in, Let υ(z)∪i represent the set of PNs, and let υ(z)∪i represent the set containing PNs. i The game value, υ(z), represents the value that does not contain PN. i The game value. The right half of the formulas in steps (3) and (5) are derived from the concept of Shapley value, in which the υ(·) function is also defined.

[0107] The above formula is similar to PN i By average Let |z|! represent the expected marginal contribution of all possible permutations to the game. |z|! represents the number of possible permutations in z∪i where i is the last element. The probability that i is the last element in the alliance z∪i is...

[0108] (8) Define the remuneration calculation scheme.

[0109] The core idea is to pay a reward strongly correlated with the contribution of nodes (PNs) to the entire R rounds of federated learning. In each round, nodes that contribute more to the global model and have a higher probability of contributing in previous rounds are selected, and the reward is accumulated over rounds. Let the set of nodes be denoted as WN. When running r-th rounds of federated learning, the total budget is B. WN i The compensation received is as follows:

[0110]

[0111] exist set up In the r-th round, PN represents i Improved model performance for the global model. It is the sum of the model performance improvements brought to the global model by all PNs in the r-th round.

[0112] (9) In the r-th federated learning round, the central node will assign the global model weight parameters ω r Send to all PNs. Then PNs use their private local data S n Update the model by running SGD to obtain local parameters. That is, PN n The loss function l based on the descent gradient will be calculated. n (·) and learning rate η. The model is updated as follows:

[0113]

[0114] (10) PNs will Send to the central node. The central node only uses PN. n In the validation dataset D v The parameters sent are used to calculate the PCV for each PN, which is expressed as PCV. n .

[0115] Finally, the set of nodes selected to participate in the next round is denoted as Winner Nodes (WNs). Then, from WNs to each i, CN aggregates the model parameters as follows.

[0116]

[0117] Furthermore, referring to the accompanying drawings. Figure 2 In the diagram, noisy nodes are marked with an asterisk and represented by dashed lines, numbered 6 to 9. Noise-free nodes are represented by solid lines. In noisy and non-independent identically distributed (ICD) scenarios, the PCV performance of all noise-free PNs is superior to that of noisy PNs. This indicates that the FFCD algorithm performs well in this scenario, effectively selecting low-noise nodes for federated learning, demonstrating its good performance. Figures 3-5 show that, for both FFCD and FedAvg algorithms, in noisy and ICD experimental settings, the sensitivity performance of the global model continuously improves with increasing round numbers. The performance of this algorithm is significantly better than FedAvg. The modified federated learning framework in this invention offers higher performance for image processing and converges faster and more easily during federated learning training.

Claims

1. A federated learning method of an image analysis model based on a noisy heterogeneous CXR image, characterized in that: The method formulates the participation order of nodes according to the different contributions of participants, and realizes the calculation of all participating nodes in one federated learning process, the method includes establishing a medical image analysis model based on CXR images, and includes the following steps: (1) Calculate the performance contribution value PCV of P n and determine the value model υ(·) of the image analysis model, the expression of the value model is as follows: υ(·) = α * Sensi + (1-α) * Speci. where Sensi denotes sensitivity, Speci denotes specificity, a is a constant factor within [0, 1], and υ(·) denotes a PN i The value brought by a round of federated learning, one permutation (2) When measuring the marginal Sensi contribution to the medical image analysis model, the definition of MissedDR is as follows: Missed DR n = 1 - Sensi n , Where TP represents the number of patients with diseases who are correctly analyzed, and FN represents the number of patients with diseases who are incorrectly analyzed; (3) For the value model, define υ(·) = MissedDR n Then PN n The marginal MissedDR contribution to the global model is computed as follows: υ(z∪n) - υ(z) = MissedDR z∪n MissedDR z = (1 - Sensi z∪n ) - (1 - Sensi z ) = Sensi z - Sensi z∪n ; (4) When measuring the marginal Speci contribution to the medical image analysis model, the definition of MisDR is as follows: MisDR n = 1 - Speci n , where TN represents the number of healthy people who are correctly diagnosed, FP represents the number of healthy people who are incorrectly diagnosed, and υ(z∪n) represents the sensitivity value of the PN n , so υ(z∪n)-υ(z) is the PN n specificity contribution to the global model, the greater the difference, the PN n greater the contribution of Speci improvement to the global model; (5) For the value model, redefine υ(·) = MisDR n represents PN n The marginal MisDR contribution to the global model is computed as follows: u(z U n) - u(z) = MisDR z∪n - MisDR z = (1 - Speci z∪n ) - (1 - Speci z ) = Speci z -Speci z∪n , If PN n is made to decrease, then Speci with PN n , i.e. Speci z∪n is high, the difference of the above formula is negative, and the greater the absolute value of the difference is, the higher the contribution of Speci is. (6) Define the performance contribution value PCV of the medical image analysis model, and the expression is as follows: PCV n = λ * PCV n + μ * S n (v), wherein In the WN r S n (v) represents a Shapley value of the form similar to r represents the node selected in the r-th round; (7) Definitions In connection with The Shapley value is defined as: wherein, denotes the set of PNs, v(z)∪i denotes the game value containing the PN i v(z) denotes the game value not containing the PN i v(z) denotes the game value not containing the PN The above formula is similar to PN i By averaging all possible permutations contribute equally to the expected margin of the game; |z|! denotes the number of possible permutations with i last in z U i, the probability of i joining the coalition z U i last is (8) Formulate a reward calculation scheme, specifically, the node set is denoted as WN, when running the r-th round of federated learning, the total budget B, WN i The obtained reward is as follows: In Let denotes the model performance improvement brought by the PNs in the rth round i the model performance improvement brought by the PNs in the rth round is the sum of the model performance improvement brought by all PNs in the rth round (9) In the r-th federated learning round, the center node sends the weight parameters ω r of the global model to all PNs, then PNs update the model with private local data S n i.e. PN n will compute the loss function l n (·) based on the descending gradient and learning rate η;​ For this, the medical image analysis model is updated as follows: (10) PNs send to the central node, which uses only the PNs n In verifying the data set D v The parameters sent above, the PCV of each PN is calculated, denoted as PCV n ; Finally, the set of nodes participating in the next round is selected as Winner Nodes (Wns), and then from WNs to each i, the central node aggregates the model parameters in the following way, and the expression is as follows:

2. The federated learning method of image analysis model based on noisy heterogeneous CXR images according to claim 1, wherein: In case v(·) = MissedDR n , if PN n makes MissedDR decrease, then Sensi with PN n , i.e. Sensi z∪n is high, then the difference of the equation is negative, and the larger the absolute value of the difference, the higher the contribution of Sensi; in case v(·) = MisDR n , if PN n makes MisDR decrease, then Speci with PN n , i.e. Speci z∪n is high, then the difference of the equation is negative.

3. The federated learning method of the noisy-based heterogeneous CXR image-based vision analysis model according to claim 1 or 2, characterized in that: The method sets α = 1 in calculating the performance contribution value PCV of P n The greater the absolute value of the difference is, the higher the Speci contribution is.

4. The federated learning method for noisy-based heterogeneous CXR image-based vision analysis model according to claim 1, wherein: In the method, the bias degree of the MissedDR diagnosis model can be realized by adjusting the value of α; When it is desired to select a low MissedDR diagnosis model, α is set to a relatively high value; When it is desired to select a low MisDR diagnosis model, α can be set to a lower value.

5. The federated learning method for noisy-based heterogeneous CXR image-based vision analysis model according to claim 1, wherein: The method for the learning and training process of the medical image analysis model is as follows: S1, the central server selects participating nodes according to the accumulated data contribution of the participating nodes in the previous rounds, and sends the global model to each selected node; S2, the selected node trains a local model according to the global model and local data; S3, the updated local model is transmitted to the central server; S4, the central server aggregates the received local models according to the data value of each participant to obtain the final model performance, and the aggregated model is used as the new global model, and the central server pays according to the cumulative contribution of each participant node.

Images