Adaptive federal learning shuffling model pulmonary tuberculosis prediction method and system, and storage medium
Through the adaptive federal learning reshuffle model, the parameter importance is evaluated using Hessian values and the Top-K ratio is dynamically adjusted, which solves the problem of inaccurate judgment of parameter importance in the existing technology, and improves the accuracy and safety of the tuberculosis prediction model.
Patent Information
- Application Number
- CN202510444905.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-09
- Publication Date
- 2025-07-29
AI Technical Summary
Existing methods only judge the importance of parameters based on the absolute value of the weights, and cannot accurately estimate the impact of weights on model performance, making the model performance lower after adding differential privacy noise.
Adaptive federated learning shuffle model is used to evaluate parameter importance by calculating the Hessian value of model weights, and dynamically adjust the Top-K ratio based on cosine similarity, select key parameters to add noise, and optimize model performance.
Improve the accuracy and robustness of the model, reduce communication bandwidth requirements, enhance privacy data protection, and improve federated learning efficiency and security.
Smart Images
Figure CN120388214A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of machine learning, and in particular, to an adaptive federated learning shuffling model tuberculosis prediction method, system, and storage medium. Background Art
[0002] Tuberculosis is a disease caused by Mycobacterium tuberculosis infection. With the increase in population density and the change of antibiotic resistance, the incidence of tuberculosis shows an upward trend in some regions. Patients often show symptoms such as persistent cough, sputum production, chest pain, weight loss, and night sweats. Chest X-rays are of great guiding value for the treatment and diagnosis of tuberculosis, and the analysis of chest X-rays helps to prevent the spread of the disease and grasp the best treatment opportunity.
[0003] With the development of machine learning technology, CNN has shown excellent performance in image classification, object detection, and segmentation. By extracting and analyzing features from chest X-rays, it can help identify tuberculosis lesion areas and provide auxiliary diagnostic support for doctors.
[0004] Federated learning technology is a machine learning method with broad prospects, which solves the security problem of training privacy data in CNN. Under the federated learning framework, each participant does not need to directly share the original data, but trains the local model and shares the model weights, effectively protecting the privacy of patients. However, studies have found that there may be a risk of privacy leakage in the model weights uploaded by users after local training, because attackers may infer the user's private data by analyzing these weights.
[0005] The existing shuffling model framework with differential privacy can better solve the above problems. However, due to the different degrees of influence of differential privacy noise on the model in different training stages, the fixed proportion of noise addition strategy may have a greater impact on the accuracy of the model. In addition, only judging the importance of parameters based on the absolute value of the weights cannot accurately estimate the impact of the parameters on the model performance, resulting in a lower performance of the model after adding differential privacy noise. Summary of the Invention
[0006] The technical problems to be solved by the present invention are:
[0007] The existing method only judges the importance of parameters based on the absolute value of the weights, and cannot accurately estimate the impact of the weights on the model performance, resulting in a lower performance of the model after adding differential privacy noise.
[0008] The technical solution adopted by the present invention to solve the above technical problems:
[0009] The present invention provides a tuberculosis prediction method based on an adaptive federated learning shuffling model, including the following steps:
[0010] Step 1: Collect the lung image dataset of the patient. The analyzer of the central server constructs a global model, initializes the weights of the global model, and sets the initial value of the Top-K ratio.
[0011] Step 2: Randomly select clients, broadcast the model weights and the Top-K ratio to the selected clients. The clients train the global model based on their local datasets, evaluate the parameter importance based on the Hessian value of the model weights, select the K most important parameters to add noise, and then send the perturbed model weights to the shuffler.
[0012] Step 3: The shuffler shuffles the model weights uploaded by the clients, eliminates the correlation between the data, and sends the shuffled model weights to the analyzer.
[0013] Step 4: The analyzer aggregates the shuffled model weights and selects the model weights with the highest accuracy to update the global model.
[0014] Step 5: Evaluate the model state. If the evaluation of the model state does not meet the conditions, calculate the cosine similarity between the current model and the previous model, and dynamically adjust the Top-K ratio according to the cosine similarity result.
[0015] Step 6: Repeat Steps 2 to 5 until the result meets the termination training conditions to obtain the tuberculosis prediction model.
[0016] Step 7: Use the tuberculosis prediction model to predict tuberculosis.
[0017] Furthermore, the evaluation of the parameter importance based on the Hessian value of the model weights in Step 2 is specifically as follows:
[0018] Calculate the parameter importance metric value M i :
[0019]
[0020] where W i represents the model weights, and H ii represents the Hessian value corresponding to the model weights;
[0021] The calculation method of the Hessian value is:
[0022]
[0023] where l represents the loss function.
[0024] Furthermore, in Step 2, the clients train the global model based on their local datasets, specifically by dividing it into two branch rounds for local training, and respectively using the parameter importance metric value M iand the model weights W i As a metric for evaluating the importance of the model.
[0025] Furthermore, Step 5 includes the following steps:
[0026] First, if the current loss function value is greater than the loss function value of the previous round, set the loss score score_loss = 1, otherwise score_loss = 0; second, calculate the average accuracy within the past training rounds, compare it with the current accuracy, if the average accuracy of the past training rounds is not lower than the current accuracy, set the accuracy score score_acc = 1, otherwise score_acc = 0; then, according to the ratio t / T of the current training round to the total number of rounds, set the round score score_t, when t / T ≥ 1 / 2, score_t = 1, otherwise score_t = 2×(t / T); finally, calculate the comprehensive model state score score_total = (score_loss + score_acc + score_t) / 3;
[0027] When the comprehensive model state score score_total of the model weights in the current round is less than the threshold, dynamically adjust the Top-K ratio, specifically:
[0028] Calculate the adjustment factor d according to the cosine similarity:
[0029]
[0030] where C t represents the cosine similarity between the current model and the previous model, C t-1 represents the cosine similarity between the previous model and its previous model, C 0 represents the cosine similarity at initialization;
[0031] Update tkr = tkr × d, where tkr is the Top-K ratio.
[0032] Furthermore, the calculation formula for the cosine similarity in Step 4 is:
[0033]
[0034] where, W t and W t-1 represent the model weights of the current round and the previous round respectively, |W t | and |W t-1 | represent the absolute values of the model weights of the current round and the previous round respectively.
[0035] Furthermore, the specific operation of selecting the K most important parameters in Step 2 to add noise is:
[0036] Add Laplace noise to the selected K most important parameters, and specify the privacy budget ∈ of the noise lt as:
[0037]
[0038] where ∈ l is the artificially set local differential privacy budget, W j is the model weight, and tkr is the Top-K ratio.
[0039] The present invention provides a tuberculosis prediction system based on an adaptive federated learning shuffling model. The system has program modules corresponding to the steps of the method described in any one of the above technical solutions, and executes the steps in the tuberculosis prediction method based on the adaptive federated learning shuffling model described above when running.
[0040] The present invention provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps in the tuberculosis prediction method based on the adaptive federated learning shuffling model described in any one of the above technical solutions when called by a processor.
[0041] Compared with the prior art, the beneficial effects of the present invention are:
[0042] The present invention proposes a dynamic adjustment strategy based on cosine similarity, enabling the model to automatically optimize the TopK ratio of the client-uploaded parameters according to the training status, thereby improving the model performance targeted. The present invention can calculate the cosine similarity of the model parameter updates, judge the convergence degree of the current training stage, and dynamically adjust the ratio of the uploaded parameters accordingly, ensuring that the key parameters are shared in time, improving the efficiency of federated learning and the overall performance of the model.
[0043] The present invention utilizes network pruning technology, comprehensively considering the second-order derivative information of the Hessian matrix and the model weights themselves, to accurately evaluate the importance of parameters. The Hessian value reflects the curvature of the loss function with respect to the parameters, and can more accurately measure the impact of each parameter on the model performance. Based on this, prune the parameters with lower importance, streamline the model structure, and reduce the computational and communication overhead.
[0044] The method of the present invention significantly improves the accuracy of the model for tuberculosis prediction. By reducing unnecessary parameter transmission, it further reduces the demand for communication bandwidth and enhances the protection of patients' private data. In addition, the dynamic adjustment and pruning strategies also improve the generalization ability and robustness of the model, providing an efficient and secure solution for federated learning in the medical field. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 Schematic diagram of the federated learning architecture in the embodiments of the present invention;
[0046] Figure 2 Flowchart of the tuberculosis prediction method for the adaptive federated learning shuffling model in the embodiments of the present invention. Detailed implementation manners
[0047] In order to enable those skilled in the art to better understand the solution of the present invention, the exemplary embodiments or examples of the present invention will be described below with reference to the accompanying drawings. Obviously, the described embodiments or examples are only a part, rather than all, of the embodiments or examples of the present invention. All other embodiments or examples obtained by those of ordinary skill in the art based on the embodiments or examples of the present invention without creative efforts shall fall within the protection scope of the present invention.
[0048] To make the above objects, features, and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0049] Specific implementation manner 1: As shown in Figures 1 to 2 The present invention provides a tuberculosis prediction method based on an adaptive federated learning shuffling model, including the following steps:
[0050] Step 1: Collect the lung X-ray image dataset of patients. The analyzer of the central server constructs a global model, initializes the global model weights, and sets the initial value of the Top-K ratio;
[0051] Step 2: The analyzer randomly selects clients according to the client selection rate set in the hyperparameters, broadcasts the model weights and the Top-K ratio to the selected clients. The clients train the global model based on the local dataset, evaluate the parameter importance based on the Hessian value of the model weights, select the K most important parameters, add noise to these parameters through an encoder, and then send the perturbed model weights to the shuffler;
[0052] Step 3: The shuffler shuffles the model weights uploaded by the clients, eliminates the correlation between the data, and sends the shuffled model weights to the analyzer;
[0053] Step 4: The analyzer aggregates the shuffled model weights and selects the model weights with the highest accuracy to update the global model;
[0054] Step 5: Evaluate the model state. If the evaluated model state does not meet the conditions, calculate the cosine similarity between the current model and the previous round model, and dynamically adjust the Top-K ratio according to the cosine similarity result;
[0055] Step 6: Repeat steps 2 to 5 until the result meets the termination training condition to obtain a tuberculosis prediction model;
[0056] Step 7: Use the tuberculosis prediction model to predict tuberculosis.
[0057] Specific implementation plan 2: The importance of parameters is evaluated by the Hessian value of the model weights in step 2, specifically:
[0058] Calculate the parameter importance metric M i :
[0059]
[0060] where W i represents the model weights, and H ii represents the Hessian value corresponding to the model weights;
[0061] The calculation method of the Hessian value is:
[0062]
[0063] Specifically,
[0064]
[0065] where l represents the loss function, and x1…x n represents each weight of the model.
[0066] Other parts of this implementation plan are the same as those of the first specific implementation plan.
[0067] Specific implementation plan 3: Step 5 includes the following steps:
[0068] Comprehensively considering the number of training rounds of the model, the change of the loss function, and the accuracy rate to evaluate the model state, the specific process is as follows: First, if the current loss function value is greater than the loss function value of the previous round, then set the loss score score_loss = 1, otherwise score_loss = 0; Second, calculate and store the average value of the accuracy rates of the most recent t - N - 1 + i times, where t is the current training round, N is the number of participating clients, and i is the client number. Calculate the average accuracy rate within the past training rounds and compare it with the current accuracy rate. If the average accuracy rate of the past training rounds is not lower than the current accuracy rate, then set the accuracy score score_acc = 1, otherwise score_acc = 0; Then, according to the ratio t / T of the current training round to the total number of rounds, set the round score score_t. When t / T ≥ 1 / 2, score_t = 1, otherwise score_t = 2×(t / T); Finally, calculate the comprehensive score score_total = (score_loss + score_acc + score_t) / 3;
[0069] When the comprehensive score score_total of the model weights in the current round is less than the threshold (set to 0.5), calculate the adjustment factor d according to the cosine similarity:
[0070]
[0071] where C t represents the cosine similarity between the current model and the previous model, C t-1 represents the cosine similarity between the previous model and its previous model, C 0 represents the cosine similarity at initialization;
[0072] Update tkr = tkr × d, where tkr is the Top - K ratio.
[0073] Other parts of this implementation plan are the same as those of the second specific implementation plan.
[0074] Specific implementation plan four: The calculation formula for the cosine similarity described in step 4 is:
[0075]
[0076] where, W t and W t-1 respectively represent the model weights of the current round and the previous round, |W t | and |W t-1 | respectively represent the absolute values of the model weights of the current round and the previous round.
[0077] Other parts of this implementation plan are the same as those of the third specific implementation plan.
[0078] Specific implementation plan five: In step 2, the K most important parameters are selected and noise is added. Specifically:
[0079] Laplace noise is added to the selected K most important parameters:
[0080] noise = Lap(μ, b)
[0081] where μ is the mean (usually 0), is the scale parameter, ∈ lt is the privacy budget, and Δf is the sensitivity (usually 1);
[0082] Specify the privacy budget ∈ lt as:
[0083]
[0084] where ∈ l is the local privacy budget, with the initial value set manually, and W j is the model weight.
[0085] The following method is used to evaluate the effect of the privacy budget: Let be the local client encoder, be the shuffler, be the analyzer; Since is an untrusted third-party server, the privacy objective of the entire model is described as there exists satisfying (∈ c , δ c )-differential privacy.
[0086] Since the aforementioned differential privacy mechanism conforms to the composition theorem of differential privacy and the privacy amplification effect of the shuffling model, let [b] be the target off-site of, given and δ cd < 2tkr, there is satisfying (∈ cd , δ cd )-differential privacy, where:
[0087]
[0088] where (∈ c , δ c ) is the privacy budget of the calculated global differential privacy, (∈ cd , δ cd ) is the global differential privacy budget of a single model weight, and δ cd is equivalent to δ c , is the local client encoder, and W j is the model weight.
[0089] The other parts of this implementation plan are the same as those of the fourth specific implementation plan.
[0090] A tuberculosis prediction method (algorithm) based on an adaptive federated learning shuffling model proposed by the present invention is the underlying technical core of the present invention, and various products can be derived based on this algorithm.
[0091] Based on the method proposed by the present invention, a tuberculosis prediction system based on an adaptive federated learning shuffling model is developed using a programming language. This system has program modules corresponding to the steps of the above technical solution, and when running, it executes the steps in the above tuberculosis prediction method based on the adaptive federated learning shuffling model.
[0092] The computer program of the developed system (software) is stored on a computer-readable storage medium. The computer program is configured to implement the steps of the above tuberculosis prediction method based on the adaptive federated learning shuffling model when called by a processor. That is, the present invention is materialized on a carrier to become a computer program product.
[0093] The various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor. The programmable processor can be a dedicated or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.
[0094] The computing programs (also referred to as programs, software, software applications, or code) in the present invention include machine instructions for a programmable processor, and these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device PLD) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.
[0095] Although the present invention is disclosed as above, the scope of protection of the present invention is not limited thereto. Those skilled in the art of the present invention can make various changes and modifications without departing from the spirit and scope of the present disclosure, and these changes and modifications will all fall within the scope of protection of the present invention.
Claims
1. A tuberculosis prediction method based on an adaptive federated learning shuffling model, characterized in that, It includes the following steps: Step 1: Collect the lung image dataset of the patient. The analyzer of the central server constructs a global model, initializes the global model weights, and sets the initial value of the Top-K ratio; Step 2: Randomly select clients, broadcast the model weights and the Top-K ratio to the selected clients. The clients train the global model in two branch rounds based on the local dataset, evaluate the parameter importance based on the model weights and their Hessian values respectively, select the K most important parameters to add noise, and then send the perturbed model weights to the shuffler; Step 3: The shuffler shuffles the model weights uploaded by the clients to eliminate the correlation between the data, and sends the shuffled model weights to the analyzer; Step 4: The analyzer aggregates the shuffled model weights and selects the model weights with the highest accuracy to update the global model; Step 5: Evaluate the model state. If the evaluation of the model state does not meet the conditions, calculate the cosine similarity between the current model and the previous model, and dynamically adjust the Top-K ratio according to the cosine similarity result; Step 6: Repeat steps 2 to 5 until the result meets the termination training condition to obtain the tuberculosis prediction model; Step 7: Use the tuberculosis prediction model to predict tuberculosis.
2. The tuberculosis prediction method based on the adaptive federated learning shuffling model according to claim 1, wherein, The evaluation of the parameter importance based on the model weights and their Hessian values in step 2 includes: Calculate the importance measure value M of the parameter i : Among them, W i represents the model weights, and H ii represents the Hessian value of the model weights; The calculation method of the Hessian value of the model weights is: where l represents the loss function; Adopt the parameter importance metric value M i and the model weight W i As the metric standard for model importance evaluation, evaluate the parameter importance.
3. The tuberculosis prediction method based on the adaptive federated learning shuffling model according to claim 2, wherein Step 5 includes the following steps: First, if the current loss function value is greater than the loss function value of the previous round, set the loss score score_loss = 1, otherwise score_loss = 0; second, calculate the average accuracy rate within the past training rounds and compare it with the current accuracy rate. If the average accuracy rate of the past training rounds is not lower than the current accuracy rate, set the accuracy score score_acc = 1, otherwise score_acc = 0; then, according to the ratio t / T of the current training round to the total number of rounds, set the round score score_t. When t / T ≥ 1 / 2, score_t = 1, otherwise score_t = 2×(t / T); finally, calculate the comprehensive model state score score_total = (score_loss + score_acc + score_t) / 3; When the comprehensive model state score score_total of the model weights in the current round is less than the threshold, dynamically adjust the Top-K ratio. Specifically: Calculate the adjustment factor d according to the cosine similarity: Among them, C t represents the cosine similarity between the current model and the previous-round model, and C t-1 represents the cosine similarity between the previous-round model and its previous-round model, and C 0 represents the cosine similarity at initialization; Update tkr = tkr×d, where tkr is the Top-K ratio.
4. The tuberculosis prediction method based on the adaptive federated learning shuffling model according to claim 3, wherein, The calculation formula of the cosine similarity in step 4 is: Among them, W t and W t-1 represent the model weights of the current round and the previous round respectively, |W t | and |W t-1 | represent the absolute values of the model weights of the current round and the previous round respectively.
5. The tuberculosis prediction method based on the adaptive federated learning shuffling model according to claim 4, wherein The selection of the K most important parameters to add noise in step 2 is specifically: Add Laplace noise to the selected K most important parameters, and specify the privacy budget ∈lt of the noise as: Wherein, ∈ l is the artificially set local differential privacy budget, W j is the model weight, and tkr is the Top-K ratio.
6. A tuberculosis prediction system based on an adaptive federated learning shuffling model, characterized in that, This system has program modules corresponding to the steps of the method described in any one of claims 1 to 5 above, and executes the steps in the above-mentioned tuberculosis prediction method based on the adaptive federated learning shuffling model when running.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program is configured to implement the steps in the tuberculosis prediction method based on the adaptive federated learning shuffling model according to any one of claims 1 to 5 when called by a processor.