Federated Medical Model Training With Quality-Weighted Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of obtaining sufficient patient data to train machine learning models for personalized medicine across diverse patient populations is hindered by ethical, legal, and administrative barriers to data sharing, leading to inefficiencies in model performance and deployment.
Innovation Solution
A distributed machine learning process, such as federated learning, is employed to train models using data from multiple clinical sites without transferring data, incorporating metadata on training quality to adjust local updates based on site-specific performance, thereby improving model accuracy and reducing bias and model drift.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data from multiple clinical sites is combined to train models, then model performance and generalizability are improved, but ethical, legal, and administrative barriers prevent data sharing
Solution Approach 1:
A central server acts as an intermediary to coordinate federated learning across multiple clinical sites. The server distributes the global model to participating sites, collects local model updates, and aggregates them into updated global models, enabling collaborative training without direct data exchange between sites.
Solution Approach 2:
The invention extracts only the necessary model parameters and metadata from local training processes, rather than transferring raw patient data. Local model updates and quality metadata are sent to the central server for aggregation, separating the essential learning signals from the sensitive data that cannot be shared.
2Productivity
If simple averaging of local model updates is used, then the process is simple and fast, but model drift and bias occur due to varying data quality across sites
Solution Approach 1:
Quality metadata is collected from each clinical site and fed back into the model aggregation process. This metadata provides feedback signals about local data quality, enabling the system to adjust aggregation weights dynamically rather than using uniform averaging, thus improving model accuracy while maintaining efficiency.
Solution Approach 2:
The aggregation process changes from fixed equal weighting to dynamic weighting based on quality metadata. The system adjusts the weight of each local update according to metrics such as data quality scores, sample sizes, and performance indicators, optimizing the contribution of each site to the global model.
3Reliability
If more training data samples are used, then model performance improves, but the time to collect and process data from multiple sites increases
Solution Approach 1:
Quality metadata is collected and evaluated during the local training process at each clinical site, before the aggregation step. This preliminary assessment of data quality enables efficient weight assignment during aggregation, reducing the time needed for centralized evaluation and speeding up the overall training cycle.
Solution Approach 2:
The federated learning process enables continuous model improvement by allowing multiple clinical sites to train simultaneously on their local data. Rather than sequential data collection and processing, the system maintains continuous learning across all participating sites, with models being updated iteratively as new local training results become available.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
According to an aspect, there is provided a method of training a model to perform a task on medical data using a distributed machine learning process whereby a global model is updated based on training performed on local copies of the model at a plurality of clinical sites. The method comprises a) sending (302) information to the plurality of clinical sites to enable each of the plurality of clinical sites to create a local copy of the model and train the respective local copy of the model on training data at the respective clinical site. The method then comprises b) receiving (304), from each of the plurality of clinical sites, i) a local update to a parameter in the model obtained by training the local copy of the model on the training data at the respective clinical site and ii) metadata related to a quality of the training performed at the respective clinical site; and c) updating (306) the parameter in the global model, based on the received local updates to the parameter and the received metadata.