Soft Federated Learning Model Parameter Weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional federated learning algorithms face challenges when dealing with outlying institutions, where model performance is poor and parameters from these institutions can 'pollute' the new model, slowing down learning across the federation, due to systematic differences in data cohorts such as scanner models, protocols, or interpretation.
Innovation Solution
The implementation of a soft federated learning method where institutions communicate directly, calculating and using influence values based on model performance across different data cohorts to weight and aggregate model parameters, allowing for tailored models suited to individual institutions without a central server, thus addressing differences in data distributions and reducing the impact of outliers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional federated learning aggregates parameters from all institutions equally weighted by cohort size, then the aggregation process is simple and fast, but outlying institutions with systematic differences pollute the new model and slow down learning
Solution Approach 1:
The patent changes the aggregation parameters from uniform weighting to influence-based weighting. Institutions are evaluated using test sets to determine their influence on the federation, and this influence metric dynamically adjusts their weight in parameter aggregation. This resolves the contradiction by allowing fast aggregation (maintaining productivity) while filtering out polluting outliers through adaptive parameter selection (improving reliability).
Solution Approach 2:
The patent introduces a feedback mechanism where model performance is continuously evaluated using test sets from various institutions. This feedback loop identifies outlying institutions and adjusts their influence weights accordingly. The feedback enables the system to maintain high training speed while automatically adapting to exclude harmful influences, thus resolving the contradiction between productivity and reliability.
2Device complexity
If a central server coordinates federated learning, then model aggregation is centralized and simple, but data privacy and control at individual institutions are reduced
Solution Approach 1:
The patent enables institutions to perform self-service by allowing them to host and coordinate federated learning processes locally. Instead of relying on a central server, institutions can independently manage their data and model training while contributing to the federation. This maintains data privacy and control (reducing loss of information) while keeping the aggregation architecture relatively simple (maintaining low device complexity).
Solution Approach 2:
The patent segments the centralized aggregation function into distributed operations across multiple institutions. Each institution can independently evaluate models using its test set and contribute to the aggregation process according to its determined influence. This segmentation eliminates the need for a central server (maintaining low complexity) while preserving institutional data sovereignty (reducing loss of information).
3Adaptability or versatility
If institutions with different scanner models and protocols are included in the federation, then data diversity increases and model generalization improves, but systematic differences create outliers that pollute the model
Solution Approach 1:
The patent changes how institution contributions are weighted based on their compatibility with the federation. By using test sets to evaluate influence, the system dynamically adjusts parameters to give more weight to institutions with compatible data distributions and less weight to outliers with systematic differences. This maintains adaptability by including diverse institutions while protecting reliability by down-weighting harmful outliers.
Solution Approach 2:
The patent applies local quality by allowing each institution to contribute differently based on its specific characteristics. Instead of treating all institutions uniformly, the system evaluates each institution's local data quality and influence using test sets. This enables the federation to maintain high adaptability by incorporating diverse local expertise while ensuring reliability by giving appropriate weight to each institution based on its actual performance and compatibility.
Data Source
AI summary
An apparatus includes processing circuitry configured to: obtain first trained parameters for a model, wherein the first trained parameters have been generated by training the model using data from a first data cohort; obtain second trained parameters for the model, wherein the second trained parameters have been generated by training the model using data from a second, different data cohort; determine a first evaluation value by inputting data from the first data cohort into a model having the first trained parameters; and determine a second evaluation value by inputting data from the first data cohort into a model having the second trained parameters.