Federated Model Contribution Evaluation for Secure Reward Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for integrating learned models across organizations in federated learning fail to appropriately evaluate the contribution of individual learning participants, leading to inadequate rewards and discouraging participation.
Innovation Solution
An evaluation system that includes parameter acquisition, integration, inference, evaluation, and output mechanisms to assess the contribution of local models through secure computation, enabling accurate reward calculation for participants.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If learned models from multiple learning devices are integrated through federated learning, then the overall model accuracy and reliability are improved, but the contribution of individual learning participants cannot be appropriately evaluated
Solution Approach 1:
The patent segments the evaluation process by calculating the contribution of each learning device separately through ablation methods. The system computes the difference between the integrated model's performance and the performance when each individual learning device's model is excluded, thereby segmenting the overall contribution into measurable individual components.
Solution Approach 2:
The patent introduces an intermediary evaluation mechanism that acts as a mediator between the integrated model and individual learning participants. This intermediary system calculates contribution metrics by systematically excluding each learning device's model from the integration and measuring the resulting performance change, thus enabling fair evaluation without compromising the integrated model's accuracy.
2Reliability
If data from multiple organizations is aggregated for model training, then the model's predictive performance is improved, but data security and privacy protection are compromised
Solution Approach 1:
The patent applies segmentation by keeping data distributed across different learning devices and organizations rather than aggregating it centrally. Each learning device maintains its own local data, and only model parameters are shared and integrated, thereby segmenting data access and maintaining security while enabling collaborative model improvement.
Solution Approach 2:
The patent introduces secure computation as an intermediary mechanism that enables model integration without direct data access. This intermediary layer allows the system to compute integrated model parameters while maintaining data privacy and security, preventing harmful data exposure while achieving improved predictive performance.
3Adaptability or versatility
If learning participants are rewarded based on contribution, then participation motivation is improved, but the complexity of calculating fair rewards increases
Solution Approach 1:
The patent applies self-service by enabling learning participants to automatically receive rewards based on their measured contribution. The system calculates contribution metrics through standardized ablation procedures and automatically distributes rewards without requiring complex manual assessment or intervention, thereby simplifying the reward calculation process while maintaining fair participation incentives.
Data Source
AI summary
An evaluation system according to the present invention includes: a memory configured to store instructions; and one or more processors. The one or more processors is configured to execute the instructions to: acquire, for a plurality of local models of learning participants for inferring a specific event, parameters of the plurality of local models; integrate the acquired parameters of the plurality of local models; execute the inference using an integrated model obtained by integrating the parameters of the plurality of local models; evaluate a contribution of each of the local models based on a result of the inference; and output the contribution.