Methods and apparatuses for performing federated learning

Abstract

A method (200) for performing Federated Learning, FL, in an FL system (100) comprising an FL collecting server (110) and a plurality of contributing servers (120a- d), wherein the method is performed by the FL collecting server (110) The method comprises receiving (210), from at least one contributing server (120a-d) of the plurality of contributing servers (120a-d), at least one first model update contribution, and at least one first assessment weight, obtaining (220) a first updated model of the global model, obtaining (230) a first performance metric of the first updated model; updating (240) the at least one reputation weight to at least one first updated reputation weight, receiving (250), from the at least one contributing server (120a-d), at least one second model update contribution, and at least one second assessment weight, and obtaining (260) a second updated model of the global model.

Description

[0001] METHODS AND APPARATUSES FOR PERFORMING FEDERATED LEARNING

[0002] TECHNICAL FIELD

[0003] The present disclosure relates to a Federated Learning (FL) collecting server, a post- hoc verifier server, and at least one aggregator server and methods performed by the FL collecting server, the post-hoc verifier server, and the at least one aggregator server. Related computer programs and computer readable storage mediums are also disclosed.

[0004] BACKGROUND

[0005] Proliferation of sensitive personal data, such as medical records, across various global locations posed significant challenges for developing Machine Learning (ML) models that require access to such data. Traditionally, centralizing data from different sources into a single location for training ML models has been a common practice. However, this approach faces obstacles, especially in terms of compliance with stringent data privacy regulations like the General Data Protection Regulation (GDPR) in Europe, which imposes strict rules on how personal data is collected, stored, and processed. To mitigate these challenges, organizations are increasingly using Federated Learning (FL) as an innovative solution. FL allows ML models to be trained across multiple contributors such as local bodes corresponding to e.g. decentralized devices or servers holding local data samples without exchanging them. Essentially, contributors compute updates to an ML model based on their data, and the updates are sent encrypted to a FL collecting server such as a central server, where they are aggregated (e.g., averaged) to update a global model.

[0006] While an FL system may enable protection of sensitive data in distributed settings, the FL system is not immune to attacks from malicious actors who aim to compromise the FL system's integrity and performance. These attacks can significantly impair the learning process by providing misleading contributions to the global model. Among the most notable of these attacks are Byzantine attacks, Model Poisoning, Sybil Attacks, Data Poisoning, and Eavesdropping. To combat these threats and attacks, mitigation techniques have been developed for detecting and either excluding or minimizing an impact of anomalous contributions providing the misleading contributions. Anomaly detection techniques are commonly employed for monitoring and identifying updates that significantly deviate from expected norms and are potential indicators of malicious activity.

[0007] The mitigation techniques, which are essentially reactive, can be limited by the sophistication of an attack. Therefore, more proactive, preventive measures based on reputation-based FL systems have been proposed as an effective complement. Reputation-based systems work by associating a reputation weight with each contributor in the FL system. The reputation weight reflects the reliability and trustworthiness of a contributor based on its past behaviors. Contributors that consistently provide updates that enhance model performance are assigned higher reputation weights, whereas those contributing faulty updates are ranked lower or even excluded. By preventing malicious actors from gaining influence over the trained ML model, the FL system may be effectively fortified. By continuously adjusting the reputation weight based on historical performance and consistency, reputation based FL systems can preclude contributors with fraudulent intent from participating further in the process, thereby safeguarding the integrity of the global model from distributed attacks.

[0008] US 2023017500 A1 refers to a method for federated learning (FL) comprising training a machine learning (ML) model collaboratively by initiating a round of FL across data parties. Each data party is allocated tokens to utilize during the training. The method further comprises maintaining, for each data party, a corresponding data usage profile indicative of an amount of data the data party consumed during the training, a corresponding participation profile indicative of an amount of data the data party provided during the training, a resource usage of the data party, and reputation of the data party.

[0009] However, challenges may arise as reputation-based FL systems have inherent limitations that can compromise their effectiveness. While reputation-based FL system may be efficient at preventing malicious contributions, they may be insufficient in dynamically assessing the quality of current contributions. This creates a gap in the ability to effectively weigh and integrate contributions into a FL model. For example, a possible limitation in a reputation-based FL system arises from that reputation weighting is primarily based on historical performance. Weighting based on the historical performance does not necessarily reflect current quality of the contribution. Discrepancies arising, for example, from the historical performance leading to low reputation, and valuable contribution of a contributor can lead to skewed model results where potentially valuable contributions are not fully realized or utilized.

[0010] SUMMARY

[0011] An object of the invention is to enable improved security in an FL system.

[0012] A first aspect of the invention relates to a method for performing Federated Learning, FL, in an FL system comprising an FL collecting server and a plurality of contributing servers, wherein the plurality of contributing servers is contributing to a global model, wherein the method is performed by the FL collecting server, the method comprising receiving, from at least one contributing server of the plurality of contributing servers, at least one first model update contribution to the global model, and at least one first assessment weight pertaining to at least one first evaluation of the at least one first model update contribution, obtaining a first updated model of the global model based on the at least one first model update contribution and at least one reputation weight of each of the at least one contributing server, wherein the at least one reputation weight is pertaining to a reliability metric of each of the at least one contributing server, obtaining a first performance metric of the first updated model, updating the at least one reputation weight to at least one first updated reputation weight based on the first performance metric, the at least one first model update contribution and the at least one first assessment weight, receiving, from the at least one contributing server, at least one second model update contribution to the global model, and at least one second assessment weight pertaining to at least one second evaluation of the at least one second model update contribution, and obtaining a second updated model of the global model based on the at least one second model update contribution and the at least one updated first reputation weight.

[0013] A second aspect of the invention relates to a method for performing Federated

[0014] Learning, FL, in an FL system comprising an FL collecting server, a post-hoc verifier server, and a plurality of contributing servers, wherein the plurality of contributing servers is contributing to a global model, wherein the method is performed by the post-hoc verifier server, the method comprising receiving, from the FL collecting server, at least one first assessment weight, and determining at least one first updated reputation weight based on a first performance metric of a first updated model and the at least one first assessment weight.

[0015] A third aspect of the invention relates to a method for performing Federated Learning, FL, in an FL system comprising an FL collecting server, at least one aggregator server and a plurality of contributing servers, wherein the plurality of contributing servers is contributing to a global model, wherein the method is performed by the at least one aggregator server, the method comprising receiving, from the FL collecting server, at least one reputation weight of each of at least one contributing server of the plurality of contributing servers, wherein the at least one reputation weight is pertaining to a reliability metric of each of the at least one contributing server, at least one first model update contribution to the global model, and at least one first assessment weight pertaining to at least one first evaluation of the at least one first model update contribution, determining, a first updated model based on the at least one reputation weight, the at least one first assessment weight, and the at least one first model update contribution, sending, to the FL collecting server, the first updated model.

[0016] A fourth aspect of the invention relates to a Federated Learning, FL, system comprising an FL collecting server being configured to perform the method according to the first aspect, a post-hoc verifier server being configured to perform the method according to the second aspect, and at least one aggregator server being configured to perform the method according to the third aspect.

[0017] A fifth aspect of the invention relates to a Federated Learning, FL, collecting server, wherein an FL system comprises the FL collecting server and a plurality of contributing servers, wherein the plurality of contributing servers is contributing to a global model, whereby the FL collecting server is configured to receive, from at least one contributing server of the plurality of contributing servers, at least one first model update contribution to the global model, and at least one first assessment weight pertaining to at least one first evaluation of the at least one first model update contribution, obtain a first updated model of the global model based on the at least one first model update contribution and at least one reputation weight of each of the at least one contributing server, wherein the at least one reputation weight is pertaining to a reliability metric of each of the at least one contributing server, obtain a first performance metric of the first updated model, update the at least one reputation weight to at least one first updated reputation weight based on the first performance metric, the at least one first model update contribution and the at least one first assessment weight, receive, from the at least one contributing server, at least one second model update contribution to the global model, and at least one second assessment weight pertaining to at least one second evaluation of the at least one second model update contribution, and obtain a second updated model of the global model based on the at least one second model update contribution and the at least one updated first reputation weight.

[0018] A sixth aspect of the invention relates to a Federated Learning, FL, collecting server, wherein an FL system comprises the FL collecting server and a plurality of contributing servers, wherein the plurality of contributing servers is contributing to a global model, wherein the FL collecting server comprises processing circuitry and a computer readable storage medium, the computer readable storage medium containing instructions executable by the processing circuitry, whereby the FL collecting server is configured to receive, from at least one contributing server of the plurality of contributing servers, at least one first model update contribution to the global model, and at least one first assessment weight pertaining to at least one first evaluation of the at least one first model update contribution, obtain a first updated model of the global model based on the at least one first model update contribution and at least one reputation weight of each of the at least one contributing server, wherein the at least one reputation weight is pertaining to a reliability metric of each of the at least one contributing server, obtain a first performance metric of the first updated model, update the at least one reputation weight to at least one first updated reputation weight based on the first performance metric, the at least one first model update contribution and the at least one first assessment weight, receive, from the at least one contributing server, at least one second model update contribution to the global model, and at least one second assessment weight pertaining to at least one second evaluation of the at least one second model update contribution, and obtain a second updated model of the global model based on the at least one second model update contribution and the at least one updated first reputation weight.

[0019] A seventh aspect of the invention relates to a post-hoc verifier server, wherein an FL system comprises an FL collecting server, the post-hoc verifier server, and a plurality of contributing servers, wherein the plurality of contributing servers is contributing to a global model, whereby the post-hoc verifier server is configured to receive, from the FL collecting server, at least one first assessment weight, and determine at least one first updated reputation weight based on a first performance metric of a first updated model and the at least one first assessment weight.

[0020] An eighth aspect of the invention relates to a post-hoc verifier server, wherein an FL system comprises an FL collecting server, the post-hoc verifier server, and a plurality of contributing servers, wherein the plurality of contributing servers is contributing to a global model, wherein the post-hoc verifier server comprises processing circuitry and a computer readable storage medium, the computer readable storage medium containing instructions executable by the processing circuitry, whereby the post-hoc verifier server is configured to receive, from the FL collecting server, at least one first assessment weight, and determine at least one first updated reputation weight based on a first performance metric of a first updated model and the at least one first assessment weight.

[0021] A ninth aspect of the invention relates to at least one aggregator server, wherein an FL system comprises an FL collecting server, the at least one aggregator server and a plurality of contributing servers, wherein the plurality of contributing servers is contributing to a global model, whereby the at least one aggregator server is configured to receive, from the FL collecting server, at least one reputation weight of each of at least one contributing server of the plurality of contributing servers, wherein the at least one reputation weight is pertaining to a reliability metric of each of the at least one contributing server, at least one first model update contribution to the global model, and at least one first assessment weight pertaining to at least one first evaluation of the at least one first model update contribution, determine, a first updated model based on the at least one reputation weight, the at least one first assessment weight, and the at least one first model update contribution, send, to the FL collecting server, the first updated model. A tenth aspect of the invention relates to at least one aggregator server, wherein an FL system comprises an FL collecting server, the at least one aggregator server and a plurality of contributing server, wherein the plurality of contributing servers is contributing to a global model, wherein the at least one aggregator server comprises processing circuitry and a computer readable storage medium, the computer readable storage medium containing instructions executable by the processing circuitry, whereby the at least one aggregator server is configured to receive, from the FL collecting server, at least one reputation weight of each of at least one contributing server of the plurality of contributing servers, wherein the at least one reputation weight is pertaining to a reliability metric of each of the at least one contributing server, at least one first model update contribution to the global model, and at least one first assessment weight pertaining to at least one first evaluation of the at least one first model update contribution, determine, a first updated model based on the at least one reputation weight, the at least one first assessment weight, and the at least one first model update contribution, send, to the FL collecting server, the first updated model.

[0022] A eleventh aspect of the invention relates to a computer program comprising instructions which, when executed on processing circuitry of a Federated Learning, FL, collecting server, cause the processing circuitry of the FL, collecting server to carry out the method according to the first aspect, processing circuitry of a post-hoc verifier server, cause the processing circuitry of the a post-hoc verifier server to carry out the method according to the second aspect, and processing circuitry of at least one aggregator server, cause the processing circuitry of the at least one aggregator server to carry out the method according to the third aspect.

[0023] A twelfth aspect of the invention relates to a tangible, non-volatile computer readable medium comprising instructions that, when executed on processing circuitry of a Federated Learning, FL, collecting server, cause the processing circuitry of the FL, collecting server to carry out the method according to the first aspect, processing circuitry of a post-hoc verifier server, cause the processing circuitry of the a post-hoc verifier server to carry out the method according to the second aspect, and processing circuitry of at least one aggregator server, cause the processing circuitry of the at least one aggregator server to carry out the method according to the third aspect.

[0024] BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a schematic diagram illustrating an example of an environment.

[0026] Figure 2A-2D are flowcharts illustrating a method 200 performed by the FL collecting server.

[0027] Figure 3 is a flowchart illustrating a method 300 performed by the post-hoc verifier server

[0028] Figure 4 is a flowchart illustrating a method 400 performed by the at least one aggregator server

[0029] Figure 5A and 5B are a signaling diagrams illustrating exemplary embodiments of the method 200, method 300 and method 400

[0030] Figure 6 illustrates a block diagram illustrating embodiments of the FL collecting server

[0031] Figure 7 illustrates a block diagram illustrating embodiments of the post-hoc verifier server

[0032] Figure 8 illustrates a block diagram illustrating embodiments of the at least one aggregator server

[0033] DETAILED DESCRIPTION

[0034] Figure 1 shows a schematic diagram illustrating an example setup of an environment in which embodiments presented herein can be applied. Figure 1 illustrates an FL system 100 comprising a FL collecting server 110, at least one contributing server 120a-d, optionally a reputation database 130, at least one aggregator server 140a-d and a post-hoc verifier server 150, and at least one contributor 120e. The plurality of contributing servers 120a-d may be communicatively connected to the FL collecting server 110, and the FL collecting server 110 may be communicatively connected to the reputation database 130, the at least one aggregator server 140a-d and the post- hoc verifier server 150 such as via external and / or internal entities using wired communications, e.g., based on Ethernet, and / or wireless communications, e.g., WiFi, and / or a cellular network corresponding to one of or a combination of 5G cellular networks, Long Term Evolution (LTE) and Evolved Packet System (EPS), LTE- advanced, Universal Mobile Telecommunications System (UMTS), or any other current or future wireless network, such as a future 3GPP 6G network. The plurality of contributing severs 120a-d is contributing to a global model, via e.g. the FL collecting server 110.

[0035] The FL collecting server 110 may correspond to a centralized FL collecting server 110 or a decentralized collecting server. The decentralized collecting server may optionally correspond to at least one of the at least one contributing server 120a-d of the plurality of the contributing server e.g. hosting a distributed ledger such as a blockchain. The decentralized collecting server may adopt a full peer-to-peer architecture e.g. using a consensus mechanism and / or a proof of state algorithm. The FL collecting server 110 may be in practice one FL collecting server host or two or more FL collecting server hosts.

[0036] The FL collecting server 110, the at least one contributing server 120a-d of the plurality of contributing server 120a-d, the post-hoc verifier server 150 and / or the at least one aggregator server 140a-d may correspond to at least one hard-ware based apparatus running a server as a software such as a web server, e.g. hosting websites and delivering web pages, a file server e.g. storing and managing access files, a database server e.g. providing database services to applications or users, an application server e.g. hosting and running application logic, a proxy server e.g. acting as intermediary for client requests to other servers, and / or a cloud server. The at least one contributing server 120a-d may correspond to one comprising server 120a or two or more contributing server 120a-d. In an example the FL system 100 may be in a cloud environment.

[0037] The reputation database 130 may correspond to a database comprising a memory, and / or to a database server hosted on a computing platform or a cloud service.

[0038] The at least one aggregator server 140a-d may be in practice one aggregator server host 140a or two or more aggregator server hosts 140a-d. In an example at least one first aggregator server host of the two or more aggregator server hosts may not be comprised in the FL system 100 and communicatively connected to the FL collecting server 110 and / or at least one second aggregator server host of the two or more aggregator server hosts.

[0039] Figure 2A-2D are flowcharts illustrating a method 200 performed by the FL collecting server 110, as shown in Figure 1 and described in the text thereto. The steps of Figure 2B continue where the steps of Figure 2A end, the steps of Figure 2C continue where the steps of Figure 2B end, and the steps of Figure 2D continue where the steps of Figure 2C end.

[0040] The FL collecting server 110 receives in a step 210 at least one first model update contribution to the global model, and at least one first assessment weight from the at one contributing server 120a-d of the plurality of contributing servers 120a-d. The at least one assessment weight pertains to at least one first evaluation of the at least one first model update contribution. In a step 220, the FL collecting server 110 obtains 220 a first updated model of the global model based on the at least one first model update contribution and at least one reputation weight of each of the at least one contributing server 120a-d. The at least one reputation weight pertains to a reliability metric of each of the at least one contributing server 120a-d. In a step 230, the FL collecting server 110 obtains a first performance metric of the first updated model. In a step 240, the FL collecting server 110 updates the at least one reputation weight to at least one first updated reputation weight based on the first performance metric, the at least one first model update contribution and the at least one first assessment weight. In a step 250, the FL collecting server 110 receives at least one second model update contribution to the global model, and at least one second assessment weight from the at least one contributing server 120a-d. The at least one second assessment weight pertains to at least one second evaluation of the at least one second model update contribution. In a step 260, the FL collecting server 110 obtains a second updated model of the global model based on the at least one second model update contribution and the at least one updated first reputation weight.

[0041] The solution presented herein addresses the above-mentioned challenges of enabling a higher security in the FL system 100 by e.g. enabling prevention of skewed model outcomes. The solution presented herein allows to dynamically assess the quality of contributions and enables an improved accurate and secure aggregation of the global model. By introducing an assessment weight, the FL system 100 is enabled to dynamically adjust a weight of each contribution based on the reputation weight and the assessment weight. The solution presented herein further enhances the FL system 100 by allowing to evolve based on real-time data rather than solely relying on a historical performance of the contributor 120e. As the contributing server 120a-d may submit a model update contribution such as the first and / or second model update contribution together with the assessment weight such as the first and / or second assessment weight, the FL collecting server 110 is enabled to verify an accuracy and reliability of each contribution. Additionally, and / or alternatively, the solution presented herein may enable obtaining inconsistencies between the assessment weight and the first performance metric enabling further a dynamic adjustment of trust levels. In an example, the contributing server 120a-d may accurately assess the model update contribution in the assessment weight. The actual accuracy may be assessed, by the FL system 100, based on the first performance and if the accuracy aligns with the actual accuracy, an accurate assessment of the contributing server 120a-d may resolve in an improvement on the reputation weight. Further, the solution presented herein may allow new or domainspecific expertise to be recognized and acknowledged, supporting a robust, adaptable FL environment maximizing the potential of the plurality of contributing server 120a-d. In an example, the method 200 may refer to a rewarding system by enabling a striving for high-quality contribution updates due to reiteration of the reputation weight based on an assessment weight and a performance metric of each cycle of contribution. In an example, the method 200 may be repetitively performed, which may enable a cyclical feedback loop encouraging accountability and continuous improvement in contributions.

[0042] By receiving the first model contribution and the at least one first assessment weight in step 210, the FL collecting server 110 may be enabled to obtain the first updated model in step 220. By obtaining the first updated model in step 220 taking the reputation weight, the first model contribution and optionally the first assessment weight into account, the FL collecting server 110, may be enabled to improve the first updated model and / or ensure the security. By obtaining the first performance metric in step 230, the FL collecting server 110 may be enabled to verify the accuracy and reliability of the first model contribution. By updating the at least one reputation weight in step 240 based on the first performance metric and the assessment weight, the FL collecting server 110 may be enabled to acknowledge the quality of first model contribution. Further by performing step 240, the FL collecting server 110 may be enabled to obtain in step 260 a second updated model based on the updated reputation weight and thus support a robust adaptable global model ensuring the security. By obtaining in step 260 the second updated model based on the updated reputation weight, the FL collecting server 110 may be enabled to efficiently prevent malicious contribution and / or to ensure a more accurate aggregation of the global model.

[0043] Referring to Figure 2A, the FL collecting server 110 may obtain in an optional step 212 the at least one reputation weight of each of the at least one contributing server 120a-d.

[0044] Optionally, the step 212 comprises an optional step 213, wherein the FL collecting server 110 sends a request, to the reputation database 130, for the at least one reputation weight.

[0045] Optionally, the step 212 comprises an optional step 214, wherein the FL collecting server 110 receives, from the reputation database 130, the at least one reputation weight.

[0046] The step 220 comprises optionally an optional step 222a. The FL collecting server 110 sends in the optional step 222a to at least one aggregator server 140a-d, the at least one reputation weight, the at least one first assessment weight, and the at least one first model update contribution.

[0047] Alternatively, to step 222a the step 220 comprises an optional step 222b. The FL collecting server 110 may send in the optional step 222b, to the at least one aggregator server 140a-d, the at least one reputation weight and the at least one first model update contribution. The step 220 comprises further optionally an optional step 224. The FL collecting server 110 may receive the first updated model from the at least one aggregator server 140a-d.

[0048] Alternatively, to step 222a or step 222b, and step 224, the step 220 may optionally comprise an optional step 226a or an optional step 226b.

[0049] In the optional step 226a, the FL collecting server 110 determines the first updated model based on the at least one reputation weight, the at least one first assessment weight, and the at least one first model update contribution.

[0050] In the optional step 226b, the FL collecting server 110 determines the first updated model based on the at least one reputation weight, and the at least one first model update contribution.

[0051] In an optional step 228, the FL collecting server 110 sends to the plurality of contributing servers 120a-d, the first updated model.

[0052] Optionally, the step 230 comprises an optional step 230a. In the optional step 230a, the FL collecting server 110 receives from at least one of the plurality of contributing servers 120a-d, feedback associated with the first performance metric.

[0053] Referring to Figure 2B, optionally, the step 240 comprises an optional step 242. In the optional step 242, the FL collecting server 110 obtains, from at least one of the plurality of contributing servers, at least one first updated reputation weight based on the first performance metric, the at least one first model update contribution and the at least one first assessment weight. Optionally obtaining is based further on the reputation weight.

[0054] The optional step 242 optionally comprises an optional step 242a and an optional step 244a, or an optional step 242b. In the optional step 242a, the FL collecting server 110 sends, to the post-hoc verifier server 150, and the at least one first assessment weight. Optionally the FL collecting server 110 sends, the reputation weight, the first performance metric, and / or the at least one first model update contribution in step 242a. In the optional step 244a, the FL collecting server 110 receives, from the post-hoc verifier server 150, the at least one first updated reputation weight. In the optional step 242b the FL collecting server 110 determines the at least one first updated reputation weight based on the first performance metric, and the at least one first assessment weight, optionally on the reputation weight and optionally the at least one first model update contribution.

[0055] Optionally, the step 240 comprises an optional step 246. In the optional step 246, the FL collecting server 110 saves the at least one first updated reputation weight in the reputation database 130.

[0056] In an optional step 252 the FL collecting server 110 obtains the at least one first updated reputation weight of each of the at least one contributing server 120a-d.

[0057] Optionally, the step 252 comprises an optional step 253, wherein the FL collecting server 110 sends a second request, to the reputation database 130, for the at least one first updated reputation weight.

[0058] Optionally, the step 252 comprises an optional step 254, wherein the FL collecting server 110 receives, from the reputation database 130, the at least one first updated reputation weight.

[0059] Referring to Figure 2C, optionally, the step 260 comprises an optional step 262a and an optional step 264, or an optional step 266a. Alternatively, the step 260 comprises an optional step 262b and the optional step 264, or an optional step 266b.

[0060] In the optional step 262a, the FL collecting server 110 sends, to at least one aggregator server 140a-d the at least one first updated reputation weight, the at least one second assessment weight, and the at least one second model update contribution. In the optional step 262b, the FL collecting server 110 sends, to at least one aggregator server 140a-d the at least one first updated reputation weight and the at least one second model update contribution.

[0061] In the optional step 264, the FL collecting server 110 receives from the at least one aggregator server 140a-d, the at least one second updated model.

[0062] In the optional step 266a, the FL collecting server 110 determines the second updated model based on the at least one first updated reputation weight, the at least one second assessment weight, and the at least one second model update contribution. In the optional step 266b, the FL collecting server 110 determines the second updated model based on the at least one first updated reputation weight, and the at least one second model update contribution.

[0063] In an optional step 268, the FL collecting server 110 sends, to the plurality of contributing servers 120a-d, the second updated model.

[0064] In an optional step 270, the FL collecting server 110 obtains a second performance metric of the second updated model.

[0065] Optionally, the optional step 270 comprises an optional step 270a, wherein the FL collecting server 110 receives, from at least one of the plurality of contributing servers 120a-d feedback associated with the second performance metric.

[0066] Referring to Figure 2D, in an optional step 280, the FL collecting server 110 updates the at least one first updated reputation weight to at least one second updated reputation weight based on the second performance metric, the at least one second model update contribution and the at least one second assessment weight.

[0067] Optionally updating is further based on the at least one first updated reputation weight.

[0068] Optionally the step 280 comprises an optional step 282. In the optional step 282, the FL collecting server 110 obtains the at least one second updated reputation weight based on the second performance metric, the at least one second model update contribution and the at least one second assessment weight. Optionally the FL collecting server 110 obtains the at least one second updated reputation weight further based on the first updated reputation weight.

[0069] Optionally the step 282 comprises an optional step 282a and an optional step 284a, or an optional step 282b. In the optional step 282a, the FL collecting server 110 sends, to a post-hoc verifier server 150, the at least one second assessment weight. Optionally the FL collecting server 110 sends further the at least one first updated reputation weight, the second performance metric, and / or the at least one second model update contribution in step 282a. In the optional step 284a, the FL collecting server 110 receives, from the post-hoc verifier server 150, the at least one second updated reputation weight. In the optional step 282b, the FL collecting server 110 determines the at least one second updated reputation weight based on the second performance metric, the at least one second assessment weight. Optionally, the FL collecting server 110 determines further based on the first updated reputation weight and / or the at least one second model update contribution in the step 282b.

[0070] Optionally the step 282 comprises a step 286 wherein the FL collecting server 110 saves the at least one second updated reputation weight in the reputation database 130.

[0071] Figure 3 is a flowchart illustrating a method 300 performed by the post-hoc verifier server 150, as shown in Figure 1 and described in the text thereto.

[0072] The post hoc verifier server receives in a step 310, from the collecting server, at least one first assessment weight. In a step 320, the post-hoc verifier server 150 determines at least one first updated reputation weight based on a first performance metric and the at least one first assessment weight.

[0073] By receiving the at least one first assessment weight in step 310, the post-hoc verifier server 150 is enabled to determine the at least one first updated reputation weight based on the first assessment weight. By determining the at least one first updated reputation weight based on the first performance metric and the at least one first assessment weight, the post-hoc verifier server 150 may verify an accuracy and / or reliability of the first model update contribution and may reflect a reliability of the at least one contributing server 120a-d in the first updated reputation weight.

[0074] The receiving step 310 may optionally comprise receiving the at least one reputation weight and / or the first performance metric. Optionally step 310 may comprise receiving the feedback associated with the first performance metric and / or the first model update contribution.

[0075] In an optional step 312 the post-hoc verifier server 150 determines the first performance metric based on a first model update contribution to the global model. Optionally determining 312 may be further based on the feedback associated with the first performance metric. In an optional step 322, the post-hoc verifier server 150 sends to the FL collecting server 110, the at least one first updated reputation weight.

[0076] In an optional step 324, alternative and / or additional to step 322, the post-hoc verifier server 150 sends to a reputation database 130 the at least one updated first reputation weight.

[0077] In an optional step 326, the post-hoc verifier server 150 receives, from the FL collecting server 110, the at least one second assessment weight.

[0078] The receiving step 326 may optionally comprise receiving the at least one first updated reputation weight and / or the second performance metric. Optionally step 326 may comprise receiving the feedback associated with the second performance metric and / or the second model update contribution.

[0079] In an optional step 327 the post-hoc verifier server 150 determines the second performance metric based on a second model update contribution to the global model. Optionally determining 327 may be further based on the feedback associated with the second performance metric.

[0080] In an optional step 328, the post-hoc verifier server 150 determines the at least one second updated reputation weight based on a second performance metric and the at least one second assessment weight.

[0081] In an optional step 330, the post-hoc verifier server 150 sends to the FL collecting server 110, the at least one second updated reputation weight.

[0082] In an optional step 332, the post-hoc verifier server 150 sends additionally and / or alternatively to step 330 the at least one second updated reputation weight.

[0083] Figure 4 is a flowchart illustrating a method 400 performed by the at least one aggregator server 140a-d, as shown in Figure 1 and described in the text thereto.

[0084] In a step 410 the at least one aggregator receives, from the FL collecting server 110 at least one reputation weight of each of at least one contributing server 120a-d of the plurality of contributing servers 120a-d, the at least one first model update contribution to the global model, and the at least one first assessment weight. In a step 420 the at least one aggregator server 140a-d determines a first updated model based on the at least one reputation weight, the at least one first assessment weight, and the at least one first model update contribution.

[0085] In a step 430, the at least one aggregator server 140a-d sends, to the FL collecting server 110 110, the first updated model.

[0086] By receiving the at least one reputation weight and the at least one first assessment weight in step 410 the at least one aggregator server 140a-d may be enabled to take the reputation and the assessment of the contributing server 120a-d into account for determining the at least one first updated model. By determining the first updated model based on the assessment weight and the reputation, the aggregator server 140a-d may be enabled to improve the robustness and decrease the risk of a skewed global model.

[0087] Referring to Figure 4, in an optional step 432 the at least one aggregator server 140a-d receives from the FL collecting server 110 at least one first updated reputation weight, the at least one second model update contribution to the global model, and at least one second assessment weight.

[0088] In an optional step 434, the aggregator server 140a-d determines the second updated model based on the at least one first updated reputation weight, the at least one second assessment weight, and the at least one second model update contribution.

[0089] In an optional step 436, the aggregator server 140a-d sends to the FL collecting server 110, the second updated model.

[0090] Figure 5A and 5B are a signaling diagrams illustrating exemplary embodiments of the method 200, method 300 and method 400, performed by the FL collecting server 110, the at least one aggregator server 140a-d, and the post-hoc verifier server 150, as shown in Figure 2, Figure 3 and Figure 4. Figure 5A and 5B refer to the FL collecting server 110, the at least one contributing server 120a-d, optionally a reputation database 130, optionally the at least one aggregator server 140a-d and optionally the post-hoc verifier server 150 as illustrated in Figure 1 and described in the text relating thereto. The steps of Figure 5B continue where the steps of Figure 5A end.

[0091] Referring to Figure 5A, a step 501 comprises sending, by the at least one contributing server 120a-d to the FL collecting server 110, the at least one first model update contribution to the global model and the at least one first assessment weight. The step 501 further comprises receiving, by the FL collecting server 110 from the at least one contributing server 120a-d, the at least one first model update contribution to the global model and the at least one first assessment weight. The step 501 may correspond to the step 210 of method 200. The at least one first model update contribution may be sent in a message such as a Structured Query Language (SQL)- based message, such as a SQL query transmitted over Transmission Control Protocol / lnternet Protocol (TCP / IP) or Unix sockets, or a NoSQL-based message transmitted over protocols such as Hypertext Transfer Protocol (HTTP), gRPC (Remote Procedure Calls), or Protocol Buffers (Protobuf). The request may be structured according to the Open Systems Interconnection (OS I) model, the International Organization for Standardization (ISO) model, and / or the TCP / IP model. For example, a transmission of these requests may utilize Ethernet at Layer 2, IP at Layer 3, UDP or TCP at Layer 4, and a nested layer encapsulating application-level content, such as SQL at Layer 5 or HTTP at Layer 7. In an example, a UDP communication packet format may include an Ethernet header of 14 bytes, an IP header of 20 bytes, a UDP header of 8 bytes, and a UDP payload of 1472 bytes. The global model may correspond to a ML model which is collaboratively trained by the plurality of contributing server 120a-d.

[0092] The at least one first model update contribution and / or the at least one second model update contribution may correspond to at least one partial model, at least one model parameter value such as updated weights and biases of the model, and / or at least one gradient value e.g. representing direction and magnitude of changes. In an example the at least one first model contribution may comprise metadata information about e.g. local training process such as a number of local training epochs, size of local dataset, local loss metrics and / or accuracy scores.

[0093] The at least one first assessment weight and / or the at least one second assessment weight may correspond to a value, metadata and / or a string indicating the at least one first evaluation and / or the at least one second evaluation of e.g. quality and / or importance evaluated by the contributing server 120a-d of the first model update contribution and / or the second model update contribution. The first and / or second assessment weight may serve to inform the FL collecting server 110 of the contributing servers’ perceived reliability and relevance of the at least one first and / or second model update contribution. The at least one first evaluation and / or the at least one second evaluation may correspond to at least one relevance obtained by the at least one contributing server 120a-d on the at least one first model update contribution and / or the at least one second model update contribution. The at least one first evaluation and / or the at least one second evaluation may have been performed by at least one contributor 120e interacting with the at least one contributing server 120a-d. The at least one contributor 120e may be human, a service or a bot interacting with the at least one contributing server 120a-d e.g. via a User Interface (III).

[0094] An optional step 502a comprises sending, by the FL collecting server 110 to the reputation database 130, the request, for the at least one reputation weight. The reputation database 130 receives the request for the at least one reputation weight. The optional step 502 may correspond to step 212 of method 200. The request may comprise e.g. an identifier of the at least one contributing server 120a-d. In an example, the request may be sent as a message such as one of the messages as described in step 501 .

[0095] An optional step 503a comprises sending, by the reputation database 130 to the FL collecting server 110, the at least one reputation weight. The step 503 comprises further receiving by the FL collecting server 110 from the reputation database 130, the at least one reputation weight. The optional step 503 may correspond to step 214 of method 200. The at least one reputation weight may be sent as a message such as one of the messages as described in step 501 .

[0096] The at least one reputation weight, the at least one updated first reputation weight and / or the at least one updated second reputation weight may correspond to a measure such as value, a string, a reliability metric pertaining to a trustworthiness and / or reliability of the at least one contributing server 120a-d. The at least one reputation weight may correspond to an initial measure such as a trust level assigned to the contribution based on past performance, e.g. providing a baseline for further evaluation.

[0097] Alternatively, to step 502a and 503a, the FL collecting server 110 may acquire the at least one reputation weight in a step 502b.

[0098] An optional step 504a comprises sending, by the FL collecting server 110 to the at least one aggregator server 140a-d the at least one reputation weight and the at least one first model update contribution. Optionally the FL collecting server 110 further sends the at least one first assessment weight to the aggregator server 140a- d. The optional step 504 comprises receiving, by the aggregator server 140a-d from the FL collecting server 110, the at least one reputation weight and the at least one first model update contribution. Optionally, the aggregator server 140a-d, receives further, from the FL collecting server 110, the at least one first assessment weight. The optional step 504a may correspond to step 222a or 222b of method 200 and step 410 of method 400. The at least one reputation weight and the at least one first model update contribution and optionally the first assessment weight may be sent as a message such as one of the messages as described in step 501 .

[0099] An optional step 505a comprises, determining by the at least one aggregator server 140a-d, the first updated model based on the at least one reputation weight, optionally the at least one first assessment weight, and the at least one first model update contribution. The step 505 may correspond to step 420 of method 400. In an example the at least one aggregator server 140a-d may determine the first updated model using a smart aggregation algorithm, at least one weighted average, a Federated Averaging (FedAvg) algorithm, a Federated Optimization (FedProx) algorithm and / or a FedNova algorithm. In case of using a FedAvg algorithm, a group of contributing model updates are randomly selected and aggregated again using a weighted average. After the selection and aggregation process is iterated with successive iterations with other random selections. The FedProx algorithm is a refinement of FedAvg, wherein during the iterations, a corrective parameter called is introduced to improve convergence. The FedNova algorithm introduces normalization and scaling techniques. The FL collecting sever may be assumed as a trusted source of the at least one aggregator server 140a-d. As the first updated model is based on the at least one reputation weight all first model update contributions of the at least one first model update contribution may be taken into account to determine the first updated model. In an example, the first model update contribution may comprise at least one model parameter.

[0100] An optional step 506a comprises sending, by the at least one aggregator server 140a-d to the FL collecting server 110 the first updated model. The optional step 506a comprises receiving, by the FL collecting server 110 from the at least one aggregator server 140a-d the first updated model. The optional step 506 may correspond to step 224 of method 200 and step 430 of method 400. The first updated model may be sent as a message such as one of the messages as described in step 501 .

[0101] The first updated model and / or the second updated model corresponds to an updated global model, at least one updated global model parameter of the global model, and / or at least one updated global gradient of the global model, based on the at least one first model update contribution and / or the at least one second model update contribution such as based on the least one partial model, the at least one model parameter value, and / or the at least one gradient value.

[0102] Alternatively, to step 504a, 505a, and 506a, the FL collecting server 110 may determine the first updated model based on the at least one reputation weight, and the at least one first model update contribution in an optional step 505b. Optionally the FL collecting server 110 determines the first model update based further on the at least one first assessment weight. The optional step 505b may correspond to the step 226a or step 226b of the method 200. In an example, the FL collecting server 110 determines the first updated model using a smart aggregation algorithm, at least one weighted average, a FedAvg algorithm, a FedProx algorithm and / or a FedNova algorithm. Optionally, the first and / or second updated model can be determined, by the FL collecting server 110 and / or the at least one aggregator server 140a-d based on a time-dependent weighting factor that dampens the contribution of the long history weighting, which can reduce the effect of history logs that may no longer be valid. Determining the first and / or the second updated model may enable consolidating diversely sourced data into the global model, wherein the global model may be enabled to be more coherent and representative while quality is maintained. In an example the FL collecting server 110 receive in an optional step 506b at least one first weighted result associated with the first model update from each of the at least one aggregator server 140a-d in response to step 504a. Each of the aggregator server 140a-d may send in the step 506b the first weighted result associated with the first model update. In the example, the FL collecting server 110 may perform step 505b of determining the first updated model based on the at least one first weighted result in response to step 506b.

[0103] The at least one first weighted result and / or at least one second weighted result may correspond to at least one part of the first model update and / or the second model update.

[0104] In an optional step 507a, the FL collecting server 110 sends, to the plurality of contributing server 120a-d, the first updated model. The plurality of contributing server 120a-d may receive the first updated model. The optional step 507a may correspond to step 228 of method 200.

[0105] In an optional step 508a, the FL collecting server 110 receives, from at least one of the plurality of contributing servers 120a-d, the feedback associated with the first performance metric. The at least one of the plurality of contributing server 120a-d may send the feedback associated with the first performance metric. The optional step 508a may correspond to step 230a of method 200.

[0106] Optionally in the optional step 507a and 508a, each contributing server of the plurality of contributing servers may deploy the first updated model using its local data. Each contributing server may run a process such as running the first updated model over its local data, running a local model associated to the first model update contribution over its local data, and comparing result. The comparison may be reflected in the feedback associated with the first performance metric. For example, a Data Dimensionality Reduction field, and / or nominally a forward selection or backward elimination may be used, wherein each contributing server may be requested to repeat running the process with an aggregated models that comprises reduced sets of contributing servers. For example, in the forward selection the contributing servers are added one at a time for each iteration of the process. Alternatively, instead in the backward elimination the contributing servers are eliminated one at a time for each iteration of the process starting from the first model update. The process may enable the FL collecting server, or the verifier server to point out which update model contributions such as the first and / or the second model update contribution have provided better or wors contributions to an overall performances.

[0107] The first performance metric and / or the second performance metric may correspond to a measure e.g. indicting how effective, accurate, speedily, scalable, and / or precise the first updated model and / or the second updated model is to achieve its intended task such as prediction and / or decision. The feedback associated with the first performance metric and / or the second performance metric may correspond to at least a part of the measure e.g. indicting how effective, accurate, speedily, scalable, and / or precise the first updated model and / or the second updated model is to achieve its intended task such as prediction and / or decision. The feedback associated with the first and / or second performance metric may optionally measure by the contributing server 120a-d by applying a selected technique and / or metric to local data of the contributing server.

[0108] Alternatively, and / or additionally to step 507a and / or 508a, the FL collecting server 110 may determine feedback associated with the first performance metric in an optional step 507b. In an example feedback associated with the first performance metric may be determined by deploying the first updated model. Alternatively, and / or additionally and / or additionally, the FL collecting server 110 may receive feedback associated with the first performance metric from a third party entity e.g. deploying the first updated model in an optional step 507c.

[0109] In an optional step 509a, the FL collecting server 110 sends to the post-hoc verifier server 150 optionally the first performance metric and / or feedback associated with the first performance metric, optionally the at least one first model update contribution and the at least one first assessment weight. Optionally the FL collecting server 110 sends further the at least one reputation weight. The post-hoc verifier server 150 receives optionally the first performance metric and / or feedback associated with the first performance metric, optionally the at least one first model update contribution, the at least one first assessment weight and optionally the at least one reputation weight from the FL collecting server 110 in the optional step 509a. The step 509a may correspond to step 242a of method 200 and step 310 of method 300. The first performance metric, optionally the at least one first model update contribution, the at least one first assessment weight and optionally the at least one reputation weight may be sent as a message such as one of the messages as described in step 501 .

[0110] In response to the optional step 509a, the post-hoc verifier server 150 may determine the first performance metric in a step 507d based on the at least one first model update contribution and optionally feedback associated with the first performance metric. The optional step 507d may correspond to step 312 of method 300.

[0111] Determining the first and / or second performance metric, by the FL collecting server 110 and / or the post-hoc verifier server 150 may comprise calculating the first and / or second performance metric using a dimensionality reduction theory such as forward feature construction or backward feature elimination and / or the feedback of the first and / or second performance metric. Based on the calculation the first and / or second performance metric may be calculated offline by progressively eliminating or adding various received models.

[0112] In an optional step 510a, the post-hoc verifier server 150 determines the at least one first updated reputation weight based on the first performance metric, optionally the at least one first model update contribution, the at least one first assessment weight and optionally the at least one reputation weight. The optional step 510a may correspond to step 320 of method 300.

[0113] In an optional step 511 , the post-hoc verifier sends to the FL collecting server 110, the at least one first updated reputation weight. The FL collecting server 110 receives from the post-hoc verifier server 150 in optional step 511 , the at least one first updated reputation weight. The optional step 511 may correspond to step 322 of method 300 and 244a of method 200.

[0114] Optionally, the FL collecting server 110 communicates with at least one third party entity for receiving additional insights related to the reputation weight and may calculate the first updated reputation weight based on the first updated reputation weight received in step 511 and / or determined in step 509b and the additional insights.

[0115] In an optional step 511a additionally and / or alternatively to step 511 , the post-hoc verifier sends to the reputation database 130, the at least one updated first reputation weight. The reputation database 130 receives, from the post-hoc verifier server 150, the at least one updated first reputation weight. The optional step 511a may correspond to step 324 of the method 300.

[0116] Alternatively, to step 509a, 507d, 510, 511 and / or 511 a, the FL collecting server 110 may determine in an optional step 509b the at least one first updated reputation weight based on the first performance metric, the at least one first assessment weight, and optionally the reputation weight and optionally the at least one first model update contribution. The step 509b may correspond to step 242b of method 200.

[0117] In an example, determining the at least one first updated reputation weight and or the at least one second updated reputation weight, by the FL collecting server 110 or by the post-hoc verifier server 150 may comprise assessing whether the first and / or second performance metric aligns with the first and / or second assessment weight. If e.g. during the assessing discrepancies between the first and / or second performance metric and the first and / or second assessment weight are found, the reputation weight may be updated according to the level of the discrepancies e.g. while ensuring that contributions such as the first and / or second model update contributions are evaluated in a transparent and systematic manner. For example, in case the discrepancies are high, the reputation weight may be lowered. Determining the at least one first and / or second updated reputation weight may comprise utilizing metrics such as validation loss, consistency checks and / or grid tests for estimating a level of contribution. Optionally the contributing servers may be involved in determining the at least one first and / or second updated reputation weight in a distributed manner e.g. via in step 507a and 508a.

[0118] In an optional step 512, the FL collecting server 110 saves the at least one first updated reputation weight in the reputation database 130. Saving in step 512 may optionally comprise sending to the reputation database 130 the at least one first updated reputation weight. The reputation database 130 may receive the at least one first updated reputation weight. The optional step 512 may correspond to step 246 of method 200.

[0119] Referring to Figure 5B, in a step 513, the at least one contributing server sends to the FL collecting server 110, the at least one second model update contribution to the global model and the at least one second assessment weight. The step 501 further comprises receiving, by the FL collecting server 110 from the at least one contributing server, the at least one second model update contribution to the global model and the at least one second assessment weight. The step 513 may correspond to the step 250 of method 200. The at least one first model update contribution may be sent as a message such as one of the messages as described in step 501.

[0120] In an optional step 514a, the FL collecting server 110 sends to the reputation database 130, the request, for the at least one first updated reputation weight. The reputation database 130 receives the request for the at least one first updated reputation weight. The optional step 502 may correspond to step 253 of method 200. The request may be sent as a message such as one of the messages as described in step 501.

[0121] In an optional step 515a the reputation database 130 sends to the FL collecting server 110, the at least one first updated reputation weight. The step 515a comprises further receiving by the FL collecting server 110 from the reputation database 130, the at least one first updated reputation weight. The optional step 515a may correspond to step 254 of method 200. The at least one first updated reputation weight may be send using a message as described in step 501 , where the same protocols and layering concepts may be used, but the type and content of the message may differ.

[0122] Alternatively, to step 514a and 515a, the FL collecting server 110 may acquire the at least one first updated reputation weight in a step 514b.

[0123] In an optional step 516a, the FL collecting server 110 sends to the at least one aggregator server 140a-d the at least one first updated reputation weight and the at least one second model update contribution. Optionally the FL collecting server 110 further sends the at least one second assessment weight to the aggregator server 140a-d. The optional step 516a comprises receiving, by the aggregator server 140a- d from the FL collecting server 110, the at least one first updated reputation weight and the at least one second model update contribution. Optionally, the aggregator server 140a-d, receives further, from the FL collecting server 110, the at least one second assessment weight. The optional step 516a may correspond to step 262a or 262b of method 200 and step 432 of method 400. The at least one first updated reputation weight and the at least one second model update contribution and optionally the second assessment weight may be sent as a message such as one of the messages as described in step 501 .

[0124] An optional step 517a comprises, determining by the at least one aggregator server 140a-d, the second updated model based on the at least one first updated reputation weight, optionally the at least one second assessment weight, and the at least one second model update contribution. The step 517 may correspond to step 434 of method 400. In an example the at least one aggregator server 140a-d may determine the second updated model a smart aggregation algorithm, at least one weighted average, a FedAvg algorithm, a FedProx algorithm and / or a FedNova algorithm. As the second updated model is based on the at least one first updated reputation weight all second model update contributions of the at least one second model update contribution may be taken into account to determine the second updated model. In an example, the second model update contribution may comprise at least one model parameter.

[0125] An optional step 518a comprises sending, by the at least one aggregator server 140a-d to the FL collecting server 110 the second updated model. The optional step 518a comprises receiving, by the FL collecting server 110 from the at least one aggregator server 140a-d the second updated model. The optional step 506 may correspond to step 264 of method 200 and step 436 of method 400. The first updated model may be sent as a message such as one of the messages as described in step 501 .

[0126] Alternatively, to step 516a, 517a, and 518a, the FL collecting server 110 may determine the second updated model based on the at least one first updated reputation weight, and the at least one second model update contribution in an optional step 516b. Optionally the FL collecting server 110 determines the second model update based further on the at least one second assessment weight. The optional step 516b may correspond to the step 266a or step 266b of the method 200. In an example, the FL collecting server 110 determines the second updated model using a a smart aggregation algorithm, at least one weighted average, a FedAvg algorithm, a FedProx algorithm and / or a FedNova algorithm.

[0127] In an example the FL collecting server 110 receive in an optional step 517b at least one second weighted result associated with the second model update from each of the at least one aggregator server 140a-d in response to step 516a. Each of the aggregator server 140a-d may send in the step 517b the second weighted result associated with the second model update. In the example, the FL collecting server 110 may perform step 516b of determining the second updated model based on the at least one second weighted result in response to step 517b.

[0128] In an optional step 519a, the FL collecting server 110 sends, to the plurality of contributing servers 120a-d, the second updated model. The plurality of contributing server 120a-d may receive the second updated model. The optional step 519a may correspond to step 268 of method 200.

[0129] In an optional step 520a, the FL collecting server 110 receives, from at least one of the plurality of contributing servers 120a-d, the feedback associated with the second performance metric. The at least one of the plurality of contributing server 120a-d may send the feedback associated with the second performance metric. The optional step 520a may correspond to step 270a of method 200.

[0130] Optionally, in the optional step 519a and 520a, each contributing server of the plurality of contributing servers may deploy the second updated model using its local data as described in step 507a and 508a. Alternatively, and / or additionally to step 519a and / or 520a, the FL collecting server 110 may determine feedback associated with the second performance metric in an optional step 519b. In an example feedback associated with the second performance metric may be determined by deploying the second updated model. Alternatively, and / or additionally and / or additionally, the FL collecting server 110 may receive feedback associated with the second performance metric from a third party entity e.g. deploying the second updated model in an optional step 519c.

[0131] In an optional step 521a, the FL collecting server 110 sends to the post-hoc verifier server 150 optionally the second performance metric and / or feedback associated with the second performance metric, optionally the at least one second model update contribution and the at least one second assessment weight. Optionally the FL collecting server 110 sends further the at least one first updated reputation weight. The post-hoc verifier server 150 receives optionally the second performance metric and / or feedback associated with the second performance metric, optionally the at least one second model update contribution, the at least one second assessment weight and optionally the at least one first updated reputation weight from the FL collecting server 110 in the optional step 521a. The step 521a may correspond to step 282a of method 200 and step 326 of method 300. The second performance metric, optionally the at least one second model update contribution, the at least one second assessment weight and optionally the at least one first updated reputation weight may be sent as a message such as one of the messages as described in step 501.

[0132] In response to the optional step 521a, the post-hoc verifier server 150 may determine the first performance metric in a step 519d based on the at least one first model update contribution and optionally feedback associated with the first performance metric. The optional step 519d may correspond to step 327 of method 300.

[0133] In an optional step 522a, the post-hoc verifier server 150 determines the at least one second updated reputation weight based on the second performance metric, optionally the at least one second model update contribution, the at least one second assessment weight and optionally the at least one first updated reputation weight. The optional step 522a may correspond to step 328 of method 300.

[0134] In an optional step 523, the post-hoc verifier server 150 sends to the FL collecting server 110, the at least one second updated reputation weight. The FL collecting server 110 receives from the post-hoc verifier server 150 in the optional step 523, the at least one second updated reputation weight. The optional step 523 may correspond to step 330 of method 300 and 284a of method 200.

[0135] In an optional step 523a additionally and / or alternatively to step 523, the post-hoc verifier server 150 sends to the reputation database 130, the at least one updated second reputation weight. The reputation database 130 receives, from the post-hoc verifier server 150, the at least one updated second reputation weight. The optional step 523a may correspond to step 332 of the method 300.

[0136] Alternatively, to step 521a, 519d, 522a, 523 and / or 523a, the FL collecting server 110 may determine in an optional step 521 b the at least one second updated reputation weight based on the second performance metric, the at least one second assessment weight, optionally the first updated reputation weight and optionally the at least one second model update contribution. The step 521b may correspond to step 282b of method 200.

[0137] Optionally, the FL collecting server 110 communicates with the at least one third party entity for receiving additional insights related to the reputation weight and may calculate the second updated reputation weight based on the second updated reputation weight received in step 523 and / or determined in step 521 b and the additional insights.

[0138] In an optional step 524, the FL collecting server 110 saves the at least one second updated reputation weight in the reputation database 130. Saving in step 524 may optionally comprise sending to the reputation database 130 the at least one second updated reputation weight. The reputation database 130 may receive the at least one second updated reputation weight. The optional step 524 may correspond to step 286 of method 200.

[0139] The terminology used herein “first” and “second” are referring to a subsequent iteration and current iteration of a variable. By iterating steps 501 to 524 for at least once, the reputation weight of the contributing server 120a-d may be more precisely and dynamically determined based on the contributing servers’ actual performance. Thus, the integrity, quality, and trustworthiness of the global model may be ensured dynamically adjusting contributors' reputations based on their actual performance. In a first embodiment, the FL system 100 comprises the FL collecting server 110 being the centralized FL collecting server 110 corresponding to the one FL collecting server host, the post-hoc verifier server 150, the aggregator server 140a-d being the one aggregator server host 140a, the contributing server 120a-d, and the reputation database 130.

[0140] In the first embodiment, the steps 501 , 502a, 503a are performed.

[0141] In the first embodiment, the step 504a is performed, wherein the FL collecting server 110 sends to the aggregating server, and the aggregating server receives from the FL collecting server 110 the at least one reputation weight, the at least one first assessment weight and the at least one first model update contribution.

[0142] In the first embodiment, the step 505a is performed, wherein the aggregator server calculates the at least one first model update, wherein the at least one first model update contribution comprises at least one first model parameter.

[0143] In an example, the aggregator server may calculate the at least one first model by calculating a first weight of each first model parameter of the at least one first model parameter.

[0144] The weight of each first model parameter may be calculated with:

[0145] . .r.T reputation_weight[il*assessment_weightri] weight_parameter [ij = — - - — — - - -

[0146] Sj iweighL[j] [1]

[0147] Wherein i corresponds to i’ths first model parameter of the at least one first model parameter, j corresponds to the j’th contribution server of the plurality of contributing server 120a-d n, reputation_weight corresponds to the reputation weight, assessment_weight corresponds to the first assessment weight and weight_parameter corresponds to the first weight of each first model parameter.

[0148] At least one first updated model parameter may be calculated with:

[0149] (parameter [j]*weight_parameter [j]) [2] Wherein parameter corresponds to the first model parameter and updated_parameter corresponds to the updated first model parameter.

[0150] The first updated model may comprise the at least one updated first model parameter.

[0151] In the first embodiment, the step 506a is performed, wherein the aggregator server sends the first updated model comprising the at least one updated first model parameter.

[0152] In the first embodiment the steps 507a, 508a, wherein the FL collecting server 110 determines the first performance metric based on the feedback associated with the first performance.

[0153] In the first embodiment the step 509a is performed, wherein the FL collecting server 110 sends to the post-verifier server and the post-verifier server receives from the FL collecting server 110 the first performance metric, the first assessment weight and the at least one first model update contribution.

[0154] In the first embodiment the step 510a is performed, wherein determining the first updated reputation weight comprises comparing the first assessment weight of the first model update contribution to the first performance metric and update the reputation weight to the first updated reputation weight based on the comparison.

[0155] In an example the FL collecting server 110 may send to the post-hoc verifier server 150 a policy comprising a threshold of the difference between the first performance metric and the first assessment weight of the first model update contribution. In that example determining comprises calculating the difference and increase or decrease the reputation weight if the difference falls below, exceeds or is equal to the threshold.

[0156] In the first embodiment, the steps 511 a, 513, 514a and 515a are performed.

[0157] In the first embodiment, the step 516a is performed, wherein the FL collecting server 110 sends to the aggregating server, and the aggregating server receives from the FL collecting server 110 the at least one first updated reputation weight, the at least one second assessment weight and the at least one second model update contribution.

[0158] In the first embodiment, the step 517a is performed, wherein the aggregator server calculates the at least one second model update, wherein the at least one second model update contribution comprises at least one second model parameter.

[0159] In the example, the aggregator server may calculate the at least one second model by calculating a second weight of each second model parameter of the at least one second model parameter.

[0160] The weight of each first model parameter may be calculated with equation [1], wherein i corresponds to i’ths second model parameter of the at least one second model parameter, j corresponds to the j’th contribution server of the plurality of contributing server 120a-d n, reputation_weight corresponds to the first updated reputation weight, assessment_weight corresponds to the second assessment weight and weight_parameter corresponds to the second weight of each second model parameter.

[0161] The at least one second updated model parameter may be calculated with equation [2], wherein parameter corresponds to the second model parameter and updated_parameter corresponds to the updated second model parameter.

[0162] The second updated model may comprise the at least one updated second model parameter.

[0163] In the first embodiment, the step 518a is performed, wherein the aggregator server sends the second updated model comprising the at least one updated second model parameter.

[0164] In the first embodiment the steps 519a, 520a, wherein the FL collecting server 110 determines the second performance metric based on the feedback associated with the second performance.

[0165] In the first embodiment the step 521a is performed, wherein the FL collecting server

[0166] 110 sends to the post-verifier server and the post-verifier server receives from the FL collecting server 110 the second performance metric, the second assessment weight and the at least one second model update contribution.

[0167] In the first embodiment the step 522a is performed, wherein determining the second updated reputation weight comprises comparing the second assessment weight of the second model update contribution to the second performance metric and update the first updated reputation weight to the second updated reputation weight based on the comparison.

[0168] In the example the FL collecting server 110 may send to the post-hoc verifier server 150 a policy comprising a threshold of the difference between the second performance metric and the second assessment weight of the second model update contribution. In that example determining comprises calculating the difference and increase or decrease the first updated reputation weight if the difference falls below, exceeds or is equal to the threshold.

[0169] In the first embodiment, the steps 523a is performed.

[0170] In a second embodiment, the FL system 100 comprises the FL collecting server 110 corresponding to the centralized FL collecting server 110 comprising the reputation database 130. The FL system 100 further comprises the at least one contributing server 120a-d corresponding to at least two contributing servers such as a first contributing server 120a and a second contributing server 120b, and the at least one aggregator server corresponding to at least two aggregator server hosts such as a first aggregator server 140a and a second aggregator server 140b.

[0171] In the second embodiment, the steps, 501a and 502b are performed, wherein the FL collecting server 110 acquires the reputation weight of each contributing servers of the two contributing servers from the reputation database 130 comprised in the FL collecting server 110.

[0172] In the second embodiment, the step 504a is performed wherein the FL collecting server 110 sends to the first aggregator server 140a, the reputation weight of the first contributing server 120a and the first model update contribution of the first contributing server 120a. The FL collecting server 110 sends to the second aggregator server 140b the reputation weight of the second contributing server 120b and the first model update contribution of the second contributing server 120b.

[0173] In the second embodiment, the step 505a is performed by each of the two aggregator servers using at least one smart aggregation algorithm, wherein the two aggregator servers each determine a part of the first updated model.

[0174] In the second embodiment, the step 506a is performed, wherein the FL collecting server 110 receives from each of the two aggregator servers the part of the first updated model.

[0175] In the second embodiment, the steps 507c, 509b and 512 are performed.

[0176] In the second embodiment, the steps 513 and 514b are performed, wherein the FL collecting server 110 acquires the first updated reputation weight of each contributing servers of the two contributing servers from the reputation database 130 comprised in the FL collecting server 110.

[0177] In the second embodiment, the step 516a is performed wherein the FL collecting server 110 sends to the first aggregator server 140a, the first updated reputation weight of the first contributing server 120a and the second model update contribution of the first contributing server 120a. The FL collecting server 110 sends to the second aggregator server 140b the first updated reputation weight of the second contributing server 120b and the second model update contribution of the second contributing server 120b.

[0178] In the second embodiment, the step 517a is performed by each of the two aggregator servers using at least one smart aggregation algorithm, wherein the two aggregator servers each determine a part of the second updated model.

[0179] In the second embodiment, the step 518a is performed, wherein the FL collecting server 110 receives from each of the two aggregator servers the part of the second updated model.

[0180] In the second embodiment, the steps 519c, 521 b and 524 are performed. In a third embodiment, the FL system 100 comprises the FL collecting server 110 being the decentralized collecting server corresponding to at least one of the at least one contributing server 120a-d. The FL system 100 further comprises the reputation database 130 and the post-hoc verifier server 150.

[0181] In the third embodiment, the steps 501 , 502a, and 503a are performed.

[0182] In the third embodiment, the step 505b is performed, wherein the FL collecting server 110 determines the first updated model based on the at least one reputation weight and the at least one first model update contribution using the at least one smart aggregation algorithm.

[0183] In the third embodiment, the step 507c is performed, wherein the FL collecting server 110 receives feedback associated with the first performance metric from the third- party entity. In an example the FL collecting server 110 sends to the third-party entity the first updated model and receives periodically feedback associated with the first performance metric.

[0184] In the third embodiment, the step 509a is performed, wherein the FL collecting server 110 sends to the post-hoc verifier server 150, and the post-hoc verifier server 150 receives from the FL collecting server 110, the at least one first assessment weight, the at least one first model update contribution and feedback associated with the first performance metric. In the example, step 509a is performed periodically in response to step 507c.

[0185] In the third embodiment, the step 507d is performed, wherein the post-hoc verifier server 150 determines the first performance metric after a predefined time threshold based on the periodically received feedback associated with the first performance metric. The post-hoc verifier server 150 determines the first performance metric using the dimensionality reduction theory.

[0186] In the third embodiment, the steps 51 Oa, 511 , 512, 513, 514a, and 515a are performed.

[0187] In the third embodiment, the step 516b is performed, wherein the FL collecting server 110 determines the second updated model based on the at least one first updated reputation weight and the at least one second model update contribution using the at least one smart aggregation algorithm.

[0188] In the third embodiment, the step 519c is performed, wherein the FL collecting server 110 receives feedback associated with the second performance metric from the third-party entity. In an example the FL collecting server 110 sends to the third-party entity the second updated model and receives periodically feedback associated with the second performance metric.

[0189] In the third embodiment, the step 521a is performed, wherein the FL collecting server 110 sends to the post-hoc verifier server 150, and the post-hoc verifier server 150 receives from the FL collecting server 110, the at least one second assessment weight, the at least one second model update contribution and feedback associated with the second performance metric. In the example, step 521a is performed periodically in response to step 519c.

[0190] In the third embodiment, the step 519d is performed, wherein the post-hoc verifier server 150 determines the second performance metric after a predefined time threshold based on the periodically received feedback associated with the second performance metric. The post-hoc verifier server 150 determines the second performance metric using the dimensionality reduction theory.

[0191] In the third embodiment, the steps 522a, 523 and 524 are performed.

[0192] In an example of deployment of an embodiment, a web interface is deployed using Hypertext Markup Language (HTML) and Cascading Style Sheets (CSS) allowing for enhanced flexibility and compatibility. The web interface enables the implementation of a customizable number of contributing servers, which are associated with independent parallel threads. The web interface allows for configuring various parameters for each contributing server 120a-d, including the reputation weight, a file containing a first / and or second model update in a dataset, and the first and / or second assessment weight. In the example, the FL system 100 implements, as micro services, various components that demonstrate outcomes of a machine learning process of the global model. The FL system 100 may facilitate an examination of quality of contributions such as the first and / or second model update contribution made by each contributing server 120a-d alongside with the first and / or second assessment weight. The examination may be used to identify any misalignments between self-evaluated contributions and their actual impact on the global model’s performance. If necessary, the FL system 100 can implement corrective measures to adjust contributors' reputations based on these evaluations. By implementing a dynamic feedback mechanism according to method 200, 300 and 400 accountability among contributors may promoted, encouraging them to provide high-quality contributions while ensuring that the global model reflects the most reliable and relevant data. The example may illustrate how integrating selfassessments and reputation mechanisms may enhance the FL system 100, which may lead to improved model accuracy and responsiveness to contributor performance. The example may demonstrate enabling creating a robust and adaptable framework for managing contributions in the FL system 100.

[0193] Figure 6 illustrates a block diagram illustrating embodiments of the FL collecting server 110 in further detail. In practice, the steps 210 to 280 of the method 200 performed by the FL collecting server 110 are performed by processing circuitry 604, embodied in one or more processors and / or microprocessors arranged to execute a computer program 601 that is downloaded to a computer program product 605, here in the form of a suitable computer readable storage medium 602 associated with the microprocessor. The computer readable storage medium 602 may be a memory, such as a random access memory (RAM) or a read-only memory (ROM), or a tangible non-volatile computer readable storage medium, such as flash memory or a hard disk drive, or any combination thereof. The computer program 601 comprises computer-executable instructions stored or downloaded to the computer readable storage medium 602 and are executable by the processing circuitry 604.

[0194] Alternatively, the computer program 601 may be transferred to the computer readable storage medium 602 using a suitable computer program product, such as a memory stick or in a memory of a device. Thus, the computer program 601 may be stored in any suitable manner in the computer program product. The processing circuity 604 is arranged to cause the FL collecting server 110 to carry out the steps 210 to 280 of method 200 in accordance with any of the of the described embodiments for steps 210 to 280. The processing circuitry 604 is in one embodiment one or more general-purpose processors wherein each one of the general purpose processors includes one or more cores, but may alternatively be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), etc. An I / O interface 603 is provided for communicating with external and / or internal entities using wired communications, e.g., based on Ethernet, and / or wireless communications, e.g., Wi-Fi, and / or a cellular network corresponding to one or a combination of 5G cellular networks, LTE, LTE-advanced, UMTS, or any other current or future wireless network, such as a future 3GPP 6G network, as long as the principles described below are applicable.

[0195] Figure 7 illustrates a block diagram illustrating embodiments of the post-hoc verifier server 150 in further detail. In practice, the steps 310 to 332 of the method 300 performed by the post-hoc verifier server 150 are performed by processing circuitry

[0196] 704, embodied in one or more processors and / or microprocessors arranged to execute a computer program 701 that is downloaded to a computer program product

[0197] 705, here in the form of a suitable computer readable storage medium 702 associated with the microprocessor. The computer readable storage medium 702 may be a memory, such as a random access memory (RAM) or a read-only memory (ROM), or a tangible non-volatile computer readable storage medium, such as flash memory or a hard disk drive, or any combination thereof. The computer program 701 comprises computer-executable instructions stored or downloaded to the computer readable storage medium 702 and are executable by the processing circuitry 704. Alternatively, the computer program 701 may be transferred to the computer readable storage medium 702 using a suitable computer program product, such as a memory stick or in a memory of a device. Thus, the computer program 701 may be stored in any suitable manner in the computer program product. The processing circuity 704 is arranged to cause the post-hoc verifier server 150 to carry out the steps 310 to 332 of method 200 in accordance with any of the of the described embodiments for steps 310 to 332. The processing circuitry 704 is in one embodiment one or more general-purpose processors wherein each one of the general purpose processors includes one or more cores, but may alternatively be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), etc. An I / O interface 703 is provided for communicating with external and / or internal entities using wired communications, e.g., based on Ethernet, and / or wireless communications, e.g., Wi-Fi, and / or a cellular network corresponding to one or a combination of 5G cellular networks, LTE, LTE-advanced, UMTS, or any other current or future wireless network, such as a future 3GPP 6G network, as long as the principles described below are applicable.

[0198] Figure 8 illustrates a block diagram illustrating embodiments of the at least one aggregator server 140a-d in further detail. In practice, the steps 410 to 436 of the method 400 performed by he at least one aggregator server 140a-d are performed by processing circuitry 804, embodied in one or more processors and / or microprocessors arranged to execute a computer program 801 that is downloaded to a computer program product 805, here in the form of a suitable computer readable storage medium 802 associated with the microprocessor. The computer readable storage medium 802 may be a memory, such as a random access memory (RAM) or a read-only memory (ROM), or a tangible non-volatile computer readable storage medium, such as flash memory or a hard disk drive, or any combination thereof. The computer program 801 comprises computer-executable instructions stored or downloaded to the computer readable storage medium 802 and are executable by the processing circuitry 804. Alternatively, the computer program 801 may be transferred to the computer readable storage medium 802 using a suitable computer program product, such as a memory stick or in a memory of a device. Thus, the computer program 801 may be stored in any suitable manner in the computer program product. The processing circuity 804 is arranged to cause the at least one aggregator server 140a-d to carry out the steps 410 to 436 of method 200 in accordance with any of the of the described embodiments for steps 410 to 436. The processing circuitry 804 is in one embodiment one or more general-purpose processors wherein each one of the general purpose processors includes one or more cores, but may alternatively be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), etc. An I / O interface 803 is provided for communicating with external and / or internal entities using wired communications, e.g., based on Ethernet, and / or wireless communications, e.g., Wi-Fi, and / or a cellular network corresponding to one or a combination of 5G cellular networks, LTE, LTE-advanced, UMTS, or any other current or future wireless network, such as a future 3GPP 6G network, as long as the principles described below are applicable.

Claims

CLAIMS1 . A method (200) for performing Federated Learning, FL, in an FL system (100) comprising an FL collecting server (110) and a plurality of contributing server (120a-d), wherein the plurality of contributing servers (120a-d) is contributing to a global model, wherein the method is performed by the FL collecting server (110), the method comprising: receiving (210), from at least one contributing server of the plurality of contributing servers (120a-d), at least one first model update contribution to the global model, and at least one first assessment weight pertaining to at least one first evaluation of the at least one first model update contribution; obtaining (220) a first updated model of the global model based on the at least one first model update contribution and at least one reputation weight of each of the at least one contributing server (120a-d), wherein the at least one reputation weight is pertaining to a reliability metric of each of the at least one contributing server (120a-d); obtaining (230) a first performance metric of the first updated model; updating (240) the at least one reputation weight to at least one first updated reputation weight based on the first performance metric, the at least one first model update contribution and the at least one first assessment weight; receiving (250), from the at least one contributing server (120a-d), at least one second model update contribution to the global model, and at least one second assessment weight pertaining to at least one second evaluation of the at least one second model update contribution; andobtaining (260) a second updated model of the global model based on the at least one second model update contribution and the at least one updated first reputation weight.

2. The method (200) according to claim 1 , wherein the method (200) comprises: obtaining (270) a second performance metric of the second updated model; and updating (280) the at least one first updated reputation weight to at least one second updated reputation weight based on the second performance metric, the at least one second model update contribution and the at least one second assessment weight.

3. The method (200) according to any of the previous claims, wherein obtaining (220) the first updated model is further based on the at least one first assessment weight, and wherein obtaining (260) the second updated model is further based on the at least one second assessment weight.

4. The method (200) according to any of the previous claims, wherein the at least one first evaluation and the at least one second evaluation have been performed by at least one contributor (120e) interacting with the at least one contributing server (120a-d).

5. The method (200) according to any of the previous claims, wherein updating (240) the at least one reputation weight comprises: obtaining (242) the at least one first updated reputation weight based on the first performance metric, the at least one first model update contribution and the at least one first assessment weight;saving (246) the at least one first updated reputation weight in a reputation database (130).

6. The method (200) according to claim 5, wherein obtaining (242) the at least one first updated reputation weight comprises sending (242a), to a post-hoc verifier server (150), the at least one first assessment weight; and receiving (244a), from the post-hoc verifier server (150), the at least one first updated reputation weight.

7. The method (200) according to claim 5, wherein obtaining (242) the at least one first updated reputation weight comprises: determining (242b) the at least one first updated reputation weight based on the first performance metric, and the at least one first assessment weight.

8. The method (200) according to any of claims 2 to 7, wherein updating (280) the at least one first updated reputation weight comprises: obtaining (282) the at least one second updated reputation weight based on the second performance metric, the at least one second model update contribution and the at least one second assessment weight; saving (286) the at least one second updated reputation weight in a reputation database (130).

9. The method (200) according to claim 8, wherein obtaining (282) the at least one second updated reputation weight comprisessending (282a), to a post-hoc verifier server (150), the at least one second assessment weight; and receiving (284a), from the post-hoc verifier server (150), the at least one second updated reputation weight.

10. The method (200) according to claim 8, wherein obtaining (282) the at least one second updated reputation weight comprises: determining (282b) the at least one second updated reputation weight based on the second performance metric, the at least one second model update contribution and the at least one second assessment weight.11 .The method (200) according to any of the previous claims, wherein the method comprises: obtaining (212) the at least one reputation weight of each of the at least one contributing server (120a-d).

12. The method (200) according to claim 11 , wherein obtaining (212) the at least one reputation weight comprises: receiving (214), from a reputation database (130), the at least one reputation weight.

13. The method (200) according to any of the previous claims, wherein obtaining (220) the first updated model comprises: sending (222a), to at least one aggregator server (140a-d), the at least one reputation weight, the at least one first assessment weight, and the at least one first model update contribution;receive (224a), from the at least one aggregator server (140a-d), the first updated model.

14. The method (200) according to claims 1 to 12, wherein obtaining (220) the first updated model comprises: determining (226a), the first updated model based on the at least one reputation weight, the at least one first assessment weight, and the at least one first model update contribution.

15. The method (200) according to any of the previous claims, wherein obtaining (260) the second updated model comprises: sending (262a), to at least one aggregator server (140a-d), the at least one first updated reputation weight, the at least one second assessment weight, and the at least one second model update contribution; receive (264), from the at least one aggregator server (140a-d), the at least one second updated model.

16. The method (200) according to claims 1 to 14, wherein obtaining (260) the second updated model comprises: determining (266a), the second updated model based on the at least one first updated reputation weight, the at least one second assessment weight, and the at least one second model update contribution.

17. The method (200) according to any of the previous claims, wherein the method (200) comprises: sending (228), to the plurality of contributing servers (120a-d), the first updated model; andsending (268), to the plurality of contributing servers (120a-d), the second updated model.

18. The method (200) according to any of the previous claims, wherein obtaining (230) the first performance metric comprises: receiving (230a), from at least one of the plurality of contributing servers (120a-d), feedback associated with the first performance metric.

19. The method (200) according to any of claims 2 to 18, wherein obtaining (270) the second performance metric comprises: receiving (270a), from at least one of the plurality of contributing servers (120a-d), feedback associated with the second performance metric.

20. A method (300) for performing Federated Learning, FL, in an FL system (100) comprising an FL collecting server (110), a post-hoc verifier server (150), and a plurality of contributing servers (120a-d), wherein the plurality of contributing servers (120a-d) is contributing to a global model, wherein the method is performed by the post-hoc verifier server (150), the method comprising: receiving (310), from the FL collecting server (110), at least one first assessment weight; and determining (320) at least one first updated reputation weight based on a first performance metric of a first updated model and the at least one first assessment weight.21 .The method (300) according to claim 20, wherein the method (300) comprises:sending (322), to the FL collecting server (110), the at least one first updated reputation weight; and / or sending (324), to a reputation database (130), the at least one updated first reputation weight.

22. The method (300) according to any of claims 20 to 21 , wherein the method (300) comprises: determining (312) the first performance metric based on a first model update contribution to the global model, wherein receiving (310) comprises receiving the first model update contribution.

23. The method (300) according to any of claims 20 to 22, wherein receiving (310) the at least one first assessment weight comprises receiving: at least one reputation weight of each of at least one contributing server of the plurality of contributing server (120a-d), wherein the at least one reputation weight is pertaining to a reliability metric of each of the at least one contributing server (120a-d), and / or a first performance metric of a first updated model of the global model, and wherein the at least one reputation weight is pertaining to a reliability metric of each of the at least one contributing server (120a-d), and wherein determining (320) the at least one first updated reputation weight comprises determining the at least one first updated reputation weight based on the at least one reputation weight.

24. The method (300) according to any of the claims 20 to 23, wherein the method comprises:receiving (326), from the FL collecting server (110), at least one second assessment weight pertaining to at least one second evaluation of the at least one second model update contribution; determining (328) the at least one second updated reputation weight based on a second performance metric of a second updated model, and the at least one second assessment weight.

25. The method (300) according to claim 24, wherein the method (300) comprises: determining (327) the second performance metric based on a second model update contribution, wherein receiving (326) comprises receiving the second model update contribution.

26. The method (300) according to any of claims 24 to 25, wherein the method (300) comprises: sending (330), to the FL collecting server (110), the at least one second updated reputation weight; and / or sending (332), to a reputation database (130), the at least one second updated reputation weight.

27. The method (300) according to any of claims 24 to 26, wherein receiving (326) the at least one second assessment weight comprises receiving at least one first updated reputation weight of each of at least one contributing server of the plurality of contributing server (120a-d), and / or a second performance metric of a second updated model, andwherein determining (328) the at least one second updated reputation weight comprises determining the at least one second updated reputation weight based on the at least one first updated reputation weight.

28. A method (400) for performing Federated Learning, FL, in an FL system (100) comprising an FL collecting server (110), at least one aggregator server (140a-d) and a plurality of contributing servers (120a-d), wherein the plurality of contributing servers (120a-d) is contributing to a global model, wherein the method is performed by the at least one aggregator server (140a-d), the method comprising: receiving (410), from the FL collecting server (110), at least one reputation weight of each of at least one contributing server of the plurality of contributing servers (120a-d), wherein the at least one reputation weight is pertaining to a reliability metric of each of the at least one contributing server (120a-d); at least one first model update contribution to the global model, and at least one first assessment weight pertaining to at least one first evaluation of the at least one first model update contribution, determining (420), a first updated model based on the at least one reputation weight, the at least one first assessment weight, and the at least one first model update contribution; sending (430), to the FL collecting server (110), the first updated model.

29. The method (400) according to claim 28, wherein the method (400) comprises: receiving (432), from the FL collecting server (110), at least one first updated reputation weight,at least one second model update contribution to the global model, at least one second assessment weight pertaining to at least one second evaluation of the at least one second model update contribution; determining (434), a second updated model based on the at least one first updated reputation weight, the at least one second assessment weight, and the at least one second model update contribution. sending (436), to the FL collecting server (110), the second updated model.

30. A Federated Learning, FL, system (100) comprising: an FL collecting server (110) being configured to perform the method according to any one of claims 1 to 19; a post-hoc verifier server (150) being configured to perform the method according to any one of claims 20 to 27; and at least one aggregator server (140a-d) being configured to perform the method according to any one of claims 28 to 29.

31. A Federated Learning, FL, collecting server, (110) wherein an FL system (100) comprises the FL collecting server (110) and a plurality of contributing servers (120a-d), wherein the plurality of contributing servers (120a-d) is contributing to a global model, whereby the FL collecting server (110) is configured to: receive, from at least one contributing server (120a-d) of the plurality of contributing servers (120a-d),at least one first model update contribution to the global model, and at least one first assessment weight pertaining to at least one first evaluation of the at least one first model update contribution; obtain a first updated model of the global model based on the at least one first model update contribution and at least one reputation weight of each of the at least one contributing server (120a-d), wherein the at least one reputation weight is pertaining to a reliability metric of each of the at least one contributing server (120a-d); obtain a first performance metric of the first updated model; update the at least one reputation weight to at least one first updated reputation weight based on the first performance metric, the at least one first model update contribution and the at least one first assessment weight; receive, from the at least one contributing server (120a-d), at least one second model update contribution to the global model, and at least one second assessment weight pertaining to at least one second evaluation of the at least one second model update contribution; and obtain a second updated model of the global model based on the at least one second model update contribution and the at least one updated first reputation weight.

32. The FL collecting server (110) according to claim 31 , configured to perform the method according to any of claims 2 to 19.

33. A Federated Learning, FL, collecting server (110), wherein an FL system (100) comprises the FL collecting server (110) and a plurality of contributing servers (120a-d), wherein the plurality of contributing servers (120a-d) is contributing to a global model, wherein the FL collecting server (110) comprises processing circuitry (604) and a computer readable storage medium (602), the computer readable storage medium (602) containing instructions executable by the processing circuitry (604), whereby the FL collecting server (110) is configured to: receive, from at least one contributing server (120a-d) of the plurality of contributing servers (120a-d), at least one first model update contribution to the global model, and at least one first assessment weight pertaining to at least one first evaluation of the at least one first model update contribution; obtain a first updated model of the global model based on the at least one first model update contribution and at least one reputation weight of each of the at least one contributing server (120a-d), wherein the at least one reputation weight is pertaining to a reliability metric of each of the at least one contributing server (120a-d); obtain a first performance metric of the first updated model; update the at least one reputation weight to at least one first updated reputation weight based on the first performance metric, the at least one first model update contribution and the at least one first assessment weight; receive, from the at least one contributing server (120a-d), at least one second model update contribution to the global model, andat least one second assessment weight pertaining to at least one second evaluation of the at least one second model update contribution; and obtain a second updated model of the global model based on the at least one second model update contribution and the at least one updated first reputation weight.

34. The FL collecting server (110) according to claim 33, configured to perform the method according to any of claims 2 to 19.

35. A post-hoc verifier server (150), wherein an FL system (100) comprises an FL collecting server (110), the post-hoc verifier server (150), and a plurality of contributing servers (120a-d), wherein the plurality of contributing servers (120a-d) is contributing to a global model, whereby the post-hoc verifier server (150) is configured to: receive, from the FL collecting server (110), at least one first assessment weight; and determine at least one first updated reputation weight based on a first performance metric of a first updated model and the at least one first assessment weight.

36. The post-hoc verifier server (150) according to claim 35 configured to perform the method according to any of claims 21 to 27.

37. A post-hoc verifier server (150), wherein an FL system (100) comprises an FL collecting server (110), the post-hoc verifier server (150), and a plurality of contributing servers (120a-d), wherein the plurality of contributing servers (120a-d) is contributing to a global model, wherein the post-hoc verifier server(150) comprises processing circuitry (704) and a computer readable storage medium (602), the computer readable storage medium (702) containing instructions executable by the processing circuitry (704), whereby the post-hoc verifier server (150) is configured to: receive, from the FL collecting server (110), at least one first assessment weight; and determine at least one first updated reputation weight based on a first performance metric of a first updated model and the at least one first assessment weight.

38. The post-hoc verifier server (150) according to claim 37 configured to perform the method according to any of claims 21 to 27.

39. At least one aggregator server (140a-d), wherein an FL system (100) comprises an FL collecting server (110), the at least one aggregator server (140a-d) and a plurality of contributing servers (120a-d), wherein the plurality of contributing servers (120a-d) is contributing to a global model, whereby the at least one aggregator server (140a-d) is configured to: receive, from the FL collecting server (110), at least one reputation weight of each of at least one contributing server (120a-d) of the plurality of contributing servers (120a-d), wherein the at least one reputation weight is pertaining to a reliability metric of each of the at least one contributing server (120a-d); at least one first model update contribution to the global model, and at least one first assessment weight pertaining to at least one first evaluation of the at least one first model update contribution,determine, a first updated model based on the at least one reputation weight, the at least one first assessment weight, and the at least one first model update contribution; send, to the FL collecting server (110), the first updated model.

40. The at least one aggregator server (140a-d), according to claim 39 configured to perform the method according to claim 40.41 . At least one aggregator server (140a-d), wherein an FL system (100) comprises an FL collecting server (110), the at least one aggregator server (140a-d) and a plurality of contributing servers (120a-d), wherein the plurality of contributing servers (120a-d) is contributing to a global model, wherein the at least one aggregator server (140a-d) comprises processing circuitry (804) and a computer readable storage medium (802), the computer readable storage medium (802) containing instructions executable by the processing circuitry (804), whereby the at least one aggregator server (140a-d) is configured to: receive, from the FL collecting server (110), at least one reputation weight of each of at least one contributing server (120a-d) of the plurality of contributing servers (120a-d), wherein the at least one reputation weight is pertaining to a reliability metric of each of the at least one contributing server (120a-d); at least one first model update contribution to the global model, and at least one first assessment weight pertaining to at least one first evaluation of the at least one first model update contribution, determine, a first updated model based on the at least one reputation weight, the at least one first assessment weight, and the at least one first model update contribution;send, to the FL collecting server (110), the first updated model.

42. The at least one aggregator server (140a-d), according to claim 41 configured to perform the method according to claim 40.

43. A computer program (601 , 701 , 801 ) comprising instructions which, when executed on processing circuitry (604) of a Federated Learning, FL, collecting server (110), cause the processing circuitry (604) of the FL, collecting server (110) to carry out the method according to any one of claims 1 to 19; processing circuitry (704) of a post-hoc verifier server (150), cause the processing circuitry (704) of the a post-hoc verifier server (150) to carry out the method according to any one of claims 20 to 27; and processing circuitry (804) of at least one aggregator server (140a-d), cause the processing circuitry (804) of the at least one aggregator server (140a- d) to carry out the method according to any one of claims 28 to 29.

44. A tangible, non-volatile computer readable medium (602, 602, 702) comprising instructions that, when executed on processing circuitry (604) of a Federated Learning, FL, collecting server (110), cause the processing circuitry (604) of the FL, collecting server (110) to carry out the method according to any one of claims 1 to 19; processing circuitry (704) of a post-hoc verifier server (150), cause the processing circuitry (704) of the a post-hoc verifier server (150) to carry out the method according to any one of claims 20 to 27; andprocessing circuitry (804) of at least one aggregator server (140a-d), cause the processing circuitry (804) of the at least one aggregator server (140a- d) to carry out the method according to any one of claims 28 to 29.

Images