Technical field

Abstract

Method comprising: obtaining a first result of inference based on test data and a first local FL model received from a FL client; obtaining a second result of inference based on the test data and a reference FL model; checking whether a deviation between the first result of inference and the second result of inference is larger than a predefined threshold; and, in response to checking that the deviation between the first result of inference and the second result of inference is larger than the threshold, at least one of: - removing the FL client from the FL training process, or - decreasing a trust score of the FL client; wherein the first local FL model and the reference FL model are obtained in a FL training process; the reference FL model is a second local FL model or a global FL model obtained from aggregating local FL models in the FL training process.

Description

[0001]Technical Field The present disclosure relates to federated learning. Abbreviations 3GPP 3rdGeneration Partnership Project 5G / 6G / 7G 5th / 6th / 7thGeneration 5GC 5thGeneration Core Network ADRF Analytics Data Repository Function AI Artificial Intelligence AI / ML Artificial Intelligence / Machine Learning AnLF Analytics Logical Function FL Federated Learning gNB Next Generation (5G) Node B ID Identifier MDAF Management Data Analytics Function ML Machine Learning MTLF Model Training Logical Function NF Network Function NR New Radio NRF Network Repository Function NWDAF Network Data Analytics Function RAN Radio Access Network RIC RAN Intelligent Controller UDM Unified Data Management UE User Equipment Glossary (taken from 3GPP R1-2205695 and 3GPP R1-2300603) Terminology Description A process of collecting data by the network nodes, management entity, or UE for the purpose of AI / ML model training, data analytics and inference A data driven algorithm that applies AI / ML techniques to generate a set of outputs based on AI / ML model A process of using a trained AI / ML model to Inference produce a set of outputs based on a set of inputs AI / ML model A subprocess of training, to evaluate the quality of AI / ML model testing A subprocess of training, to evaluate the performance of a final AI / ML model using a dataset different from one used for model training and validation. Differently from AI / ML model validation, testing does not assume subsequent tuning of the model. An AI / ML Model whose inference is performed entirely at the UE An AI / ML Model whose inference is performed entirely at the network A UE-side (AI / ML) model or a Network-side (AI / ML) model Model monitoring A procedure that monitors the inference performance of the AI / ML model Functionality A process / method of identifying an AI / ML identification functionality for the common understanding between the NW and the UE Note: Information regarding the AI / ML functionality may be shared during functionality identification. Federated learning / A machine learning technique that trains an AI / ML federated training model across multiple decentralized edge nodes (e.g., UEs, gNBs) each performing local model training using local data samples. The technique requires multiple interactions of the model, but no exchange of local data samples. Background Many applications in mobile networks require a large amount of data from multiple distributed sources like UEs or distributed gNBs to be used to train a single common model. To minimize the data exchange between the distributed units from where the data is generated and the centralized unit(s) where the common model is created, the concept of Federated learning (FL) may be applied. FL is a form of machine learning where, instead of model training at a single node, different versions of the model are trained at the different distributed hosts (FL clients). This is different from distributed machine learning, where a single ML model is trained at distributed nodes to use computation power of different nodes. In other words, FL is different from distributed learning in the sense that: 1) each distributed node in a FL scenario has its own local training data which may not come from the same distribution as the local training data at other nodes; 2) each node computes parameters for its local ML model and 3) the central host (FL server) does not compute a version or part of the model but combines parameters of all the distributed models to generate a main model (aggregation of the local models to a global model). An objective of this approach is to keep the training dataset where it is generated and perform the model training locally at each individual learner in the federation. After training a local model, each individual learner transfers its local model parameters, instead of the (raw) training dataset, to an aggregating unit. The aggregating unit utilizes the local model parameters to update a global model which may eventually be fed back to the local learners for further iterations until the global model converges. As a result, each local learner benefits from the datasets of the other local learners only through the global model, shared by the aggregator, without explicitly accessing high volume of (potentially privacy- sensitive) data available at each of the other local learners. Summarizing, FL training process can be explained by the following main steps: ^ Initialization: A machine learning model (e.g., linear regression, neural network) is chosen to be trained on local nodes and initialized. ^ Client selection: a fraction of local nodes may be selected to start training on local data. The selected nodes acquire the current statistical model while the others wait for the next federated round. ^ Reporting and Aggregation: each selected node sends its local model to the central function (FL server) for aggregation. The central function aggregates the received models and sends back the model updates to the nodes. ^ Termination: once a pre-defined termination criterion is met (e.g., a maximum number of iterations is reached), the central function aggregates the updates and finalizes the global model. Some example deployment options for FL server and FL client in 3GPP networks are shown in Table 1. FL Server (or aggregator) FL Client Option 1 Cross-domain management domain Individual management domain(s) (cross-domain MDAF or cross- domain MLEntity) Option 2 RAN management domain (RAN gNodeB(s) MDAF or MLEntity) Option 3 Core management domain (Core NWDAF(s) MDAF or MLEntity) Option 4 Central NWDAF Distributed NWDAF(s) Option 5 NWDAF UE(s) Option 6 gNodeB UE(s) Option 7 Non-real-time RIC (rApp) Real-time RIC(s) (xApp(s)) Option 8 Real-time RIC (xApp) gNodeB(s) Table 1: Example FL deployment options in 3GPP networks Summary It is an object to improve the prior art. According to a first aspect, there is provided an apparatus comprising: one or more processors, and memory storing instructions that, when executed by the one or more processors, cause the apparatus to perform: obtaining a first result of inference based on test data and a first local federated learning model; obtaining a second result of inference based on the test data and a reference federated learning model different from the first local federated learning model; checking whether a first deviation between the first result of inference and the second result of inference is larger than a predefined first deviation threshold; and, in response to checking that the first deviation between the first result of inference and the second result of inference is larger than the predefined first deviation threshold, at least one of: - removing a first federated learning client from the federated learning training process, or - decreasing a trust score of the first federated learning client; wherein the first local federated learning model and the reference federated learning model are obtained in a federated learning training process; the reference federated learning model is a second local federated learning model or a global federated learning model obtained from aggregating local federated learning models in the federated learning training process; the local federated learning models include the first local federated learning model and the second local federated learning model; for the aggregating, each of the local federated learning models is received from a respective one of plural federated learning clients involved in the federated learning training process; and the first local federated learning model is received from the first federated learning client. The instructions, when executed by the one or more processors, may further cause the apparatus to perform the obtaining the first result of inference by computing the first result of inference using the test data in the first local federated learning model received from the first federated learning client. The instructions, when executed by the one or more processors, may further cause the apparatus to perform the obtaining the first result of inference by sending the test data to the first federated learning client; instructing the first federated learning client to compute inference using the test data in the first local federated learning model; and receiving the first result of inference from the first federated learning client in response to the instructing the first federated learning client. The instructions, when executed by the one or more processors, may further cause the apparatus to perform checking whether a second deviation between the computed first result of inference and the received first result of the inference is larger than a predefined second deviation threshold; and, in response to checking that the second deviation is larger than the predefined second deviation threshold, at least one of: - removing the first federated learning client from the federated learning training process; or - decreasing the trust score of the first federated learning client. The instructions, when executed by the one or more processors, may further cause the apparatus to perform selecting the plural federated learning clients involved in the federated learning training process based on the trust scores of plural potential federated learning clients such that a potential federated learning client of the potential federated learning clients with a higher trust score is selected for being involved in the federated learning training process with a higher probability than a potential federated learning client of the potential federated learning clients with a lower trust score. The instructions, when executed by the one or more processors, may further cause the apparatus to perform checking whether the first local federated learning model causes a first metric to define a quality of the global federated learning model being less than a respective predefined quality threshold; and inhibiting the checking whether the first deviation is larger than the predefined first deviation threshold in response to checking that the first local federated learning model causes the first metric to define the quality of the global federated learning model being less than the respective predefined quality threshold. The first result of inference may be a second metric to define a quality of the first local federated learning model and the second result of inference is the second metric to define the quality of the global federated learning model. The respective one of the first metric to define the quality and the second metric to define the quality may comprise at least one of an accuracy, a precision, or a recall of the respective federated learning model. The instructions, when executed by the one or more processors, may further cause the apparatus to perform checking whether the federated learning training process is tagged as being critical; and inhibiting the checking whether the first deviation is larger than the predefined first deviation threshold in response to checking that the federated learning training process is not tagged as being critical. The instructions, when executed by the one or more processors, may further cause the apparatus to perform requesting, from each of one or more potential federated learning clients, a respective first portion of the test data; and receiving, from at least a subset of the one or more potential federated learning clients, a respective second portion of the test data in response to the requesting the respective first portion of the test data, wherein the second portion is equal to or smaller than the first portion. The instructions, when executed by the one or more processors, may further cause the apparatus to perform checking, for each of the one or more potential federated learning clients, whether the respective second portion of the test data is smaller than the respective first portion of the test data; and reducing, for each of the one or more potential federated learning clients, a priority to select the respective potential federated learning client as one of the plural federated learning clients in response to checking that the respective second portion of the test data is smaller than the respective first portion of the test data or that the respective potential federated learning client does not provide any second portion of the test data. The instructions, when executed by the one or more processors, may further cause the apparatus to perform requesting a data repository function to provide the test data; and receiving the test data in response to the requesting the data repository function. The instructions, when executed by the one or more processors, may further cause the apparatus to perform the obtaining the second result of inference by computing the second result of inference using the test data in the global federated learning model. The instructions, when executed by the one or more processors, may further cause the apparatus to perform the obtaining the second result of inference by: selecting a second federated learning client among the plural federated learning clients; and computing the second result of inference using the test data in the second local federated learning model; wherein, the second federated learning client is different from the first federated learning client; and for the aggregating, the second local federated learning model is received from the second federated learning client. The instructions, when executed by the one or more processors, may further cause the apparatus to perform the obtaining the second result of inference by: selecting a second federated learning client among the plural federated learning clients; sending the test data to the second federated learning client; and instructing the second federated learning client to compute inference using the test data in the second local federated learning model; receiving the second result of inference from the second federated learning client in response to the instructing the second federated learning client; wherein the second federated learning client is different from the first federated learning client; and for the aggregating, the second local federated learning model is received from the second federated learning client. The instructions, when executed by the one or more processors, may further cause the apparatus to perform the selecting the second federated learning client such that a trust score of the second federated learning client is higher than a trust score of the first federated learning client. The instructions, when executed by the one or more processors, may further cause the apparatus to perform suspending the first federated learning client from the federated learning training process before the first federated learning client is instructed to compute inference using the test data in the first local federated learning model; and suspending the second federated learning client from the federated learning training process before the second federated learning client is instructed to compute inference using the test data in the second local federated learning model. The first federated learning client may be a terminal. The apparatus may be comprised by a network data analytics function. According to a second aspect, there is provided a method comprising: obtaining a first result of inference based on test data and a first local federated learning model; obtaining a second result of inference based on the test data and a reference federated learning model different from the first local federated learning model; checking whether a first deviation between the first result of inference and the second result of inference is larger than a predefined first deviation threshold; and, in response to checking that the first deviation between the first result of inference and the second result of inference is larger than the predefined first deviation threshold, at least one of: - removing a first federated learning client from the federated learning training process, or - decreasing a trust score of the first federated learning client; wherein the first local federated learning model and the reference federated learning model are obtained in a federated learning training process; the reference federated learning model is a second local federated learning model or a global federated learning model obtained from aggregating local federated learning models in the federated learning training process; the local federated learning models include the first local federated learning model and the second local federated learning model; for the aggregating, each of the local federated learning models is received from a respective one of plural federated learning clients involved in the federated learning training process; and the first local federated learning model is received from the first federated learning client. The first result of inference may be obtained by computing the first result of inference using the test data in the first local federated learning model received from the first federated learning client. The first result of inference may be obtained by sending the test data to the first federated learning client; instructing the first federated learning client to compute inference using the test data in the first local federated learning model; and receiving the first result of inference from the first federated learning client in response to the instructing the first federated learning client. The method may further comprise checking whether a second deviation between the computed first result of inference and the received first result of the inference is larger than a predefined second deviation threshold; and, in response to checking that the second deviation is larger than the predefined second deviation threshold, at least one of: - removing the first federated learning client from the federated learning training process; or - decreasing the trust score of the first federated learning client. The method may further comprise selecting the plural federated learning clients involved in the federated learning training process based on the trust scores of plural potential federated learning clients such that a potential federated learning client of the potential federated learning clients with a higher trust score is selected for being involved in the federated learning training process with a higher probability than a potential federated learning client of the potential federated learning clients with a lower trust score. The method may further comprise checking whether the first local federated learning model causes a first metric to define a quality of the global federated learning model being less than a respective predefined quality threshold; and inhibiting the checking whether the first deviation is larger than the predefined first deviation threshold in response to checking that the first local federated learning model causes the first metric to define the quality of the global federated learning model being less than the respective predefined quality threshold. The first result of inference may be a second metric to define a quality of the first local federated learning model and the second result of inference is the second metric to define the quality of the global federated learning model. The respective one of the first metric to define the quality and the second metric to define the quality may comprise at least one of an accuracy, a precision, or a recall of the respective federated learning model. The method may further comprise checking whether the federated learning training process is tagged as being critical; and inhibiting the checking whether the first deviation is larger than the predefined first deviation threshold in response to checking that the federated learning training process is not tagged as being critical. The method may further comprise requesting, from each of one or more potential federated learning clients, a respective first portion of the test data; and receiving, from at least a subset of the one or more potential federated learning clients, a respective second portion of the test data in response to the requesting the respective first portion of the test data, wherein the second portion is equal to or smaller than the first portion. The method may further comprise checking, for each of the one or more potential federated learning clients, whether the respective second portion of the test data is smaller than the respective first portion of the test data; and reducing, for each of the one or more potential federated learning clients, a priority to select the respective potential federated learning client as one of the plural federated learning clients in response to checking that the respective second portion of the test data is smaller than the respective first portion of the test data or that the respective potential federated learning client does not provide any second portion of the test data. The method may further comprise requesting a data repository function to provide the test data; and receiving the test data in response to the requesting the data repository function. The second result of inference may be obtained by computing the second result of inference using the test data in the global federated learning model. The second result of inference may be obtained by: selecting a second federated learning client among the plural federated learning clients; and computing the second result of inference using the test data in the second local federated learning model; wherein, the second federated learning client is different from the first federated learning client; and for the aggregating, the second local federated learning model is received from the second federated learning client. The second result of inference may be obtained by: selecting a second federated learning client among the plural federated learning clients; sending the test data to the second federated learning client; and instructing the second federated learning client to compute inference using the test data in the second local federated learning model; receiving the second result of inference from the second federated learning client in response to the instructing the second federated learning client; wherein the second federated learning client is different from the first federated learning client; and for the aggregating, the second local federated learning model is received from the second federated learning client. The selecting the second federated learning client may be performed such that a trust score of the second federated learning client is higher than a trust score of the first federated learning client. The method may further comprise suspending the first federated learning client from the federated learning training process before the first federated learning client is instructed to compute inference using the test data in the first local federated learning model; and suspending the second federated learning client from the federated learning training process before the second federated learning client is instructed to compute inference using the test data in the second local federated learning model. The first federated learning client may be a terminal. The method may be performed by a network data analytics function. The method may be a method of federated learning. According to a third aspect, there is provided a computer program product comprising a set of instructions which, when executed on an apparatus, is configured to cause the apparatus to carry out the method according to the second aspect. The computer program product may be embodied as a computer-readable medium or directly loadable into a computer. According to some example embodiments, at least one of the following advantages may be achieved: ^ Data and / or model poisoning attacks in federated learning may be detected; ^ Data and / or model poisoning attacks in federated learning may be mitigated; ^ Flexible assignment of tasks, e.g. dependent on capabilities of the FL server and the FL clients. It is to be understood that any of the above modifications can be applied singly or in combination to the respective aspects to which they refer, unless they are explicitly stated as excluding alternatives. Brief description of the drawings Further details, features, objects, and advantages are apparent from the following detailed description of the preferred example embodiments which is to be taken in conjunction with the appended drawings, wherein: Fig.1 (comprising Figs.1A and 1B) shows a message sequence chart according to some example embodiments; Fig. 2 shows a message sequence chart according to some example embodiments; Fig. 3 shows a message sequence chart according to some example embodiments; Fig.4 shows an apparatus according to an example embodiment; Fig.5 shows a method according to an example embodiment; and Fig.6 shows an apparatus according to an example embodiment. Detailed description of certain example embodiments Herein below, certain example embodiments are described in detail with reference to the accompanying drawings, wherein the features of the example embodiments can be freely combined with each other unless otherwise described. However, it is to be expressly understood that the description of certain example embodiments is given by way of example only, and that it is by no way intended to be understood as limiting the disclosure to the disclosed details. Moreover, it is to be understood that the apparatus is configured to perform the corresponding method, although in some cases only the apparatus or only the method are described. The very nature of FL introduces new threat vectors such as data poisoning and model poisoning. In particular (but not only), if UEs are involved as FL clients in FL training, the threat surface is essentially enlarged. Therefore, such potential attacks should be detected and, if possible, mitigated. In Federated learning, since the FL clients train the model using their local data and only send a model parameter to the FL server (aggregator), if a FL client is malicious, it can train the model using tampered data and can then send the model parameter to the FL server. This can then lead to adversarial attack wherein the global model that will then be generated by the FL server can misclassify the entities during inference in a way the malicious FL client(s) intended to. The problem may occur in different FL deployment options, for example in a telecommunication network. Some example embodiments provide a mechanism to detect if there is any attack using poisoned local training data by a malicious FL client, thereby corrupting the global model which may be sent to all the FL clients. In some example embodiments, security test data are stored at the ADRF (or any other NF) for any given analytics ID. If a FL consumer (for instance NWDAF AnLF) requests for a secured FL service, or if the FL server discovers that the analytics ID for which FL is performed is critical, defensive FL procedure is initiated. In some example embodiments, the defensive FL procedure may be executed as a default, regardless from any request for it or from the analytics ID. In the defensive FL procedure, first the FL server monitors at least one of the accuracy, precision, and recall of the FL global model for every FL round for each FL client update. Typically, the FL server monitors accuracy, precision, and recall of the FL global model. Accuracy, precision, and recall are metrics (quality metrics) to determine how good the FL global model (or any other machine learning model) reflects the reality. They are defined as follows: Accuracy = Correct predictions / total prediction number If the FL server detects an abnormal deviation in the metric values in subsequent FL rounds, FL server initiates a challenge to at least one of the FL clients using the test data stored. Typically, the at least one FL client is suspected to cause the abnormal deviation in the metric values. For the challenge, test data (which are assumed to be not tampered) are given to the local FL model of the at least one FL client to compute an inference using the test data. There are two options for computing the inference using the test data: - The FL server sends the test data to the FL client, which computes the inference and provides the result of the inference back to the FL server; or - The FL server computes the inference itself on the local FL model received from the FL client. Which of the options is selected may depend on the implementation. For example, the capability (e.g. processing power, memory,…) of the FL server and the FL client may be taken into account. These capabilities may depend on the type of network function. For example, a UE may have a smaller processing power and / or memory than a NWDAF. In some example embodiments, the selected option may be predefined. The outcome of the challenge is an inference result which the FL server compares with an inference result expected for the test data. In the case of a significant deviation from the expected inference result (i.e., the deviation is larger than a deviation threshold), FL sever may remove the respective FL client(s) from the FL process. In addition or alternatively, FL server may trigger NRF to decrease the trust score of the FL client (which is typically set during the FL client registration). In some example embodiments, the trust score may be decreased only if, after removal of the suspicious FL client(s), there is no longer a reduction of the metric values in the next FL rounds. In some example embodiments, the defensive procedure may not comprise the monitoring of the metric values of the global FL model prior to challenging one or more of the FL clients. For example, the FL server may select randomly, from time to time, some FL clients and initiate the challenge to the selected FL clients. Some example embodiments are now described at greater detail with reference to Figs.1 to 3. Note that each of these example embodiments uses only one of the implementation options for the challenge discussed hereinabove (computing the inference using the test data on the FL client, or computing the inference using the test data on the FL server), but in general, the other option is feasible, too. Fig. 1 shows an example embodiment completely within 5GC.For example, an NWDAF MTLF may act as FL server (aggregator), and other NWDAF MTLFs may act as FL clients. As another example, NWDAF (or another 5GC NF) may act as FL server (aggregator), and UEs may act as FL clients. The test data may be stored on ADRF or at another suitable data storage. For this purpose, metadata – e.g. “securityTestData” – is added to this test data in the ADRF which indicates that the test data may not be used any other entity, but only by an authorized FL Server in the case an attack is suspected. Furthermore, the profile of FL Clients (in this case NWDAF MTLFs) stored at the NRF may be enhanced to also include a FL Trust Score. The FL Trust Score can then be modified (decreased) by the FL Server in case of a detection or suspicion of an attack by the respective FL client. The FL trust score may be used to select FL clients participating in a FL training: the higher the trust score of a potential FL client, the higher the probability that the potential FL client is selected to be involved in the FL training process. The actions in Fig.1 are as follows: Action 1: An analytics consumer (e.g. NWDAF AnLF) sends a request for analytics to FL server (e.g. NWDAF MTLF). In particular, AnLF may request subscription for a service (e.g. “secured FL service”) where the defensive FL procedure is employed. This service instructs the FL server to endeavor to make sure that the FL model provided to the AnLF is not tampered with or attacked by any malicious entity during the model training phase. Action 2a: FL server selects the FL clients and initiates the FL process. For example, FL server may use the trust score for selecting the FL clients. Action 2b: In case there is a request for “secured FL service“ by the AnLF, or if FL server considers the analytics ID request by AnLF to be critical (for instance NF load prediction), FL server initiates the defensive FL procedures Actions 3,4: FL Server sends a request for FL tagged security test data collection for the given analytics ID to ADRF. If present, ADRF sends the data as a response Action 5: FL Server initiates the FL process at the FL clients. (In some example embodiments, actions 3, 4 may follow action 5, or these actions may be performed fully or partly in parallel). Action 6: Once the FL process is initiated, FL server monitors the quality metrics (accuracy, precision, recall (or at least one of them)) with every FL round per FL client Action 7: In the case there is a decrease in the quality metrics after receiving model updates from specific FL client(s) in a particular FL round, FL server adds those FL client(s) in a suspicious list Action 8: FL Server then sends the test data received in Action 4 to the suspicious FL client(s) to get the inference from their local FL model. This acts as a challenge to the suspicious FL client to re-ensure that they are genuine and are not attackers. Action 9: FL server in parallel also computes the inference using the test data in the global aggregated model. Action 10: The suspicious FL Client(s) sends the inference result for the given test data to the FL server. Action 11: FL server analyses the inferences of all the suspicious FL clients. More specifically, FL server compares the inference results of the suspicious FL client(s) with the inference result obtained in action 9. Based on the deviation between the inference results (if present), FL server discards one or more of the suspicious FL clients with the highest deviation (inference deviation) from the ongoing FL process. FL Server now resumes the monitoring of the accuracy, precision, recall per each remaining FL round again. Action 12: In case the accuracy no longer drops, FL Server indicates to the NRF to decrease the trust score for the suspicious FL client(s) which led to the decrease in accuracy, precision, recall. Some example embodiments may additionally prevent the suspicious FL client from hiding to be a malicious FL client in the following way: Before providing the result of inference based on the test data, the suspicious FL client may retrain the model again with the genuine local data, and then use this re-trained model to provide the inference based on the test data back to the FL server. The FL server can detect this kind of cheating as follows: The FL server may additionally compute the inference using the test data in the local FL model received from the suspicious FL client and compare this inference result with the inference output based on the test data received from the suspicious FL client. If this deviation is larger than a certain threshold, FL server may assume that the suspicious FL client is malicious and perform the actions 11 and / or 12 of Fig.1. In some example embodiments, it is sufficient that one of the deviations of the inference results (between the local FL model at the suspicious FL client and the global FL model, and between the local FL model at the FL client and the local FL model from the suspicious FL client at the FL server) is larger than the respective threshold, to cause the FL server to discard the suspicious FL client from the FL training and / or to decrease the trust score of the suspicious FL client. In some example embodiments, both of these deviations must be larger than the respective threshold to cause the FL server to discard the suspicious FL client from the FL training and / or to decrease the trust score of the suspicious FL client. In some example embodiments, it is sufficient that one of the deviations of the inference results is larger than the respective threshold, to cause the FL server to discard the suspicious FL client from the FL training, but both of the deviations must be larger than the respective threshold to cause the FL server to decrease the trust score of the suspicious FL client, or vice versa. Fig. 2 shows another example embodiment. As one option, the example embodiment may be completely within 5GC, wherein an NWDAF MTLF acts as FL server (aggregator), and other NWDAF MTLFs act as FL clients. As another option, the FL server may be any 5GC NF (such as a NWDAF), and the FL clients may be UEs. Different from the example embodiment of Fig. 1, in the present example embodiment, the test data are obtained from other FL clients, in particular from such FL clients which the FL server considers to be trusted. In addition, different to Fig.1, the inference results of the challenge are computed at the FL server using the test data from the other FL clients in the local FL model received from the suspicious FL client for the aggregating. The actions in Fig.2 are as follows: Action 1: Analytics consumer (AnLF) MTLF FL server – Analytics Request for secured FL service. The request may be a subscription. Start: FL client selection Action 2: MTLF FL server potential MTLF FL client(s) – Request a portion (x%) of test data for the analytics ID Action 3: Potential MTLF FL client(s) FL server – Receive y% of test data for the analytics ID. Some FL clients may be willing to share only less than x% of test data, or some may not be willing to share any test data. Therefore, for each of the FL clients: y% ≤ x%. The test data are historical data and are assumed to be labelled. Action 4: MTLF FL server – Perform FL clients selection among the potential FL clients. In the selection, FL server may take into account the portion (y%) of test data received from each potential FL client in action 3. For example, FL server may assume that those potential FL clients willing to provide a larger portion (y%) of the test data are more likely genuine clients than those potential FL clients not willing to provide any test data or only a smaller portion (y%) of the test data. Thus, FL server may prioritize the potential FL clients sharing more test data in the FL client selection. FL server may use other selection criteria instead of or in addition to the portion (y%) of test data received from each potential FL client. End: FL client selection Start: FL training process (iterative) Action 5: FL server initiates FL process at selected FL clients. Action 6: MTLF FL server requests and receives the trust score of the selected FL clients from the NRF. In addition, MTLF FL server may assign a preliminary reputation score for each selected FL client (e.g., 0 for the FL client who doesn’t share any test data, 0.1 for the FL client sharing less than x% of test data and 0.2 for the FL client sharing x% of sharing data). Based on the trust score and the preliminary reputation score, MTLF FL server may assign an initial reputation score for each selected FL client. The reputation score may be considered as a locally valid trust score. In some example embodiments, MTLF may use only one of the trust score and the preliminary reputation score for the initial reputation score. Actions 5 and 6 may be performed in an arbitrary sequence. They may be performed fully or partly in parallel. Action 7: FL clients send locally trained model to FL server. Action 8: MTLF FL server – Determine (test) the quality metrics (accuracy, precision, recall, or at least one of them) of the local model reported by each FL client using the test data (test dataset) which was provided jointly by several (or all) of the potential FL clients. Action 9: MTLF FL server – Aggregate the local models received from the FL client(s) to generate a global model Action 10: MTLF FL server - Determine (test) the quality metrics of the aggregated global model using the test data (test dataset) which was provided jointly by several (or all) of the FL clients Action 11: MTLF FL server - Compare the quality metrics obtained in action 8 and action 10 for each FL client. and accordingly increase or decrease the reputation score by e.g., 0.1. I.e., if the quality metrics of the local model is substantially equal to that of the global model (the deviation is less than a predefined first threshold) the reputation score may be increased, if it is substantially different (the deviation is larger than a predefined second threshold equal to or larger than the predefined first threshold) the reputation score may be decreased. Action 12: MTLF FL server MTLF FL client(s) – Notify the FL client(s) that have crossed a predefined first lowest reputation score threshold (e.g., -1) indicating that they cannot participate in the FL training process from next iteration on. Alternatively, there may be a warning action prior to removing the MTLF FL client(s) from the FL training process. Action 13: MTLF FL server NRF – Decrease the trust score for the NWDAF MTLF that has crossed a predefined second lowest reputation score threshold. The predefined second lowest reputation score threshold may be the same or different from the predefined first lowest reputation score threshold. For example, the predefined first lowest reputation score threshold may be lower than the predefined second lowest reputation score threshold. In this case, a FL client may still participate in the FL training process but its trust score is reduced. End: FL training process (iterative) In some example embodiments, the trust score may not be considered such that action 6 and action 13 may be omitted. In some example embodiments, In some example embodiments, the reputation score is not used. For example, in such example embodiments, the trust score may be reduced if the quality metrics of the local model is substantially different from that of the global model. Fig.3 shows another example embodiment, where the FL server is in 5GC or RAN, and the FL clients are preferably UEs. In the example embodiments of Figs.1 and 2, the aggregated FL model was used as a reference model. In Fig.3, the local FL model of another FL client (preferably of a trusted FL client) is used as the reference model which is compared with the local FL model of the suspicious FL client (using the test data). The actions are as follows: Action 2a: After a consumer requested a secure FL service (action 1 in Fig.1, not shown in Fig. 3), FL server selects some FL clients (e.g. based on their trust scores) and initiates the FL process with the selected FL clients. The selected FL clients may be considered as a FL group. Action 2b: FL server detects that one of the FL clients (here: UE1) causes the quality metrics to decrease such that UE1 is a suspicious FL client and selected for a challenge. In addition, FL server suspends UE1 from participating in the FL training process (suspend it from FL group). Actions 3, 4: FL server requests test data from a data repository (e.g. ADRF) and receives the same in response. Action 4b: FL server selects another FL client (here: UE3, a “ggod” UE) to provide its local FL model as the reference model. Preferably, UE3 is a trusted UE. For example, the trust score of UE3 may be higher than the trust score of UE1. If necessary for performance reasons, FL server may suspend UE3 from participating in the FL training process, too. The pair of actions 3 and 4 and the action 4b may be performed in an arbitrary sequence and fully or partly in parallel. They may be performed fully or partly in parallel. Actions 3, 4 may be performed prior to action 2b or fully or partly in parallel thereto. Action 5: FL server sends the test data to UE1 and UE3 and instructs UE1 and UE3 to compute inference using the test data in the respective local FL model. Action 6: UE1 and UE3 send back the respective result of inference to the FL server. Action 7: FL server compares the inference results of the suspicious FL client UE1 with the inference result of the trusted UE3. Action 8: If the deviation of the inference results is larger than a predefined threshold, FL server discards the suspicious FL client UE1 from the ongoing FL process. FL Server now resumes the monitoring of the quality metrics (accuracy, precision, recall) per each remaining FL round again. FL server may add UE3 back to the FL training cycle if UE3 was suspended. In addition or instead of discarding UE1, FL server may decrease the trust score of UE1. In some example embodiments (not shown in a figure), if FL server detects that one of the UEs (UE1) causes a lowering of the quality metrics, FL server may stop the FL process at all the FL clients (UEs) in action 2b of Fig.3. Then, FL server has to initiate the FL process at UE1 and the selected “good” UE (UE3) for performing inference using the test data (after action 4b in Fig.3). Fig.4 shows an apparatus according to an example embodiment. The apparatus may be a Fl server or an element thereof. Fig.5 shows a method according to an example embodiment. The apparatus according to Fig.4 may perform the method of Fig.5 but is not limited to this method. The method of Fig.5 may be performed by the apparatus of Fig.4 but is not limited to being performed by this apparatus. The apparatus comprises first means for obtaining 110, second means for obtaining 120, means for checking 130, and at least one of means for removing 140 or means for decreasing 150. The first means for obtaining 110, second means for obtaining 120, means for checking 130, means for removing 140, and means for decreasing 150 may be a first obtaining means, second obtaining means, checking means, removing means, and decreasing means, respectively. The first means for obtaining 110, second means for obtaining 120, means for checking 130, means for removing 140, and means for decreasing 150 may be a first obtainer, second obtainer, checker, remover, and decreaser, respectively. The first means for obtaining 110, second means for obtaining 120, means for checking 130, means for removing 140, and means for decreasing 150 may be a first obtaining processor, second obtaining processor, checking processor, removing processor, and decreasing processor, respectively. The first means for obtaining 110 obtains a first result of inference based on test data and a first local FL model (S110). The second means for obtaining 120 obtains a second result of inference based on the test data and a reference FL model different from the first local FL model (S120). The first local FL model and the reference FL model are obtained in a FL training process. The reference FL model is a second local FL model or a global FL model obtained from aggregating local FL models in the FL training process. The local FL models include the first local FL model and the second local FL model. For the aggregating, each of the local FL models is received from a respective one of plural FL clients involved in the FL training process; in particular, the first local FL model is received from the first local FL client. S110 and S120 may be performed in an arbitrary sequence. They may be performed fully or partly in parallel. The means for checking 130 checks whether a deviation between the first result of inference and the second result of inference is larger than a predefined deviation threshold (S130). In response to checking that the deviation between the first result of inference and the second result of inference is larger than the predefined deviation threshold (S130 = yes), at least one of the following may happen: - The means for removing 140, if present in the apparatus, may remove the first FL client from the FL training process (S140); or - The means for decreasing 150, if present in the apparatus, decreases a trust score of the first FL client (S150). Fig.6 shows an apparatus according to an example embodiment. The apparatus comprises at least one processor 810, at least one memory 820 storing instructions that, when executed by the at least one processor 810, cause the apparatus at least to perform the method according to Fig. 5 and related description. Some example embodiments are explained with respect to 5G (NR). However, other example embodiments may be employed in other 3GPP generations, such as 4G, 6G, 7G, etc., in other wireless or wired communication networks, and in other systems employing federated learning. A UE is an example of a terminal. Other examples are MTC devices. Each of the terminals may be implemented as a smartphone, a mobile phone, a laptop, a sensor device etc. One piece of information may be transmitted in one or plural messages from one entity to another entity. Each of these messages may comprise further (different) pieces of information. Names of network elements, network functions, protocols, and methods are based on current standards. In other versions or other technologies, the names of these network elements and / or network functions and / or protocols and / or methods may be different, as long as they provide a corresponding functionality. The same applies correspondingly to the terminal. If not otherwise stated or otherwise made clear from the context, the statement that two entities are different means that they perform different functions. It does not necessarily mean that they are based on different hardware. That is, each of the entities described in the present description may be based on a different hardware, or some or all of the entities may be based on the same hardware. It does not necessarily mean that they are based on different software. That is, each of the entities described in the present description may be based on different software, or some or all of the entities may be based on the same software. Each of the entities described in the present description may be deployed in the cloud. According to the above description, it should thus be apparent that example embodiments provide, for example, a FL server or a component thereof, an apparatus embodying the same, a method for controlling and / or operating the same, and computer program(s) controlling and / or operating the same as well as mediums carrying such computer program(s) and forming computer program product(s). Implementations of any of the above described blocks, apparatuses, systems, techniques or methods include, as non-limiting examples, implementations as hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof. Each of the entities described in the present description may be embodied in the cloud. It is to be understood that what is described above is what is presently considered the preferred example embodiments. However, it should be noted that the description of the preferred example embodiments is given by way of example only and that various modifications may be made without departing from the scope of the disclosure as defined by the appended claims. The terms “first X” and “second X” include the options that “first X” is the same as “second X” and that “first X” is different from “second X”, unless otherwise specified. As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements are joined by “and” or “or”, mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements.

Claims

1. Apparatus comprising: one or more processors, and memory storing instructions that, when executed by the one or more processors, cause the apparatus to perform: obtaining a first result of inference based on test data and a first local federated learning model; obtaining a second result of inference based on the test data and a reference federated learning model different from the first local federated learning model; checking whether a first deviation between the first result of inference and the second result of inference is larger than a predefined first deviation threshold; and, in response to checking that the first deviation between the first result of inference and the second result of inference is larger than the predefined first deviation threshold, at least one of: - removing a first federated learning client from the federated learning training process, or - decreasing a trust score of the first federated learning client; wherein the first local federated learning model and the reference federated learning model are obtained in a federated learning training process; the reference federated learning model is a second local federated learning model or a global federated learning model obtained from aggregating local federated learning models in the federated learning training process; the local federated learning models include the first local federated learning model and the second local federated learning model; for the aggregating, each of the local federated learning models is received from a respective one of plural federated learning clients involved in the federated learning training process; and the first local federated learning model is received from the first local federated learning client.

2. The apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, further cause the apparatus to perform the obtaining the first result of inference bycomputing the first result of inference using the test data in the first local federated learning model received from the first federated learning client.

3. The apparatus according to any of claims 1 and 2, wherein the instructions, when executed by the one or more processors, further cause the apparatus to perform the obtaining the first result of inference by sending the test data to the first federated learning client; instructing the first federated learning client to compute inference using the test data in the first local federated learning model; and receiving the first result of inference from the first federated learning client in response to the instructing the first federated learning client.

4. The apparatus according to claim 3 dependent on claim 2, wherein the instructions, when executed by the one or more processors, further cause the apparatus to perform checking whether a second deviation between the computed first result of inference and the received first result of the inference is larger than a predefined second deviation threshold; and, in response to checking that the second deviation is larger than the predefined second deviation threshold, at least one of: - removing the first federated learning client from the federated learning training process; or - decreasing the trust score of the first federated learning client.

5. The apparatus according to any of claims 1 to 4, wherein the instructions, when executed by the one or more processors, further cause the apparatus to perform selecting the plural federated learning clients involved in the federated learning training process based on the trust scores of plural potential federated learning clients such that a potential federated learning client of the potential federated learning clients with a higher trust score is selected for being involved in the federated learning training process with a higher probability than a potential federated learning client of the potential federated learning clients with a lower trust score.

6. The apparatus according to any of claims 1 to 5, wherein the instructions, when executed by the one or more processors, further cause the apparatus to perform checking whether the first local federated learning model causes a first metric to define a quality of the global federated learning model being less than a respective predefined quality threshold; and inhibiting the checking whether the first deviation is larger than the predefined first deviation threshold in response to checking that the first local federated learning model causes the first metric to define the quality of the global federated learning model being less than the respective predefined quality threshold.

7. The apparatus according to any of claims 1 to 6, wherein the first result of inference is a second metric to define a quality of the first local federated learning model and the second result of inference is the second metric to define the quality of the global federated learning model.

8. The apparatus according to any of claims 6 and 7, wherein the respective one of the first metric to define the quality and the second metric to define the quality comprise at least one of an accuracy, a precision, or a recall of the respective federated learning model.

9. The apparatus according to any of claims 1 to 8, wherein the instructions, when executed by the one or more processors, further cause the apparatus to perform checking whether the federated learning training process is tagged as being critical; and inhibiting the checking whether the first deviation is larger than the predefined first deviation threshold in response to checking that the federated learning training process is not tagged as being critical.

10. The apparatus according to any of claims 1 to 9, wherein the instructions, when executed by the one or more processors, further cause the apparatus to perform requesting, from each of one or more potential federated learning clients, a respective first portion of the test data; andreceiving, from at least a subset of the one or more potential federated learning clients, a respective second portion of the test data in response to the requesting the respective first portion of the test data, wherein the second portion is equal to or smaller than the first portion.

11. The apparatus according to claim 10, wherein the instructions, when executed by the one or more processors, further cause the apparatus to perform checking, for each of the one or more potential federated learning clients, whether the respective second portion of the test data is smaller than the respective first portion of the test data; and reducing, for each of the one or more potential federated learning clients, a priority to select the respective potential federated learning client as one of the plural federated learning clients in response to checking that the respective second portion of the test data is smaller than the respective first portion of the test data or that the respective potential federated learning client does not provide any second portion of the test data.

11. The apparatus according to any of claims 1 to 9, wherein the instructions, when executed by the one or more processors, further cause the apparatus to perform requesting a data repository function to provide the test data; and receiving the test data in response to the requesting the data repository function.

12. The apparatus according to any of claims 1 to 11, wherein the instructions, when executed by the one or more processors, further cause the apparatus to perform the obtaining the second result of inference by computing the second result of inference using the test data in the global federated learning model.

13. The apparatus according to any of claims 1 to 12, wherein the instructions, when executed by the one or more processors, further cause the apparatus to perform the obtaining the second result of inference by: selecting a second federated learning client among the plural federated learning clients; andcomputing the second result of inference using the test data in the second local federated learning model; wherein, the second federated learning client is different from the first federated learning client; and for the aggregating, the second local federated learning model is received from the second federated learning client.

14. The apparatus according to any of claims 1 to 13, wherein the instructions, when executed by the one or more processors, further cause the apparatus to perform the obtaining the second result of inference by: selecting a second federated learning client among the plural federated learning clients; sending the test data to the second federated learning client; and instructing the second federated learning client to compute inference using the test data in the second local federated learning model; receiving the second result of inference from the second federated learning client in response to the instructing the second federated learning client; wherein the second federated learning client is different from the first federated learning client; and for the aggregating, the second local federated learning model is received from the second federated learning client.

15. The apparatus according to any of claims 13 and 14, wherein the instructions, when executed by the one or more processors, further cause the apparatus to perform the selecting the second federated learning client such that a trust score of the second federated learning client is higher than a trust score of the first federated learning client.

16. The apparatus according to any of claims 13 to 15, wherein the instructions, when executed by the one or more processors, further cause the apparatus to performsuspending the first federated learning client from the federated learning training process before the first federated learning client is instructed to compute inference using the test data in the first local federated learning model; and suspending the second federated learning client from the federated learning training process before the second federated learning client is instructed to compute inference using the test data in the second local federated learning model.

17. The apparatus according to any of claims 1 to 16, wherein at least one of the following: - the first federated learning client is a terminal; or - the apparatus is comprised by a network data analytics function.

18. Method comprising: obtaining a first result of inference based on test data and a first local federated learning model; obtaining a second result of inference based on the test data and a reference federated learning model different from the first local federated learning model; checking whether a first deviation between the first result of inference and the second result of inference is larger than a predefined first deviation threshold; and, in response to checking that the first deviation between the first result of inference and the second result of inference is larger than the predefined first deviation threshold, at least one of: - removing a first federated learning client from the federated learning training process, or - decreasing a trust score of the first federated learning client; wherein the first local federated learning model and the reference federated learning model are obtained in a federated learning training process; the reference federated learning model is a second local federated learning model or a global federated learning model obtained from aggregating local federated learning models in the federated learning training process;the local federated learning models include the first local federated learning model and the second local federated learning model; for the aggregating, each of the local federated learning models is received from a respective one of plural federated learning clients involved in the federated learning training process; and the first local federated learning model is received from the first local federated learning client.

19. A computer program product comprising a set of instructions which, when executed on an apparatus, is configured to cause the apparatus to carry out the method according to claim 18.

20. The computer program product according to claim 19, embodied as a computer-readable medium or directly loadable into a computer.

Images