Method for collaborative training of ml model based on relay assisted federated learning with over-the-air computation and data privacy

Abstract

A method is provided performed by a network node without a direct communication link with a plurality of UEs to collaboratively train a machine learning, ML, model based on a relay assisted distributed ML with over-the-air computation and data privacy. The method includes receiving (506) further modified local ML model gradients from a relay over wireless channels. A further modified local ML model gradient includes a local ML model gradient modified by a respective UE with a first artificial noise for data privacy which is further modified by the relay with a second artificial noise for data privacy. The method further includes aggregating (508) the further modified local ML model gradients based on an over-the-air computation; constructing (510) an updated ML model; and performing (512) one of (i) transmitting the updated ML model to the relay, or (ii) configure variables and start a next round of the collaborative training.

Description

METHOD FOR COLLABORATIVE TRAINING OF ML MODEL BASED ON RELAY ASSISTED FEDERATED LEARNING WITH OVER-THE-AIR COMPUTATION AND DATA PRIVACY TECHNICAL FIELD

[0001] The present disclosure relates generally to methods for collaborative training of a machine learning (ML) model based on relay assisted federated learning with over-the-air computation and data privacy, and related methods and apparatus. BACKGROUND

[0002] In fifth generation (5G) and future cellular communications, the adoption of artificial intelligence (AI) / ML applications can face many challenges, such as: data silos, computation capacity, learning accuracy, channel fading, and privacy concerns. Data privacy has become a priority for both businesses and individuals, for example.

[0003] Federated learning can enable AI / ML model training at a network edge by exploiting large scale distributed data and compute resources. Federated learning differs from classical AI / ML in four main domains: data privacy (e.g., no end-user data leaves the user device), data distribution (e.g., data can be IID or non-IID), learning time (e.g., the communication time between a user device and central server may be too long to provide a satisfactory user experience), and aggregation of data (e.g., some privacy notions and rules are violated when user data aggregation occurs in a central server).

[0004] Federated learning involves different devices to upload and aggregate parameters iteratively to train a global ML model, for example. In such a scenario, distributed devices (e.g., mobile devices or workers) collaborate to train a common AI / ML model under the coordination of an access point (AP) or parameter server. SUMMARY

[0005] While federated learning may restrict explicit data sharing so that confidentiality and some privacy associated with use cases may be preserved, certain challenges still exist. In critical applications, for example, over a private wireless network or a fixed wireless network, privacy issues still may exist in federated learning, even if the ML models are trained by sharing local model updates (e.g., gradient information) instead of the raw data. For example, by exploiting parameter differences between training and uploading, privacy can be disclosed or divulgated. Moreover, privacy information also may be obtained through a ML model inversion attack.

[0006] Certain aspects of the disclosure and their embodiments may provide solutions to these or other challenges.

[0007] Some embodiments provide a system for collaborative training of a ML model based on a relay assisted federated learning with over-the-air computation and data privacy. The system includes a plurality of user equipment, UEs, in a federated learning environment respectively configured to (i) communicate over a plurality of wireless channels, (ii) receive, from a relay, the ML model and a third subset of variables from a plurality of variables, (iii) train the ML model with local data and obtain a local ML model gradient, (iv) modify the local ML gradient with a first transmit power level from the third subset of variables and a first artificial noise for data privacy, and (v) transmit the modified local ML model gradient to the relay. The system further includes a network node without a direct communication link with the plurality of UEs. The network node is configured to (i) transmit, to the relay, the ML model and the plurality of variables, (ii) receive, from the relay, over a plurality of respective wireless channels respective further modified local ML model gradients including a second artificial noise for data privacy, (iii) aggregate the further modified local ML model gradients based on an over-the-air computation for wireless aggregation, (iv) construct an updated ML model based on the aggregated further modified local ML model gradients, and (v) perform one of (a) transmit the updated ML model to the relay when a convergence or a maximum learning round is reached, or (b) start a next round of the collaborative training. The system further includes a relay configured to (i) receive the plurality of variables and the ML model from the network node, (ii) extract from the plurality of variables a second subset of variables, (iii) transmit to the plurality of UEs the third subset of variables and the ML model, (iv) receive the modified local ML model gradients transmitted from UEs over a plurality of respective wireless channels, (v) further modify the respective modified local ML gradients with the second subset of variables to add a second artificial noise for data privacy and to scale the received modified local gradients, and (vi) transmit the respective further modified local ML model gradients to the network node.

[0008] Other embodiments provide a method performed by a network node without a direct communication link with a plurality of UEs to collaboratively train a ML model based on a relay assisted federated learning with over-the-air computation and data privacy. The method includes receiving a plurality of further modified local ML model gradients from a relay over respective wireless channels of a plurality of wireless channels. A further modified local ML model gradient includes a local ML model gradient modified by a respective UE with a first artificial noise for data privacy which is further modified by the relay with a second artificial noise for data privacy. The method further includes aggregating the plurality of further modified local ML modelgradients based on an over-the-air computation for wireless aggregation. The method further includes constructing an updated ML model based on the aggregated further modified local gradients; and performing one of (i) transmitting the updated ML model to the relay, or (ii) configure a plurality of variables and start a next round of the collaborative training.

[0009] Some embodiments provide a method performed by a relay to collaboratively train a ML model based on relay assisted distributed machine learning with over-the-air computation and data privacy. The method includes extracting from a plurality of variables a second subset of variables; and transmitting to a plurality of UEs the ML model and a third subset of variables from the plurality of variables. The method further includes receiving a modified local ML model gradient from respective UEs over a plurality of respective wireless channels. The modified local ML gradient includes a local ML model gradient modified by a respective UE with a first artificial noise for data privacy. The method further includes further modifying the respective modified local ML gradients with the second subset of variables to add a second artificial noise for data privacy and to scale the received modified local gradients; and transmitting the respective further modified local ML model gradients to a network node.

[0010] Other embodiments provide a method performed by a UE without a direct communication link with a network node to collaboratively train a ML model based on a relay assisted distributed machine learning with over-the-air computation and data privacy. The method includes receiving over a wireless channel, from a relay, the ML model and a third subset of variables from a plurality of variables; and training the ML model with local data to obtain a local ML model gradient. The method further includes modifying the local ML gradient with a first transmit power level from the third subset of variables and a first artificial noise for data privacy; and transmitting the modified local ML model gradient to the relay.

[0011] Still other embodiments provide a network node configured to perform collaborative training of a ML model based on relay assisted federated learning with over-the-air computation and data privacy. The network node includes processing circuitry; and memory coupled with the processing circuitry. The memory includes instructions that when executed by the processing circuitry causes the network node to perform operations. The operations include to receive a plurality of further modified local ML model gradients from a relay over respective wireless channels of a plurality of wireless channels. A further modified local ML model gradient includes a local ML model gradient modified by a respective UE with a first artificial noise for data privacy which is further modified by the relay with a second artificial noise for data privacy. The operations further include to aggregate the plurality of further modified local ML model gradients based on an over-the-air computation for wireless aggregation; to construct an updated ML model based onthe aggregated further modified local gradients; and to perform one of (i) transmitting the updated ML model to the relay, or (ii) configure a plurality of variables and start a next round of the collaborative training.

[0012] Some embodiments provide a non-transitory computer readable medium including program code to be executed by processing circuitry of a network node configured to perform collaborative training of a ML model based on relay assisted federated learning with over-the-air computation and data privacy. Execution of the program code causes the program code to perform operations including to receive a plurality of further modified local ML model gradients from a relay over respective wireless channels of a plurality of wireless channels. A further modified local ML model gradient includes a local ML model gradient modified by a respective UE with a first artificial noise for data privacy which is further modified by the relay with a second artificial noise for data privacy. The operations further include to aggregate the plurality of further modified local ML model gradients based on an over-the-air computation for wireless aggregation; to construct an updated ML model based on the aggregated further modified local gradients; and to perform one of (i) transmitting the updated ML model to the relay, or (ii) configure a plurality of variables and start a next round of the collaborative training.

[0013] Yet other embodiments provide a relay configured to perform collaborative training of a ML model based on relay assisted distributed machine learning with over-the-air computation and data privacy. The relay includes processing circuitry, and memory coupled with the processing circuitry. The memory includes instructions that when executed by the processing circuitry causes the relay to perform operations. The operations include to extract from a plurality of variables a second subset of variables; to transmit to a plurality of UEs the ML model and a third subset of variables from the plurality of variables; and to receive a modified local ML model gradient from respective UEs over a plurality of respective wireless channels. The modified local ML gradient includes a local ML model gradient modified by a respective UE with a first artificial noise for data privacy. The operations further include to further modify the respective modified local ML gradients with the second subset of variables to add a second artificial noise for data privacy and to scale the received modified local gradients; and to transmit the respective further modified local ML model gradients to a network node.

[0014] Other embodiments provide a non-transitory computer readable medium including program code to be executed by processing circuitry of a relay configured to perform collaborative training of a ML model based on relay assisted distributed machine learning with over-the-air computation and data privacy. Execution of the program code causes the program code to perform operations. The operations include to extract from a plurality of variables a second subset ofvariables; to transmit to a plurality of UEs the ML model and a third subset of variables from the plurality of variables; and to receive a modified local ML model gradient from respective UEs over a plurality of respective wireless channels. The modified local ML gradient includes a local ML model gradient modified by a respective UE with a first artificial noise for data privacy. The operations further include to further modify the respective modified local ML gradients with the second subset of variables to add a second artificial noise for data privacy and to scale the received modified local gradients; and to transmit the respective further modified local ML model gradients to a network node.

[0015] Still other embodiments provide a UE configured to perform collaborative training of a ML model based on relay assisted distributed machine learning with over-the-air computation and data privacy. The UE includes processing circuitry, and memory coupled with the processing circuitry. The memory includes instructions that when executed by the processing circuitry causes the UE to perform operations. The operations include to receive over a wireless channel, from a relay, the ML model and a third subset of variables from a plurality of variables; and to train the ML model with local data to obtain a local ML model gradient. The operations further include to modify the local ML gradient with a first transmit power level from the third subset of variables and a first artificial noise for data privacy; and to transmit the modified local ML model gradient to the relay.

[0016] Other embodiments provide a non-transitory computer readable medium including program code to be executed by processing circuitry of a UE configured to perform collaborative training of a ML model based on relay assisted distributed machine learning with over-the-air computation and data privacy. Execution of the program code causes the program code to perform operations. The operations include to receive over a wireless channel, from a relay, the ML model and a third subset of variables from a plurality of variables; and to train the ML model with local data to obtain a local ML model gradient. The operations further include to modify the local ML gradient with a first transmit power level from the third subset of variables and a first artificial noise for data privacy; and to transmit the modified local ML model gradient to the relay.

[0017] Certain embodiments may provide one or more of the following technical advantage(s). Based on inclusion of operations for collaboratively training a ML model based on a relay assisted federated learning with over-the-air computation and data privacy, an area in a network having restricted coverage can be accessed to enable collaborative federated learning. As a consequence, prediction error may be improved and a gap in optimality may be decreased; privacy may be enhanced; and learning accuracy may be increased as the number of UEs increases, among other technical advantages discussed herein.BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this application, illustrate certain non-limiting embodiments of the present disclosure. In the drawings:

[0019] Figure 1 is a schematic diagram of a system in accordance with some embodiments;

[0020] Figure 2 is a schematic diagram of an overview of a relay assisted private federated learning system in accordance with some embodiments;

[0021] Figure 3 is a schematic diagram of operations executed at a network node, a relay, and respective UEs in accordance with some embodiments;

[0022] Figure 4 is a flowchart of an example training process in accordance with some embodiments;

[0023] Figure 5 is a flowchart illustrating example operations of a network node in accordance with some embodiments;

[0024] Figure 6 is a flowchart illustrating example operations of a relay in accordance with some embodiments;

[0025] Figure 7 is a flowchart illustrating example operations of a UE in accordance with some embodiments;

[0026] Figure 8 is a block diagram of a communication system in accordance with some embodiments;

[0027] Figure 9 is a block diagram of a user equipment (UE) in accordance with some embodiments;

[0028] Figure 10 is a block diagram of a network node in accordance with some embodiments;

[0029] Figure 11 is a block diagram of a host in accordance with some embodiments;

[0030] Figure 12 is a block diagram of a virtualization environment in accordance with some embodiments;

[0031] Figure 13 is a communication diagram of a host communicating via a network node and relay with a UE over a wireless connection in accordance with some embodiments; and

[0032] Figures 14-18 are plots of results from an example simulation in accordance with some embodiments. DETAILED DESCRIPTION

[0033] Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Embodiments are provided by way of example to conveythe scope of the subject matter to those skilled in the art, in which examples of embodiments of the present disclosure are shown. Inventive concepts may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. Components from one embodiment may be tacitly assumed to be present / used in another embodiment.

[0034] Over-the-air computation (AirComp) may allow fast data aggregation, specifically by exploiting the superposition property of a wireless multiple access channel. A technical advantage of AirComp may be to enable energy and spectrum efficiency (e.g., efficient use of available wireless spectrum) in large scale aggregation.

[0035] Differential Privacy (DP) refers to a theoretical framework to analyze and try to guarantee privacy. When DP is deployed, random noise can be added to transmitted data to try to achieve a certain privacy level. However, random perturbations need to be carefully designed to avoid undesired impact on learning accuracy. DP may be used to try to enforce privacy guarantees in ML. When DP is used, there may be a tradeoff between privacy and learning accuracy. For the standardization of cooperative communications, the Third Generation Partnership Project (3GPP), for example, has standardized radio relay technologies by specifying Layer 1 Relay, Layer 2 Relay and Layer 3 Relay. Moreover, relay technology for long term evolution-advanced (LTE-A) may improve cell edge throughput and cell coverage.

[0036] As discussed herein, however, data privacy, learning accuracy, and / or communication efficiency still may be concerns in distributed artificial intelligence, such as federated learning.

[0037] Certain aspects of the disclosure and their embodiments may provide solutions to these or other challenges.

[0038] Figure 1 is a schematic diagram of a system 100 in accordance with some embodiments. The system 100, in some embodiments, is provided for collaborative training of a ML model based on relay assisted federated learning with AirComp and data privacy. The system includes a plurality of user equipment (UEs) 102 in a federated learning environment respectively configured to (i) communicate over a plurality of wireless channels (such as frequency channels), (ii) receive, from a relay, the ML model and a third subset of variables from a plurality of variables, (iii) train the ML model with local data and obtain a local ML model gradient, (iv) modify the local ML gradient with a first transmit power level from the third subset of variables and a first artificial noise for data privacy, and (v) transmit the modified local ML model gradient to the relay.

[0039] The system 100 further includes a network node 106 without a direct communication link with the plurality of UEs 102. The network node 106 is configured to (i) transmit, to a relay 104, the ML model and the plurality of variables, (ii) receive, from the relay 104, over a plurality of respective wireless channels respective further modified local ML model gradients including a second artificial noise for data privacy, (iii) aggregate the further modified local ML model gradients based on an AirComp for wireless aggregation, (iv) construct an updated ML model based on the aggregated further modified local ML model gradients, and (v) perform one of (a) transmit the updated ML model to the relay 104 when a convergence or a maximum learning round is reached, or (b) start a next round of the collaborative training.

[0040] The system 100 further includes the relay 104 configured to (i) receive the plurality of variables and the ML model from the network node 106, (ii) extract from the plurality of variables a second subset of variables, (iii) transmit to the plurality of UEs 102 the third subset of variables and the ML model, (iv) receive the modified local ML model gradients transmitted from UEs 102 over a plurality of respective wireless channels, (v) further modify the respective modified local ML gradients with the second subset of variables to add a second artificial noise for data privacy and to scale the received modified local gradients, and (vi) transmit the respective further modified local ML model gradients to the network node 106.

[0041] The plurality of variables can include (i) a first subset of receive scalars for the network node 106 to apply to aggregate the respective further modified local ML model gradients, (ii) the second subset of transmit scalars for the relay 104 to add the second artificial noise and to scale the received modified local gradients, and (iii) the third subset of transmit power levels at the respective UEs 102. Further, before starting the next round of collaborative learning, the network node 106 can be further configured to configure the plurality of variables comprising the first subset of receive scalars for the network node 106, the second subset of transmit scalars for the relay 104, and the third subset of transmit power levels at the respective UEs 102.

[0042] In one example, the system 100 includes K UEs 102, a network node 106 that is an AP, and a relay 104 deployed between the UEs 102 and the AP 106. The AP 106 can be a 5G radio access (e.g., gNodeB (gNB)), or a fixed wireless network node with a large amount of computing resources. In this example, the UEs 102 in this example can transmit data over multi-channels of M frequencies and there is no direct link between the UEs 102 and the AP 106.

[0043] As shown in Figure 1, the architecture of system 100 can implement AirComp, relay 104, DP, and multi-channel transmission, and may address the following: - Data privacy during federated learning in a hostile environment, for example;- Federated learning model training accuracy and effectiveness, especially when a direct link between the UEs 102 and the AP 106 may not be good or is lacking; and / or - Improvement of communication efficiency with AirComp and enhancement of a communication link by multi-channel transmission.

[0044] For example, model training accuracy and data privacy may be enhanced based on multiple UEs 102 that rely on the AP 106 to collaboratively train an AI / ML model (collectively referred to herein as an “ML model”) in adverse conditions. In other words, the UEs 102 in this example are unable to directly connect to the AP 106. Federated learning, which refers to a specific category of distributed ML, can be used to train a ML model using decentralized data residing on the UEs 102 (e.g., mobile phones). Optimization and design of system 100 may result in increased ML model accuracy and data privacy.

[0045] System 100, thus, can enable relay 104 assisted private federated learning in a multi- channel scenario including the following: (1) fast ML model aggregation based on use of AirComp as the multiple access strategy to try to achieve an efficient utilization of the wireless spectrum and reduction of transmission delay; (2) quality of communication links may be enhanced by deploying relay 104. Relay 104 may act as an advanced communication and cost-effective technique to combat the effects of deep channel fading and blockage in the wireless systems; and / or (3) DP can be employed as a mathematical framework to try to guarantee data privacy, using two steps of artificial noises added at the UEs 102 and at the relay 104.

[0046] System 100 optimization for the private federated learning may be accomplished based on AirComp. An objective can include to minimize an average optimality gap under three constraints: a power constraint at each UE 102, a power constraint at the relay 104, and the ( ^^, ^^)- DP constraint at each UE 102.

[0047] As discussed further herein, system 100 optimization may be split into N (number of the learning rounds) separated optimization problems to address non-convexity of optimization at each learning round. At each learning round, the optimization variables can be determined by solving three minimization problems as discussed further herein. At the AP 106, and in each learning round n, three sets of optimization variables are obtained: (1) The received scalars at the AP 106: ^ ^^^^^^ ^, (2) the transmit scalars at the relay 104: ^ ^^^^^^,^, ^^^^^ଶ,^^, and (3) the transmit power atthe UEs 102:^^^^ ^^^ ^ ^^^^^, ^^, ^^^^, ^^ ^.

[0048] analysis and validation of convergence performance is discussed herein from the view of an average optimality gap. For example, privacy performance is derived based on ( ^^, ^^)-DP; and overall system 100 optimization to minimize the average optimality gap iterativelyperformed during learning rounds is analyzed and shows that the system 100 outperforms other approaches.

[0049] Examples include relay 104 assisted federated learning in a multi-channel environment with AirComp for wireless data aggregation.

[0050] Examples further include DP to protect data sharing between the UEs 102 and an AP 106. Gaussian noises are added at the UEs 102 and the relay 104 to enhance privacy in the multi- channel scenario.

[0051] In some examples, a direct link between the UEs 102 and the AP 106 is not accessible. Relay 104 is used to combat channel fading and, thus, enhance the quality of communication links.

[0052] Examples further include performing gradient aggregation based on the AirComp in the multi-channel configuration.

[0053] AirComp may allow fast data aggregation and computation resource reduction in the multi-channel configuration.

[0054] As previously referenced, based on inclusion of operations for collaboratively training a ML model based on a relay assisted federated learning with over-the-air computation and data privacy, an area in a network having restricted coverage can be accessed to enable collaborative federated learning. Moreover, some operations of the present disclosure may be performed by / with UEs having a wide range of capabilities. As a consequence of the operations, one or more of the following technical advantages may be provided: ^ Improved prediction error and low optimality gap: Improved learning performance due to using optimized resource allocation and multi-channel transmission. ^ Addressing a non-convex optimization problem: It can be challenging to obtain an optimal outcome for non-convex problems. Based on use of an alternating minimization approach, non-convexity may be addressed. ^ Enhanced privacy: Operations of examples include a larger privacy level,. As a consequence, a higher learning accuracy may be achieved in comparison to fixed allocation schemes and / or a single channel scheme, for example. ^ Increased leaning accuracy as the number of UEs increases: The ML model aggregation may be improved with an increase in the number of UEs based on AirComp. ^ Optimality gap may be signal-to-noise (SNR) dependent: a higher SNR may yield better channel quality, which may lead to higher accuracy of the aggregation. ^ Vertical Federated Learning: Operations of examples herein may be extrapolated to vertical federated learning where features are heterogeneously distributed across UEs. Forexample, operations of examples herein may be suitable in the context of healthcare where data privacy is of high importance. ^ Application to tactical communications: Operations of examples herein may realize tactical communication over conventional wireless networks such as 5G, while ensuring a reliable level of privacy. ^ Cloud based implementation as a service: The system and operations of examples herein may be generalized to various types of collaborative computing applications as a service, such as privacy critical applications (e.g., healthcare). ^ Generalization and Application Specific: The system and operations of examples herein may be deployed based on any type of existing AI / ML process and deployed in a federated learning environment. ^ Solution during a catastrophe: The system and operations of examples herein may be deployed / used during a catastrophe since the system and operations can be deployed in adverse conditions. ^ AirComp may improve energy efficiency: The system and operations of examples herein may allow fast data aggregation; and improve energy efficiency as less resources may be used. For example, when a channel condition is stationary, operations may skip the system optimization process (e.g., in some learning rounds) which may result in computation resource saving.

[0055] In one example, a cluster of K UEs 102 are involved in an application to train a ML model in a collaborative manner. In this example, the communication link between the UEs 102 and the AP 106 cannot be established due to various conditions (e.g., a remote mining site, blockage, deep fading, drastic weather conditions, accident, catastrophe, etc.). A dedicated relay 104 is integrated into the system 100 to forward the data from the UEs 102 to the AP 106 and vice versa over wireless channels.

[0056] Figure 2 is a schematic diagram of an overview of a relay assisted private federated learning system 100 in accordance with some examples of the present disclosure. The set of UEs 102a-102k is denoted by ^^ ൌ ^1 ,2, … , ^^^; and there are M channels. The set of channels is denoted by ℳ ൌ^1 ,2, … , ^^^, andis a frequency channel.

[0057] A first use case in this system example is a mining industrial environment. In this example use case, 5G connectivity supports mine operators by integrating safe and smart unmanned mines. The mining industrial site is equipped with a 5G slicing, delivering ultra-high- definition video and sub-millisecond latency. As the mining site is strategic and in a remote location (e.g., a crushing area), the mining site integrates relay 104 (e.g., an operation center) toallow connection to the closest network node 106 (e.g., AP or parameter server). The UEs 102 in this example are unmanned vehicles and can participate in a federated learning, offering faster automation, optimizing applications and data application such as: analyzing waiting and material unloading times. System developments in this example are made using a powerful security system to fight cyberattacks and to protect private mining operation.

[0058] A second use case in this system example is federated learning in healthcare for researchers to access robust and numerous datasets, while maintaining patient privacy.5G network slicing is included to generalize operations to a broader population. Remote populations or medical centers are included where a connection to AP 106 is not possible. Relay 104, in this example, helps to implement federated learning to avoid cybersecurity vulnerabilities along with potential violations of patient privacy.

[0059] A third use case in this system example is for a natural disaster. In this example, operations are activated in a natural disaster scenario. Due to deep fading and blockage, the system 100 is activated so that AP 106 in the disaster area is transparently converted into a relay 104. This can allow UEs 102 to train a ML model (e.g., a vision or pollution ML model) in a federated learning environment with high privacy. Such a system 100 may avoid routing humans into affected areas or may result in efficient use of emergency capabilities.

[0060] Various examples herein include federated learning ML model training in a wireless and adverse communication scenario, such as to address or minimize data privacy leakage; accelerate ML model aggregation at the AP 106; and / or efficiently utilize resources. An adverse communication scenario herein refers to a network where a direct link between the UEs 102 and the AP 106 is not available; and relay 104 is used to assist the communication link.

[0061] Federated learning is now discussed further in the context of Figure 2. Given security concerns, federated learning may be expected to provide increased privacy. However, during transmission of gradients from UEs 102 to the AP 106, for example, advanced cyber techniques can be used to retrieve data.

[0062] In the example system in Figure 2, differential privacy can be used to protect data and avoid or minimize privacy leakage. Additionally, wireless data aggregation may be achieved using AirComp.

[0063] A local dataset at UE 102 ^^ ^^ ^^ is denoted by ^^^ൌ^^^^, ^^^^^ೖ^ୀ^, where ^^^is the ^^-th data sample, ^^^is the associated ground-truth label, and ^^^of ^^^. The local loss function at UE 102 k is given by equation (1) below: ^^^^ ^^^ ൌ^∑^௨,௩^ఢ ^^ೖ^^^ ^^; ^^, ^^^ , (1)

[0064] where ^^ ^^ ^^ௗis the ML model vector with a dimension ^^, ^^^ ^^; ^^, ^^^ is the sample- wise loss function measuring the prediction error of the ML model vector ^^ for sample ^^, with respect to the label ^^. The global loss function can be expressed as follows in equation (2): ^^^ ^^^ ൌ^^^^^∑^^ୀ^^^^^^^^ ^^^ , (2)

[0065] of the global dataset ^^ ൌ⋃^^ୀ^^^^. In this example, the size ofis identical, so that ^^^ൌ ^^.

[0066] An objective of the learning process is to minimize the global loss function, which can be written as follows in equation (3): ^^∗ൌ argmin ^^^ ^^^ . (3) ఠ

[0067] The problem defined in equation (3) is addressed in a distributed manner in the context of federated learning. The learning process has N learning rounds in total. At the ^^-th learning round, the network node 106 broadcasts the global model ^^^^^to UEs 102 in downlink. Assuming each UE 102 k receives the ML model vector without based on the local dataset and thereceived ML model, the respective UEs 102 calculate a local gradient 202 as follows in equation (4): ∇^^^൫ ^^^^^൯ ൌ^^ೖ∑^௨,௩^ఢ ^^ೖ ∇ ^^^൫ ^^^^^; ^^, ^^൯. (4)

[0068] over the wireless channels. The true global gradient ∇ ^^^൫ ^^^^^൯ can be expressed as follows in equation (5): ∇^^^൫ ^^^^^൯ ൌ^^^^^∑ ^^ୀ^ ^^^∇ ^^^^^^^. (5)

[0069] UEs 102, the network node 106 obtains an estimated global gradient ∇^^^^൫ ^^^^^൯. Based on the gradient descent method, the global ML model is updated as follows in equation (6): ^^^^ା^^ൌ ^^^^^െ ^^∇^^^൫^^^^^൯, (6)

[0070]

[0071] A training process in this example is iteratively executed until a convergence condition is reached or the maximal number of rounds N is reached. The process of uploading of the local gradient 202 may expose data to some privacy leakage issues (e.g., disclosure of dataset statistics). Thus, differential privacy is included to protect the data privacy (e.g., ensure privacy).

[0072] In some scenarios, the network node 106 may attempt to obtain the UEs 102 information by observing the signal vector ^^. Based on two neighboring global datasets ^^ and ^^ᇱ,for the test variable ^^, the two probabilities Pr^ ^^| ^^^ and Pr^ ^^| ^^ᇱ^ are used in this example to characterize the differential privacy.

[0073] || ^^^െ ^^ଶ||^denotes the cardinality of set difference between two sets ^^^and ^^ଶ. It is assumed in this example that ^^ ൌ⋃^^ୀ^^^^and ^^ᇱൌ⋃^^ୀ^^^^ᇱare two neighboring global datasets. Thus, || ^^ െ ^^ᇱ|| ൌ 1 for UE 102 k, and || ^^ െ ^^ᇱ|^^ ^ ^ ^|^ൌ 0,∀ ^^ ് ^^.

[0074] For any ^^ ^ 0 and ^^ ∈ ^0, 1^, a protocol is ( ^^, ^^)-differentially private, when there isthe following inequality as shown below in equation (7):Pr^ ^^| ^^^ ^ ^^ఌ Pr^ ^^| ^^ᇱ^ ^ ^^ . (7)

[0075] For ( ^^, ^^)-DP, from equation (7), the differential privacy loss is defined as the log- likelihood ratio as follows in equation (8): ℒ^୰ᇲ^ ^^^ ൌ ^൫ ^^ห ^^൯^^, ^^^ ^^^୰^ ^^^ ^^ᇱ^. (8)

[0076] A. Roth, “The algorithmic foundation of differentialprivacy”, Sc., vol.9, no.3-4, p.211-404 (August 2014), for any twoneighboring datasets ^^ and ^^ᇱ, ( ^^, ^^)-DP can guarantee the following as shown in equation (9):Pr ^|ℒ ^^, ^^ᇲ^ ^^^|^ ^^^ ^ 1 െ ^^ . (9)

[0077] For a private federated learning, a smaller ^^ indicates a higher privacy guarantee.

[0078] This example further includes three assumptions on the loss function to achieve expected results.

[0079] First assumption (smoothness): The global loss function ^^^ ^^^ is Lipschitz continuous with a constant ^^ ^ 0, meaning as follows in equation (10): ^^^ ^^^ ^ ^^^ ^^ᇱ^ ^ ∇ ^^^ ^^ᇱ^்^ ^^ െ ^^ᇱ^ ^ఒ ᇱ ଶ ᇱ ௗଶ ∙ ‖ ^^ െ ^^ ‖ ,∀ ^^, ^^ ∈ ℝ . (10)

[0080] value of the global loss function ^^^ ^^^, then it satisfies the PL condition with a non-negative constant ^^ as shown below in equation (11): ‖∇ ^^^ω^‖ଶ^ 2 ∙ ^^ ∙^^^^^^^െ ^^∗^. (11)

[0081] loss function’s gradient grows faster than a quadratic function when it is away from the optimal value.

[0082] Third assumption (Bounded Gradient): At the ^^-th learning round, given a data sample ^ ^^, ^^^ for user ^^, the Euclidian norm of the gradient of the loss function ^^൫^^^^^; ^^, ^^൯is upper bounded by a constant ^^^^^as shown in equation (12):ฮ∇ ^^൫^^^^^; ^^, ^^൯ฮ^ ^^^^^ . (12)

[0083] The communication medium in this example in Figure 2 is wireless multiple access channel. That is, the communication medium between the UEs 102 and the network node 106 is a wireless channel. In the uplink transmission, AirComp is used as the multiple access strategy. At the n-th learning round, UEs 102 communicate with the network node 106 with the aid of the relay 104 in a time slot. Each time slot is further divided into two phases: a first phase in uplink and a second phase in uplink.

[0084] During the first phase in uplink, to preserve the privacy, respective UEs 102 k transmit a noisy local gradient 200, 202 on channel m to the relay 104 as shown in equation (13): ^^^^^^,^ൌ ^ ^^^^^ ^^^^,^^^^^ ^ ^^^^^ ^^^^,^^^^,^,∀ ^^ ∈ ^^, ^^ ∈ℳ , (13) [0085 m for the local gradient ^^^∇ ^^ ^^^^൯ the distribution of ^^^0,ௗallocated transmit power for AN ^^^^^,^. Accordingly, the power constraint102 k is given by equation (14): ∑ெ^ୀ^ ^^^^^^^,^ ൫^^^^^൯ଶ^ ^^^^^^,^ ^^ ^^^^^௫ , (14)

[0086] for a UE 102

[0087] The received signal at the relay 104 on channel m is given by equation (15): ^^^^^ൌ ∑^ ^^^ ^^^ ^^^^,^ ^ୀ^ℎ^,^^^^,^^ ^^^,^, ^^ ∈ℳ, (15)

[0088] UE 102 k and the relay 104 on m, and ^^^^^^,^is the additive white gaussian noise (AWGN) vector at the relay 104 on channel m ఙమof ^^^0,ௗ^^^.

[0089] During in uplink, the relay 104 scale^^^s the received signal ^^^,^on channel m by the transmit scalar ^^^^^ ^^^^,^and adds an artificial noise ^^^,^with the distribution of ^^ 0,^ௗ ^^^. Then the transmit signal at the relay 104 on channel ^^follows in equation (16) ^^^^^ൌ ^^^^^^^^^^^ ^^^^^^ ^^^^^^,^ ^,^ ^,^ ଶ, ^,^, ^^ ∈ ℳ , (16)

[0090] ^^

[0091] Finally, the power constraint at the relay 104 is as follows in equation (17): ^^^ ଶ ^^^ ଶ ^ ^ ଶ ^ ^ ^ ^ ଶ∑ெ^ ^^ ^ ൬∑^^ℎ ^ ^ ^^^൫ ^^^^^൯ ^ ^^^^ ^ ^^ଶ^ ^ ^ ^^^^ ^^^^௫,

[0092] at the AP 106 on channel ^^ is written as equation (18):^^^^^ൌ ℎ^^^ ^^^ ^^^^^,^^^^,^^ ^^^,^, ^^ ∈ ℳ , (18)

[0093] ^^^ ାbetween the relay 104 and the AP 106channel ^^ vector at AP 106 on channel ^^, with the distribution మ^^^0,ఙௗ ^^^.After receiving the signal ^^^^^^ from the relay 104, the network node 106 applies thereceived scalars ^ ^^^^^^ to obtain the est^^^ ^ ^ ^^^^ imated global gradient update as: ^^ ൌ^∑^ୀ^^^^, which is detailed as follows in equation (19):^^^^^ൌ^ ∑ெ^^^^^ ^^^^^ୀ^ ^^^^. (19)

[0095] of operations executed at a network node 106 (e.g., an AP or a, a and respective UEs 102 according to some examples.

[0096] Operations 300, 306, 312, and 318 are receive operations. In operation 300, as shown by the uplink arrow from operation 311 of the relay 104 to operation 300 of the network node 106, the network node 106 receives from relay 104 the noisy local gradients. In operation 306, as shown by the downlink arrow from operation 304 of network node 106 to operation 306 of relay 104, relay 104 receives from network node 106 optimization variables and the global ML model. In operation 312, as shown by the uplink arow from operation 322 of UE 102 to operation 312 of relay 104, relay 104 receives from respective UEs 102 noisy local gradients. In operation 318, as shown by the downlink arrow from operation 310 of relay 104 to operation 318 of respective UEs 102, respective UEs 102 receive from relay 104 the optimization variables and the global ML model.

[0097] Operations 302, 308, 310, 314, and 320 are processing operations. In operation 302, network node 106 optimizes the optimization variables in system 100 based on aggregating and obtaining the estimated global gradient, updating the global ML model, and checking for convergence. In operation 308, relay 104 extracts relay related optimization variables and keeps the remaining optimization variables. In operation 314, relay 104, multiplies the signal received in operation 312 with a designed transmit scalar; and adds artificial noise with the designed transmit scalar. In operation 320, respective UEs 102 compute the local gradient using the global ML model and a local dataset; multiply the computed local gradient with a designed transmit power; and add artificial noise with the designed transmit power.

[0098] Operations 304, 310, 316, and 322 are transmit operations. In operation 304, network node 106 transmits the global ML model and optimization variables to relay 104. In operation 310, relay 104 transmits the remaining optimization variables and the global ML model to UEs 102. Inoperation 311, relay 104 transmits the noisy gradients to network node 106. In operation 322, respective UEs 102 transmit data (that is noisy local gradient) to relay 104.

[0099] Convergence at network node 106 is now discussed further. At the n-th learning round, the estimated global gradient ^^^^^can be decomposed as follows in equation (20): ^^^^^ൌ^ ெ ^^^ ^^^ ^^^ ^^^^∑^ୀ^^^^^^^,^ℎ^,^ ℎ^,^ ^^ ^^^^^ ^^^^,^ ^^^ ^ ^ ^^^^^ ^^^^,^ ^^^,^ ^ ^

[0100] data from UEs 102, the secondoptimality gap between the function ^^^ ^^^ேା^^^ value after N learning rounds and the optimal function value ^^^∗^. When the first, second, and third assumptions discussed herein hold, then the average is upper boundedby the following equation (21): ^^^ ^^൫ ^^^ ^^^1^൯൧ െ ^^∗^^1 െ ^^ ^^^ே൫ ^^^ ^^൫ ^^^1^൯൧ െ ^^∗൯ ^∑ே^ୀ^^1 െ ^^ ^^^ேି^^ఒఓమ ఛ ଶଶ^ ଶ|^^ ^^ െround.To achieve the ( ^^, ^^ )-DP, the optimization variables ^ ^^^^^ ^^^ ^^^ ^^^ ^^^^,^, ^^^,^, ^^^, ^^^,^, ^^ଶ,^^ should be properly designed. Therefore, to guarantee the ( ^^, ^^)- 102, the followingcondition is satisfied as shown in equation (22): ଶ ଶ √ଶ ∑ಾ^సభ ^^^^ ^^^^^ ఎ^^^^ ଶൌparticularly, to minimize the average optimality gap in equation (21), under the power constraint in equation (14) at respective UEs 102, the power constraint in equation (17) at the relay 104 and the ( ^^, ^^ )-DP constraint in equation (22), at respective UEs 102. Moreover, the optimization variables ^ ^^^^^, ^^^^^, ^^^^^, ^^^^^, ^^^^^are jointly optimized. Accordingly, a first optimization problemequation (23):^^^ ^^^m^^in^ ^ ^ ^ ^∑ே^ୀ^^1 െ ^^ ^^^ି^^ఒఓమ^ఛ| ^^ ^^ଶെ ^^|^∆^^^(23) ^^ೖ,^ , ^ೖ,^ , ^ ^ ^ ଶ ଶ^ , ^భ,^ , ^మ,^ ^in the objective^^^^∆^^^ൌ1^^^ ^^^ ^^^ ^ ^^^ 2^^, ^ ^^ െ 1^ ^ ^^^ ^^^^2^^2 ^^ ^^, ^^^^It can be optimizationproblems to minimize ∆ at each learning round n. Each optimization problem to minimize ∆^^is non-convex and challenging to solve. Thus, in this example, an alternating minimizationapproach is used to address the non-convexity.

[0106] A first optimization step is the optimization of the receive scalars at the network node 106: ^ ^^^^^^ ^

[0107] At each iteration of minimizing ∆^^^for the ^^ -th learning round, given theoptimization variables^^^^^^^,^, ^^^^^^,^, ^^^^^^,^, ^^^^^ଶ,^ ^, the receive scalars^^^^^^^^at the network node 106are optimized basedproblem as follows.

[0108] min^^^^^^∆ (24) ^^^^^^ ^^ ^^ ^^ ^^. ^22^,∀ ^^ ∈ ^^. (24-i)

[0110] In the second problem, ∆^^^is convex with respect to ^ ^^^^^^ ^ . The constraint in EQ. (24-i) is non-convex with respect to ^ ^^^^^^ ^. The second problem in this example is solved in three stages: ^ Stage 1: Transformation of the constraint to derive a new constraint using the successive convex approximation (SCA) method. A method can be used as described in Beck, A. Ben- Tal, and L. Tetrusahvili, “A sequential parametric convex approximation method with applications to non-convex truss topology design problems”, J. Glob. Optimization, vol. 47, no.1, pp.29-51 (2010).^ Stage 2: Derive a new convex optimization problem, described further herein as a third problem (Problem 3). ^ Stage 3: Solve the new convex problem (Problem 3) using the existing method.

[0111] The third problem (Problem 3) can be defined as:min∆^^^^(25) ^^^^^^

[0112] 0,∀ ^^ ∈ ^^, (25-i)

[0113] of ^^^^^ଶ,^^^ ^^^^^ is given byெ^^ଶ,^^^ ^^^^^^^^ ൌ√2 ^௧^^ଶ-1Process 1: Optimizing Receive Scalars ^ ^^^^^^ ^ 1 Set t = 0, and initialize ^^^^^^,^, ∀ ^^ ∈ ^^; 2 repeat 3, 4, 5 3 t = t +1 4Replace^^^^^^ ^^^^,௧^with^ ^^^^ ^^^௧ൠat the (t – 1)-th iteration;5 Obtain the solutions ^ ^ ^^^^^^ ^ ^^௧ൠ for Problem 3; 6until ห ^ഥ^^௧^െ ^ഥ^^௧ି^^ห ^ ^^^;Output: The converged solutions for ^ ^^^^^^ ^

[0115] A second optimization step is the optimization of transmit scalars at relay 104: ^ ^^^^^ ^ ^^^, ^^^^.

[0116] At each iteration of minimizing ∆^^^for the ^^ -th learning round, given the optimization variables ^ ^^^^^ ^^^ ^^^ ^^^ ^ ^^,^, ^^^,^, ^^^^, the scalars ^ ^^^^,^, ^^ଶ,^^ at the relay 104, in this example, are optimized based on Problem 4 below: ^^ ^^ ^^ ^^^^^^ ^^^ ^^^^,^ , ^^ଶ,^ ^∆toof the item െ^ΓΘ^^^. Thus, a SCA method is employed. Similar toStep 1, three stages are to iteratively solve the following Problem 5: ^^^^^^^^, ^^^^^ ^^^^^∆^^ ^^ ^^. ^17^ (26)^^ ^ 0,∀ ^^ ∈ ^^,

[0118] where the expressions of ^^^^^ ^^^ସ,^^^ ^^^,^, ^^ଶ,^^^ is given by ^^ ^^ ^^^^ ^^^ସ,^^^,^, ^^ଶ,^^^ ^ଶ

[0119] A detailed procedure of optimizing ^ ^^^^^ ^^^^,^, ^^ଶ,^^ can be performed in like Process 1 described herein.

[0120] A third optimization step is the optimization of the transmit power at UEs 102: ^ ^^^^^, ^^^^^.

[0121] At each iteration of minimizing ∆^^^for the ^^ -th learning round, given the optimization variables ^ ^^^^^ ^^^ ^^^ ^^^ ^^^^ , ^^^,^, ^^ଶ,^^ , the transmit power ^ ^^^,^, ^^^,^^ at UEs 102 can be optimally determined Problem 6:^^^^^^ ^ ^ ^^^^,^ , ^^^^,^ ^∆^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^14^,∀ ^^ ^^^^. ^^. ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^. ^17^^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^ ^^

[0122] Problem 6 is non-convex with^. After conducting a changeof variables ( ^^^^^ൌ ^ ^^^^^^,^) and re-new optimization problem given in Problem 7 below: ^^ ^^ ^^ ^^^^^^ ^^^ ^^^^,^ , ^^^,^ ^∆tothose of skill in the art.

[0124] An overall example process of solving Problem 1 in an iterative manner is summarized below in PROCESS 2: Process 2: Process for Solving Problem 1 1 Set l = 0, and initialize optimization variables ^ ^^^^^ ^^^ ^^^ ^^^ ^^^^ , ^^^,^, ^^ଶ,^, ^^^,^, ^^^,^^; 2 repeat 3, 4, 5, 6 3 l = l +1 4 Solve Problem 3 to obtain ^ ^^^^^^ ^; 5 Solve Problem 5 to obtain ^ ^^^^^ ^^^^,^, ^^ଶ,^^; 6 Solve Problem 7 to obtain ^ ^^^^^ ^^^^,^, ^^^,^^ 7untilห^^^^^ െ ^^^^ି^^ห^ ^^ଶor l ^ ^^^^௫;Output: The converged solutions

[0125] Figure 4 is a flowchart of an example training process. An application 400 is located in the cloud, for example, and requests training of a ML model, using data distributed across several UEs 102. In this example, the UEs 102 do not have direct access to network node 106 (e.g., AP 106 in this example). A relay 104 is provided to allow connections between the AP 106 and the UEs 102. The AP 106 launches the federated training as follows.

[0126] In step 0, Application 400 sends a request to AP 106 for federated learning training. The application 400 also specifies a UE 102 cluster and a dataset.

[0127] The AP 106, in step 1, gathers the parameters for the request (e.g., list of available UEs, ML model type, etc.), sets an initial learning round, obtains network parameters (e.g., from the relay 104), initializes the optimization variables (e.g., receive scalars at the AP 106, transmit power at respective UEs 102, and the transmit scalars at the relay 104), and initializes the global ML model.

[0128] In step 2, AP 106 transmits the optimization variables^^^^^^ ^^^ ^^^ ^^^^,^, ^^^,^, ^^^,^, ^^ଶ,^ ^and the global ML model ^^^^^to the relay 104 at the n-th learning

[0129] The relay 104, in step 3, receives the transmitted data from the AP 106. The relay 104 extracts its optimization variables ^ ^^^^^^,^, ^^^^^ଶ,^^. Relay 104 transmits the remaining optimization variables^^^^^^^,^, ^^^^^^,^ ^together^^^ML model ^^ to UEs 102.

[0130] In step 4, respective UEs 102 receive theML model ^^^^^together with the optimization variables ^ ^^^^^ ^^^^,^, ^^^,^^. Respective UEs 102 then uses theML model ^^^^^and its local dataset to train and obtain the local model (gradient): ^^^^^^ ൌ ∇ ^^^൫ ^^^^^൯.

[0131] Respective UEs 102, in step 5, allocate the corresponding power ^^^^^^,^to the local gradient ^^^^^. The respective UEs 102 also add the^^^^ designed artificial noise ^^^,^with its corresponding power ^^^^^^,^to the local gradient. Then, the resulting signal (local gradient and noise) is sent to the relay 104: ^^^^^ൌ ^ ^^^^^^^^^^^ ^ ^^^^^^^^^^^^ ∈ ^^ ∈

[0132] In step 6, relay(local gradient and noise): ^^^^^ൌ ℎ^^^^^^^^^ ^^^^^on ea^^^^^,^ ^,^ ^,^ch channel. The relay 104 multiplies the received signal ^^^with thescalar ^^^^^and adds an artificial noise ^^^^^^,^with thescalar ^^^^^. The relay 104^^^ ^^^ ^^^ ^^^ ^^^ଶ,^sends ^^ ൌ ^^ ^^ ^ ^^ ^^ m to the AP 106.

[0133] The AP 106, in step 7, receives the signal ^^^^^^ on channel m and applies ^ ^^^^^^ ^ to aggregate the data: ^^^^^ൌ^^∑ெ ^^^ ^^^^ୀ^^^^^^^. The AP the updated global ML model^^^^ା^^. At this the AP 106 checks if the maximum learning round is reached, or ifconvergence is reached. If convergence is reached, or if the maximum learning round is reached, then the AP 106 will send the global ML model back to the application 400 and the training process will end.

[0134] In step 8, AP 106 computes the optimization variables ^ ^^^^ା^^^,^, ^^^^ା^^^,^, ^^^^ା^^^,^, ^^^^ା^^ଶ,^, ^^^^ା^^^^. The AP 106 starts the next learning round and therefore carried out iteratively until the ML model converges or the

[0135] Further in step 8, and for each learning round, a system optimization process is performed. In this example, channel state information (CSI) is available and the channel is not stationary. In other words, the CSI changes at each learning round.

[0136] Alternatively, the channel is stationary and does not change much over several learning rounds. In this alternative, step 8 can be avoided during these rounds. The method of this example can use a process discussed in Adriana Dapena et al, “Detection of Channel Variation to Improve Channel Estimation Methods”, Circuits Systems Signal Processing 33(8): 2605-2623 (2014), for example, to monitor the CSI. The CSI is retrieved from the relay 104 and UEs 102. If the variation of CSI is less than a threshold, then the system optimization process is not executed. The AP 106 only transmits the global ML model to the relay 104. The optimization variables from the previous learning round are still valid and, therefore, still can be used in the current learning round. This may save computation resources, reduce the latency, and ,thus, reduce training time.

[0137] As shown, Figure 4 includes uplink arrows from step 5 to step 6; from step 6, to step 7; and from the decision on convergence to step 0 are for uplink. Figure 4 further includes downlink arrows from step 2 to step 3; and from step 3 to step 4. Figure 4 further includes arrows for internal processing from step 0 to step 1; from step 1 to step 2; from the decision on convergence to step 8; and from step 8 to step 2.

[0138] Operations of a network node can be performed by the network node 1000 of Figure 10. Operations of the network node are now discussed with reference to the flow chart of Figure 5 according to some embodiments of the present disclosure. For example, modules may be stored in memory 1004 of Figure 10, and these modules may provide instructions so that when the instructions of a module are executed by respective network node processing circuitry 1002, network node 1000 performs respective operations of the flow chart.

[0139] In some embodiments, a method is performed by a network node without a direct communication link with a plurality of UEs to collaboratively train a ML model based on a relay assisted federated learning with AirComp and data privacy. The method includes receiving (operation 506 in Figure 5) a plurality of further modified local ML model gradients from a relay over respective wireless channels of a plurality of wireless channels. A further modified local ML model gradient includes a local ML model gradient modified by a respective UE with a first artificial noise for data privacy which is further modified by the relay with a second artificial noise for data privacy. The method further includes aggregating (operation 508 in Figure 5) the plurality of further modified local ML model gradients based on an over-the-air computation for wireless aggregation. The method further includes constructing (510) an updated ML model based on the aggregated further modified local gradients; and performing (512) one of (i) transmitting the updated ML model to the relay, or (ii) configure a plurality of variables and start a next round of the collaborative training.

[0140] In some embodiment, the data privacy includes a differential privacy to protect data shared between the network node and the plurality of UEs via the relay.

[0141] The first artificial noise can include Gaussian noise added at respective UEs and the second artificial noise can include Gaussian noise added at the relay to enhance data privacy in the plurality of wireless channels.

[0142] In some embodiments, a direct communication link between the network node and the plurality of UEs is not accessible and the relay is used at least to (i) address wireless channel fading and / or (ii) assist communication between the relay and the plurality of UEs and between the relay and the network node.

[0143] In some embodiments, the method further includes transmitting (operation 504 in Figure 5), to the relay, the ML model and the plurality of variables.

[0144] The plurality of variables can include (i) a first subset of the plurality of variables including receive scalars for the network node to apply to aggregate the respective further modified local ML model gradients, (ii) a second subset of the plurality of variables including transmit scalars for the relay to add the second artificial noise and to scale the received modified local gradients, and (iii) a third subset of the plurality of variables including transmit power levels at the respective UEs.

[0145] Configuring the plurality of variables can include to configure a first subset of the plurality of variables including receive scalars for the network node, configure a second subset of the plurality of variables including transmit scalars for the relay, and configure a third subset of the plurality of variables including power levels at the respective UEs.

[0146] In some embodiments, the method further includes receiving (operation 500 in Figure 5) a request for distributed training of the ML model; and initializing (operation 502 in Figure 5) the ML model including initialization of the plurality of variables including (i) a set of receive scalars at the network node, (ii) a set of transmit scalars at the relay, and (iii) a set of transmit power per UE for a plurality UEs.

[0147] Various operations from the flow chart of Figure 5 may be optional with respect to some embodiments of network nodes and related methods. For example the operations of blocks 500, 502, and 504 may be optional.

[0148] Operations of a relay can be performed by the network node 1000 of Figure 10. Operations of the relay are now discussed with reference to the flow chart of Figure 6 according to some embodiments of the present disclosure. For example, modules may be stored in memory 1004 of Figure 10, and these modules may provide instructions so that when the instructions of a module are executed by respective relay processing circuitry 1002, network node 1000 performs respective operations of the flow chart.

[0149] In some embodiments, a method is performed by a relay to collaboratively train a ML model based on relay assisted federated learning with AirComp and data privacy. The method includes extracting (operation 602 in Figure 6) from a plurality of variables a second subset of variables; and transmitting (operation 604 in Figure 6) to a plurality of UE the ML model and a third subset of variables from the plurality of variables. The method further includes receiving (operation 606 in Figure 6) a modified local ML model gradient from respective UEs over a plurality of respective wireless channels. The modified local ML gradient includes a local ML model gradient modified by a respective UE with a first artificial noise for data privacy. The method further includes further modifying (operation 608 in Figure 6) the respective modified local ML gradients with the second subset of variables to add a second artificial noise for data privacy and to scale the received modified local gradients; and transmitting (operation 610 in Figure 6) the respective further modified local ML model gradients to a network node.

[0150] The data privacy can include a differential privacy to protect data shared between the network node and the plurality of UEs via the relay.

[0151] The first artificial noise can include Gaussian noise added at respective UEs and the second artificial noise can include Gaussian noise added at the relay to enhance data privacy in the plurality of wireless channels.

[0152] In some embodiments, a direct communication link between the network node and the plurality of UEs is not accessible and the relay is used at least to (i) address wireless channel fadingand / or (ii) assist communication between the relay and the plurality of UEs and between the relay and the network node.

[0153] In some embodiments, the method further includes receiving (operation 600 in Figure 6), from the network node, the ML model and the plurality of variables.

[0154] The plurality of variables can include (i) a first subset of the plurality of variables including receive scalars for the network node to apply to aggregate the respective further modified local ML model gradients, (ii) a second subset of the plurality of variables including transmit scalars for the relay to add the second artificial noise and to scale the modified local gradients, and (iii) a third subset of the plurality of variables including transmit power levels at the respective UEs.

[0155] Various operations from the flow chart of Figure 6 may be optional with respect to some embodiments of relays and related methods. For example the operations of block 600 may be optional.

[0156] Operations of a UE can be performed by the UE 900 of Figure 9. Operations of the UE are now discussed with reference to the flow chart of Figure 7 according to some embodiments of the present disclosure. For example, modules may be stored in memory 910 of Figure 9, and these modules may provide instructions so that when the instructions of a module are executed by respective UE processing circuitry 902, UE 900 performs respective operations of the flow chart.

[0157] In some embodiments, a method is performed by a UE without a direct communication link with a network node to collaboratively train a ML model based on a relay assisted federated learning with over-the-air computation and data privacy. The method includes receiving (operation 700 in Figure 7) over a wireless channel, from a relay, the ML model and a third subset of variables from a plurality of variables; and training (operation 702 in Figure 7) the ML model with local data to obtain a local ML model gradient. The method further includes modifying (operation 704 in Figure 7) the local ML gradient with a first transmit power level from the third subset of variables and a first artificial noise for data privacy; and transmitting (operation 706 in Figure 7) the modified local ML model gradient to the relay.

[0158] The data privacy can include a differential privacy to protect data shared between the network node and the UE via the relay.

[0159] The first artificial noise can include Gaussian noise added at the UE to enhance data privacy in the wireless channel.

[0160] In some embodiments, a direct communication link between the network node and the UE is not accessible and the relay is used at least to (i) address wireless channel fading and / or (ii) assist communication between the relay and the UE and between the relay and the network node.

[0161] The plurality of variables can include (i) a first subset of the plurality of variables including receive scalars for the network node to apply to aggregate the respective further modified local ML model gradients, (ii) a second subset of the plurality of variables including transmit scalars for the relay to add a second artificial noise and to scale the modified local gradients, and (iii) the third subset of the plurality of variables including transmit power levels at the respective UEs in a plurality of UEs.

[0162] Figure 8 shows an example of a communication system 800 in accordance with some embodiments.

[0163] In the example, the communication system 800 (also referred to herein as a network) includes a telecommunication network 802 that includes an access network 804, such as a radio access network (RAN), and a core network 806, which includes one or more core network nodes 808. The access network 804 includes one or more access network nodes, such as network node 810a and relay network node 810b (one or more of which may be generally referred to as network nodes 810), or any other similar Third Generation Partnership (3GPP) access node or non-3GPP access point. The network nodes 810 facilitate direct or indirect connection of UE as shown, such as by connecting UEs 812a, 812b, 812c, and 812d (one or more of which may be generally referred to as UEs 812) to the core network 806 over one or more wireless connections.

[0164] Example wireless communications over a wireless connection include transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for conveying information without the use of wires, cables, or other material conductors. Moreover, in different embodiments, the communication system 800 may include any number of wired or wireless networks, computing devices, network nodes, UEs, and / or any other components or systems that may facilitate or participate in the communication of data and / or signals whether via wired or wireless connections. The communication system 800 may include and / or interface with any type of communication, telecommunication, data, cellular, radio network, and / or other similar type of system.

[0165] The UEs 812 may be any of a wide variety of communication devices, including wireless devices arranged, configured, and / or operable to communicate wirelessly with the network nodes 810 and other communication devices. Similarly, the network nodes 810 are arranged, capable, configured, and / or operable to communicate directly or indirectly with the UEs 812 and / or with other network nodes or equipment in the telecommunication network 802 to enable and / or provide network access, such as wireless network access, and / or to perform other functions, such as administration in the telecommunication network 802.

[0166] In the depicted example, the core network 806 connects the network nodes 810 to one or more hosts, such as host 816. These connections may be direct or indirect via one or more intermediary networks or devices. In other examples, network nodes may be directly coupled to hosts. The core network 806 includes one more core network nodes (e.g., core network node 808) that are structured with hardware and software components. Features of these components may be substantially similar to those described with respect to the UEs, network nodes, and / or hosts, such that the descriptions thereof are generally applicable to the corresponding components of the core network node 808. Example core network nodes include functions of one or more of a Mobile Switching Center (MSC), Mobility Management Entity (MME), Home Subscriber Server (HSS), Access and Mobility Management Function (AMF), Session Management Function (SMF), Authentication Server Function (AUSF), Subscription Identifier De-concealing function (SIDF), Unified Data Management (UDM), Security Edge Protection Proxy (SEPP), Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0167] The host 816 may be under the ownership or control of a service provider other than an operator or provider of the access network 804 and / or the telecommunication network 802, and may be operated by the service provider or on behalf of the service provider. The host 816 may host a variety of applications to provide one or more service. Examples of such applications include live and pre-recorded audio / video content, data collection services such as retrieving and compiling data on various ambient conditions detected by a plurality of UEs, analytics functionality, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and surveillance center, or any other such function performed by a server.

[0168] As a whole, the communication system 800 of Figure 8 enables connectivity between the UEs, network nodes, and hosts. In that sense, the communication system may be configured to operate according to predefined rules or procedures, such as specific standards that include, but are not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standard (e.g., 6G); wireless local area network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards (WiFi); and / or any other appropriate wireless communication standard, such as the Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and / or any low-power wide-area network (LPWAN) standards such as LoRa and Sigfox.

[0169] In some examples, the telecommunication network 802 is a cellular network that implements 3GPP standardized features. Accordingly, the telecommunications network 802 maysupport network slicing to provide different logical networks to different devices that are connected to the telecommunication network 802. For example, the telecommunications network 802 may provide URLLC services to some UEs, while providing eMBB services to other UEs, and / or mMTC / Massive IoT services to yet further UEs.

[0170] In some examples, the UEs 812 are configured to transmit and / or receive information without direct human interaction. For instance, a UE may be designed to transmit information to the access network 804 on a predetermined schedule, when triggered by an internal or external event, or in response to requests from the access network 804. Additionally, a UE may be configured for operating in single- or multi-RAT or multi-standard mode. For example, a UE may operate with any one or combination of Wi-Fi, NR and LTE, i.e. being configured for multi-radio dual connectivity (MR-DC), such as E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network) New Radio – Dual Connectivity (EN-DC).

[0171] In the example, the hub 814 optionally communicates with the access network 804 to facilitate indirect communication between one or more UEs (e.g., UE 812c and / or 812d) and network nodes (e.g., network node 810b). In some examples, the hub 814 may be a controller, router, content source and analytics, or any of the other communication devices described herein regarding UEs. For example, the hub 814 may be a broadband router enabling access to the core network 806 via the relay network node 810B for the UEs. As another example, the hub 814 may be a controller that sends commands or instructions to one or more actuators in the UEs. Commands or instructions may be received from the UEs, network nodes 810, or by executable code, script, process, or other instructions in the hub 814. As another example, the hub 814 may be a data collector that acts as temporary storage for UE data and, in some embodiments, may perform analysis or other processing of the data. As another example, the hub 814 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker or other media delivery device, the hub 814 may retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, which the hub 814 then provides to the UE either directly, after performing local processing, and / or after adding additional local content. In still another example, the hub 814 acts as a proxy server or orchestrator for the UEs, in particular if one or more of the UEs are low energy IoT devices.

[0172] The hub 814 may have a constant / persistent or intermittent connection to the network node 810b. The hub 814 may also allow for a different communication scheme and / or schedule between the hub 814 and UEs (e.g., UE 812c and / or 812d), and between the hub 814 and the core network 806 via the relay network node 810B. In some scenarios, UEs may establish a wireless connection with the network nodes 810 via relay network node 810B while still connected via thehub 814 via a wired or wireless connection. In some embodiments, the hub 814 may be a dedicated hub – that is, a hub whose primary function is to route communications to / from the UEs from / to the relay network node 810b. In other embodiments, the hub 814 may be a non-dedicated hub – that is, a device which is capable of operating to route communications between the UEs and relay network node 810b, but which is additionally capable of operating as a communication start and / or end point for certain data channels.

[0173] Figure 9 shows a UE 900 in accordance with some embodiments. As used herein, a UE refers to a device capable, configured, arranged and / or operable to communicate wirelessly with network nodes and / or other UEs. Examples of a UE include, but are not limited to, a smart phone, mobile phone, cell phone, voice over IP (VoIP) phone, wireless local loop phone, desktop computer, personal digital assistant (PDA), wireless cameras, gaming console or device, music storage device, playback appliance, wearable terminal device, wireless endpoint, mobile station, tablet, laptop, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), smart device, wireless customer-premise equipment (CPE), vehicle, vehicle-mounted or vehicle embedded / integrated wireless device, etc. Other examples include any UE identified by the 3rd Generation Partnership Project (3GPP), including a narrow band internet of things (NB-IoT) UE, a machine type communication (MTC) UE, and / or an enhanced MTC (eMTC) UE.

[0174] A UE may support device-to-device (D2D) communication, for example by implementing a 3GPP standard for sidelink communication, Dedicated Short-Range Communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to- everything (V2X). In other examples, a UE may not necessarily have a user in the sense of a human user who owns and / or operates the relevant device. Instead, a UE may represent a device that is intended for sale to, or operation by, a human user but which may not, or which may not initially, be associated with a specific human user (e.g., a smart sprinkler controller). Alternatively, a UE may represent a device that is not intended for sale to, or operation by, an end user but which may be associated with or operated for the benefit of a user (e.g., a smart power meter).

[0175] The UE 900 includes processing circuitry 902 that is operatively coupled via a bus 904 to an input / output interface 906, a power source 908, a memory 910, a communication interface 912, and / or any other component, or any combination thereof. Certain UEs may utilize all or a subset of the components shown in Figure 9. The level of integration between the components may vary from one UE to another UE. Further, certain UEs may contain multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0176] The processing circuitry 902 is configured to process instructions and data and may be configured to implement any sequential state machine operative to execute instructions stored asmachine-readable computer programs in the memory 910. The processing circuitry 902 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, field- programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), etc.); programmable logic together with appropriate firmware; one or more stored computer programs, general-purpose processors, such as a microprocessor or digital signal processor (DSP), together with appropriate software; or any combination of the above. For example, the processing circuitry 902 may include multiple central processing units (CPUs).

[0177] In the example, the input / output interface 906 may be configured to provide an interface or interfaces to an input device, output device, or one or more input and / or output devices. Examples of an output device include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smartcard, another output device, or any combination thereof. An input device may allow a user to capture information into the UE 900. Examples of an input device include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a web camera, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smartcard, and the like. The presence-sensitive display may include a capacitive or resistive touch sensor to sense input from a user. A sensor may be, for instance, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, an optical sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as an input device. For example, a Universal Serial Bus (USB) port may be used to provide an input device and an output device.

[0178] In some embodiments, the power source 908 is structured as a battery or battery pack. Other types of power sources, such as an external power source (e.g., an electricity outlet), photovoltaic device, or power cell, may be used. The power source 908 may further include power circuitry for delivering power from the power source 908 itself, and / or an external power source, to the various parts of the UE 900 via input circuitry or an interface such as an electrical power cable. Delivering power may be, for example, for charging of the power source 908. Power circuitry may perform any formatting, converting, or other modification to the power from the power source 908 to make the power suitable for the respective components of the UE 900 to which power is supplied.

[0179] The memory 910 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disks, optical disks, hard disks, removable cartridges, flash drives, and so forth. In one example, the memory 910 includes one or more application programs 914,such as an operating system, web browser application, a widget, gadget engine, or other application, and corresponding data 916. The memory 910 may store, for use by the UE 900, any of a variety of various operating systems or combinations of operating systems.

[0180] The memory 910 may be configured to include a number of physical drive units, such as redundant array of independent disks (RAID), flash memory, USB flash drive, external hard disk drive, thumb drive, pen drive, key drive, high-density digital versatile disc (HD-DVD) optical disc drive, internal hard disk drive, Blu-Ray optical disc drive, holographic digital data storage (HDDS) optical disc drive, external mini-dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro-DIMM SDRAM, smartcard memory such as tamper resistant module in the form of a universal integrated circuit card (UICC) including one or more subscriber identity modules (SIMs), such as a USIM and / or ISIM, other memory, or any combination thereof. The UICC may for example be an embedded UICC (eUICC), integrated UICC (iUICC) or a removable UICC commonly known as ‘SIM card.’ The memory 910 may allow the UE 900 to access instructions, application programs and the like, stored on transitory or non-transitory memory media, to off-load data, or to upload data. An article of manufacture, such as one utilizing a communication system may be tangibly embodied as or in the memory 910, which may be or comprise a device-readable storage medium.

[0181] The processing circuitry 902 may be configured to communicate with an access network or other relay network node using the communication interface 912. The communication interface 912 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 922. The communication interface 912 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., another UE or a relay network node). Each transceiver may include a transmitter 918 and / or a receiver 920 appropriate to provide network communications (e.g., optical, electrical, frequency allocations, and so forth). Moreover, the transmitter 918 and receiver 920 may be coupled to one or more antennas (e.g., antenna 922) and may share circuit components, software or firmware, or alternatively be implemented separately.

[0182] In the illustrated embodiment, communication functions of the communication interface 912 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short- range communications such as Bluetooth, near-field communication, location-based communication such as the use of the global positioning system (GPS) to determine a location, another like communication function, or any combination thereof. Communications may beimplemented in according to one or more communication protocols and / or standards, such as IEEE 802.11, Code Division Multiplexing Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, transmission control protocol / internet protocol (TCP / IP), synchronous optical networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), and so forth.

[0183] Regardless of the type of sensor, a UE may provide an output of data captured by its sensors, through its communication interface 912, via a wireless connection to a relay network node. The output may be periodic (e.g., once every 15 minutes if it reports the sensed temperature), random (e.g., to even out the load from reporting from several sensors), in response to a triggering event (e.g., when moisture is detected an alert is sent), in response to a request (e.g., a user initiated request), or a continuous stream (e.g., a live video feed of a patient).

[0184] As another example, a UE comprises an actuator, a motor, or a switch, related to a communication interface configured to receive wireless input from a relay network node via a wireless connection. In response to the received wireless input the states of the actuator, the motor, or the switch may change. For example, the UE may comprise a motor that adjusts the control surfaces or rotors of a drone in flight according to the received input or to a robotic arm performing a medical procedure according to the received input.

[0185] A UE, when in the form of an Internet of Things (IoT) device, may be a device for use in one or more application domains, these domains comprising, but not limited to, city, wearable technology, extended industrial application, and healthcare. Non-limiting examples of such an IoT device are a device which is or which is embedded in: a connected refrigerator or freezer, a TV, a connected lighting device, an electricity meter, a robot vacuum cleaner, a voice controlled smart speaker, a home security camera, a motion detector, a thermostat, a smoke detector, a door / window sensor, a flood / moisture sensor, an electrical door lock, a connected doorbell, an air conditioning system like a heat pump, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smart watch, a fitness tracker, a head-mounted display for Augmented Reality (AR) or Virtual Reality (VR), a wearable for tactile augmentation or sensory enhancement, a water sprinkler, an animal- or item- tracking device, a sensor for monitoring a plant or animal, an industrial robot, an Unmanned Aerial Vehicle (UAV), and any kind of medical device, like a heart rate monitor or a remote controlled surgical robot. A UE in the form of an IoT device comprises circuitry and / or software in dependence of the intended application of the IoT device in addition to other components as described in relation to the UE 900 shown in Figure 9.

[0186] As yet another specific example, in an IoT scenario, a UE may represent a machine or other device that performs monitoring and / or measurements, and transmits the results of such monitoring and / or measurements to another UE and / or a relay network node. The UE may in this case be an M2M device, which may in a 3GPP context be referred to as an MTC device. As one particular example, the UE may implement the 3GPP NB-IoT standard. In other scenarios, a UE may represent a vehicle, such as a car, a bus, a truck, a ship and an airplane, or other equipment that is capable of monitoring and / or reporting on its operational status or other functions associated with its operation.

[0187] In practice, any number of UEs may be used together with respect to a single use case. For example, a first UE might be or be integrated in a drone and provide the drone’s speed information (obtained through a speed sensor) to a second UE that is a remote controller operating the drone. When the user makes changes from the remote controller, the first UE may adjust the throttle on the drone (e.g. by controlling an actuator) to increase or decrease the drone’s speed. The first and / or the second UE can also include more than one of the functionalities described above. For example, a UE might comprise the sensor and the actuator, and handle communication of data for both the speed sensor and the actuators.

[0188] Figure 10 shows a network node 1000 in accordance with some embodiments. As used herein, network node refers to equipment capable, configured, arranged and / or operable to communicate directly or indirectly with a UE and / or with other network nodes or equipment, in a telecommunication network. Examples of network nodes include, but are not limited to, computing devices, APs (e.g., radio access points), parameter servers, relay network nodes, base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs) and NR NodeBs (gNBs)), O- RAN nodes or components of an O-RAN node (e.g., O-RU, O-DU, O-CU).

[0189] Base stations may be categorized based on the amount of coverage they provide (or, stated differently, their transmit power level) and so, depending on the provided amount of coverage, may be referred to as femto base stations, pico base stations, micro base stations, or macro base stations. A base station may be a relay node or a relay donor node controlling a relay. A network node may also include one or more (or all) parts of a distributed radio base station such as centralized digital units, distributed units (e.g., in an O-RAN access node) and / or remote radio units (RRUs), sometimes referred to as Remote Radio Heads (RRHs). Such remote radio units may or may not be integrated with an antenna as an antenna integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[0190] Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, multi-standard radio (MSR) equipment such as MSR BSs, network controllers suchas radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi-cell / multicast coordination entities (MCEs), Operation and Maintenance (O&M) nodes, Operations Support System (OSS) nodes, Self-Organizing Network (SON) nodes, positioning nodes (e.g., Evolved Serving Mobile Location Centers (E-SMLCs)), and / or Minimization of Drive Tests (MDTs).

[0191] The network node 1000 includes a processing circuitry 1002, a memory 1004, a communication interface 1006, and a power source 1008. The network node 1000 may be composed of multiple physically separate components (e.g., a NodeB component and a RNC component, or a BTS component and a BSC component, etc.), which may each have their own respective components. In certain scenarios in which the network node 1000 comprises multiple separate components (e.g., BTS and BSC components), one or more of the separate components may be shared among several network nodes. For example, a single RNC may control multiple NodeBs. In such a scenario, each unique NodeB and RNC pair, may in some instances be considered a single separate network node. In some embodiments, the network node 1000 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memory 1004 for different RATs) and some components may be reused (e.g., a same antenna 1010 may be shared by different RATs). The network node 1000 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node 1000, for example GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, Radio Frequency Identification (RFID) or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chip or set of chips and other components within network node 1000.

[0192] The processing circuitry 1002 may comprise a combination of one or more of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and / or encoded logic operable to provide, either alone or in conjunction with other network node 1000 components, such as the memory 1004, to provide network node 1000 functionality.

[0193] In some embodiments, the processing circuitry 1002 includes a system on a chip (SOC). In some embodiments, the processing circuitry 1002 includes one or more of radio frequency (RF) transceiver circuitry 1012 and baseband processing circuitry 1014. In some embodiments, the radio frequency (RF) transceiver circuitry 1012 and the baseband processing circuitry 1014 may be on separate chips (or sets of chips), boards, or units, such as radio units anddigital units. In alternative embodiments, part or all of RF transceiver circuitry 1012 and baseband processing circuitry 1014 may be on the same chip or set of chips, boards, or units.

[0194] The memory 1004 may comprise any form of volatile or non-volatile computer- readable memory including, without limitation, persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (for example, a hard disk), removable storage media (for example, a flash drive, a Compact Disk (CD) or a Digital Video Disk (DVD)), and / or any other volatile or non-volatile, non-transitory device-readable and / or computer-executable memory devices that store information, data, and / or instructions that may be used by the processing circuitry 1002. The memory 1004 may store any suitable instructions, data, or information, including a computer program, software, an application including one or more of logic, rules, code, tables, and / or other instructions capable of being executed by the processing circuitry 1002 and utilized by the network node 1000. The memory 1004 may be used to store any calculations made by the processing circuitry 1002 and / or any data received via the communication interface 1006. In some embodiments, the processing circuitry 1002 and memory 1004 is integrated.

[0195] The communication interface 1006 is used in wired or wireless communication of signaling and / or data between a network node, access network, a relay network node, and / or UE. As illustrated, the communication interface 1006 comprises port(s) / terminal(s) 1016 to send and receive data, for example to and from a network over a wired connection. The communication interface 1006 also includes radio front-end circuitry 1018 that may be coupled to, or in certain embodiments a part of, the antenna 1010. Radio front-end circuitry 1018 comprises filters 1020 and amplifiers 1022. The radio front-end circuitry 1018 may be connected to an antenna 1010 and processing circuitry 1002. The radio front-end circuitry may be configured to condition signals communicated between antenna 1010 and processing circuitry 1002. The radio front-end circuitry 1018 may receive digital data that is to be sent out to other network nodes (e.g., a relay network node) or UEs via a wireless connection. The radio front-end circuitry 1018 may convert the digital data into a radio signal having the appropriate channel and bandwidth parameters using a combination of filters 1020 and / or amplifiers 1022. The radio signal may then be transmitted via the antenna 1010. Similarly, when receiving data, the antenna 1010 may collect radio signals which are then converted into digital data by the radio front-end circuitry 1018. The digital data may be passed to the processing circuitry 1002. In other embodiments, the communication interface may comprise different components and / or different combinations of components.

[0196] In certain alternative embodiments, the network node 1000 does not include separate radio front-end circuitry 1018, instead, the processing circuitry 1002 includes radio front-endcircuitry and is connected to the antenna 1010. Similarly, in some embodiments, all or some of the RF transceiver circuitry 1012 is part of the communication interface 1006. In still other embodiments, the communication interface 1006 includes one or more ports or terminals 1016, the radio front-end circuitry 1018, and the RF transceiver circuitry 1012, as part of a radio unit (not shown), and the communication interface 1006 communicates with the baseband processing circuitry 1014, which is part of a digital unit (not shown).

[0197] The antenna 1010 may include one or more antennas, or antenna arrays, configured to send and / or receive wireless signals. The antenna 1010 may be coupled to the radio front-end circuitry 1018 and may be any type of antenna capable of transmitting and receiving data and / or signals wirelessly. In certain embodiments, the antenna 1010 is separate from the network node 1000 and connectable to the network node 1000 through an interface or port.

[0198] The antenna 1010, communication interface 1006, and / or the processing circuitry 1002 may be configured to perform any receiving operations and / or certain obtaining operations described herein as being performed by the network node or relay network node. Any information, data and / or signals may be received from a UE, another network node, relay network node, and / or any other network equipment. Similarly, the antenna 1010, the communication interface 1006, and / or the processing circuitry 1002 may be configured to perform any transmitting operations described herein as being performed by the network node. Any information, data and / or signals may be transmitted to a UE, another network node (e.g., a relay network node) and / or any other network equipment.

[0199] The power source 1008 provides power to the various components of network node 1000 in a form suitable for the respective components (e.g., at a voltage and current level needed for each respective component). The power source 1008 may further comprise, or be coupled to, power management circuitry to supply the components of the network node 1000 with power for performing the functionality described herein. For example, the network node 1000 may be connectable to an external power source (e.g., the power grid, an electricity outlet) via an input circuitry or interface such as an electrical cable, whereby the external power source supplies power to power circuitry of the power source 1008. As a further example, the power source 1008 may comprise a source of power in the form of a battery or battery pack which is connected to, or integrated in, power circuitry. The battery may provide backup power should the external power source fail.

[0200] Embodiments of the network node 1000 may include additional components beyond those shown in Figure 10 for providing certain aspects of the network node’s functionality, including any of the functionality described herein and / or any functionality necessary to supportthe subject matter described herein. For example, the network node 1000 may include user interface equipment to allow input of information into the network node 1000 and to allow output of information from the network node 1000. This may allow a user to perform diagnostic, maintenance, repair, and other administrative functions for the network node 1000.

[0201] Figure 11 is a block diagram of a host 1100, which may be an embodiment of the host 816 of Figure 8, in accordance with various aspects described herein. As used herein, the host 1100 may be or comprise various combinations hardware and / or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, container, or processing resources in a server farm. The host 1100 may provide one or more services to one or more UEs (e.g., via relay network node 810B).

[0202] The host 1100 includes processing circuitry 1102 that is operatively coupled via a bus 1104 to an input / output interface 1106, a network interface 1108, a power source 1110, and a memory 1112. Other components may be included in other embodiments. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as Figures 9 and 10, such that the descriptions thereof are generally applicable to the corresponding components of host 1100.

[0203] The memory 1112 may include one or more computer programs including one or more host application programs 1114 and data 1116, which may include user data, e.g., data generated by a UE for the host 1100 or data generated by the host 1100 for a UE. Embodiments of the host 1100 may utilize only a subset or all of the components shown. The host application programs 1114 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for multiple different classes, types, or implementations of UEs (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). The host application programs 1114 may also provide for user authentication and licensing checks and may periodically report health, routes, and content availability to a central node, such as a device in or on the edge of a core network. Accordingly, the host 1100 may select and / or indicate a different host for over-the-top services for a UE. The host application programs 1114 may support various protocols, such as the HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.

[0204] Figure 12 is a block diagram illustrating a virtualization environment 1200 in which functions implemented by some embodiments may be virtualized. In the present context,virtualizing means creating virtual versions of apparatuses or devices which may include virtualizing hardware platforms, storage devices and networking resources. As used herein, virtualization can be applied to any device described herein, or components thereof, and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 1200 hosted by one or more of hardware nodes, such as a hardware computing device that operates as a computing device, network node, UE, core network node, or host. Further, in embodiments in which the virtual node does not require radio connectivity (e.g., a core network node or host), then the node may be entirely virtualized. In some embodiments, the virtualization environment 1200 includes components defined by the O-RAN Alliance, such as an O-Cloud environment orchestrated by a SMO 1210 Framework via an O-2 interface.

[0205] Applications 1202 (which may alternatively be called software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) are run in the virtualization environment 1200 to implement some of the features, functions, and / or benefits of some of the embodiments disclosed herein.

[0206] Hardware 1204 includes processing circuitry, memory that stores software and / or instructions executable by hardware processing circuitry, and / or other hardware devices as described herein, such as a network interface, input / output interface, and so forth. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1206 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 1208a and 1208b (one or more of which may be generally referred to as VMs 1208), and / or perform any of the functions, features and / or benefits described in relation with some embodiments described herein. The virtualization layer 1206 may present a virtual operating platform that appears like networking hardware to the VMs 1208.

[0207] The VMs 1208 comprise virtual processing, virtual memory, virtual networking or interface and virtual storage, and may be run by a corresponding virtualization layer 1206. Different embodiments of the instance of a virtual appliance 1202 may be implemented on one or more of VMs 1208, and the implementations may be made in different ways. Virtualization of the hardware is in some contexts referred to as network function virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry standard high volume server hardware, physical switches, and physical storage, which can be located in data centers, and customer premise equipment.

[0208] In the context of NFV, a VM 1208 may be a software implementation of a physical machine that runs programs as if they were executing on a physical, non-virtualized machine. Each of the VMs 1208, and that part of hardware 1204 that executes that VM, be it hardware dedicated to that VM and / or hardware shared by that VM with others of the VMs, forms separate virtual network elements. Still in the context of NFV, a virtual network function is responsible for handling specific network functions that run in one or more VMs 1208 on top of the hardware 1204 and corresponds to the application 1202.

[0209] Hardware 1204 may be implemented in a standalone network node with generic or specific components. Hardware 1204 may implement some functions via virtualization. Alternatively, hardware 1204 may be part of a larger cluster of hardware (e.g. such as in a data center or CPE) where many hardware nodes work together and are managed via management and orchestration (SMO) 1210, which, among others, oversees lifecycle management of applications 1202. In some embodiments, hardware 1204 is coupled to one or more radio units that each include one or more transmitters and one or more receivers that may be coupled to one or more antennas. Radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with the virtual components to provide a virtual node with radio capabilities, such as a radio access node or a base station. In some embodiments, some signaling can be provided with the use of a control system 1212 which may alternatively be used for communication between hardware nodes and radio units.

[0210] Figure 13 shows a communication diagram of a host 1302 communicating via a network node 1304 with a UE 1306 over a partially wireless connection in accordance with some embodiments. Example implementations, in accordance with various embodiments, of the UE (such as a UE 812a of Figure 8 and / or UE 900 of Figure 9), network node (such as network node 808 / 810A of Figure 8 and / or network node 1000 of Figure 10), and host (such as host 816 of Figure 8 and / or host 1100 of Figure 11) discussed in the preceding paragraphs will now be described with reference to Figure 13.

[0211] Like host 1100, embodiments of host 1302 include hardware, such as a communication interface, processing circuitry, and memory. The host 1302 also includes software, which is stored in or accessible by the host 1302 and executable by the processing circuitry. The software includes a host application that may be operable to provide a service to a remote user, such as the UE 1306 connecting via an over-the-top (OTT) connection 1350 extending between the UE 1306 and host 1302, or between UE 1306 and network node 1304 over wireless connection 1370 via a relay. In providing the service to the remote user, a host application may provide user data which is transmitted using the OTT connection 1350 or via the connections 1360, 1370.

[0212] The network node 1304 includes hardware enabling it to communicate with the host 1302 and the UE 1306 via a relay over wireless connection 1370. The connection 1360 may be direct or pass through a core network (like core network 806 of Figure 8) and / or one or more other intermediate networks, such as one or more public, private, or hosted networks. For example, an intermediate network may be a backbone network or the Internet.

[0213] The UE 1306 includes hardware and software, which is stored in or accessible by UE 1306 and executable by the UE’s processing circuitry. The software includes a client application, such as a web browser or operator-specific “app” that may be operable to provide a service to a human or non-human user via UE 1306 with the support of the host 1302. In the host 1302, an executing host application may communicate with the executing client application via the OTT connection 1350 terminating at the UE 1306 and host 1302; or wireless connection via relay 1370. In providing the service to the user, the UE's client application may receive request data from the host's host application and provide user data in response to the request data. The OTT connection 1350 or wireless connection 1370 may transfer both the request data and the user data. The UE's client application may interact with the user to generate the user data that it provides to the host application through the OTT connection 1350 or the wireless connection 1370.

[0214] The OTT connection 1350 may extend via a connection 1360 between the host 1302 and the network node 1304 and between the network node 1304 and the UE 1306 via a wireless connection 1370 via a relay to provide the connection between the host 1302 and the UE 1306. The connection 1360 and wireless connection 1370, over which the OTT connection 1350 may be provided, have been drawn abstractly to illustrate the communication between the host 1302 and the UE 1306 via a relay and the network node 1304, without explicit reference to any intermediary devices and the precise routing of messages via these devices.

[0215] As an example of transmitting data via the OTT connection 1350, in step 1308, the host 1302 provides user data, which may be performed by executing a host application. In some embodiments, the user data is associated with a particular human user interacting with the UE 1306. In other embodiments, the user data is associated with a UE 1306 that shares data with the host 1302 without explicit human interaction. In step 1310, the host 1302 initiates a transmission carrying the user data towards the UE 1306. The host 1302 may initiate the transmission responsive to a request transmitted by the UE 1306. The request may be caused by human interaction with the UE 1306 or by operation of the client application executing on the UE 1306. The transmission may pass via the network node 1304, in accordance with the teachings of the embodiments described throughout this disclosure. Accordingly, in step 1312, the network node 1304 transmits to the UE 1306 via a relay the user data that was carried in the transmission that the host 1302initiated, in accordance with the teachings of the embodiments described throughout this disclosure. In step 1314, the UE 1306 receives the user data carried in the transmission, which may be performed by a client application executed on the UE 1306 associated with the host application executed by the host 1302.

[0216] In some examples, the UE 1306 executes a client application which provides user data to the host 1302. The user data may be provided in reaction or response to the data received from the host 1302. Accordingly, in step 1316, the UE 1306 may provide user data, which may be performed by executing the client application. In providing the user data, the client application may further consider user input received from the user via an input / output interface of the UE 1306. Regardless of the specific manner in which the user data was provided, the UE 1306 initiates, in step 1318, transmission of the user data towards the host 1302 via the relay and network node 1304. In step 1320, in accordance with the teachings of the embodiments described throughout this disclosure, the network node 1304 receives user data from the UE 1306 via the relay and initiates transmission of the received user data towards the host 1302. In step 1322, the host 1302 receives the user data carried in the transmission initiated by the UE 1306.

[0217] One or more of the various embodiments improve the performance of OTT services provided to the UE 1306 using the OTT connection 1350. More precisely, the teachings of these embodiments may improve computation resources and thereby provide benefits such as better responsiveness and enhanced privacy.

[0218] In an example scenario, factory status information may be collected and analyzed by the host 1302. As another example, the host 1302 may process audio and video data which may have been retrieved from a UE for use in creating maps. As another example, the host 1302 may collect and analyze real-time data to assist in controlling vehicle congestion (e.g., controlling traffic lights). As another example, the host 1302 may store surveillance video uploaded by a UE. As another example, the host 1302 may store or control access to media content such as video, audio, VR or AR which it can broadcast, multicast or unicast to UEs. As other examples, the host 1302 may be used for energy pricing, remote control of non-time critical electrical load to balance power generation needs, location services, presentation services (such as compiling diagrams etc. from data collected from remote devices), or any other function of collecting, retrieving, storing, analyzing and / or transmitting data.

[0219] Although the network nodes, relays, and UEs described herein may include the illustrated combination of hardware components, other embodiments may comprise network nodes, relays, and UEs with different combinations of components. It is to be understood that these network nodes, relays, and UEs may comprise any suitable combination of hardware and / orsoftware needed to perform the tasks, features, functions and methods disclosed herein. Determining, calculating, obtaining or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting the obtained information into other information, comparing the obtained information or converted information to information stored in the network nodes, relays, and UEs, and / or performing one or more operations based on the obtained information or converted information, and as a result of said processing making a determination. Moreover, while components are depicted as single boxes located within a larger box, or nested within multiple boxes, in practice, network nodes, relays, and UEs may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned between separate components. For example, a communication interface may be configured to include any of the components described herein, and / or the functionality of the components may be partitioned between the processor and the network interface. In another example, non-computationally intensive functions of any of such components may be implemented in software or firmware and computationally intensive functions may be implemented in hardware.

[0220] In certain embodiments, some or all of the functionality described herein may be provided by processing circuitry executing instructions stored on in memory, which in certain embodiments may be a computer program product in the form of a non-transitory computer- readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner. In any of those particular embodiments, whether executing instructions stored on a non-transitory computer-readable storage medium or not, the processing circuitry can be configured to perform the described functionality. The benefits provided by such functionality are not limited to the processing circuitry alone or to other components of the computing device, but are enjoyed by the computing device as a whole, and / or by end users and a wireless network generally.

[0221] In certain embodiments, a network node (106, 810A, 1000) is provided. The network node is configured to perform collaborative training of ML model based on relay assisted federated learning with AirComp and data privacy. The network node includes processing circuitry 1002; and memory 1004 coupled with the processing circuitry. The memory includes instructions that when executed by the processing circuitry causes the network node to perform operations. The operations include to perform some or all of the functionality described herein.

[0222] In certain embodiments, a non-transitory computer readable medium 1004 including program code to be executed by processing circuitry 1002 of a network node 1000 is configuredto perform collaborative training of ML model based on relay assisted federated learning with AirComp and data privacy. Execution of the program code causes the program code to perform operations. The operations include to perform some or all of the functionality described herein.

[0223] In certain embodiments, a relay (104, 810B, 1000) is provided. The relay is configured to perform collaborative training of a ML model based on relay assisted federated learning with AirComp and data privacy. The relay includes processing circuitry 1002; and memory 1004 coupled with the processing circuitry. The memory includes instructions that when executed by the processing circuitry causes the relay to perform operations. The operations include to perform some or all of the functionality described herein.

[0224] In certain embodiments, a non-transitory computer readable medium 1004 including program code to be executed by processing circuitry 1002 of a relay 1000 is configured to perform collaborative training of ML model based on relay assisted federated learning with AirComp and data privacy. Execution of the program code causes the program code to perform operations. The operations include to perform some or all of the functionality described herein.

[0225] In certain embodiments, a UE (102, 812, 900) is provided. The UE is configured to perform collaborative training of a ML model based on relay assisted federated learning with AirComp and data privacy. The UE includes processing circuitry 902; and memory 910 coupled with the processing circuitry. The memory includes instructions that when executed by the processing circuitry causes the UE to perform operations. The operations include to perform some or all of the functionality described herein.

[0226] In certain embodiments, a non-transitory computer readable medium 910 including program code to be executed by processing circuitry 902 of a UE 900 is configured to perform collaborative training of ML model based on relay assisted federated learning with AirComp and data privacy. Execution of the program code causes the program code to perform operations. The operations include to perform some or all of the functionality described herein.

[0227] A simulation of an example was conducted, along with an analysis of performance. The performance analysis includes a learning algorithm of ridge regression for which the loss function ^^^ ^^: ^^, ^^^ ൌ ‖ ^^்^^ െ ^^‖ଶ^ ^^ ^^^ ^^^, a regularization function of type: ^^^ ^^^ ൌ ‖ ^^‖ଶ, an hyperparameter ^^ ൌ 5 ൈ 10ିହ. The data points were uniformly distributed over ^^ ൌ 10 UEs 102. The size of the data set at each UE 102 ^^ is ^^^ൌ 1000,∀ ^^ ∈ ^^. For this example validation, the smoothness parameters and the PL parameters were set to be the largest and smallest eigen ^ values of the data Gramian matrix^^ ^ 2 ^^ ^^, where ^^ ൌ ^ ^^்^, ^^ଶ, .. , ^^^^^^൧ . The optimal globalML model was computed as: ^^∗ൌ ^ ^^்^^ ^ 2 ^^ ^^ ^^^ି^^^்^^, where ^^ ൌ ^^்௧^௧ ^ ^, ^^ଶ, .. , ^^^^^^൧. The upper bound of the local ^^^^^^^ ^^ ^

[0228] Further, thewords, ℎ^,^and ℎ^,^are Gaussian random variables with zero mean and unit variance. Each UE 102 k’s maximum transmit power is ^^^^^௫ൌ 5 ^^,∀ ^^ ∈ ^^. The maximum transmit power of relay 104 is ^^^^^௫ൌ 5 ^^. The maximum signal-to-noise ratio (SNR) at UE 102 k and the relay 104 were ^^^௫ ^^ೌ^^ೌೖఙమൌ 30 ^^ ^^ and ^^ ^^ ^^^ൌ^^ ^set to ^^ ^^ ^^ ൌ^^௫ఙమൌ 30 ^^ ^^, respectively. The tolerance ^^^, ^^ଶand the 0.001 ൌ ^^ଶ. The number of channels was ^^ ൌ 2. Thewas ^^ ൌ 100. A variable learning rate was usedwhich was set to ^^ ൌ^^ାଷ. The differential privacy parameters were set to ^^ ൌ 10 and ^^ ൌ 0.01. The training process was averaged by running 10 times.

[0229] The were compared with four other approaches: ^ A single channel scheme: In this approach, there was only one available channel, that is ^^ ൌ 1. ^A fixed allocation scheme 1: Uniform combining factors were used at the AP 106. ^^^^^^ ൌ0.2, and the transmit power and the noise power were identically chosen at each UE 102.The fixed scalar ^^ଶ,^ൌ 0 was used, meaning the relay 104 did not add artificial noise, with a learning rate of ^^ ൌ 0.5. ^ A fixed allocation scheme 2: here, ^^ଶ,^ൌ 0.01 and ^^^,^ൌ 0. In other words, artificial noise was added at the relay 104, but UEs 102 did not add artificial noise. ^ A without DP scheme: In this approach, the DP constraint is not considered.

[0230] Results of the example simulation are shown in Figures 14-18.

[0231] Referring first to Figure 14, the impact of the learning round N is illustrated. The optimality gaps of the example process of the present disclosure and the four other approaches are shown versus the number of learning rounds N. As shown in Figure 14, the optimality gap decreases to a small value approaching 0 for the example process as the number of learning rounds N is increased. Thus, a function value close to an optimal value may be obtained with the example process.

[0232] Prediction error versus the number of learning rounds N is depicted in Figure 15. As shown, an increase of the number of learning rounds N leads to an error floor of the example process. The example process outperformed the two fixed allocation schemes and the singlechannel and achieved a lower optimality gap and prediction error. Thus, the impact of using multiples channels and optimized resource allocation of the example process is beneficial.

[0233] The impact of the privacy level ^^ is demonstrated in Figure 16. By varying the privacy level ^^ from 5 to 25, the obtained optimality gap after N learning rounds was computed. When the privacy level ^^ becomes larger, a lower optimality gap is reached. This indicates that under a larger privacy level, the example process achieves higher learning accuracy and a slower convergence rate. The example process had a better performance than the fixed allocation schemes and the single channel scheme. As shown, when ^^ is large, the learning performance of the example process is almost the same as that of the approach without considering DP constraint.

[0234] In Figure 17, the impact of the number of UEs 102 on the learning performance is shown. Each UE 102 in this example simulation had the same size of data samples. By changing the number of UEs 102 from 5 to 25, the optimality gap was evaluated. As shown, when the number of UEs 102 is increased, the example process generally achieved better learning accuracy. More data samples were used to improve the accuracy of the ML model aggregation at the AP 106, which demonstrates a benefit of over-the-air computation.

[0235] Figure 18 illustrates the optimality gap versus the maximum SNR at the UEs 102 and the relay 104. As shown, the maximum SNR was varied from 10dB to 30dB by adjusting the value of the noise variance ^^ଶ. As shown in Figure 18, the optimality gap decreases when there is a higher maximum SNR because a larger SNR yields a better channel condition, which leads to a higher accuracy of data aggregation.

[0236] In some examples, the method described herein can be deployed as a cloud based service to provide a mechanism or strategy when conditions permit a federated learning system.

[0237] Further definitions and embodiments are discussed below.

[0238] In the above-description of certain embodiments of the present disclosure, it is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which concepts of the present disclosure belong. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of this specification and the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0239] When an element is referred to as being “connected”, “coupled”, “responsive”, or variants thereof to another element, it can be directly connected, coupled, or responsive to the otherelement or intervening elements may be present. In contrast, when an element is referred to as being “directly connected”, “directly coupled”, “directly responsive”, or variants thereof to another element, there are no intervening elements present. Like numbers refer to like elements throughout. Furthermore, “coupled”, “connected”, “responsive”, or variants thereof as used herein may include wirelessly coupled, connected, or responsive. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. Well-known functions or constructions may not be described in detail for brevity and / or clarity. The term “and / or” (abbreviated “ / ”) includes any and all combinations of one or more of the associated listed items.

[0240] It will be understood that although the terms first, second, third, etc. may be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another element / operation. Thus a first element / operation in some embodiments could be termed a second element / operation in other embodiments without departing from the teachings of concepts of the present disclosure. The same reference numerals or the same reference designators denote the same or similar elements throughout the specification.

[0241] As used herein, the terms “comprise”, “comprising”, “comprises”, “include”, “including”, “includes”, “have”, “has”, “having”, or variants thereof are open-ended, and include one or more stated features, integers, elements, steps, components, or functions but does not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions, or groups thereof. Furthermore, as used herein, the common abbreviation “e.g.”, which derives from the Latin phrase “exempli gratia,” may be used to introduce or specify a general example or examples of a previously mentioned item, and is not intended to be limiting of such item. The common abbreviation “i.e.”, which derives from the Latin phrase “id est,” may be used to specify a particular item from a more general recitation.

[0242] Example embodiments are described herein with reference to block diagrams and / or flowchart illustrations of computer-implemented methods, apparatus (systems and / or devices) and / or computer program products. It is understood that a block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by computer program instructions that are performed by one or more computer circuits. These computer program instructions may be provided to a processor circuit of a general purpose computer circuit, special purpose computer circuit, and / or other programmable data processing circuit to produce a machine, such that the instructions, which execute via the processor of the computer and / or other programmable data processing apparatus,transform and control transistors, values stored in memory locations, and other hardware components within such circuitry to implement the functions / acts specified in the block diagrams and / or flowchart block or blocks, and thereby create means (functionality) and / or structure for implementing the functions / acts specified in the block diagrams and / or flowchart block(s).

[0243] These computer program instructions may also be stored in a tangible computer- readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions which implement the functions / acts specified in the block diagrams and / or flowchart block or blocks. Accordingly, embodiments of the present disclosure may be embodied in hardware and / or in software (including firmware, resident software, micro-code, etc.) that runs on a processor such as a digital signal processor, which may collectively be referred to as “circuitry,” “a module” or variants thereof.

[0244] It should also be noted that in some alternate implementations, the functions / acts noted in the blocks may occur out of the order noted in the flowcharts. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality / acts involved. Moreover, the functionality of a given block of the flowcharts and / or block diagrams may be separated into multiple blocks and / or the functionality of two or more blocks of the flowcharts and / or block diagrams may be at least partially integrated. Finally, other blocks may be added / inserted between the blocks that are illustrated, and / or blocks / operations may be omitted without departing from the scope of the present disclosure. Moreover, although some of the diagrams include arrows on communication paths to show a primary direction of communication, it is to be understood that communication may occur in the opposite direction to the depicted arrows.

[0245] Many variations and modifications can be made to the embodiments without substantially departing from the principles of the present disclosure. All such variations and modifications are intended to be included herein within the scope of present disclosure. Accordingly, the above disclosed subject matter is to be considered illustrative, and not restrictive, and the examples of embodiments are intended to cover all such modifications, enhancements, and other embodiments, which fall within the spirit and scope of the present disclosure. Thus, to the maximum extent allowed by law, the scope of the present disclosure is to be determined by the broadest permissible interpretation of the present disclosure including the examples of embodiments and their equivalents, and shall not be restricted or limited by the foregoing detailed description.

Claims

What is claimed is:

1. A system (100) for collaborative training of a machine learning, ML, model based on a relay assisted federated learning with over-the-air computation and data privacy, the system comprising: a plurality of user equipment, UEs, (102) in a federated learning environment respectively configured to (i) communicate over a plurality of wireless channels, (ii) receive, from a relay, the ML model and a third subset of variables from a plurality of variables, (iii) train the ML model with local data and obtain a local ML model gradient, (iv) modify the local ML gradient with a first transmit power level from the third subset of variables and a first artificial noise for data privacy, and (v) transmit the modified local ML model gradient to the relay; a network node (106) without a direct communication link with the plurality of UEs, the network node configured to (i) transmit, to the relay, the ML model and the plurality of variables, (ii) receive, from the relay, over a plurality of respective wireless channels respective further modified local ML model gradients comprising a second artificial noise for data privacy, (iii) aggregate the further modified local ML model gradients based on an over-the-air computation for wireless aggregation, (iv) construct an updated ML model based on the aggregated further modified local ML model gradients, and (v) perform one of (a) transmit the updated ML model to the relay when a convergence or a maximum learning round is reached, or (b) start a next round of the collaborative training; and a relay (104) configured to (i) receive the plurality of variables and the ML model from the network node, (ii) extract from the plurality of variables a second subset of variables, (iii) transmit to the plurality of UEs the third subset of variables and the ML model, (iv) receive the modified local ML model gradients transmitted from UEs over a plurality of respective wireless channels, (v) further modify the respective modified local ML gradients with the second subset of variables to add a second artificial noise for data privacy and to scale the received modified local gradients, and (vi) transmit the respective further modified local ML model gradients to the network node.

2. The system (100) of Claim 1, wherein the plurality of variables comprise (i) a first subset of receive scalars for the network node to apply to aggregate the respective further modified local ML model gradients, (ii) the second subset of transmit scalars for the relay to add the second artificial noise and to scale the receivedmodified local gradients, and (iii) the third subset of transmit power levels at the respective UEs, and before starting the next round of collaborative learning, the network node is further configured to configure the plurality of variables comprising the first subset of receive scalars for the network node, the second subset of transmit scalars for the relay, and the third subset of transmit power levels at the respective UEs.

3. A method performed by a network node without a direct communication link with a plurality of user equipment, UEs, to collaboratively train a machine learning, ML, model based on a relay assisted distributed machine learning with over-the-air computation and data privacy, the method comprising: receiving (506) a plurality of further modified local ML model gradients from a relay over respective wireless channels of a plurality of wireless channels, wherein a further modified local ML model gradient comprises a local ML model gradient modified by a respective UE with a first artificial noise for data privacy which is further modified by the relay with a second artificial noise for data privacy; aggregating (508) the plurality of further modified local ML model gradients based on an over-the-air computation for wireless aggregation; constructing (510) an updated ML model based on the aggregated further modified local gradients; and performing (512) one of (i) transmitting the updated ML model to the relay, or (ii) configure a plurality of variables and start a next round of the collaborative training.

4. The method of Claim 3, wherein the data privacy comprises a differential privacy to protect data shared between the network node and the plurality of UEs via the relay.

5. The method of any one of Claims 3 to 4, wherein the first artificial noise comprises Gaussian noise added at respective UEs and the second artificial noise comprises Gaussian noise added at the relay to enhance data privacy in the plurality of wireless channels.

6. The method of any one of Claims 3 to 5, wherein a direct communication link between the network node and the plurality of UEs is not accessible and the relay is used at least to (i) address wireless channel fading and / or (ii) assist communication between the relay and the plurality of UEs and between the relay and the network node.

7. The method of any one of Claims 3 to 6, further comprising: transmitting (504), to the relay, the ML model and the plurality of variables.

8. The method of any one of Claims 3 to 7, wherein the plurality of variables comprise (i) a first subset of the plurality of variables comprising receive scalars for the network node to apply to aggregate the respective further modified local ML model gradients, (ii) a second subset of the plurality of variables comprising transmit scalars for the relay to add the second artificial noise and to scale the received modified local gradients, and (iii) a third subset of the plurality of variables comprising transmit power levels at the respective UEs.

9. The method of any one of Claims 3 to 8, wherein configure the plurality of variables comprises configure a first subset of the plurality of variables comprising receive scalars for the network node, configure a second subset of the plurality of variables comprising transmit scalars for the relay, and configure a third subset of the plurality of variables comprising power levels at the respective UEs.

10. The method of any one of Claims 3 to 9, further comprising: receiving (500) a request for distributed training of the ML model; and initializing (502) the ML model including initialization of the plurality of variables comprising (i) a set of receive scalars at the network node, (ii) a set of transmit scalars at the relay, and (iii) a set of transmit power per user equipment, UE, for a plurality UEs.

11. A method performed by a relay to collaboratively train a machine learning, ML, model based on relay assisted distributed machine learning with over-the-air computation and data privacy, the method comprising: extracting (602) from a plurality of variables a second subset of variables; transmitting (604) to a plurality of user equipment, UEs, the ML model and a third subset of variables from the plurality of variables; receiving (606) a modified local ML model gradient from respective UEs over a plurality of respective wireless channels, wherein the modified local ML gradient comprises a local ML model gradient modified by a respective UE with a first artificial noise for data privacy;further modifying (608) the respective modified local ML gradients with the second subset of variables to add a second artificial noise for data privacy and to scale the received modified local gradients; and transmitting (610) the respective further modified local ML model gradients to a network node.

12. The method of Claim 11, wherein the data privacy comprises a differential privacy to protect data shared between the network node and the plurality of UEs via the relay.

13. The method of any one of Claims 11 to 12, wherein the first artificial noise comprises Gaussian noise added at respective UEs and the second artificial noise comprises Gaussian noise added at the relay to enhance data privacy in the plurality of wireless channels.

14. The method of any one of Claims 11 to 13, wherein a direct communication link between the network node and the plurality of UEs is not accessible and the relay is used at least to (i) address wireless channel fading and / or (ii) assist communication between the relay and the plurality of UEs and between the relay and the network node.

15. The method of any one of Claims 11 to 14, further comprising: receiving (600), from the network node, the ML model and the plurality of variables.

16. The method of any one of Claims 11 to 15, wherein the plurality of variables comprise (i) a first subset of the plurality of variables comprising receive scalars for the network node to apply to aggregate the respective further modified local ML model gradients, (ii) a second subset of the plurality of variables comprising transmit scalars for the relay to add the second artificial noise and to scale the received modified local gradients, and (iii) a third subset of the plurality of variables comprising transmit power levels at the respective UEs.

17. A method performed by a user equipment, UE, without a direct communication link with a network node to collaboratively train a machine learning, ML, model based on a relay assisted distributed machine learning with over-the-air computation and data privacy, the method comprising: receiving (700) over a wireless channel, from a relay, the ML model and a third subset of variables from a plurality of variables;training (702) the ML model with local data to obtain a local ML model gradient; modifying (704) the local ML gradient with a first transmit power level from the third subset of variables and a first artificial noise for data privacy; and transmitting (706) the modified local ML model gradient to the relay.

18. The method of Claim 17, wherein the data privacy comprises a differential privacy to protect data shared between the network node and the UE via the relay.

19. The method of any one of Claims 17 to 18, wherein the first artificial noise comprises Gaussian noise added at the UE to enhance data privacy in the wireless channel.

20. The method of any one of Claims 17 to 19, wherein a direct communication link between the network node and the UE is not accessible and the relay is used at least to (i) address wireless channel fading and / or (ii) assist communication between the relay and the UE and between the relay and the network node.

21. The method of any one of Claims 17 to 20, wherein the plurality of variables comprise (i) a first subset of the plurality of variables comprising receive scalars for the network node to apply to aggregate the respective further modified local ML model gradients, (ii) a second subset of the plurality of variables comprising transmit scalars for the relay to add a second artificial noise and to scale the modified local gradients, and (iii) the third subset of the plurality of variables comprising transmit power levels at the respective UEs in a plurality of UEs.

22. A network node (106, 810A, 1000) configured to perform collaborative training of a machine learning, ML, model based on relay assisted distributed machine learning with over-the- air computation and data privacy, the network node comprising: processing circuitry (1002); memory (1004) coupled with the processing circuitry, wherein the memory includes instructions that when executed by the processing circuitry causes the network node to perform operations comprising: receive a plurality of further modified local ML model gradients from a relay over respective wireless channels of a plurality of wireless channels, wherein a further modified local ML model gradient comprises a local ML model gradient modified by a respective UE with afirst artificial noise for data privacy which is further modified by the relay with a second artificial noise for data privacy; aggregate the plurality of further modified local ML model gradients based on an over- the-air computation for wireless aggregation; construct an updated ML model based on the aggregated further modified local gradients; and perform one of (i) transmitting the updated ML model to the relay, or (ii) configure a plurality of variables and start a next round of the collaborative training.

23. The network node of Claim 22, wherein the operations further comprise any of the operations of Claims 3-10.

24. A non-transitory computer readable medium (1004) including program code to be executed by processing circuitry (1002) of a network node (106, 810A, 1000) configured to perform collaborative training of a machine learning, ML, model based on relay assisted distributed machine learning with over-the-air computation and data privacy, whereby execution of the program code causes the program code to perform operations comprising: receive a plurality of further modified local ML model gradients from a relay over respective wireless channels of a plurality of wireless channels, wherein a further modified local ML model gradient comprises a local ML model gradient modified by a respective UE with a first artificial noise for data privacy which is further modified by the relay with a second artificial noise for data privacy; aggregate the plurality of further modified local ML model gradients based on an over- the-air computation for wireless aggregation; construct an updated ML model based on the aggregated further modified local gradients; and perform one of (i) transmitting the updated ML model to the relay, or (ii) configure a plurality of variables and start a next round of the collaborative training.

25. The non-transitory computer readable medium of Claim 13, the operations further comprising any of the operations of Claims 3-10.

26. A relay (104, 810B, 1000) configured to perform collaborative training of a machine learning, ML, model based on relay assisted distributed machine learning with over-the-air computation and data privacy, the relay comprising: processing circuitry (1002); memory (1004) coupled with the processing circuitry, wherein the memory includes instructions that when executed by the processing circuitry causes the relay to perform operations comprising: extract from a plurality of variables a second subset of variables; transmit to a plurality of user equipment, UEs, the ML model and a third subset of variables from the plurality of variables; receive a modified local ML model gradient from respective UEs over a plurality of respective wireless channels, wherein the modified local ML gradient comprises a local ML model gradient modified by a respective UE with a first artificial noise for data privacy; further modify the respective modified local ML gradients with the second subset of variables to add a second artificial noise for data privacy and to scale the received modified local gradients; and transmit the respective further modified local ML model gradients to a network node.

27. The relay of Claim 26, wherein the operations further comprise any of the operations of Claims 12-16.

28. A non-transitory computer readable medium (1004) including program code to be executed by processing circuitry (1002) of a relay (104, 810B, 1000) configured to perform collaborative training of a machine learning, ML, model based on relay assisted distributed machine learning with over-the-air computation and data privacy, whereby execution of the program code causes the program code to perform operations comprising: extract from a plurality of variables a second subset of variables; transmit to a plurality of user equipment, UEs, the ML model and a third subset of variables from the plurality of variables; receive a modified local ML model gradient from respective UEs over a plurality of respective wireless channels, wherein the modified local ML gradient comprises a local ML model gradient modified by a respective UE with a first artificial noise for data privacy;further modify the respective modified local ML gradients with the second subset of variables to add a second artificial noise for data privacy and to scale the received modified local gradients; and transmit the respective further modified local ML model gradients to a network node.

29. The non-transitory computer readable medium of Claim 28, the operations further comprising any of the operations of Claims 12-16.

30. A user equipment, UE, (102, 812, 900) configured to perform collaborative training of a machine learning, ML, model based on relay assisted distributed machine learning with over-the-air computation and data privacy, the UE comprising: processing circuitry (902); memory (910) coupled with the processing circuitry, wherein the memory includes instructions that when executed by the processing circuitry causes the UE to perform operations comprising: receive over a wireless channel, from a relay, the ML model and a third subset of variables from a plurality of variables; train the ML model with local data to obtain a local ML model gradient; modify the local ML gradient with a first transmit power level from the third subset of variables and a first artificial noise for data privacy; and transmit the modified local ML model gradient to the relay.

31. The network node of Claim 30, wherein the operations further comprise any of the operations of Claims 18-21.

32. A non-transitory computer readable medium (910) including program code to be executed by processing circuitry (902) of a user equipment, UE, (102, 812, 900) configured to perform collaborative training of a machine learning, ML, model based on relay assisted distributed machine learning with over-the-air computation and data privacy, whereby execution of the program code causes the program code to perform operations comprising: receive over a wireless channel, from a relay, the ML model and a third subset of variables from a plurality of variables; train the ML model with local data to obtain a local ML model gradient;modify the local ML gradient with a first transmit power level from the third subset of variables and a first artificial noise for data privacy; and transmit the modified local ML model gradient to the relay.

33. The non-transitory computer readable medium of Claim 32, the operations further comprising any of the operations of Claims 18-21.

Images