First node, communications system and methods performed thereby, for handling a machine learning model

Abstract

A first node (111) determines (504, 505) second (122) and third nodes (123) out of at least a subset of a set of nodes (121). Each of the nodes in the set (121) manages an agent capable of training a machine learning model. The nodes have a capability to connect in a fully decentralized setting. The first node (111) obtains (506, 507), using a decentralized consensus mechanism, a first a second indication of the heterogeneity of the sets of data used by, and the respective compute capability of, each of the respective agents managed by the second nodes (122) and the third nodes (123). The first node (111) determines (508) whether to group the nodes (122, 123) into a same group to train a common model, or to refrain from grouping the nodes (122, 123), based on a difference between the first and second indications, and outputs (509) an indication of the result.

Description

[0001] FIRST NODE, COMMUNICATIONS SYSTEM AND METHODS PERFORMED THEREBY, FOR HANDLING A MACHINE LEARNING MODEL

[0002] TECHNICAL FIELD

[0003] The present disclosure relates generally to a first node and methods performed thereby for handling a machine learning model. The present disclosure also relates generally to a communications system and methods performed thereby, for handling the machine learning model. The present disclosure also relates generally to computer programs and computer- readable storage mediums, having stored thereon the computer programs to carry out these methods.

[0004] BACKGROUND

[0005] Computer systems in a communications network or communications system may comprise one or more nodes. A node may comprise one or more processors which, together with computer program code may perform different functions and actions, a memory, a receiving port, and a sending port. A node may be, for example, a server. Nodes may perform their functions entirely on the cloud.

[0006] Computer systems may be comprised in a telecommunications network. The telecommunications network, communications network, or wireless communications network, may cover a geographical area which may be divided into cell areas, each cell area being served by a type of node, a network node in the Radio Access Network (RAN), radio network node or Transmission Point (TP), for example, an access node such as a Base Station (BS), e.g., a Radio Base Station (RBS), which sometimes may be referred to as e.g., gNB, evolved Node B (“eNB”), “eNodeB”, “NodeB”, “B node”, or Base Transceiver Station (BTS), depending on the technology and terminology used. The base stations may be of different classes such as e.g., Wide Area Base Stations, Medium Range Base Stations, Local Area Base Stations and Home Base Stations, based on transmission power and thereby also cell size. A cell may be understood to be the geographical area where radio coverage may be provided by the base station at a base station site. One base station, situated on the base station site, may serve one or several cells. Further, each base station may support one or several communication technologies. The telecommunications network may also comprise network nodes which may serve receiving nodes, such as user equipments or devices, with serving beams.

[0007] The standardization organization Third Generation Partnership Project (3GPP) is currently in the process of specifying a New Radio Interface called Next Generation Radio or New Radio (NR), as well as a Fifth Generation (5G) Packet Core Network, which may be referred to as 5G Core Network (5GC). The advantages of 5G NR may include higher bandwidth, more resources, low latency and network slicing. 5G may provide services to various applications, such as enhanced Mobile Broad Band (eMBB), machine to Machine type communication (mMTC), Ultra Reliable Low Latency Communication (URLLC), etc.

[0008] 5G may be understood to bring in sizeable flexibility with technological advancements along with innovations of cloud and Artificial Intelligence (Al). This may be understood to bring a whole new set of opportunities in the enterprise segment.

[0009] Machine Learning

[0010] Machine learning (ML) may be understood as the study of computer algorithms that may improve automatically through experience. It is seen as a part of Artificial Intelligence (Al). ML algorithms may build a model based on sample data, known as "training data", in order to make predictions or decisions without being explicitly programmed to do so. ML algorithms may be used in a wide variety of applications, such as email filtering and computer vision, where it may be difficult or unfeasible to develop conventional algorithms to perform the needed tasks.

[0011] There may be basically three types of ML Algorithms: Supervised Learning, Unsupervised Learning, and Reinforcement Learning (RL).

[0012] Supervised Learning algorithms may comprise a target / outcome variable, or dependent variable, which may have to be predicted from a given set of predictors, that is, independent variables. Using this set of variables, a function may be generated that may map inputs to desired outputs. The training process may continue until the model may achieve a desired level of accuracy on the training data. Once an ML model may have been trained, an inference process may begin, whereby new data may be run through the ML model to calculate an output. Examples of Supervised Learning may be Regression, Decision Tree, Random Forest, KNN, Logistic Regression etc.

[0013] In Unsupervised Learning algorithms, there may be no target or outcome variable to predict / estimate. It may be used for clustering a population into different groups, which may be widely used for segmenting customers in different groups for specific intervention. Examples of Unsupervised Learning may be K-means, mean-shift clustering, Density-Based Spatial Clustering of Applications with Noise (DBSCAN), Expectation-Maximization (EM) Clustering using Gaussian Mixture Models (GMM), Agglomerative Hierarchical Clustering, etc....

[0014] Cluster analysis or clustering may be understood as an ML technique which may comprise grouping a set of objects in such a way that objects in the same group, which may be called a cluster, may be understood to be more similar, in some sense, to each other than to those in other groups, that is, other clusters. It may be understood as a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including pattern recognition, image analysis, information retrieval, bioinformatics, data compression, computer graphics and ML. Using an RL algorithm, a machine may be trained to make specific decisions. It may be understood to work as follows: the machine may be exposed to an environment where it may train itself continually using trial and error. This machine may learn from past experience and may try to capture the best possible knowledge to make accurate decisions. An example of RL may be a Markov Decision Process (MDP). The training using RL may comprise generating an ML model. To train such an ML model, an agent, given a state of the environment, may take an action in this environment and receive a reward. The action may result in a new state of the environment. This process may be repeated in a loop. Over time, the agent may learn to take actions that may result in larger immediate and future rewards, meaning that it may be understood to be in the best interest of the agent not to take the action that may only lead to the highest reward in the next state, but the action that may cumulatively lead to the highest reward in the next state and in a future number of states.

[0015] The agent may comprise a neural network which may input the state and may produce an action. There may be several ML algorithms that may be used for training the network of the agent, e.g., policy-learning based, such as actor-critic approaches, or value-based learning, such as deep-q networks.

[0016] Within the AI / ML field, several new decentralized algorithms are being developed with which autonomous agents, which may be also referred to as clients, which may be for example mobile users, smart devices, cameras in a network, or other sensors, may collaboratively solve large-scale inference and learning tasks. In the past, much focus has been on the federated learning setup, a schematic representation of which is depicted in Figure 1 , where a number of agents may participate in the training of a machine learning model residing in a central parameter server. Here, each agent may be understood to have its own local training data but may be understood to not share this data with the server. Instead, only updates to the model may be shared, and subsequently aggregated by the server.

[0017] In the last few years, there has been increased interest in fully decentralized learning setups, where there may be understood to be no central parameter server. Instead, all agents may have their own instance of the learning model and, similar to the federated learning, each agent may have its own private training data. The agents may be connected according to a connectivity graph, such that each agent may have a number of neighbors. The assumption here may be understood to be that either there may be a reliable link between two agents, or there may not be. The existence of a link may be represented by corresponding edge of the connectivity graph. The training may proceed by exchanging updates between the agents and their neighbors. Figure 2 is a schematic diagram illustrating a non-limiting example of such a fully distributed learning setup, without a central parameter server. Here, all agents may have a local copy of the model, and cooperate in training the model through the exchange of messages directly between the agents. In some examples of the fully decentralized setup, there may be, in addition, a coordinating authority (CA) that may coordinate the learning activity. Typically, the links between agents and their neighbors may have high capacity, e.g., facilitated by millimeter wave or Terahertz communications, whereas the links between agents and the CA may have small bandwidth, e.g., facilitated through a sub-6 GHz cellular network or similar. For instance, they may be connected with technologies such as NR sidelink, Bluetooth or WiFi. These connections may provide very high bitrate, for example, if operating at millimeter wave or Terahertz frequencies. Yet, for privacy reasons, the agents may be unwilling to share raw data with one another, or with the CA, so communication between agents may be restricted to local updates to the model. Therefore, it may be understood to be desirable to carry out the model update communication via device-to-device links between the agents and use the connections to the CA only to coordinate the transmissions and for control signalling. Note that the amounts of data may be quite large, e.g., some emerging models for natural language processing may have 1 trillion parameters. Figure 3 is a schematic diagram illustrating a nonlimiting example of such a fully distributed learning setup, without a central parameter server, but with a CA. Here, all agents may have a local copy of the model, and may cooperate in training the model through the exchange of messages directly between the agents. The CA may only assist in coordination of the transmissions.

[0018] The decentralized learning may be understood to function according to an iterative process, which may be as follows. In a first step, the agents may broadcast their current estimate of the learning model to their neighbors. In a second step, each agent may perform a consensus update, substantially averaging its own local model estimate with those received in the first step from its neighbors. In a third step, each agent may compute a local model update, based on its own private training data. This may take place, for example, using standard stochastic gradient optimization methods. Subsequently, each agent may update its model parameter estimate by adding this model update.

[0019] Each round comprising these three steps may be called a "global iteration". Several global iterations may be performed sequentially, until a convergence criterion may be met, for example, that the difference between two consecutive iterations may be less than a predetermined threshold.

[0020] As compared to the alternative case with server-based decentralized learning, the fully decentralized architecture may be understood to be more resilient to single-node failure and may provide better scalability in terms of communication design by allowing concurrent transmissions among spatially separated nodes.

[0021] Decentralized learning setup - mathematical details To define notation, if TV is the number of agents, the machine learning model may be parameterized by a vector z, and „(z) may denote the local objective function at agent n, defined as: where Snmay be understood to be the set of training data at agent n and / jj(z) may be understood to be the loss function evaluated at the j-th data sample of agent n, given the current model parameter vector z.

[0022] The objectives may be understood to be to learn the vector z, which may be referred to as z*, that may minimize £n / n(z) with respect to z, and to make sure that all agents may obtain that optimal vector z*. G may be understood to be a connectivity graph that may define how the agents may be interconnected. G may be assumed to change with time, e.g., as a function of mobility, and may be known to the CA and to all agents.

[0023] The decentralized learning may seek to find z* by having each node keeping track of a local estimate of the model, denoted by znfor agent n. Mathematically, the learning steps may be as follows. First, each of the N agents may broadcast its current model parameter estimate, zn, to its neighbors, as defined by G. Second, each agent may perform a consensus update,

[0024] Zn — ' ^ni^i’ i= neighbors n) where {wni} may be understood to be a set of pre-determined weights, and where the set of neighbors may optionally include the value of node n itself, zn. The matrix W formed by assembling these weights into an N x N array may be typically taken as W = I - eL, where e may be understood to be a positive constant and L may be understood to be the Laplacian matrix associated with the graph G. Many other choices may be understood to be possible. Third, based on its local training data, each agent n may compute its local model update, which may be added to zn. A common choice may be to use stochastic gradient descent to compute the local model updates. The learning algorithm may then become a decentralized stochastic gradient descent (DSGD). In this case, the detailed update may be: where y may be understood to be a constant and V may denote the gradient operator;

[0025] Vf„(z)|znmay denote the gradient of (. ) with respect to z evaluated at zn. The execution of these three steps may be referred to as a "global iteration". Ultimately, performing several global iterations may lead to consensus to the optimal parameter vector, that is, the algorithm may converge eventually, such that z„ = z* at all agents. Basically, the decentralized learning setting may require similar convergence conditions as the centralized case, plus a few additional conditions such as the network being connected, the mixing matrix being symmetric and doubly stochastic, and the second largest magnitude of the eigenvalues of the mixing matrix being strictly smaller than 1.

[0026] Some existing approaches using decentralized learning include a consensus-based Federated Learning method [2], where a consensus mechanism is described in a setup that does not contain a parameter server but, instead, every participant, or agent, may build a model using the input, model parameters, from nearby agents.

[0027] In another approach [3], a “Fog Learning” method is described, which may be understood to be in a distributed learning setup, where different devices may have different capabilities. To solve this challenge, devices may be clustered based on their capability.

[0028] In yet another approach [4], a Federated Learning setup is described which may make use of a (Graph Convolutional Network) GCN layer to learn the relationship between different devices in terms of their data.

[0029] Existing methods of decentralized learning may lead to poor training performance and slow convergence of models.

[0030] SUMMARY

[0031] As part of the development of embodiments herein, one or more problems with the existing technology will first be identified and discussed.

[0032] One issue in decentralized learning may occur when the datasets among agents are heterogenous. This may cause the following problems. One problem may be data imbalance. If the distribution of data across different agents is not balanced, some nodes may have significantly more, or less, data than others. For example, this may happen when some agents may only have samples with a certain label in a classification problem. This may lead to poor training performance and slow convergence of models since agents with less data may be more prone to overfitting in their local model update. Another problem may be due to computation resources. Heterogeneous data may require different computational resources for processing, which may be difficult to coordinate among different agents. If a group of agents are having similar amounts of data, the computational overhead may be likely to be similar for all agents. Hence, it may be simpler for the central authority to set training hyperparameters. Yet another problem may be due to system and / or device heterogeneity. The participating devices may have different capabilities in terms of compute, storage and availability, for example, they may be engaged with other tasks, which may make some devices respond faster to a request to learn a model than others. If this is untreated, slow devices, mixed with fast devices may slow-down the overall process.

[0033] One approach may be to allow agents to share statistics and compute capabilities on the dataset prior to performing the decentralized learning. However, it may be sensitive for agents to disclose information regarding their private training data to anyone. When there is no method to find to what extent the data possessed by different agents are heterogenous, it may not be possible to identify groups of agents within which data may be more homogeneous. This may be understood to imply that use cases where such a grouping may be used for example to activate split learning, or to determine that the identified groups may have to train separate models, are not feasible.

[0034] Embodiments herein may address the problems of the existing methods described by providing a method for handling a machine learning model in a communications system.

[0035] According to a first aspect of embodiments herein, the object is achieved by a computer- implemented method, performed by a first node. The method is for handling a machine learning model. The first node operates in a communications system comprising a set of nodes. The first node determines one or more second nodes out of at least a subset of the set of nodes. Each of the nodes in the set of nodes manages a respective agent capable of training a respective machine learning model with a respective set of data. The nodes in the set of nodes have a capability to connect, directly or indirectly, via resources according to one or more communication subgraphs in a fully decentralized setting. The first node determines one or more third nodes out of at least the subset of the set of nodes. The first node obtains, using a decentralized consensus mechanism, a first indication of a heterogeneity of the respective sets of data used by, and a respective compute capability of, each of the respective agents managed by the one or more second nodes. The first node obtains, using the decentralized consensus mechanism, a second indication of the heterogeneity of the sets of data used by, and the respective compute capability of, each of the respective agents managed by the one or more third nodes. The first node determines whether to group the one or more second nodes and the one or more third nodes into a same group, to train a common machine learning model, or to refrain from grouping the one or more second nodes and the one or more third nodes to train into the same group to train the common machine learning model. The determining of whether to group or not is based on a difference between the first indication and the second indication with respect to a threshold. The first node then outputs a third indication indicating a result of the determining of whether to cluster or not.

[0036] According to a second aspect of embodiments herein, the object is achieved by a computer-implemented method, performed by the communications system comprising the set of nodes. The method is for handling the machine learning model. The first node is comprised in the communications system. At least the first node determines the one or more second nodes out of at least the subset of the set of nodes. Each of the nodes in the set of nodes manages the respective agent capable of training the respective machine learning model with the respective set of data. The nodes in the set of nodes have the capability to connect, directly or indirectly, via the resources according to the one or more communication subgraphs in the fully decentralized setting. At least the first node determines the one or more third nodes out of at least the subset of the set of nodes. At least the first node obtains, using the decentralized consensus mechanism, the first indication of the heterogeneity of the respective sets of data used by, and the respective compute capability of, each of the respective agents managed by the one or more second nodes. At least the first node obtains, using the decentralized consensus mechanism, the second indication of the heterogeneity of the sets of data used by, and the respective compute capability of, each of the respective agents managed by the one or more third nodes. At least the first node determines whether to group the one or more second nodes and the one or more third nodes into the same group to train the common machine learning model, or to refrain from grouping the one or more second nodes and the one or more third nodes to train into the same group to train the common machine learning model. The determining of whether to group or not is based on the difference between the first indication and the second indication with respect to the threshold. At least the first node outputs the third indication indicating the result of the determining in of whether to cluster or not.

[0037] According to a third aspect of embodiments herein, the object is achieved by the first node. The first node may be understood to be for handling the machine learning model. The first node is configured to operate in the communications system configured to comprise the set of nodes. The first node is configured to determine the one or more second nodes out of at least the subset of the set of nodes. Each of the nodes in the set of nodes is configured to manage the respective agent capable of training the respective machine learning model with the respective set of data. The nodes in the set of nodes are configured to have the capability to connect, directly or indirectly, via the resources according to the one or more communication subgraphs in the fully decentralized setting. The first node is also configured to determine the one or more third nodes out of at least the subset of the set of nodes. The first node is also configured to obtain, using the decentralized consensus mechanism, the first indication of the heterogeneity of the respective sets of data configured to be used by, and the respective compute capability of, each of the respective agents configured to be managed by the one or more second nodes. The first node is further configured to obtain, using the decentralized consensus mechanism, the second indication of the heterogeneity of the sets of data configured to be used by, and the respective compute capability of, each of the respective agents configured to be managed by the one or more third nodes. The first node is also configured to determine whether to group the one or more second nodes and the one or more third nodes into the same group to train the common machine learning model, or to refrain from grouping the one or more second nodes and the one or more third nodes to train into the same group to train the common machine learning model. The determining of whether to group or not is configured to be based on the difference between the first indication and the second indication with respect to the threshold. The first node is further configured to output the third indication configured to indicate the result of the determining of whether to cluster or not.

[0038] According to a fourth aspect of embodiments herein, the object is achieved by the communications system. The communications system may be understood to be for handling the machine learning model. The communications system is configured to comprise at least the first node comprised in the communications system. The communications system is configured to, by at least the first node comprised in the communications system, determine the one or more second nodes out of at least the subset of the set of nodes. Each of the nodes in the set of nodes is configured to manage the respective agent capable of training the respective machine learning model with the respective set of data. The nodes in the set of nodes are configured to have the capability to connect, directly or indirectly, via the resources according to the one or more communication subgraphs in the fully decentralized setting. The communications system is configured to, by at least the first node comprised in the communications system, determine the one or more third nodes out of at least the subset of the set of nodes. The communications system is configured to, by at least the first node comprised in the communications system, obtain, using the decentralized consensus mechanism, the first indication of the heterogeneity of the respective sets of data configured to be used by, and the respective compute capability of, each of the respective agents configured to be managed by the one or more second nodes. The communications system is configured to, by at least the first node comprised in the communications system, obtain, using the decentralized consensus mechanism, the second indication of the heterogeneity of the sets of data configured to be used by, and the respective compute capability of, each of the respective agents configured to be managed by the one or more third nodes. The communications system is configured to, by at least the first node comprised in the communications system, determine whether to group the one or more second nodes and the one or more third nodes into the same group to train the common machine learning model, or to refrain from grouping the one or more second nodes and the one or more third nodes to train into the same group to train the common machine learning model. The determining of whether to group or not is configured to be based on the difference between the first indication and the second indication with respect to the threshold. The communications system is configured to, by at least the first node comprised in the communications system, output the third indication configured to indicate the result of the determining of whether to cluster or not. According to a fifth aspect of embodiments herein, the object is achieved by a computer program, comprising instructions which, when executed on at least one processing circuitry, cause the at least one processing circuitry to carry out the method performed by the first node.

[0039] According to a sixth aspect of embodiments herein, the object is achieved by a computer-readable storage medium, having stored thereon the computer program, comprising instructions which, when executed on at least one processing circuitry, cause the at least one processing circuitry to carry out the method performed by the first node.

[0040] According to a seventh aspect of embodiments herein, the object is achieved by a computer program, comprising instructions which, when executed on at least one processing circuitry, cause the at least one processing circuitry to carry out the method performed by the communications system.

[0041] According to an eighth aspect of embodiments herein, the object is achieved by a computer-readable storage medium, having stored thereon the computer program, comprising instructions which, when executed on at least one processing circuitry, cause the at least one processing circuitry to carry out the method performed by the communications system.

[0042] By the first node obtaining the first indication and the second indication, the first node may be enabled to infer knowledge of the statistics of the training data and system capabilities possessed by the one or more second nodes and the one or more third nodes, respectively, without transmission of large amounts or private information to the first node. By the first node obtaining the first indication and the second indication, the first node may be particularly enabled to know the heterogeneity of the data and the compute capability of, each of the respective agents managed by the one or more second nodes and the one or more third nodes, respectively. This may be understood to enable that the common machine learning model may be split, such that agents may be grouped into different groups that may train different models. For example initially, all agents may train the same common machine learning model, but by using the method according to embodiments herein, it may be discovered that a first subset of the agents may have data and system capability with much different statistics than agents in a second subset. It may then be decided that the first subset may train a first new model while the second subset may train a second new model. This way, splitting the nominal models into two, each one may be trained with more homogeneous data and types of nodes. This may be understood to result in improved performance, for instance, faster convergence and / or less resource usage, of machine learning models that may be trained in situations with heterogeneous data in fully decentralized learning.

[0043] BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Examples of embodiments herein are described in more detail with reference to the accompanying drawings, according to the following description. Figure 1 is a schematic diagram illustrating a non-limiting example of a Federated Learning setup, according to existing methods.

[0045] Figure 2 is a schematic diagram illustrating a non-limiting example of a fully distributed learning setup, according to existing methods.

[0046] Figure 3 is a schematic diagram illustrating a non-limiting example of a fully distributed learning setup with a coordinating authority (CA), according to existing methods.

[0047] Figure 4 is a schematic diagram illustrating a non-limiting example, of a communications system, according to embodiments herein.

[0048] Figure 5 is a flowchart depicting a method in a first node, according to embodiments herein. Figure 6 is a flowchart depicting a method in a communications system, according to embodiments herein.

[0049] Figure 7 illustrates aspects of a method in a communications system, according to embodiments herein, wherein panel a) depicts a gradient vector and panel b) depicts a training data distribution.

[0050] Figure 8 is a schematic diagram depicting a non-limiting example of a communications system, according to a device-to-device example of embodiments herein.

[0051] Figure 9 is a signalling diagram depicting a non-limiting example of the method performed in the communications system depicted in Figure 8, according to embodiments herein.

[0052] Figure 10 is a schematic block diagram illustrating an embodiment of a first node, according to embodiments herein.

[0053] Figure 11 is a schematic block diagram illustrating an embodiment of a communications system, according to embodiments herein.

[0054] DETAILED DESCRIPTION

[0055] Certain aspects of the present disclosure and their embodiments address the challenges identified in the Background and Summary sections with the existing methods and provide solutions to the challenges discussed.

[0056] Embodiments herein may be understood to relate to detecting heterogeneity in fully decentralized learning. Further particularly, embodiments herein may be understood to enable to provide a decentralized method for inferring a level of heterogeneity of data and system capability possessed by the agents in a fully decentralized learning setup, without disclosing private dataset information. According to some embodiments herein, the agents may be grouped according to data and system heterogeneity. The grouping of agents may then be used in various learning tasks.

[0057] Some of the embodiments contemplated will now be described more fully hereinafter with reference to the accompanying drawings, in which examples are shown. In this section, the embodiments herein will be illustrated in more detail by a number of exemplary embodiments. Other embodiments, however, are contained within the scope of the subject matter disclosed herein. The disclosed subject matter should not be construed as limited to only the embodiments set forth herein; rather, these embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art. It should be noted that the exemplary embodiments herein are not mutually exclusive. Components from one embodiment may be tacitly assumed to be present in another embodiment and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments.

[0058] Several embodiments and examples are comprised herein. It should be noted that the embodiments and / or examples herein are not mutually exclusive. Components from one embodiment or example may be tacitly assumed to be present in another embodiment or example and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments and / or examples.

[0059] Figure 4 depicts a non-limiting example of a communications system 100, in which embodiments herein may be implemented. The communications system 100 may be a telecommunications system, sometimes also referred to as a cellular radio system, cellular network or wireless communications system or network. In some examples, the communications system 100 may comprise network nodes which may serve receiving nodes, such as wireless devices, with serving beams. In some examples, the communications system 100 may be a 5G system, or Next Gen network or newer system, such as, e.g., a sixth generation 6G system. The telecommunications system may also, or alternatively, support other technologies, such as LTE, e.g., LTE Frequency Division Duplex (FDD), LTE Time Division Duplex (TDD), LTE Half-Duplex Frequency Division Duplex (HD-FDD), LTE operating in an unlicensed band, etc. The telecommunications system may additionally support other technologies, such as Wideband Code Division Multiple Access (WCDMA), Universal Terrestrial Radio Access (UTRA) TDD, Global System for Mobile communication (GSM) / Enhanced Data Rate for GSM Evolution (EDGE) Radio Access Network (GERAN) network, Ultra-Mobile Broadband (UMB), EDGE network, network comprising of any combination of Radio Access Technologies (RATs) such as e.g., Multi-Standard Radio (MSR) base stations, multi-RAT base stations etc., any 3rd Generation Partnership Project (3GPP) cellular network, Wireless Local Area Network / s (WLAN) or WiFi network / s, Worldwide Interoperability for Microwave Access (WiMax), Internet of Things (loT), Machine Type Communication (MTC), IEEE 802.15.4-based low-power short-range networks such as IPv6 over Low-Power Wireless Personal Area Networks (6LowPAN), Zigbee, Z-Wave, Bluetooth Low Energy (BLE), or any cellular network or system. The communications system 100 comprises a first node 111. The communications system 100 may comprise at least another node 112. In some examples, such as in the nonlimiting example depicted in Figure 4, the first node 111 and the another node 112 may be comprised a set of nodes 121 comprised in the communications system 100. In other examples, the first node 111 may be a different node from the set of nodes 121. For example the first node 111 may be a coordinating authority of the set of nodes 121. The set of nodes 121 may comprise one or more second nodes 122 and one or more third nodes 123. In the non-limiting example of Figure 4, the set of nodes 121 comprises five nodes. Particularly, the one or more second nodes 122, in this non-limiting example depicted in Figure 4, comprise the first node 111 and the another node 112. The one or more third nodes 123, in this nonlimiting example depicted in Figure 4, comprise the first node 111 , the another node 112 and a further node 133. The set of nodes 121 , in this non-limiting example depicted in Figure 4, may comprise a fourth node 134 and a fifth node 135. It may be understood that this is for illustration purposes and non-limiting. In other examples, the set of nodes 121 may comprise fewer or additional nodes.

[0060] Any of the first node 111 and the set of nodes 121 , such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123, may be understood, respectively, as a first computer system, a set of computer systems, such as a another computer system, one or more second computer systems, and one or more third computer systems. Any of the first node 111 and the set of nodes 121 , such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123, may have a capability to perform machine-implemented learning procedures, which may be also referred to as “machine learning” (ML).

[0061] Each of the nodes in the set of nodes 121, which is some examples may include the first node 111, may manage a respective agent capable of training a respective machine learning model with a respective set of data.

[0062] Any of the first node 111 and the set of nodes 121 , such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123, may have a capability to manage an artificial neural network, that is, e.g., a respective artificial neural network. The artificial neural network may be understood as a machine learning framework, which may comprise a collection of connected sub-nodes, where in each sub-node or perceptron, there may be an elementary decision unit. Each such sub-node may have one or more inputs and an output. The input to a sub-node may be from the output of another sub-node or from a data source. Each of the sub-nodes and connections may have certain weights or parameters associated with it. In order to solve a decision task, the weights may be learnt or optimized over a data set which may be representative of the decision task. The most commonly used sub-node may have each input separately weighted, and the sum may be passed through a non-linear function which may be known as an activation function. The nature of the connections and the sub-node may determine the type of the neural network, for example a feedforward network, recurrent neural network etc. That any of the first node 111 and the set of nodes 121, such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123 may have the capability to manage the artificial neural network may be understood herein as having the capability to store the training data set and the models that may result from the machine learning, to train a machine learning model, and once the machine learning model may have been trained, to use this model for prediction. Any of the first node 111 and the set of nodes 121 , such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123 may be understood as a node having a capability to use, e.g., execute, the machine learning model, once the machine learning model may have already been trained.

[0063] Any of the first node 111 and the set of nodes 121 , such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123 used for training the artificial neural network may, for example, support running python / Java with Tensorflow or Pytorch, Theano etc... Any of the first node 111 and the set of nodes 121 , such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123 may also have GPU capabilities.

[0064] Any of the first node 111 and the set of nodes 121 , such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123 may be implemented as a standalone server in e.g., a host computer in the cloud 115. In other examples, any of the first node 111 and the set of nodes 121 , such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123 may be a distributed node or and may perform some of its respective functions locally, e.g., by a client manager, and some of its functions in the cloud, by e.g., a server manager. In other examples, Any of the first node 111 and the set of nodes 121 , such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123 may perform its functions partially in the cloud, in collaboration or collocated with a radio network node. Yet in other examples, any of the first node 111 and the set of nodes 121, such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123 may also be implemented as partially collaborating with processing resources in a server farm. Any of the first node 111 and the set of nodes 121, such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123 may be under the ownership or control of a service provider or may be operated by the service provider, or on behalf of the service provider.

[0065] In preferred embodiments, such as the non-limiting example depicted in Figure 4, any of the first node 111 and the set of nodes 121 , such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123 may be a device, e.g., a wireless device or wireless communication device, such as a 5G UE, or a UE, which may also be known as e.g., mobile terminal, wireless terminal and / or mobile station, a Customer Premises Equipment (CPE) a mobile telephone, cellular telephone, or laptop with wireless capability, just to mention some further examples. Any of the first node 111 and the set of nodes 121 , such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123 as a device may be, for example, portable, pocket-storable, hand-held, computer- comprised, or a vehicle-mounted mobile device, enabled to communicate voice and / or data, via the RAN, with another entity, such as a server, a laptop, a Personal Digital Assistant (PDA), or a tablet, Machine-to-Machine (M2M) device, device equipped with a wireless interface, such as a printer or a file storage device, modem, sensor, camera, loT device, or any other radio network unit capable of communicating over a radio link in a communications system. Any of the first node 111 and the set of nodes 121 , such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123 as a device may be enabled to communicate wirelessly in the communications system 100. The communication may be performed e.g., via a RAN, and possibly the one or more core networks, which may be comprised within the communications system 100.

[0066] Particularly, any of the first node 111 and the set of nodes 121 , such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123 may have a capability to connect, directly or indirectly, via resources, according to one or more communication subgraphs in a fully decentralized setting.

[0067] In other examples, any of the first node 111 and the set of nodes 121, such as the another node 112, the one or more second nodes 122 and the one or more third nodes 123 may be a radio network node, such as the radio network node 140 described next.

[0068] The communications system 100 may comprise a radio network node 140, as depicted in the non-limiting example of Figure 4. That is, a transmission point such as a radio base station, for example a gNB, or any other network node with similar features capable of serving a device or user equipment, such as a wireless device or a machine type communication device, in the communications system 100. In some examples, the network node 140 may be a distributed node, and may partially perform its functions in collaboration with a virtual node in the cloud. The network node 140 may be directly connected to one or more core networks, e.g., to one or more network nodes in the one or more core networks.

[0069] A geographical area, which in some embodiments may be divided into cell areas, may be served by the radio network node 140. The radio network node 140 may be of different classes, such as, e.g., macro base station, home base station or pico base station, based on transmission power and thereby also cell size. In some examples, the radio network node 140 may serve receiving nodes with serving beams. The radio network node 140 may support one or several communication technologies, and its name may depend on the technology and terminology used.

[0070] The first node 111 may be configured to communicate within the communications system 100 with the another node 112 over a first link 151 , e.g., a radio link. The first node 111 may be configured to communicate within the communications system 100 with the further node 133 over a second link 152, e.g., a radio link. The another node 112 may be configured to communicate within the communications system 100 with the further node 113 over a third link 153, e.g., a radio link. The another node 112 may be configured to communicate within the communications system 100 with the fifth node 135 over a respective fourth link 154, e.g., a radio link. The further node 113 may be configured to communicate within the communications system 100 with the fifth node 135 over a fifth link 155, e.g., a radio link. The fourth node 134 may be configured to communicate within the communications system 100 with the further node 113 over a sixth link 156, e.g., a radio link. The fourth node 134 may be configured to communicate within the communications system 100 with the fifth node 135 over a seventh link 157, e.g., a radio link. The radio network node 140 may be configured to communicate within the communications system 100 with the first node 111 over an eighth link 158, e.g., a radio link. The radio network node 140 may be configured to communicate within the communications system 100 with the further node 113 over a ninth link 159, e.g., a radio link. The radio network node 140 may be configured to communicate within the communications system 100 with the another node 112 over a tenth link 160, e.g., a radio link. The radio network node 140 may be configured to communicate within the communications system 100 with the fifth node 135 over an eleventh link 161, e.g., a radio link. The radio network node 140 may be configured to communicate within the communications system 100 with the fourth node 114 over a twelfth link 162, e.g., a radio link.

[0071] Any of the first link 151, the second link 152, the third link 153, the fourth link 154, the fifth link 155, the sixth link 156, the seventh link 157, the eighth link 158, the ninth link 159, the tenth link 160, the eleventh link 161 and the twelfth link 162 may be a direct link or may be comprised of a plurality of individual links, wherein it may go via one or more computer systems or one or more core networks in the communications system 100, which are not depicted in Figure 4, or it may go via an optional intermediate network. The intermediate network may be one of, or a combination of more than one of, a public, private or hosted network; the intermediate network, if any, may be a backbone network or the Internet; in particular, the intermediate network may comprise two or more sub-networks, which is not shown in Figure 4.

[0072] In general, the usage of “first”, “second”, “third”, “fourth”, “fifth”, “sixth”, “seventh”, “eighth”, “ninth”, “tenth” and / or “eleventh” herein may be understood to be an arbitrary way to denote different elements or entities, and may be understood to not confer a cumulative or chronological character to the nouns they modify.

[0073] Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Other embodiments, however, are contained within the scope of the subject matter disclosed herein, the disclosed subject matter should not be construed as limited to only the embodiments set forth herein; rather, these embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art.

[0074] Embodiments of a computer-implemented method, performed by the first node 111 , will now be described with reference to the flowchart depicted in Figure 5. The method is for handling a machine learning model. The first node 111 operates in the communications system 100 comprising the set of nodes 121.

[0075] In some examples, the first node 111 may be comprised in the set of nodes 121.

[0076] In some examples, the first node 111 may be a coordinating authority, separate from the set of nodes 121.

[0077] Several embodiments are comprised herein. In some embodiments all the actions may be performed. In some embodiments, some actions may be optional. In Figure 5, optional actions are indicated with dashed lines. It should be noted that the examples herein are not mutually exclusive. Components from one embodiment may be tacitly assumed to be present in another embodiment and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments. One or more embodiments may be combined, where applicable. All possible combinations are not described to simplify the description. It may be noted that the Actions described in reference to Figure 5 may be performed in a different order than that depicted in Figure 5.

[0078] Action 501

[0079] As stated earlier, each of the nodes in the set of nodes 121 manages a respective agent capable of training a respective machine learning model with a respective set of data. That is, each of the nodes in the set of nodes 121 may manage its own agent which may be able to train a local copy of a machine learning model with its own set of data.

[0080] That each of the nodes in the set of nodes 121 manages a respective agent may be understood to mean that each of the nodes may run its own agent.

[0081] The first node 111 may be understood to lack access to the respective sets of data. In examples wherein the first node 111 may be one of the nodes in the set of nodes 121 , it may be understood that the first node 111 may be understood to lack access to the respective sets of data of the other nodes in the set of nodes 121. This may be understood to ensure privacy. Each node in the set of nodes 121 may be understood to lack access to the respective set of data of other nodes in the set of nodes 121.

[0082] In this Action 501 , the first node 111 may determine at least a subset of the set of nodes 121.

[0083] In some embodiments, the determining in this Action 501 of at least the subset of the set of nodes 121 may be performed by detecting the subset of the set of nodes 121. The detecting may be performed by radio detection. The detecting may be based on the first node 111 having received high enough Reference Signals Received Power (RSRP) and / or Reference Signal Received Quality (RSRQ) measurements from nearby nodes, e.g., other nearby nodes, in the set of nodes 121. That is, RSRP and / or RSRQ over a certain threshold.

[0084] In some embodiments, the determining in this Action 501 of at least the subset of the set of nodes 121 may be performed based on a connectivity criterion. The connectivity criterion may be, for example, a possibility to establish a Device to Device (D2D) connection via the help of a radio network node, e.g., a gNB, which may establish the communication channel and a capability in the node of the set of nodes 121, e.g., a UE, to send signals to another node of the set of nodes 121, e.g., another UE, without the need for the radio network node to route such signals indirectly. Other shortage communication mechanisms may apply.

[0085] Action 502

[0086] The nodes in the set of nodes 121 have a capability to connect, directly or indirectly, via resources according to one or more communication subgraphs in a fully decentralized setting. In this Action 502, the first node 111 may initiate allocation of communication channels with each of the nodes in the subset of the set of nodes 121. That is, the first node 111 may initiate the physical channel allocation with the subset of the set of nodes 121 it may have detected in Action 501. The resources may be understood to be time-frequency radio resources.

[0087] Initiating may be understood as triggering, enabling, facilitating or starting.

[0088] The initiating in this Action 502 of the allocation may be performed by sending an allocation request to the radio network node 140.

[0089] For example, the nodes in the set of nodes 121 may be connected through side-links connections which may be handled by the radio network node 140, e.g. a gNB. In such a setup, the radio network node 140 may for example handle the control plane traffic for all the nodes in the set of nodes 121, and each node may directly communicate with the other nodes in the set of nodes 121 to share user plane traffic, such as, for example, model updates.

[0090] A non-limiting example of an initial communication subgraph, which may be also referred to as an initial connectivity graph, e.g., G for an example setup wherein the subset of the set of nodes 121 may comprise three nodes which may be UEs, UE1, UE2 and UE3, for uplink and downlink may be as shown in Table 1 :

[0091] Table 1

[0092] The initial connectivity graph may be fully connected, which may be understood to mean that the maximum number of physical channels both for uplink and downlink may need to be allocated in this Action 502 for this communication to take place.

[0093] The radio network node 140 may then be enabled to imitate communication channels for available frequencies, and handle control plane traffic.

[0094] Action 503

[0095] In this Action 503, the first node 111 may exchange a respective estimate of a common machine learning model with the other nodes in the determined subset of the set of nodes 121.

[0096] The exchanging in this Action 503 of the respective estimate may be performed by broadcasting the respective estimate by the first node 111 and receiving a respective broadcast comprising the respective estimate by each of the other nodes in the determined subset of the set of nodes 121.

[0097] For example to exchange the respective estimate of the common machine learning model in this Action 503 may comprise broadcasting its model parameters to the other nodes in the determined subset of the set of nodes 121.

[0098] The first node 111 may refrain from determining any parameters for the respective machine learning models with the proviso the first node 111 may be a separate node from the set of nodes 111.

[0099] Action 504

[0100] In this Action 504, the first node 111 determines one or more second nodes 122 out of at least the subset of the set of nodes 121. As mentioned earlier, each of the nodes in the set of nodes 121 manages the respective agent capable of training the respective machine learning model with the respective set of data. Also a mentioned earlier, the nodes in the set of nodes 121 have the capability to connect, directly or indirectly, via resources according to one or more communication subgraphs in the fully decentralized setting.

[0101] Determining may be understood as calculating, deriving, selecting or similar.

[0102] In some embodiments, the determining 504 of the one or more second nodes 122 may be initially performed one of: randomly and based on one or more first criteria. The one or more first criteria may be, for example, having received the respective estimate of the common machine learning model from the one or more second nodes 122 within a first period of time. In other examples, the one or more criteria may be receiving an indication of the one or more second nodes 122 from another entity in the communications system 100, e.g., an actor, such as a Network function consumer, which may expect a task to be learned jointly by multiple UEs. That is, in such examples the model architecture may be already known.

[0103] In some embodiments, the determining in this Action 504 of the one or more second nodes 122 may be based on the exchanged respective estimates of the common machine learning model.

[0104] Action 505

[0105] In this Action 505, the first node 111 determines one or more third nodes 123 out of at least the subset of the set of nodes 121 . The one or more third nodes 123, as a group, may be understood to be different than the one or more second nodes 122. However, in some examples, the one or more second nodes 122 may partially overlap with the one or more third nodes 123. In particular examples, the one or more second nodes 112 and the one or more third nodes 123 may have a node in common. In other examples, the one or more second nodes 112 and the one or more third nodes 123 may not share any common node.

[0106] In some embodiments, the determining 504 of the one or more second nodes 122 and the determining 505 of the one or more third nodes 123 may be initially performed one of: randomly and based on the one or more first criteria. The one or more first criteria may be, for example, having received the respective estimate of the common machine learning model from the one or more third nodes 123 within a second period of time. In other examples, as explained before, the model architecture may be already known, and the one or more criteria may be receiving an indication of the one or more third nodes 123 from the another entity in the communications system 100. In some examples, the one or more criteria may comprise receiving certain assistance from the radio network node 140, e.g., a gNB, since the radio network node 140 may be considered a facilitator that may allocate communication channels between nodes and as such, always influence on how communication may need to be established, that is, which communication channel to use.

[0107] In some embodiments, the determining in this Action 505 of the one or more third nodes 123 may be based on the exchanged respective estimates of the common machine learning model.

[0108] Action 506

[0109] In this Action 506, the first node 111 obtains, using a decentralized consensus mechanism, a first indication of a heterogeneity of the respective sets of data used by, and a respective compute capability of, each of the respective agents managed by the one or more second nodes 122.

[0110] Compute capability may be understood as processing capability (e.g., CPU specification, available memory, total memory, available storage, total storage), presence of hardware acceleration mechanisms, etc.

[0111] A consensus mechanism may be understood as a protocol that may be used to identify an agreement between two or more nodes on a single data input. More details about the consensus mechanism used here are provided below.

[0112] In some embodiments, the obtaining in this Action 506 of the first indication may comprise receiving the first indication.

[0113] In some embodiments, the first node 111 may obtain the first indication in this Action 506 by calculating the first indication. The first indication may be, e.g., a score.

[0114] In some embodiments, the obtaining in this Action 506 of the first indication may be based on the exchanged respective estimates of the common machine learning model. For example, the first node 111 may calculate the first indication using one or more of the model parameters that may have been broadcasted by the one or more second nodes 122 and received by the first node 111.

[0115] As mentioned above, according to embodiments herein, the first node 111 , which in some non-limiting examples may be, or may act as, a central unit of the communications system 100, may infer various properties of the respective machine learning model trained by the respective agent managed by each of the nodes in the one or more second nodes 122, that is, a local model at each agent. Specifically, in this Action 506, the first node 111 , e.g., the central unit, may infer the level of data and system heterogeneity among the agents, which in this Action 506, may be understood to be the one or more second nodes 122, quantified, for example, in terms of an empirical statistical distribution of the gradient updates for a particular subset of parameter vector components and the similarity of the compute properties of the one or more second nodes 122. In other words, the first indication may indicate the heterogeneity of the respective sets of data used by each of the respective agents managed by the one or more second nodes 122 by being a quantification of an empirical statistical distribution of the gradient updates for a particular subset of parameter vector components. The first indication may also indicate the heterogeneity of the respective compute capability of each of the respective agents managed by the one or more second nodes 122, that is, the similarity of the compute properties of the one or more second nodes 122.

[0116] In many cases, the first node 111 may only be interested in coarse estimates of heterogeneity and hence may need only to obtain rough estimates of said quantities.

[0117] More specifically, the consensus mechanism may be used to compute statistics of the quantities of interest. For example, the nodes in the set of nodes 121 may be indexed by {j} and a quantity xtassociated with all nodes may be considered. xtmay be, for example, in a given global iteration, a particular component of the model held by node i , that is, zfabove, or it may be a particular component of the gradient update at node i (Vffz^Zi above), with compute capability of c. It may also be a combination of the two. This may be understood to mean that these scores may measure the difference among the gradients as well, and not just the difference between the model vectors alone. For generality, X{ may be used herein for the, e.g., per-component, quantity whose difference among the nodes, that is, agents, may need to be examined. C may either be a vector detailing the specifications of a given node, e.g., a device, or it may also be an estimation of the expectation of the node to deliver a model, learned in a certain timeframe. The expectation of the node to deliver the model may be understood to refer that it may be understood that even if two or more nodes may be considered to be homogeneous while performing the proposed checks, there may still be some unforeseen element, e.g., a software failure, that may impact their behavior during the training process.

[0118] The consensus mechanism to estimate statistics

[0119] For an arbitrary function g(. ), and a graph G', a consensus averaging l / N')ig xi') may be estimated by using a consensus mechanism, where N' may be understood to be the number of one or more second nodes 122 in G'. The consensus averaging may converge linearly, and the convergence speed may depend on the spectral properties of a Laplacian associated with the connectivity subgraph of the one or more second nodes 122. Specifically, if, for example, a standard decentralized average consensus algorithm is used, where the weight matrix may be taken to be W = I - eL, for some appropriately selected number e, where L may be understood to be the graph Laplacian, and I may be understood to be an identity matrix, that is, a diagonal matrix where the elements may be 1 , then the convergence speed may be dictated by the second-largest eigenvalue of W, which may be understood to equal 1 - eA2, where A2may be the second-smallest eigenvalue of the Laplacian [1], As one example, the arithmetic average may be estimated by taking g(x) = x. As another example, the variance may be estimated, by first computing the second moment using g(x) = x2and then subtracting the squared arithmetic average. As another example, higher-order moments may be estimated, e.g., Kurtosis. Such moments may give information about how “heavy” the tail of the statistical distribution may be. It may be understood that the larger tail, the larger high-order moments. Here, G' may be the whole G, or a part of it, depending on what inference may need to be eventually made.

[0120] The consensus averaging to obtain the estimate of l / N')ig xi') may be implemented in terms of a standard consensus algorithm that may iteratively compute, by passing messages along the edges of G', where the initial values of each qtmay be set to #(%;). wnimay be understood to be a weight of the corresponding #(%;). The weight may be, for example, a function of communication quality. Variations on this standard consensus iteration may also be used, for example an accelerated consensus algorithm may be used.

[0121] In one example, rather than estimating the average over the nodes of the graph G', verage may be estimated over the edges of the graph G', where L here may be understood to be the number of edges of G'. In a graph comprising nodes and edges, a node may connect with one or more nodes using edges. Every edge may have a quantity or a weight. As the formula may be understood to indicate, the average over the edges may be defined as the sum of a quantity associated with the edges, the sum being taken over all edges, divided by the number of edges. This may be understood to stand in contrast to the average over the nodes, which may be instead the sum of a node-based quantity divided by the number of nodes. This sample average may be estimated through a two-step consensus algorithm, e.g., such as that described by Chen- Larsson in “Consensus-Based Distributed Computation of Link-Based Network Metrics”, IEEE Signal Processing Letters, Volume: 28, 2021 , for any polynomial function In particular, with g(x,y) = (x - y)2, the so-called total variation (TV) may be obtained, which may be understood to be the squared differences between the node values averaged over all links.

[0122] In one example, xtmay be taken to be a particular model vector component at node, e.g., agent, i.

[0123] In one example, xtmay be taken to be the gradient vector update of a particular model vector component at agent i. A gradient vector may be understood as a vector which may comprise the partial derivatives of a multi-variate function.

[0124] In some embodiments, the parameters of a respective model of the one or more second nodes 122 and the one or more third nodes 123 may have been updated within at least one of the one or more second nodes 122 and the one or more third nodes 123, according to the decentralized consensus mechanism.

[0125] Numerical Examples

[0126] In what follows, a heterogeneity score (HS), as an example of the first indication, may be understood to denote the result of the consensus mechanism when applied to a particular subgraph. In obtaining HS, f (. ) may be selected, for example, to be a moment of a predefined order. That is, in some embodiments, the first indication may be based on a moment of a pre-defined order. A moment may be understood as a relationship as expressed in a graph which may describe how nodes, e.g., UEs, may connect to each other, that is, nodes and edges. The nodes, e.g., UEs, may be understood to be moving and that may be understood to affect how they communicate due to interference and their communication channel. A moment of pre-defined order may be understood as a snapshot of the current location and the properties of the communication channel and the compute availability of the nodes.

[0127] As such, heterogeneity may go beyond data similarity. Data similarity may be assessed by way of distance functions, e.g., cosine similarity, 11 , 12 norms, but also via f-divergences such as KL divergence or Hellnger divergence if the probability distribution of two different datasets may be being compared or, in other words, the probability that an event from one distribution may occur in other. However, beyond that, the difference in the capabilities of the nodes, e.g., devices, may also be understood to be considered here, such as compute capabilities, but also spare capacity to take new tasks given the current tasks. The latter may be understood to be relevant since mixing heterogeneous nodes, e.g., devices, in the same collaborative process may introduce stranglers that may slow down the learning task.

[0128] Action 507

[0129] In this Action 507, the first node 111 obtains, using the decentralized consensus mechanism, a second indication of the heterogeneity of the sets of data used by, and the respective compute capability of, each of the respective agents managed by the one or more third nodes 123.

[0130] Action 507 may be understood to be performed in a similar manner as described for Action 506, but in reference to the second indication and the one or more third nodes 123.

[0131] In some embodiments, the obtaining in this Action 506 of the first indication and the obtaining in Action 507 of the second indication may comprise receiving the first indication and the second indication, respectively.

[0132] In some embodiments, the obtaining in this Action 507 of the second indication may be based on the exchanged respective estimates of the common machine learning model.

[0133] In some embodiments, the determining in Action 504 of the one or more second nodes 122, the determining in Action 505 of the one or more third nodes 123, the obtaining in Action 506 of the first indication and the obtaining in this Action 507 of the second indication may be based on the exchanged respective estimates of the common machine learning model. This maybe because the determining in Action 504 and Action 505 may be based on when the first node 111 may receive the respective estimates of the one or more second nodes 122 and the one or more third nodes 123, respectively, and the first indication and the second indication may be calculated using the received respective estimates of the one or more second nodes 122 and the one or more third nodes 123. In some embodiments, the obtaining in Action 506 of the first indication and the obtaining in this Action 507 of the second indication may comprise: i) performing a consensus update of the respective estimate by the first node 111 and the respective estimate by each of the one or more second nodes 122 and one or more third nodes 123, respectively, ii) performing a local model update with the respective data of the first node 111 , and iii) computing the first indication and the second indication, respectively, as scores computed by an average decentralized consensus mechanism within each of the one or more second nodes 122 and the one or more third nodes 123.

[0134] In some embodiments, each of the first indication and the second indication may be based on a moment of a pre-defined order.

[0135] Action 508

[0136] In this Action 508, the first node 111 determines whether to group the one or more second nodes 122 and the one or more third nodes 123 into a same group, e.g., cluster, to train the common machine learning model, or to refrain from grouping the one or more second nodes 122 and the one or more third nodes 123 to train into the same group to train the common machine learning model. The determining in this Action 508 of whether to group or not is based on a difference between the first indication and the second indication with respect to a threshold.

[0137] In one example, one of the above consensus algorithms may be run multiple times on different graphs and the results may be compared. For example, the algorithm may be run on a first subgraph, e.g., Gltof G defined by a first set of nodes, such as the one or more second nodes 122, and a second subgraph, e.g., G2, of G defined by a second set of nodes, such as the one or more third nodes 123. This may give two results R and R2, that may be, for example a statistical moment of a pre-defined order. If xtis taken to contain the gradient vector update, or model vector component, then the results R and R2obtained from the respective graphs G}and G2may be compared. According to this Action 508, if R exceeds R2in magnitude by a predetermined amount, it may be concluded that the data possessed by the nodes in G}is more heterogeneous than the data possessed by the nodes in G2, and vice versa.

[0138] Merging two clusters of nodes, e.g., agents, into one group

[0139] For an initial cluster of nodes S, the HS value may be computed within the cluster, denoted by If by merging another clusters' into S, the new HS value, denoted 7SUS / , is smaller than 7S, then two clusters may be merged and a new cluster may be obtained with smaller HS. This community merging step may be relevant to a decentralized machine learning setting, where in each communication round, it may be desired to schedule a subset of nodes, which may be understood to manage agents, to train a model jointly with more homogeneous data and system representation. System homogeneity may be understood to be also relevant to consider in this context since, even if data may be similar between the clusters, the nodes, e.g., devices, may be very different, which may be understood to mean that slow nodes, may be mixed together with fast nodes, thus slowing down the process. If fast nodes are mixed together with fast nodes, and slow nodes with slow, at least a learned model may be obtained from the first cluster without waiting for the second.

[0140] An example of possible application may be stated as follows. The aim may be understood to be to find a cluster of N nodes with homogeneous data representation by looking at the gradient vectors after model training.

[0141] One possible algorithm may be as follows. In a first step, according to Action 504, the first node 111 may start from a random link in a network, where S may denote the two nodes of this link, e.g., the one or more second nodes 122. The first node 111 may then, according to Action 506, compute the HS value, denoted Next, the first node 111 may, according to Action 505, select two, or more, but the required computation time may increase, random of similar compute capability nodes and n2that may be connected to the nodes in S. The first node 111 may then, according to Action 507, compute the HS values and Visun^, and, according to this Action 508, choose the node that may give the smaller HS and merge it into S. Then step 2 may be repeated until the cluster size N may be reached.

[0142] Bottom-up building of node, e.g., agent, clusters based on data heterogeneity

[0143] Given a large cluster of connected nodes S, the first node 111 may find a smaller subset of connected nodes S' with similar system capability that may represent the statistical similarity of the data samples within the larger clusters. Once this smaller representative cluster may be identified, the decentralized machine learning algorithms may be implemented within S' instead of S, to reduce communication costs and energy consumption.

[0144] A possible non-limiting example of such an algorithm may be that, in a first step, agents managed by nodes may perform collaborative training of a machine learning model over locally available data. After a certain number of rounds, the algorithm may be stopped. The agents may obtain the gradient vectors. In a second step, the first node 111 , according to Action 504, may start from a random link in a network, JV that may be used to denote the subgraph that may consist of the two second nodes 122 and the link between them. The first node 111 may then, according to Action 506, compute the HS value of the subgraph, y|Arj. In a third step, the first node 111, in accordance with Action 505, may select two, or more, but the required computation time may increase, random nodes n±and n2that may be connected to the nodes in S. The first node 111 then, in accordance with Action 507, may compute and F{ un2), choose the node that may give the smaller HS and merge it into JV. In a fourth step, step 2 may be repeated in accordance with action 510, until the cluster size N may be reached.

[0145] Top-down splitting of a group of agents into two clusters

[0146] For an initial cluster of nodes S with HS value 7S, the first node 111 may split it into two clusters Si , in accordance with Action 504, and S2and in accordance with Action 505, and the first node 111 may compute the HSs 7S1, in accordance with Action 506, and VS2, in accordance with Action 507. If both 7S1and VS2are smaller than Vs, as determined in accordance with this Action 508, then it may be understood to be implied that two clusters after splitting may be more homogeneous in terms of the node attribute distribution within each cluster.

[0147] An extension of this algorithm may be to remove a random node from S at a time, in accordance with Action 504, and later Action 505, and, in accordance with Action 508, compare the HSs, as obtained in accordance with Action 506 and Action 507, before and after removing the node. If the HS become smaller by removing the node, then it may be understood to mean that the selected node may be understood to be an outlier and may not be included in the community.

[0148] Use cases

[0149] When the groups may have been identified, each group may perform the following tasks. In some non-limiting examples, each group may train an ML-model, the common machine learning model, where the common machine learning model trained within the group may be part of a local update in a federated learning process. For example, in federated learning, it may happen that, over iterations, the gradients may start to become very similar because the common machine learning model may converge. Therefore, new clusters may be obtained to reduce the volume of data as the common machine learning model may mature. In other nonlimiting examples, each group may perform split learning, decentralized estimation with local observations, decentralized hypothesis testing etc....

[0150] Additional examples

[0151] In one example, the {%f} that may be examined using the method may be only a subset of the components of the total model vectors or gradient update vectors. For example, this subset may be selected based on an importance score. Any of the nodes in the set of nodes 121 may identify that some model components, e.g., layers in a Deep Neural Network (DNN) may be more important than others, and instruct the set of nodes 121 , e.g., agents, to only consider those components when computing the heterogeneity metric. For instance, in some systems, some parameters may not be trainable, so it may be possible to then choose only those parameters that may be trained.

[0152] In one example, the first node 111 , e.g., central unit, may have an interest in determining how far from convergence to consensus on a common machine learning model the agents may be. This may be quantified, for example, in terms of the empirical statistical distribution, or some property of it, e.g., the maximum or root-mean-square deviation from the average, of the agents' local model vectors, that is the vectors of the respective machine learning models of the nodes. This information may be obtained using the same mechanism as disclosed above and then used to determine, for example, whether the learning process or consensus formation process may need to stop.

[0153] Action 509

[0154] In this Action 509, the first node 111 outputs a third indication indicating a result of the determining 508 of whether to cluster or not.

[0155] Outputting may be understood as providing, sending or similar.

[0156] In some embodiments, the third indication may be at least one of: a) an instruction instructing the one or more second nodes 122 and the one or more third nodes 123 to train respective machine learning models, wherein the training may be fully decentralized learning, and b) a request to the radio network node 140 to deallocate allocated resources between nodes excluded from a cluster, wherein the result of the determining 508 may have been to not cluster, and wherein the allocated resources may be a subset of the resources connecting the set of nodes 121. That is, the request to the radio network node 140 to deallocate allocated resources may be a request for the radio network node 140 to tear down the uplink / downlink. The radio network node 140 may then be enabled to release the allocated resource.

[0157] In some of the latter embodiments, the one or more communication subgraphs may be updated based on the deallocation of resources. Following the example provided in Table 1 above, the updated connectivity subgraph that may result from a non-limiting iteration may be that as shown in Table 2:

[0158] Table 2

[0159] Action 510

[0160] In this Action 510, the first node 111 may iterate the method by one of: adding and subtracting an additional node to at least one of the one or more second nodes 122 and the one or more third nodes 123, until a consensus criterion may be reached. In at least one of the iterations, at least one of the first indication and the second indication may be based on computations performed within at least one of the one or more second nodes 122 and the one or more third nodes 123, according to the decentralized consensus mechanism.

[0161] In every iteration, the additional node added or subtracted may be selected one of: randomly and based on one or more second criteria. The one or more second criteria may be, for example, pathloss, signal strength, RSRP, RSRQ, etc.

[0162] In some embodiments, the respective data may have been compressed in a decentralized manner, and the first node 111 for at least one of the clusters may use only the respective data from one agent.

[0163] In some embodiments, the training may be federated learning, and the first node 111 for at least one of the clusters may use only the respective data from one agent.

[0164] Embodiments of a computer-implemented method, performed by the communications system 100 comprising the set of nodes 121, will now be described with reference to the flowchart depicted in Figure 6. The method is for handling the machine learning model. The first node 111 is comprised in the communications system 100.

[0165] In some examples, the first node 111 may be comprised in the set of nodes 121.

[0166] In some examples, the first node 111 may be a coordinating authority, separate from the set of nodes 121.

[0167] Several embodiments are comprised herein. In some embodiments all the actions may be performed. In some embodiments, some embodiments of the actions may be optional. In Figure 6, optional actions are indicated with dashed lines. It should be noted that the examples herein are not mutually exclusive. Components from one embodiment may be tacitly assumed to be present in another embodiment and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments. One or more embodiments may be combined, where applicable. All possible combinations are not described to simplify the description. The detailed description of some of the following corresponds to the same references provided above, in relation to the actions described for the first node 111 , and will thus not be repeated here. For example to exchange the respective estimate of the common machine learning model may comprise broadcasting model parameters to the other nodes in the determined subset of the set of nodes 121.

[0168] Actions 601-603 and Actions 607-613 are performed by at least the first node 111 comprised in the communications system 100.

[0169] In some embodiments, the method may be performed by at least a plurality of nodes in the set of nodes 121. In some embodiments, the method, as e.g., described in Actions 601-603 and Actions 607-613, may be performed by at least a plurality of nodes in the set of nodes 121.

[0170] Action 601

[0171] In this Action 601 , at least the first node 111 may determine at least a subset of the set of nodes 121.

[0172] This Action may be understood to be performed as described for Action 501.

[0173] Action 602

[0174] In this Action 602, at least the first node 111 may initiate the allocation of the communication channels with each of the nodes in the subset of the set of nodes 121.

[0175] This Action may be understood to be performed as described for Action 502.

[0176] Action 603

[0177] In this Action 603, at least the first node 111 may exchange the respective estimate of the common machine learning model with the other nodes in the determined subset of the set of nodes 121.

[0178] In some embodiments, at least one of the following may apply: a) the determining in Action 601 of at least the subset of the set of nodes 121 may be performed by detecting the subset of the set of nodes 121 , b) the determining in Action 601 of at least the subset of the set of nodes 121 may be performed based on the connectivity criterion, c) the initiating in Action 602 of the allocation may be performed by sending the allocation request to the radio network node 140, and d) the exchanging in this Action 603 of the respective estimate may be performed by broadcasting the respective estimate by the first node 111 and receiving the respective broadcast comprising the respective estimate by each of the other nodes in the determined subset of the set of nodes 121.

[0179] This Action may be understood to be performed as described for Action 503.

[0180] Action 604

[0181] In this Action 604, at least another node 112 in the set of nodes 121 may receive the respective estimate. That is, the respective estimate that may have been exchanged, e.g., broadcasted, by the first node 111 in Action 603.

[0182] Action 605

[0183] In this Action 605, at least another node 112 in the set of nodes 121 may determine, respective one or more second nodes 122 out of at least a subset of the set of nodes 121. This Action may be understood to be performed as described for Action 504 for the first node 111.

[0184] Action 606

[0185] In this Action 606, at least another node 112 in the set of nodes 121 may send the respective estimate of the of the common machine learning model to the other nodes in the determined subset of the set of nodes 121 . This may be performed by broadcasting. The other nodes may comprise the first node 111.

[0186] Action 607

[0187] In this Action 607, at least the first node 111 determines the one or more second nodes 122 out of at least the subset of the set of nodes 121. Each of the nodes in the set of nodes 121 manages the respective agent capable of training the respective machine learning model with the respective set of data. The nodes in the set of nodes 121 have the capability to connect, directly or indirectly, via the resources according to the one or more communication subgraphs in the fully decentralized setting.

[0188] This Action may be understood to be performed as described for Action 504.

[0189] Action 608

[0190] In this Action 608, at least the first node 111 determines the one or more third nodes 123 out of at least the subset of the set of nodes 121 .

[0191] This Action may be understood to be performed as described for Action 505.

[0192] Action 609

[0193] In this Action 609, at least the first node 111 obtains, using the decentralized consensus mechanism, the first indication of the heterogeneity of the respective sets of data used by, and the respective compute capability of, each of the respective agents managed by the one or more second nodes 122.

[0194] This Action may be understood to be performed as described for Action 506.

[0195] Action 610

[0196] In this Action 610, at least the first node 111 obtains, using the decentralized consensus mechanism, the second indication of the heterogeneity of the sets of data used by, and the respective compute capability of, each of the respective agents managed by the one or more third nodes 123.

[0197] In some embodiments, the determining in Action 607 of the one or more second nodes

[0198] 122, the determining in Action 608 of the one or more third nodes 123, the obtaining in Action 609 of the first indication and the obtaining in this Action 610 of the second indication may be based on the exchanged respective estimates of the common machine learning model.

[0199] In some embodiments, the obtaining in Action 609 of the first indication and the obtaining in this Action 610 of the second indication may comprise: i) performing a consensus update of the respective estimate by the first node 111 and the respective estimate by each of the one or more second nodes 122 and one or more third nodes 123, respectively, ii) performing a local model update with the respective data of the first node 111 , and iii) computing the first indication and the second indication, respectively, as scores computed by the average decentralized consensus mechanism within each of the one or more second nodes 122 and the one or more third nodes 123.

[0200] In some embodiments, each of the first indication and the second indication may be based on a moment of a pre-defined order.

[0201] This Action may be understood to be performed as described for Action 507.

[0202] Action 611

[0203] In this Action 611, at least the first node 111 determines whether to group the one or more second nodes 122 and the one or more third nodes 123 into the same group to train the common machine learning model, or to refrain from grouping the one or more second nodes 122 and the one or more third nodes 123 to train into the same group to train the common machine learning model. The determining in this Action 611 of whether to group or not is based on the difference between the first indication and the second indication with respect to the threshold.

[0204] This Action may be understood to be performed as described for Action 508.

[0205] Action 612

[0206] In this Action 612, at least the first node 111 outputs the third indication indicating the result of the determining in Action 611 of whether to cluster or not.

[0207] In some embodiments, the third indication may be at least one of: a) the instruction instructing the one or more second nodes 122 and the one or more third nodes 123 to train the respective machine learning models, wherein the training may be fully decentralized learning, and b) the request to the radio network node 140 to deallocate the allocated resources between the nodes excluded from a cluster, wherein the result of the determining 611 has been to not cluster, and wherein the allocated resources may be the subset of the resources connecting the set of nodes 121.

[0208] In some embodiments, the respective data may have been compressed in a decentralized manner, and wherein the first node 111 for at least one of the clusters uses only the respective data from one agent. In some embodiments, the one or more communication subgraphs may be updated based on the deallocation of resources.

[0209] In some embodiments, the training may be federated learning, and the first node 111 for at least one of the clusters may uses only the respective data from one agent.

[0210] This Action may be understood to be performed as described for Action 509.

[0211] Action 613

[0212] In this Action 613, at least the first node 111 may iterate the method by one of: adding and subtracting an additional node to at least one of the one or more second nodes 122 and the one or more third nodes 123, until the consensus criterion may be reached.

[0213] In some embodiments, in at least one of the iterations, at least one of the first indication and the second indication may be based on computations performed within at least one of the one or more second nodes 122 and the one or more third nodes 123, according to the decentralized consensus mechanism.

[0214] In some embodiments, at least one of the following may apply: a) the first node 111 may lack access to the respective sets of data, b) each node in the set of nodes 121 may lack access to the respective set of data of other nodes in the set of nodes 121 , c) the obtaining in Action 609 of the first indication and the obtaining in Action 610 of the second indication may comprise receiving the first indication and the second indication, respectively, d) the first node 111 may refrain from determining any parameters for the respective machine learning models with the proviso the first node 111 may be a separate node from the set of nodes 111 , e) the determining in Action 607 of one or more second nodes 122 and the determining in Action 608 of the one or more third nodes 123 may be initially performed one of: randomly and based on the one or more first criteria, and f) in every iteration, the additional node added or subtracted may be selected one of: randomly and based on the one or more second criteria.

[0215] In some embodiments, the parameters of a respective model of the one or more second nodes 122 and the one or more third nodes 123 may have been updated within at least one of the one or more second nodes 122 and the one or more third nodes 123, according to the decentralized consensus mechanism.

[0216] This Action may be understood to be performed as described for Action 510.

[0217] Figure 7 depicts a graphic representation showing a non-limiting example of a gradient vector, in panel a), and training data distribution, in panel b, when applying decentralized learning to solve a non-linear regression problem. In this example, there are 20 nodes in the set of nodes 121 , and each node manages a respective agent n that may hold a set of local data samples {(Xn(0,yn(0)}J=?. The connectivity graph may be understood to be an instance of an Erdbs-Reniy graph, but with a constraint enforced such that it may be connected. Each training data sample (x,y) may be generated by a polynomial function y = ax2+ bx + c + w, where w may be understood to be additive White Gaussian noise. A scenario with extremely heterogeneous data may be considered, wherein each agent n may only contain training data samples for a limited range of x e [x”in, xax. The goal of training may be understood to be to estimate the parameter vector [a, b, c] , and the final gradient vector of each agent after convergence may be stored for testing the clustering method described herein.

[0218] The method may start from a randomly chosen pair of connected nodes, that is, agents, in accordance to Action 504, and the HS value may be computed according to Action 506, then merging more agents into the cluster, in accordance to Action 505, until reaching the maximum size of 10. The gradient vector, depicted on panel a) of Figure 7, and the training data distribution, depicted on panel b) of Figure 7, of the clustered nodes are depicted as stars. In panel a), the horizontal axis, the vertical axis, and the depth axis represent, respectively, the parameters of the gradient vector, in the case a, b, c, assuming a 3rdgradient vector. In panel b), the horizontal axis and the vertical axis represent, respectively, samples, wherein x is the input and y is the input, assuming a model is trying to learn a quadratic function as determined by a second-order polynomial. The results may be understood to validate that the clustered agents depicted as stars have similar training data, as compared to the rest of the agents in the network, depicted as crosses.

[0219] In conclusion, from this example depicted in Figure 7, it may be appreciated that the methods described in relation to Figure 5 and Figure 6 may be understood to correctly identify a subset of agents that have similar gradients, as shown in panel a). This subset of agents may also turn out to have similar ranges of their input data, as shown in panel b), which may be understood to illustrate the working principle of embodiments herein.

[0220] Device-to-device (D2D) example

[0221] According to examples of embodiments herein, the nodes in the set of nodes 121 may be devices, e.g., UEs, which may be capable of D2D communication. To illustrate the D2D based example, a setup may be considered, such as the one shown in Figure 8, where 3 UEs are connected through side-links connections which are handled by the radio network node 140, here a gNB. In a learning setup according to embodiments herein, UEs UE1, UE2 and UE3 may be understood to be nodes representing participating agents and the gNB may be the radio network node 140, representing the central unit. In this case, the gNB may be handling the control plane traffic for all UEs and each UE may directly communicate with the other UE to share user plane traffic, such as, model updates. The first node 111 may be understood to be UE1 in this example, and the another node 112 may be UE2.

[0222] The initial connectivity graph G for this setup for uplink and downlink may be as shown in

[0223] Table 3:

[0224] Table 3

[0225] The initial connectivity graph may be fully connected, which may be understood to mean that the maximum number of physical channels both for uplink and downlink may need to be allocated for this communication to take place. Using an approach according to embodiments described herein, the number of these allocations may be reduced and may thus allow for the released / unused spectrum to be used for other allocations. The process is illustrated in Figure 9. The physical channel allocation may be assumed to be initiated by the first node 111 , UE1 , according to Action 502 and Action 602, which may be understood to imply that UE1, according to Action 501 and Action 601 , may have received high enough RSRP and / or RSRQ measurements from nearby UEs, e.g., UE2 and UE3. As such, the gNB may imitate communication channels for available frequencies, as indicated by steps 2, 3, 4, 5, and handle control plane traffic. In 6, according to Action 503, Action 603 and Action 604, UE1 may broadcast its model parameters to UE2 and in 7, according to Action 605, UE2 may perform a consensus_update which yields VS[1,2], In 8, according to Action 503 and Action 606, UE2 may broadcast its model parameters to UE1 and UE1 in 9, according to Action 504, Action 506, Action 607 and Action 609, may produce VS(2,1). In Step 10, according to Action 503, Action 603, and Action 606, UE3 may broadcast model parameters to UE1 and in 11 , according to Action 505, Action 507, Action 606 and Action 610, UE1 may compute new VS’(1,2,3) consensus_update which may include the input from all 3 UEs. Depending on the comparison of VS(1 ,2) and VS(1 ,2,3) performed according to Action 508 and Action 611, UE1 may, in accordance with Action 509 and Action 612, request for the gNB to tear down the uplink / downlink, in the interest of this example uplink may be chosen, between UE3 and UE1 may thus release the allocated resource. The updated connectivity graph is shown in Table 4:

[0226] Table 4 As an overview of some aspects of this disclosure, in a particular example, embodiments herein may be understood to be drawn to a system comprising agents and a central unit. The agents may be capable of communicating device-to-device according to a connectivity graph. The central unit may determine a first set of agents. The central unit may determine a second set of agents. The second set may constitute a subset of the first set. The first set of agents may compute a first data and system heterogeneity score. The second set of agents may compute a second data and system heterogeneity score. The central unit may compare the difference between the first and second heterogeneity scores to a threshold, and if the difference exceeds the threshold, may define a first cooperation cluster comprising the agents in the first set and a second cooperation cluster comprising the agents in the second set.

[0227] In some particular examples, the heterogeneity scores may be computed by an average decentralized consensus mechanism within each respective set of agents.

[0228] In some particular examples, the system may perform fully decentralized learning and the network may instruct the first and second cluster to train individual models.

[0229] In some particular examples, the system may perform federated learning, and the network for at least one of the clusters may only use data from one agent in that cluster in order to reduce the amount of data as the model may mature.

[0230] In some particular examples, the system may perform, e.g., decentralized, compression of data, and the network for at least one of the clusters may only use data from one agent in that cluster.

[0231] In another particular example, embodiments herein may be understood to be drawn to a system comprising agents and a central unit. The agents may be capable of communicating device-to-device according to a connectivity graph. The central unit may obtain a first set of agents. The central unit may obtain a second set of agents. The second set may constitute a subset of the first set. The first set of agents may compute a first data and system heterogeneity score. The second set of agents may compute a second data and system heterogeneity score. The central unit may compare the difference between the first and second heterogeneity scores to a threshold, and if the difference is less than the threshold, declaring acceptable difference in heterogeneity.

[0232] Certain embodiments herein may provide one or more of the following technical advantage(s). Embodiments herein may be understood to enable that the first node 111, e.g., the central unit, may infer knowledge of the statistics of the training data and system capabilities possessed by the nodes in the set of nodes 121, e.g., managing respective agents, without transmission of large amounts or private information to the first node 111 , e.g., central unit. This may be understood to result in improved performance, for instance, faster convergence and / or less resource usage, of machine learning models that may be trained in situations with heterogeneous data in fully decentralized learning.

[0233] The setup in embodiments herein may be understood to be a decentralized setting and, as such, different from earlier approaches such as [2], [3] and [4], In [3], data heterogeneity in the datasets is not considered. Similarly to [3] and [4], embodiments herein may take into consideration the capability of the devices and the data similarity, respectively. However, unlike [3] and [4], embodiments herein may be understood to do that in a decentralized setting which may be understood to lack a parameter server. Embodiments herein may be understood to use a consensus mechanism that may converge linearly and may allow the participants to cluster themselves in data and device capability aware clusters.

[0234] A difference with [1] may be understood to be that, in embodiments herein, the consensus mechanism may be used to determine resource allocation of physical channels in a device-to-device setup for nodes managing agents that may be collaboratively training a common machine learning model, thus prioritizing connections between nodes managing agents that may belong to the same group, while deprioritizing others to enhance the resource allocation in a setup where the central unit may manage the physical channels between different nodes managing agents, but also in a setup where nodes managing agents may manage their physical links autonomously.

[0235] Figure 10 depicts an example of the arrangement that the first node 111 may comprise to perform the method described in Figure 5, and / or Figures 6-9. The first node 111 may be understood to be for handling the machine learning model. The first node 111 is configured to operate in the communications system 100 configured to comprise the set of nodes 121.

[0236] In some examples, the first node 111 may be configured to be comprised in the set of nodes 121.

[0237] In some examples, the first node 111 may be configured to be a coordinating authority, separate from the set of nodes 121.

[0238] Several embodiments are comprised herein. It should be noted that the examples herein are not mutually exclusive. One or more embodiments may be combined, where applicable. All possible combinations are not described to simplify the description. Components from one embodiment may be tacitly assumed to be present in another embodiment and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments. The detailed description of some of the following corresponds to the same references provided above, in relation to the actions described for the first node 111 , and will thus not be repeated here. For example to exchange the respective estimate of the common machine learning model may be configured to comprise broadcasting model parameters to the other nodes in the subset of the set of nodes 121 configured to be determined.

[0239] In Figure 10, optional units are indicated with dashed boxes.

[0240] The first node 111 is configured to determine the one or more second nodes 122 out of at least the subset of the set of nodes 121. Each of the nodes in the set of nodes 121 is configured to manage the respective agent capable of training the respective machine learning model with the respective set of data. The nodes in the set of nodes 121 are configured to have the capability to connect, directly or indirectly, via the resources according to the one or more communication subgraphs in the fully decentralized setting.

[0241] The first node 111 is also configured to determine the one or more third nodes 123 out of at least the subset of the set of nodes 121.

[0242] The first node 111 is also configured to obtain, using the decentralized consensus mechanism, the first indication of the heterogeneity of the respective sets of data configured to be used by, and the respective compute capability of, each of the respective agents configured to be managed by the one or more second nodes 122.

[0243] The first node 111 is also configured to obtain, using the decentralized consensus mechanism, the second indication of the heterogeneity of the sets of data configured to be used by, and the respective compute capability of, each of the respective agents configured to be managed by the one or more third nodes 123.

[0244] The first node 111 is also configured to determine whether to group the one or more second nodes 122 and the one or more third nodes 123 into the same group to train the common machine learning model, or to refrain from grouping the one or more second nodes 122 and the one or more third nodes 123 to train into the same group to train the common machine learning model. The determining of whether to group or not is configured to be based on the difference between the first indication and the second indication with respect to the threshold.

[0245] The first node 111 is also configured to output the third indication configured to indicate the result of the determining of whether to cluster or not.

[0246] The first node 111 may be also configured with the following three configurations.

[0247] In some embodiments, the first node 111 may be further configured to determine at least the subset of the set of nodes 121.

[0248] In some embodiments, the first node 111 may be further configured to initiate the allocation of the communication channels with each of the nodes in the subset of the set of nodes 121.

[0249] In some embodiments, the first node 111 may be further configured to exchange the respective estimate of the common machine learning model with the other nodes in the subset of the set of nodes 121 configured to be determined. The determining of the one or more second nodes 122, the determining of the one or more third nodes 123, the obtaining of the first indication and the obtaining of the second indication may be configured to be based on the respective estimates of the common machine learning model configured to be exchanged.

[0250] In some embodiments, at least one of the following may apply: a) the determining of at least the subset of the set of nodes 121 may be configured to be performed by detecting the subset of the set of nodes 121 , b) the determining of at least the subset of the set of nodes 121 may be configured to be performed based on the connectivity criterion, c) the initiating of the allocation may be configured to be performed by sending the allocation request to the radio network node 140, and d) the exchanging of the respective estimate may be configured to be performed by broadcasting the respective estimate by the first node 111 and receiving the respective broadcast comprising the respective estimate by each of the other nodes in the subset of the set of nodes 121 configured to be determined.

[0251] In some embodiments, the third indication may be configured to be at least one of: a) the instruction configured to instruct the one or more second nodes 122 and the one or more third nodes 123 to train the respective machine learning models, wherein the training may be configured to be fully decentralized learning, and b) the request to the radio network node 140 to deallocate allocated resources between nodes excluded from the cluster, wherein the result of the determining may be configured to have been to not cluster, and wherein the allocated resources may be configured to be the subset of the resources connecting the set of nodes 121.

[0252] In some embodiments, the one or more communication subgraphs may be configured to be updated based on the deallocation of resources.

[0253] In some embodiments, the training may be configured to be federated learning, and the first node 111 for at least one of the clusters may be configured to use only the respective data from one agent.

[0254] In some embodiments, the obtaining of the first indication and the obtaining of the second indication may be configured to comprise: i) performing the consensus update of the respective estimate by the first node 111 and the respective estimate by each of the one or more second nodes 122 and the one or more third nodes 123, respectively, ii) performing the local model update with the respective data of the first node 111 , and iii) computing the first indication and the second indication, respectively, as scores computed by the average decentralized consensus mechanism within each of the one or more second nodes 122 and the one or more third nodes 123.

[0255] In some embodiments, the first node 111 may be further configured to iterate the method configured to be performed according to the configurations of any of the preceding paragraphs by one of: adding and subtracting an additional node to at least one of the one or more second nodes 122 and the one or more third nodes 123, until a consensus criterion may be reached. In some embodiments, in at least one of the iterations, at least one of the first indication and the second indication may be configured to be based on the computations configured to be performed within at least one of the one or more second nodes 122 and the one or more third nodes 123, according to the decentralized consensus mechanism.

[0256] In some embodiments, at least one of the following may apply: a) the first node 111 may be configured to lack access to the respective sets of data, b) each node in the set of nodes 121 may be configured to lack access to the respective set of data of other nodes in the set of nodes 121 , c) the obtaining of the first indication and the obtaining of the second indication may be configured to comprise receiving the first indication and the second indication, respectively, d) the first node 111 may be configured to refrain from determining any parameters for the respective machine learning models with the proviso the first node 111 is configured to be a separate node from the set of nodes 111, e) the determining of the one or more second nodes 122 and the determining of the one or more third nodes 123 may be configured to be initially performed one of: randomly and based on the one or more first criteria, and f) in every iteration, the additional node added or subtracted may be configured to be selected one of: randomly and based on the one or more second criteria.

[0257] In some embodiments, the parameters of the respective model of the one or more second nodes 122 and the one or more third nodes 123 may be configured to have been updated within at least one of the one or more second nodes 122 and the one or more third nodes 123, according to the decentralized consensus mechanism.

[0258] In some embodiments, each of the first indication and the second indication may be configured to be based on the moment of the pre-defined order.

[0259] In some embodiments, the respective data may be configured to have been compressed in a decentralized manner. The first node 111 for at least one of the clusters may be configured to use only the respective data from one agent.

[0260] The embodiments herein in the first node 111 may be implemented through one or more processors, such as a processing circuitry 1001 in the first node 111 depicted in Figure 10, together with computer program code for performing the functions and actions of the embodiments herein. A processor, as used herein, may be understood to be a hardware component. The program code mentioned above may also be provided as a computer program product, for instance in the form of a data carrier carrying computer program code for performing the embodiments herein when being loaded into the first node 111. One such carrier may be in the form of a CD ROM disc. It is however feasible with other data carriers such as a memory stick. The computer program code may furthermore be provided as pure program code on a server and downloaded to the first node 111.

[0261] The first node 111 may further comprise a memory 1002 comprising one or more memory units. The memory 1002 is arranged to be used to store obtained information, store data, configurations, schedulings, and applications etc. to perform the methods herein when being executed in the first node 111.

[0262] In some embodiments, the first node 111 may receive information from, e.g., any of the nodes in the set of devices 121 , the another node 112, the radio network node 140 and / or another structure in the communications system 100, through a receiving port 1003. In some embodiments, the receiving port 1003 may be, for example, connected to one or more antennas in first node 111. In other embodiments, the first node 111 may receive information from another structure in the communications system 100 through the receiving port 1003. Since the receiving port 1003 may be in communication with the processing circuitry 1001, the receiving port 1003 may then send the received information to the processing circuitry 1001. The receiving port 1003 may also be configured to receive other information.

[0263] The processing circuitry 1001 in the first node 111 may be further configured to transmit or send information to e.g., any of the nodes in the set of devices 121 , the another node 112, the radio network node 140 and / or another structure in the communications system 100, through a sending port 1004, which may be in communication with the processing circuitry 1001, and the memory 1002.

[0264] Those skilled in the art will also appreciate that the units comprised within the first node 111 described above as being configured to perform different actions, may refer to a combination of analog and digital circuits, and / or one or more processors configured with software and / or firmware, e.g., stored in memory, that, when executed by the one or more processors such as the processing circuitry 1001, perform as described above. One or more of these processors, as well as the other digital hardware, may be included in a single Application-Specific Integrated Circuit (ASIC), or several processors and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a System-on-a-Chip (SoC).

[0265] The first node 111 may be configured to perform any of the Actions described in relation to Figure 5 and / or Figures 6-9, e.g., by means of the processing circuitry 1001 within the first node 111, configured to perform any of such actions.

[0266] Also, in some embodiments, different units comprised within the first node 111 may be configured to perform the different actions described above, implemented as one or more applications running on one or more processors such as the processing circuitry 1001.

[0267] Thus, the methods according to the embodiments described herein for the first node 111 may be respectively implemented by means of a computer program 1005 product, comprising instructions, i.e., software code portions, which, when executed on at least one processing circuitry 1001 , cause the at least one processing circuitry 1001 to carry out the actions described herein, as performed by the first node 111. The computer program 1005 product may be stored on a computer-readable storage medium 1006. The computer- readable storage medium 1006, having stored thereon the computer program 1005, may comprise instructions which, when executed on at least one processing circuitry 1001, cause the at least one processing circuitry 1001 to carry out the actions described herein, as performed by the first node 111. In some embodiments, the computer-readable storage medium 1006 may be a non-transitory computer-readable storage medium, such as a CD ROM disc, or a memory stick. In other embodiments, the computer program 1005 product may be stored on a carrier containing the computer program 1005 just described, wherein the carrier is one of an electronic signal, optical signal, radio signal, or the computer-readable storage medium 1006, as described above.

[0268] The first node 111 may comprise a communication interface configured to facilitate, or an interface unit to facilitate, communications between the first node 111 and other nodes or devices, e.g. any of the nodes in the set of devices 121 , the another node 112, the radio network node 140 and / or another structure in the communications system 100, The interface may, for example, include a transceiver configured to transmit and receive radio signals over an air interface in accordance with a suitable standard.

[0269] In other embodiments, the first node 111 may comprise a radio circuitry 1007, which may comprise e.g., the receiving port 1003 and the sending port 1004.

[0270] The radio circuitry 1007 may be configured to set up and maintain at least a wireless connection with any of the nodes in the set of devices 121 , the another node 112, the radio network node 140 and / or another structure in the communications system 100. Circuitry may be understood herein as a hardware component.

[0271] Hence, embodiments herein also relate to the first node 111 operative to operate in the communications system 100. The first node 111 may comprise the processing circuitry 1001 and the memory 1002, said memory 1002 containing instructions executable by said processing circuitry 1001 , whereby the first node 111 is further operative to perform the actions described herein in relation to the first node 111, e.g., in Figure 5, and / or Figures 6-9.

[0272] Figure 11 depicts an example of the arrangement that the communications system 100 may comprise to perform the method described in Figure 6 and / or Figures 7-9. The communications system 100 is configured to comprise the set of nodes 121. The communications system 100 may be understood to be for handling the machine learning model. The communications system 100 is configured to comprise at least the first node 111 comprised in the communications system 100.

[0273] In some examples, the first node 111 may be configured to be comprised in the set of nodes 121.

[0274] In some examples, the first node 111 may be configured to be a coordinating authority, separate from the set of nodes 121. Several embodiments are comprised herein. It should be noted that the examples herein are not mutually exclusive. One or more embodiments may be combined, where applicable. All possible combinations are not described to simplify the description. Components from one embodiment may be tacitly assumed to be present in another embodiment and it will be obvious to a person skilled in the art how those components may be used in the other exemplary embodiments. The detailed description of some of the following corresponds to the same references provided above, in relation to the actions described for the second node 112, and will thus not be repeated here. For example to exchange the respective estimate of the common machine learning model may be configured to comprise broadcasting model parameters to the other nodes in the subset of the set of nodes 121 configured to be determined.

[0275] In Figure 11, optional units are indicated with dashed boxes.

[0276] The communications system 100 is configured to, by at least the first node 111 comprised in the communications system 100, determine the one or more second nodes 122 out of at least the subset of the set of nodes 121. Each of the nodes in the set of nodes 121 is configured to manage the respective agent capable of training the respective machine learning model with the respective set of data. The nodes in the set of nodes 121 are configured to have the capability to connect, directly or indirectly, via the resources according to the one or more communication subgraphs in the fully decentralized setting.

[0277] The communications system 100 is configured to, by at least the first node 111 comprised in the communications system 100, determine the one or more third nodes 123 out of at least the subset of the set of nodes 121.

[0278] The communications system 100 is configured to, by at least the first node 111 comprised in the communications system 100, obtain, using the decentralized consensus mechanism, the first indication of the heterogeneity of the respective sets of data configured to be used by, and the respective compute capability of, each of the respective agents configured to be managed by the one or more second nodes 122.

[0279] The communications system 100 is configured to, by at least the first node 111 comprised in the communications system 100, obtain, using the decentralized consensus mechanism, the second indication of the heterogeneity of the sets of data configured to be used by, and the respective compute capability of, each of the respective agents configured to be managed by the one or more third nodes 123.

[0280] The communications system 100 is configured to, by at least the first node 111 comprised in the communications system 100, determine whether to group the one or more second nodes 122 and the one or more third nodes 123 into the same group to train the common machine learning model, or to refrain from grouping the one or more second nodes 122 and the one or more third nodes 123 to train into the same group to train the common machine learning model. The determining of whether to group or not is configured to be based on the difference between the first indication and the second indication with respect to the threshold.

[0281] The communications system 100 is configured to, by at least the first node 111 comprised in the communications system 100, output the third indication configured to indicate the result of the determining of whether to cluster or not.

[0282] The communications system 100 may be also configured with the following three configurations.

[0283] In some embodiments, the communications system 100 may be further configured to, by at least the first node 111 , determine at least the subset of the set of nodes 121.

[0284] In some embodiments, the communications system 100 may be further configured to, by at least the first node 111 , initiate the allocation of the communication channels with each of the nodes in the subset of the set of nodes 121.

[0285] In some embodiments, the communications system 100 may be further configured to, by at least the first node 111 , exchange the respective estimate of the common machine learning model with the other nodes in the subset of the set of nodes 121 configured to be determined. The determining of the one or more second nodes 122, the determining of the one or more third nodes 123, the obtaining of the first indication and the obtaining of the second indication may be configured to be based on the respective estimates of the common machine learning model configured to be exchanged.

[0286] In some embodiments, at least one of the following may apply: a) the determining of at least the subset of the set of nodes 121 may be configured to be performed by detecting the subset of the set of nodes 121 , b) the determining of at least the subset of the set of nodes 121 may be configured to be performed based on the connectivity criterion, c) the initiating of the allocation may be configured to be performed by sending the allocation request to the radio network node 140, and d) the exchanging of the respective estimate may be configured to be performed by broadcasting the respective estimate by the first node 111 and receiving the respective broadcast comprising the respective estimate by each of the other nodes in the subset of the set of nodes 121 configured to be determined.

[0287] In some embodiments, the third indication may be configured to be at least one of: a) the instruction configured to instruct the one or more second nodes 122 and the one or more third nodes 123 to train the respective machine learning models, wherein the training may be configured to be fully decentralized learning, and b) the request to the radio network node 140 to deallocate allocated resources between nodes excluded from the cluster, wherein the result of the determining may be configured to have been to not cluster, and wherein the resources configured to be allocated may be configured to be the subset of the resources connecting the set of nodes 121. In some embodiments, the one or more communication subgraphs may be configured to be updated based on the deallocation of resources.

[0288] In some embodiments, the training may be configured to be federated learning, and the first node 111 for at least one of the clusters may be configured to use only the respective data from one agent.

[0289] In some embodiments, the obtaining of the first indication and the obtaining of the second indication may be configured to comprise: i) performing the consensus update of the respective estimate by the first node 111 and the respective estimate by each of the one or more second nodes 122 and the one or more third nodes 123, respectively, ii) performing the local model update with the respective data of the first node 111 , and iii) computing the first indication and the second indication, respectively, as scores computed by the average decentralized consensus mechanism within each of the one or more second nodes 122 and the one or more third nodes 123.

[0290] In some embodiments, the communications system 100 may be further configured to, by at least the first node 111, iterate the method configured to be performed according to the configurations of any of the preceding paragraphs by one of: adding and subtracting an additional node to at least one of the one or more second nodes 122 and the one or more third nodes 123, until a consensus criterion may be reached.

[0291] In some embodiments, in at least one of the iterations, at least one of the first indication and the second indication may be configured to be based on the computations configured to be performed within at least one of the one or more second nodes 122 and the one or more third nodes 123, according to the decentralized consensus mechanism.

[0292] In some embodiments, at least one of the following may apply: a) the first node 111 may be configured to lack access to the respective sets of data, b) each node in the set of nodes 121 may be configured to lack access to the respective set of data of other nodes in the set of nodes 121, c) the obtaining of the first indication and the obtaining of the second indication may be configured to comprise receiving the first indication and the second indication, respectively, d) the first node 111 may be configured to refrain from determining any parameters for the respective machine learning models with the proviso the first node 111 is configured to be a separate node from the set of nodes 111, e) the determining of the one or more second nodes 122 and the determining of the one or more third nodes 123 may be configured to be initially performed one of: randomly and based on the one or more first criteria, and f) in every iteration, the additional node added or subtracted may be configured to be selected one of: randomly and based on the one or more second criteria.

[0293] In some embodiments, the parameters of the respective model of the one or more second nodes 122 and the one or more third nodes 123 may be configured to have been updated within at least one of the one or more second nodes 122 and the one or more third nodes 123, according to the decentralized consensus mechanism.

[0294] In some embodiments, each of the first indication and the second indication may be configured to be based on the moment of the pre-defined order.

[0295] In some embodiments, the respective data may be configured to have been compressed in a decentralized manner. The first node 111 for at least one of the clusters may be configured to use only the respective data from one agent.

[0296] In some embodiments, at least a plurality of nodes in the set of nodes 121 may be configured according to any of the preceding paragraphs.

[0297] In some embodiments, the communications system 100 may be further configured with the following three configurations.

[0298] In some embodiments, the communications system 100 may be further configured to, by at least another node 112 in the set of nodes 121 , receive the respective estimate.

[0299] In some embodiments, the communications system 100 may be further configured to, by at least another node 112 in the set of nodes 121, determine the respective one or more second nodes 122 out of at least a subset of the set of nodes 121.

[0300] In some embodiments, the communications system 100 may be further configured to, by at least another node 112 in the set of nodes 121, send the respective estimate of the of the common machine learning model to the other nodes in the determined subset of the set of nodes 121.

[0301] The embodiments herein in the communications system 100 may be implemented by a first node 111 as described in relation to Figure 10. The description of the arrangement of the first node 111 will therefore not be repeated in relation to Figure 11, but may be understood to equally apply.

[0302] The embodiments herein in the communications system 100 may further implemented by the another node 112.

[0303] The embodiments herein in the another node 112 may be implemented through one or more processors, such as a processing circuitry 1101 in the another node 112 depicted in Figure 11, together with computer program code for performing the functions and actions of the embodiments herein. A processor, as used herein, may be understood to be a hardware component. The program code mentioned above may also be provided as a computer program product, for instance in the form of a data carrier carrying computer program code for performing the embodiments herein when being loaded into the another node 112. One such carrier may be in the form of a CD ROM disc. It is however feasible with other data carriers such as a memory stick. The computer program code may furthermore be provided as pure program code on a server and downloaded to the another node 112. The another node 112 may further comprise a memory 1102 comprising one or more memory units. The memory 1102 is arranged to be used to store obtained information, store data, configurations, schedulings, and applications etc. to perform the methods herein when being executed in the another node 112.

[0304] In some embodiments, the another node 112 may receive information from, e.g., the first node 111 , any of the other nodes in the set of devices 121, the radio network node 140 and / or another structure in the communications system 100, through a receiving port 1103. In some embodiments, the receiving port 1103 may be, for example, connected to one or more antennas in another node 112. In other embodiments, the another node 112 may receive information from another structure in the wireless communications system 100 through the receiving port 1103. Since the receiving port 1103 may be in communication with the processing circuitry 1101 , the receiving port 1103 may then send the received information to the processing circuitry 1101. The receiving port 1103 may also be configured to receive other information.

[0305] The processing circuitry 1101 in the another node 112 may be further configured to transmit or send information to e.g., the first node 111, any of the other nodes in the set of devices 121, the radio network node 140 and / or another structure in the communications system 100, through a sending port 1104, which may be in communication with the processing circuitry 1101, and the memory 1102.

[0306] Those skilled in the art will also appreciate that the units comprised within the another node 112 described above as being configured to perform different actions, may refer to a combination of analog and digital circuits, and / or one or more processors configured with software and / or firmware, e.g., stored in memory, that, when executed by the one or more processors such as the processing circuitry 1101, perform as described above. One or more of these processors, as well as the other digital hardware, may be included in a single Application-Specific Integrated Circuit (ASIC), or several processors and various digital hardware may be distributed among several separate components, whether individually packaged or assembled into a System-on-a-Chip (SoC).

[0307] The another node 112 may be configured to perform any of the Actions described in relation to Figure 6 and / or Figures 7-9, e.g., by means of the processing circuitry 1001 within the first node 111, configured to perform any of such actions.

[0308] Also, in some embodiments, different units comprised within the first node 111 may be configured to perform the different actions described above, implemented as one or more applications running on one or more processors such as the processing circuitry 1001.

[0309] Thus, the methods according to the embodiments described herein for the another node 112 may be respectively implemented by means of a computer program 1105 product, comprising instructions, i.e., software code portions, which, when executed on at least one processing circuitry 1101 , cause the at least one processing circuitry 1101 to carry out the actions described herein, as performed by the another node 112. The computer program 1105 product may be stored on a computer-readable storage medium 1106. The computer- readable storage medium 1106, having stored thereon the computer program 1105, may comprise instructions which, when executed on at least one processing circuitry 1101, cause the at least one processing circuitry 1101 to carry out the actions described herein, as performed by the another node 112. In some embodiments, the computer-readable storage medium 1106 may be a non-transitory computer-readable storage medium, such as a CD ROM disc, or a memory stick. In other embodiments, the computer program 1105 product may be stored on a carrier containing the computer program 1105 just described, wherein the carrier is one of an electronic signal, optical signal, radio signal, or the computer-readable storage medium 1106, as described above.

[0310] The another node 112 may comprise a communication interface configured to facilitate, or an interface unit to facilitate, communications between the another node 112 and other nodes or devices, e.g., the first node 111, any of the other nodes in the set of devices 121 , the radio network node 140 and / or another structure in the communications system 100. The interface may, for example, include a transceiver configured to transmit and receive radio signals over an air interface in accordance with a suitable standard.

[0311] In other embodiments, the another node 112 may comprise a radio circuitry 1107, which may comprise e.g., the receiving port 1103 and the sending port 1104.

[0312] The radio circuitry 1107 may be configured to set up and maintain at least a wireless connection with the first node 111, any of the other nodes in the set of devices 121 , the radio network node 140 and / or another structure in the communications system 100. Circuitry may be understood herein as a hardware component.

[0313] Hence, embodiments herein also relate to the another node 112 operative to operate in the wireless communications system 100. The another node 112 may comprise the processing circuitry 1101 and the memory 1102, said memory 1102 containing instructions executable by said processing circuitry 1101 , whereby the another node 112 is further operative to perform the actions described herein in relation to the another node 112, e.g., in Figure 6 and / or Figures 7-9.

[0314] When using the word "comprise" or “comprising”, it shall be interpreted as non- limiting, i.e. , meaning "consist at least of".

[0315] The embodiments herein are not limited to the above-described preferred embodiments. Various alternatives, modifications and equivalents may be used. Therefore, the above embodiments should not be taken as limiting the scope of the invention. Generally, all terms used herein are to be interpreted according to their ordinary meaning in the relevant technical field, unless a different meaning is clearly given and / or is implied from the context in which it is used. All references to a / an / the element, apparatus, component, means, step, etc. are to be interpreted openly as referring to at least one instance of the element, apparatus, component, means, step, etc., unless explicitly stated otherwise. The steps of any methods disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as following or preceding another step and / or where it is implicit that a step must follow or precede another step. Any feature of any of the embodiments disclosed herein may be applied to any other embodiment, wherever appropriate. Likewise, any advantage of any of the embodiments may apply to any other embodiments, and vice versa. Other objectives, features and advantages of the enclosed embodiments will be apparent from the following description.

[0316] As used herein, the expression “at least one of:” followed by a list of alternatives separated by commas, and wherein the last alternative is preceded by the “and” term, may be understood to mean that only one of the list of alternatives may apply, more than one of the list of alternatives may apply or all of the list of alternatives may apply. This expression may be understood to be equivalent to the expression “at least one of:” followed by a list of alternatives separated by commas, and wherein the last alternative is preceded by the “or” term.

[0317] Any of the terms processor and circuitry may be understood herein as a hardware component.

[0318] As used herein, the expression “in some embodiments” has been used to indicate that the features of the embodiment described may be combined with any other embodiment or example disclosed herein.

[0319] As used herein, the expression “in some examples” has been used to indicate that the features of the example described may be combined with any other embodiment or example disclosed herein.

[0320] REFERENCES

[0321] 1. Fast linear iterations for distributed averaging, https: / / www.sciencedirect.com / science / article / pii / S0167691104000398.

[0322] 2. https: / / arxiv.org / pdf / 1912.13163.pdf.

[0323] 3. Hosseinalipour et.al - From Federated to Fog Learning: Distributed Machine Learning over Heterogeneous Wireless Networks.

[0324] 4. Wang et.al - Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms and Implementation.

Claims

CLAIMS:

1. A computer-implemented method performed by a first node (111), the method being for handling a machine learning model, the first node (111) operating in a communications system (100) comprising a set of nodes (121), the method comprising:- determining (504) one or more second nodes (122) out of at least a subset of the set of nodes (121), each of the nodes in the set of nodes (121) managing a respective agent capable of training a respective machine learning model with a respective set of data, and wherein the nodes in the set of nodes (121) have a capability to connect, directly or indirectly, via resources according to one or more communication subgraphs in a fully decentralized setting,- determining (505) one or more third nodes (123) out of at least the subset of the set of nodes (121),- obtaining (506), using a decentralized consensus mechanism, a first indication of a heterogeneity of the respective sets of data used by, and a respective compute capability of, each of the respective agents managed by the one or more second nodes (122),- obtaining (507), using the decentralized consensus mechanism, a second indication of the heterogeneity of the sets of data used by, and the respective compute capability of, each of the respective agents managed by the one or more third nodes (123),- determining (508) whether to group the one or more second nodes (122) and the one or more third nodes (123) into a same group to train a common machine learning model, or to refrain from grouping the one or more second nodes (122) and the one or more third nodes (123) to train into the same group to train the common machine learning model, the determining (508) of whether to group or not being based on a difference between the first indication and the second indication with respect to a threshold, and- outputting (509) a third indication indicating a result of the determining (508) of whether to cluster or not.

2. The method according to claim 1 , further comprising:- determining (501) at least a subset of the set of nodes (121),- initiating (502) allocation of communication channels with each of the nodes in the subset of the set of nodes (121),- exchanging (503) a respective estimate of the common machine learning model with the other nodes in the determined subset of the set of nodes (121), andwherein the determining (504) of the one or more second nodes (122), the determining (505) of the one or more third nodes (123), the obtaining (506) of the first indication and the obtaining (507) of the second indication is based on the exchanged respective estimates of the common machine learning model.

3. The method according to claim 2, wherein at least one of:- the determining (501) of at least the subset of the set of nodes (121) is performed by detecting the subset of the set of nodes (121),- the determining (501) of at least the subset of the set of nodes (121) is performed based on a connectivity criterion,- the initiating (502) of the allocation is performed by sending an allocation request to a radio network node (140), and- the exchanging (503) of the respective estimate is performed by broadcasting the respective estimate by the first node (111) and receiving a respective broadcast comprising the respective estimate by each of the other nodes in the determined subset of the set of nodes (121).

4. The method according to claim 3, wherein the third indication is at least one of:- an instruction instructing the one or more second nodes (122) and the one or more third nodes (123) to train respective machine learning models, wherein the training is fully decentralized learning, and- a request to the radio network node (140) to deallocate allocated resources between nodes excluded from a cluster, wherein the result of the determining (508) has been to not cluster, and wherein the allocated resources are a subset of the resources connecting the set of nodes (121).

5. The method according to claim 4, wherein the one or more communication subgraphs are updated based on the deallocation of resources.

6. The method according to any of claims 4-5, wherein the training is federated learning, and wherein the first node (111) for at least one of the clusters uses only the respective data from one agent.

7. The method according to any of claims 2-6, wherein the obtaining (506) of the first indication and the obtaining (507) of the second indication comprises:i) performing a consensus update of the respective estimate by the first node (111) and the respective estimate by each of the one or more second nodes (122) and one or more third nodes (123), respectively, ii) performing a local model update with the respective data of the first node (111), and iii) computing the first indication and the second indication, respectively, as scores computed by an average decentralized consensus mechanism within each of the one or more second nodes (122) and the one or more third nodes (123).

8. The method according to any of claims 1-7, further comprising:- iterating (510) the method by one of: adding and subtracting an additional node to at least one of the one or more second nodes (122) and the one or more third nodes (123), until a consensus criterion is reached.

9. The method according to claim 8, wherein in at least one of the iterations, at least one of the first indication and the second indication are based on computations performed within at least one of the one or more second nodes (122) and the one or more third nodes (123), according to the decentralized consensus mechanism.

10. The method according to any of claims 8-9, wherein at least one of:- the first node (111) lacks access to the respective sets of data,- each node in the set of nodes (121) lacks access to the respective set of data of other nodes in the set of nodes (121),- the obtaining (506) of the first indication and the obtaining (507) of the second indication comprises receiving the first indication and the second indication, respectively,- the first node (111) refrains from determining any parameters for the respective machine learning models with the proviso the first node (111) is a separate node from the set of nodes (111),- the determining (504) of the one or more second nodes (122) and the determining (505) of the one or more third nodes (123) is initially performed one of: randomly and based on one or more first criteria, and- in every iteration, the additional node added or subtracted is selected one of: randomly and based on one or more second criteria.

11. The method according to claims 9 and 10, wherein the parameters of a respective model of the one or more second nodes (122) and the one or more third nodes (123)have been updated within at least one of the one or more second nodes (122) and the one or more third nodes (123), according to the decentralized consensus mechanism.

12. The method according to any of claims 1-11 , wherein each of the first indication and the second indication are based on a moment of a pre-defined order.

13. The method according to any of claims 1-12, wherein the respective data has been compressed in a decentralized manner, and wherein the first node (111) for at least one of the clusters uses only the respective data from one agent.

14. A computer-implemented method performed by a communications system (100) comprising a set of nodes (121), the method being for handling a machine learning model, the method comprising, by at least a first node (111) comprised in the communications system (100):- determining (607) one or more second nodes (122) out of at least a subset of the set of nodes (121), each of the nodes in the set of nodes (121) managing a respective agent capable of training a respective machine learning model with a respective set of data, and wherein the nodes in the set of nodes (121) have a capability to connect, directly or indirectly, via resources according to one or more communication subgraphs in a fully decentralized setting,- determining (608) one or more third nodes (123) out of at least the subset of the set of nodes (121),- obtaining (609), using a decentralized consensus mechanism, a first indication of a heterogeneity of the respective sets of data used by, and a respective compute capability of, each of the respective agents managed by the one or more second nodes (122),- obtaining (610), using the decentralized consensus mechanism, a second indication of the heterogeneity of the sets of data used by, and the respective compute capability of, each of the respective agents managed by the one or more third nodes (123),- determining (611) whether to group the one or more second nodes (122) and the one or more third nodes (123) into a same group to train a common machine learning model, or to refrain from grouping the one or more second nodes (122) and the one or more third nodes (123) to train into the same group to train the common machine learning model, the determining (611) of whether to group or not being based on a difference between the first indication and the second indication with respect to a threshold, andoutputting (612) a third indication indicating a result of the determining (611) of whether to cluster or not.

15. The method according to claim 14, further comprising:- determining (601) at least a subset of the set of nodes (121),- initiating (602) allocation of communication channels with each of the nodes in the subset of the set of nodes (121),- exchanging (603) a respective estimate of the common machine learning model with the other nodes in the determined subset of the set of nodes (121), and wherein the determining (607) of the one or more second nodes (122), the determining (608) of the one or more third nodes (123), the obtaining (609) of the first indication and the obtaining (610) of the second indication is based on the exchanged respective estimates of the common machine learning model.

16. The method according to claim 15, wherein at least one of:- the determining (601) of at least the subset of the set of nodes (121) is performed by detecting the subset of the set of nodes (121),- the determining (601) of at least the subset of the set of nodes (121) is performed based on a connectivity criterion,- the initiating (602) of the allocation is performed by sending an allocation request to a radio network node (140), and- the exchanging (603) of the respective estimate is performed by broadcasting the respective estimate by the first node (111) and receiving a respective broadcast comprising the respective estimate by each of the other nodes in the determined subset of the set of nodes (121).

17. The method according to claim 16, wherein the third indication is at least one of:- an instruction instructing the one or more second nodes (122) and the one or more third nodes (123) to train respective machine learning models, wherein the training is fully decentralized learning, and- a request to the radio network node (140) to deallocate allocated resources between nodes excluded from a cluster, wherein the result of the determining (611) has been to not cluster, and wherein the allocated resources are a subset of the resources connecting the set of nodes (121).

18. The method according to claim 14, wherein the one or more communication subgraphs are updated based on the deallocation of resources.

19. The method according to any of claims 17-18, wherein the training is federated learning, and wherein the first node (111) for at least one of the clusters uses only the respective data from one agent.

20. The method according to any of claims 15-19, wherein the obtaining (609) of the first indication and the obtaining (610) of the second indication comprises: i) performing a consensus update of the respective estimate by the first node (111) and the respective estimate by each of the one or more second nodes (122) and one or more third nodes (123), respectively, ii) performing a local model update with the respective data of the first node (111), and iii) computing the first indication and the second indication, respectively, as scores computed by an average decentralized consensus mechanism within each of the one or more second nodes (122) and the one or more third nodes (123).

21. The method according to any of claims 14-20, further comprising:- iterating (613) the method by one of: adding and subtracting an additional node to at least one of the one or more second nodes (122) and the one or more third nodes (123), until a consensus criterion is reached.

22. The method according to claim 21, wherein in at least one of the iterations, at least one of the first indication and the second indication are based on computations performed within at least one of the one or more second nodes (122) and the one or more third nodes (123), according to the decentralized consensus mechanism.

23. The method according to any of claims 21-22, wherein at least one of:- the first node (111) lacks access to the respective sets of data,- each node in the set of nodes (121) lacks access to the respective set of data of other nodes in the set of nodes (121),- the obtaining (609) of the first indication and the obtaining (610) of the second indication comprises receiving the first indication and the second indication, respectively,- the first node (111) refrains from determining any parameters for the respective machine learning models with the proviso the first node (111) is a separate node from the set of nodes (111),- the determining (607) of the one or more second nodes (122) and the determining (608) of the one or more third nodes (123) is initially performed one of: randomly and based on one or more first criteria, and- in every iteration, the additional node added or subtracted is selected one of: randomly and based on one or more second criteria.

24. The method according to claims 22 and 23, wherein the parameters of a respective model of the one or more second nodes (122) and the one or more third nodes (123) have been updated within at least one of the one or more second nodes (122) and the one or more third nodes (123), according to the decentralized consensus mechanism.

25. The method according to any of claims 14-24, wherein each of the first indication and the second indication are based on a moment of a pre-defined order.

26. The method according to any of claims 14-25, wherein the respective data has been compressed in a decentralized manner, and wherein the first node (111) for at least one of the clusters uses only the respective data from one agent.

27. The method according to any of claims 14-26, wherein the method is performed by at least a plurality of nodes in the set of nodes (121).

28. The method according to any of claims 14-27, further comprising, by at least another node (112) in the set of nodes (121):- receiving (604) the respective estimate,- determining (605) respective one or more second nodes (122) out of at least a subset of the set of nodes (121), and- sending (606) the respective estimate of the of the common machine learning model to the other nodes in the determined subset of the set of nodes (121).

29. A first node (111), for handling a machine learning model, the first node (111) being configured to operate in a communications system (100) configured to comprise a set of nodes (121), the first node (111) being further configured to:- determine one or more second nodes (122) out of at least a subset of the set of nodes (121), each of the nodes in the set of nodes (121) being configured to manage a respective agent capable of training a respective machine learning model with a respective set of data, and wherein the nodes in the set of nodes (121) are configured to have a capability to connect, directly or indirectly, viaresources according to one or more communication subgraphs in a fully decentralized setting,- determine one or more third nodes (123) out of at least the subset of the set of nodes (121),- obtain, using a decentralized consensus mechanism, a first indication of a heterogeneity of the respective sets of data configured to be used by, and a respective compute capability of, each of the respective agents configured to be managed by the one or more second nodes (122),- obtain, using the decentralized consensus mechanism, a second indication of the heterogeneity of the sets of data configured to be used by, and the respective compute capability of, each of the respective agents configured to be managed by the one or more third nodes (123),- determine whether to group the one or more second nodes (122) and the one or more third nodes (123) into a same group to train a common machine learning model, or to refrain from grouping the one or more second nodes (122) and the one or more third nodes (123) to train into the same group to train the common machine learning model, the determining of whether to group or not being configured to be based on a difference between the first indication and the second indication with respect to a threshold, and- output a third indication configured to indicate a result of the determining of whether to cluster or not.

30. The first node (111) according to claim 29, being further configured to:- determine at least a subset of the set of nodes (121),- initiate allocation of communication channels with each of the nodes in the subset of the set of nodes (121),- exchange a respective estimate of the common machine learning model with the other nodes in the subset of the set of nodes (121) configured to be determined, and wherein the determining of the one or more second nodes (122), the determining of the one or more third nodes (123), the obtaining of the first indication and the obtaining of the second indication is configured to be based on the respective estimates of the common machine learning model configured to be exchanged.

31. The first node (111) according to claim 30, wherein at least one of:- the determining of at least the subset of the set of nodes (121) is configured to be performed by detecting the subset of the set of nodes (121),- the determining of at least the subset of the set of nodes (121) is configured to be performed based on a connectivity criterion,- the initiating of the allocation is configured to be performed by sending an allocation request to a radio network node (140), and- the exchanging of the respective estimate is configured to be performed by broadcasting the respective estimate by the first node (111) and receiving a respective broadcast comprising the respective estimate by each of the other nodes in the subset of the set of nodes (121) configured to be determined.

32. The first node (111) according to claim 31 , wherein the third indication is configured to be at least one of:- an instruction configured to instruct the one or more second nodes (122) and the one or more third nodes (123) to train respective machine learning models, wherein the training is configured to be fully decentralized learning, and- a request to the radio network node (140) to deallocate allocated resources between nodes excluded from a cluster, wherein the result of the determining is configured to have been to not cluster, and wherein the allocated resources are configured to be a subset of the resources connecting the set of nodes (121).

33. The first node (111) according to claim 32, wherein the one or more communication subgraphs are configured to be updated based on the deallocation of resources.

34. The first node (111) according to any of claims 32-33, wherein the training is configured to be federated learning, and wherein the first node (111) for at least one of the clusters is configured to use only the respective data from one agent.

35. The first node (111) according to any of claims 29-34, wherein the obtaining of the first indication and the obtaining of the second indication is configured to comprise: i) performing a consensus update of the respective estimate by the first node (111) and the respective estimate by each of the one or more second nodes (122) and one or more third nodes (123), respectively, ii) performing a local model update with the respective data of the first node (111), and iii) computing the first indication and the second indication, respectively, as scores computed by an average decentralized consensus mechanism within each of the one or more second nodes (122) and the one or more third nodes (123).

36. The first node (111) according to any of claims 29-35, being further configured to:- iterate the method configured to be performed according to the configurations of any of claims 29-35 by one of: adding and subtracting an additional node to at least one of the one or more second nodes (122) and the one or more third nodes (123), until a consensus criterion is reached.

37. The first node (111) according to claim 36, wherein in at least one of the iterations, at least one of the first indication and the second indication are configured to be based on computations configured to be performed within at least one of the one or more second nodes (122) and the one or more third nodes (123), according to the decentralized consensus mechanism.

38. The first node (111) according to any of claims 36-37, wherein at least one of:- the first node (111) is configured to lack access to the respective sets of data,- each node in the set of nodes (121) is configured to lack access to the respective set of data of other nodes in the set of nodes (121),- the obtaining of the first indication and the obtaining of the second indication is configured to comprise receiving the first indication and the second indication, respectively,- the first node (111) is configured to refrain from determining any parameters for the respective machine learning models with the proviso the first node (111) is configured to be a separate node from the set of nodes (111),- the determining of the one or more second nodes (122) and the determining of the one or more third nodes (123) is configured to be initially performed one of: randomly and based on one or more first criteria, and- in every iteration, the additional node added or subtracted is configured to be selected one of: randomly and based on one or more second criteria.

39. The first node (111) according to claims 37 and 38, wherein the parameters of a respective model of the one or more second nodes (122) and the one or more third nodes (123) are configured to have been updated within at least one of the one or more second nodes (122) and the one or more third nodes (123), according to the decentralized consensus mechanism.

40. The first node (111) according to any of claims 29-39, wherein each of the first indication and the second indication are configured to be based on a moment of a predefined order.

41. The first node (111) according to any of claims 29-40, wherein the respective data is configured to have been compressed in a decentralized manner, and wherein the first node (111) for at least one of the clusters is configured to use only the respective data from one agent.

42. A communications system (100) configured to comprise a set of nodes (121), the communications system (100) being for handling a machine learning model, the communications system (100) being configured to, by at least a first node (111) comprised in the communications system (100):- determine one or more second nodes (122) out of at least a subset of the set of nodes (121), each of the nodes in the set of nodes (121) being configured to manage a respective agent capable of training a respective machine learning model with a respective set of data, and wherein the nodes in the set of nodes (121) are configured to have a capability to connect, directly or indirectly, via resources according to one or more communication subgraphs in a fully decentralized setting,- determine one or more third nodes (123) out of at least the subset of the set of nodes (121),- obtain, using a decentralized consensus mechanism, a first indication of a heterogeneity of the respective sets of data configured to be used by, and a respective compute capability of, each of the respective agents configured to be managed by the one or more second nodes (122),- obtain, using the decentralized consensus mechanism, a second indication of the heterogeneity of the sets of data configured to be used by, and the respective compute capability of, each of the respective agents configured to be managed by the one or more third nodes (123),- determine whether to group the one or more second nodes (122) and the one or more third nodes (123) into a same group to train a common machine learning model, or to refrain from grouping the one or more second nodes (122) and the one or more third nodes (123) to train into the same group to train the common machine learning model, the determining of whether to group or not being configured to be based on a difference between the first indication and the second indication with respect to a threshold, and- output a third indication configured to indicate a result of the determining of whether to cluster or not.

43. The communications system (100) according to claim 42, being further configured to, by at least the first node (111):- determine at least a subset of the set of nodes (121),- initiate allocation of communication channels with each of the nodes in the subset of the set of nodes (121),- exchange a respective estimate of the common machine learning model with the other nodes in the subset of the set of nodes (121) configured to be determined, and wherein the determining of the one or more second nodes (122), the determining of the one or more third nodes (123), the obtaining of the first indication and the obtaining of the second indication is configured to be based on the respective estimates of the common machine learning model configured to be exchanged.

44. The communications system (100) according to claim 43, wherein at least one of:- the determining of at least the subset of the set of nodes (121) is configured to be performed by detecting the subset of the set of nodes (121),- the determining of at least the subset of the set of nodes (121) is configured to be performed based on a connectivity criterion,- the initiating of the allocation is configured to be performed by sending an allocation request to a radio network node (140), and- the exchanging of the respective estimate is configured to be performed by broadcasting the respective estimate by the first node (111) and receiving a respective broadcast comprising the respective estimate by each of the other nodes in the subset of the set of nodes (121) configured to be determined.

45. The communications system (100) according to claim 44, wherein the third indication is configured to be at least one of:- an instruction configured to instruct the one or more second nodes (122) and the one or more third nodes (123) to train respective machine learning models, wherein the training is configured to be fully decentralized learning, and- a request to the radio network node (140) to deallocate allocated resources between nodes excluded from a cluster, wherein the result of the determining is configured to have been to not cluster, and wherein the resources configured to be allocated are configured to be a subset of the resources configured to be connecting the set of nodes (121).

46. The communications system (100) according to claim 42, wherein the one or more communication subgraphs are configured to be updated based on the deallocation of resources.

47. The communications system (100) according to any of claims 45-46, wherein the training is configured to be federated learning, and wherein the first node (111) for at least one of the clusters is configured to use only the respective data from one agent.

48. The communications system (100) according to any of claims 43-47, wherein the obtaining of the first indication and the obtaining of the second indication is configured to comprise: i) performing a consensus update of the respective estimate by the first node (111) and the respective estimate by each of the one or more second nodes (122) and one or more third nodes (123), respectively, ii) performing a local model update with the respective data of the first node (111), and iii) computing the first indication and the second indication, respectively, as scores configured to be computed by an average decentralized consensus mechanism within each of the one or more second nodes (122) and the one or more third nodes (123).

49. The communications system (100) according to any of claims 42-48, being further configured to, by at least the first node (111):- iterate the method configured to be performed according to the configurations of any of claims 42-48 by one of: adding and subtracting an additional node to at least one of the one or more second nodes (122) and the one or more third nodes (123), until a consensus criterion is reached.

50. The communications system (100) according to claim 49, wherein in at least one of the iterations, at least one of the first indication and the second indication are configured to be based on computations configured to be performed within at least one of the one or more second nodes (122) and the one or more third nodes (123), according to the decentralized consensus mechanism.

51. The communications system (100) according to any of claims 49-50, wherein at least one of: the first node (111) is configured to lack access to the respective sets of data,- each node in the set of nodes (121) is configured to lack access to the respective set of data of other nodes in the set of nodes (121),- the obtaining of the first indication and the obtaining of the second indication is configured to comprise receiving the first indication and the second indication, respectively,- the first node (111) is configured to refrain from determining any parameters for the respective machine learning models with the proviso the first node (111) is configured to be a separate node from the set of nodes (111),- the determining of the one or more second nodes (122) and the determining of the one or more third nodes (123) is configured to be initially performed one of: randomly and based on one or more first criteria, and- in every iteration, the additional node added or subtracted is configured to be selected one of: randomly and based on one or more second criteria.

52. The communications system (100) according to claims 50 and 51, wherein the parameters of a respective model of the one or more second nodes (122) and the one or more third nodes (123) are configured to have been updated within at least one of the one or more second nodes (122) and the one or more third nodes (123), according to the decentralized consensus mechanism.

53. The communications system (100) according to any of claims 42-52, wherein each of the first indication and the second indication are configured to be based on a moment of a pre-defined order.

54. The communications system (100) according to any of claims 42-53, wherein the respective data is configured to have been compressed in a decentralized manner, and wherein the first node (111) for at least one of the clusters is configured to use only the respective data from one agent.

55. The communications system (100) according to any of claims 42-54, wherein at least a plurality of nodes in the set of nodes (121) are configured according to any of claims 42-54.

56. The communications system (100) according to any of claims 42-55, further configured to, by at least another node (112) in the set of nodes (121):- receive the respective estimate,- determine respective one or more second nodes (122) out of at least a subset of the set of nodes (121), and- send the respective estimate of the of the common machine learning model to the other nodes in the determined subset of the set of nodes (121).

57. A computer program (1005), comprising instructions which, when executed on at least one processing circuitry (1001), cause the at least one processing circuitry (1001) to carry out the method according to any of claims 1-13.

58. A computer-readable storage medium (1006), having stored thereon a computer program (1005), comprising instructions which, when executed on at least one processing circuitry (1001), cause the at least one processing circuitry (1001) to carry out the method according to any of claims 1-13.

59. A computer program (1005), comprising instructions which, when executed on at least one processing circuitry (1001), cause the at least one processing circuitry (1001) to carry out the method according to any of claims 14-28.

60. A computer-readable storage medium (1006), having stored thereon a computer program (1005), comprising instructions which, when executed on at least one processing circuitry (1001), cause the at least one processing circuitry (1001) to carry out the method according to any of claims 14-28.

Images