Managing distributed network functions in core network

Abstract

The solution discussed provides a framework for managing distributed network functions in a core network. The framework may coordinate distributed network functions in a core network using multi-agent federated reinforcement learning. The framework may include a server network function responsible for providing a global machine learning model to one or more local network functions and managing the local network functions using a feedback mechanism. The local network function may perform local training and apply feedback from the server network function.

Description

Technical Field

[0001] Various example embodiments generally relate to the field of telecommunication systems. In particular, some example embodiments relate to solutions for managing distributed network functions in a core network. Background Art

[0002] In recent years, the number of devices connected to the Internet has gradually increased. New generations of mobile communication networks provide more reliable real-time access and significantly less latency than earlier generations. In order to improve the quality of experience (QoS) and quality of service (QoS) of end users, a new network function (NF) called network data analysis function (NWDAF) is defined in the 5G core architecture. NWDAF is a 3GPP standard method responsible for providing machine intelligence in 5G. NWDAF is able to collect data from any NF, operation, implementation and management (OAM) system or even user equipment on the edge.

[0003] There can be multiple NWDAFs in the 5G core to provide information to NF, Application Function (AF), or OAM. These NWDAFs can be located on the edge near the core network functions. Network functions can subscribe to the edge NWDAF that generates analysis and provide ultra-low latency application scenarios to improve QoE and QoS.

[0004] One possible solution for coordinating multiple NWDAF instances is to apply centralized learning in the context of the core network. Based on this approach, the edge NWDAFs can preprocess their local datasets in a parallel manner. Afterwards, the edge NWDAFs can send their preprocessed data and the analysis they provide to a defined central NWDAF. The central NWDAF acts like a server and processes the data collected from the local NWDAFs and proposes a fully trained model. Although centralized learning algorithms have advantages in processing large amounts of data, there are many disadvantages in the solution, such as:

[0005] - Latency: Sending all data to and processing all data on the central NWDAF takes a significant amount of time in each iteration.

[0006] - Low security and privacy: local datasets are exposed to the server. In addition, sharing local datasets between local agents and the central NWDAF makes the coordination framework vulnerable to security attacks such as phishing or fake data injection attacks (FDIA).

[0007] - High implementation cost: Processing all network data on the server and sending a large amount of local data to the server is extremely expensive in terms of implementation cost and energy consumption.

[0008] - Lack of robustness: In case of unpredictable events, the framework will have catastrophic results due to lack of control in the learning process. Summary of the invention

[0009] This Summary is provided to introduce some concepts in a simplified form that are further described below. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0010] Example embodiments may provide a multi-agent federated reinforcement learning solution to coordinate distributed network functions, such as NWDAF in a core network. This may enable efficient architecture implementation. The benefit may be achieved by the features of the independent claims. Further implementation forms are provided in the dependent claims, the description and the drawings.

[0011] According to a first aspect, a device may include at least one processor; and at least one memory storing instructions, which, when executed by the at least one processor, cause the device to at least perform: applying an iterative distributed machine learning model training process between the server network function and at least one local network function by a server network function, wherein the server network function initially provides an initial global machine learning model to at least one local network function, and the initial global machine learning model will be trained by at least one local network function through the corresponding local data of at least one local network function; obtaining new parameter data associated with the trained initial global machine learning model from each of the at least one local network function, and the trained initial global machine learning model is trained by each local network function in its most recent iteration; updating the initial global machine learning model based at least in part on the new parameter data obtained from at least one local network function; generating feedback data for at least one local network function based at least in part on the updated initial global machine learning model, and the feedback data of the local network function reflects the individual performance of the local network function in its most recent iteration; and sending the feedback data to at least one local network function.

[0012] In an example embodiment of the first aspect, the instructions, when executed by at least one processor, cause the device to perform: sending an initialization message to each local network function of at least one network function to start an iterative distributed machine learning model training process, the initialization message sent to the local network function including encrypted parameter data defined for the local network function, the encrypted parameter data being used to encrypt parameter data sent by the local network function; and decrypting new parameter data obtained from the local network function using the encrypted parameter data associated with the local network function.

[0013] In an example embodiment of the first aspect, the instructions, when executed by at least one processor, cause the device to perform: after each iteration, generating weight data associated with at least one local network function based at least in part on individual behavior of the at least one local network function, the weight data providing a weight value for each of the at least one local network function for weighting new parameter data associated with the local network function.

[0014] In an example embodiment of the first aspect, when executed by at least one processor, the apparatus is caused to perform: aggregating new parameter data obtained from at least one local network function using a weighted joint average and current weight data associated with at least one local network function.

[0015] In an example embodiment of the first aspect, the instructions, when executed by at least one processor, cause the device to perform: receiving an evaluation result from each of at least one local network function, the evaluation result indicating the performance of the local network function on its local data; and generating feedback data using the evaluation result.

[0016] In an example embodiment of the first aspect, the server network function comprises a network data analysis function.

[0017] According to the second aspect, a device may include: at least one processor; and at least one memory storing instructions, which, when executed by the at least one processor, cause the device to at least perform: applying an iterative distributed machine learning model training process between a server network function and a local network function by a local network function, wherein an initial global machine learning model is initially obtained from the server network function; obtaining local data; obtaining feedback data from the server network function, wherein the feedback data reflects the performance of the local network function in its most recent iteration; and training the initial global machine learning model based at least in part on the feedback data and the local data.

[0018] In an example embodiment of the second aspect, the instructions, when executed by at least one processor, cause the device to perform: receiving an initialization message from a server network function to start an iterative distributed machine learning model training process, the initialization message including encrypted parameter data defined for a local network function; obtaining new parameter data associated with a trained initial global machine learning model of the local network function; encrypting the new parameter data using the encrypted parameter data; and sending the encrypted new parameter data to the server network function.

[0019] In an example embodiment of the second aspect, the instructions, when executed by at least one processor, cause the device to perform: evaluating the predictive analysis using local data to provide an evaluation result; and sending the evaluation result to the server network function.

[0020] In an example embodiment of the second aspect, the instructions, when executed by at least one processor, cause the device to perform: obtaining new parameter data based on the trained initial global machine learning model; and sending the new parameter data to a server network function.

[0021] In an exemplary embodiment of the second aspect, the local network function includes: a network data analysis function.

[0022] According to a third aspect, a method may include: applying an iterative distributed machine learning model training process between a server network function and at least one local network function, the server network function initially providing an initial global machine learning model to at least one local network function, the initial global machine learning model to be trained by at least one local network function through corresponding local data of at least one local network function; obtaining new parameter data associated with the trained initial global machine learning model from each of the at least one local network function, the trained initial global machine learning model being trained by each local network function in its most recent iteration; updating the initial global machine learning model based at least in part on the new parameter data obtained from at least one local network function; generating feedback data for at least one local network function based at least in part on the updated initial global machine learning model, the feedback data of the local network function reflecting the individual performance of the local network function in its most recent iteration; and sending the feedback data to at least one local network function.

[0023] In an example embodiment of the third aspect, the method also includes: sending an initialization message to each local network function of at least one network function to start an iterative distributed machine learning model training process, the initialization message sent to the local network function including encrypted parameter data defined for the local network function, the encrypted parameter data being used to encrypt parameter data sent by the local network function; and decrypting new parameter data obtained from the local network function using the encrypted parameter data associated with the local network function.

[0024] In an example embodiment of the third aspect, the method further comprises: generating weight data associated with at least one local network function based at least in part on individual behavior of at least one local network function, the weight data providing a weight value for each of the at least one local network function for weighting new parameter data associated with the local network function.

[0025] In an example embodiment of the third aspect, the method further comprises aggregating new parameter data obtained from the at least one local network function using a weighted joint average and current weight data associated with the at least one local network function.

[0026] In an example embodiment of the third aspect, an evaluation result is received from each of the at least one local network function, the evaluation result indicating a performance of the local network function on its local data; and feedback data is generated using the evaluation result.

[0027] In an exemplary embodiment of the third aspect, the server network function includes: a network data analysis function.

[0028] According to a fourth aspect, a method may include: applying, by a local network function, an iterative distributed machine learning model training process between a server network function and a local network function, an initial global machine learning model being initially obtained from the server network function; obtaining local data; obtaining feedback data from the server network function, the feedback data reflecting the performance of the local network function in its most recent iteration; and training the initial global machine learning model based at least in part on the feedback data and the local data.

[0029] In an example embodiment of the fourth aspect, the method also includes: receiving an initialization message from a server network function to start an iterative distributed machine learning model training process, the initialization message including encrypted parameter data defined for a local network function; obtaining new parameter data associated with a trained initial global machine learning model of the local network function; encrypting the new parameter data using the encrypted parameter data; and sending the encrypted new parameter data to the server network function.

[0030] In an example embodiment of the fourth aspect, the method further comprises: evaluating the predictive analysis using local data to provide an evaluation result; and sending the evaluation result to the server network function.

[0031] In an example embodiment of the fourth aspect, the method further includes: acquiring new parameter data based on the trained initial global machine learning model; and sending the new parameter data to the server network function.

[0032] In an exemplary embodiment of the fourth aspect, the server network function includes: a network data analysis function.

[0033] According to the fifth aspect, a computer program may include instructions, which, when executed by an apparatus, cause the apparatus to perform at least the following items: applying an iterative distributed machine learning model training process by a server network function between the server network function and at least one local network function, the server network function initially providing an initial global machine learning model to at least one local network function, the initial global machine learning model to be trained by at least one local network function through local data of at least one local network function; obtaining new parameter data associated with the trained initial global machine learning model from each of at least one local network function, the trained initial global machine learning model being trained by each local network function in its most recent iteration; updating the initial global machine learning model based at least in part on the new parameter data obtained from at least one local network function; generating feedback data for at least one local network function based at least in part on the updated initial global machine learning model, the feedback data of the local network function reflecting the individual performance of the local network function in its most recent iteration; and sending the feedback data to at least one local network function.

[0034] According to a sixth aspect, a computer program may include instructions which, when executed by a device, cause the device to perform at least the following items: apply an iterative distributed machine learning model training process between a server network function and a local network function by a local network function, wherein an initial global machine learning model is initially obtained from the server network function; obtain local data; obtain feedback data from the server network function, wherein the feedback data reflects the performance of the local network function in its most recent iteration; and train the initial global machine learning model based at least in part on the feedback data and the local data.

[0035] According to the seventh aspect, a computer-readable medium may include a program, which includes instructions, which when executed by an apparatus, causes the apparatus to perform at least the following items: applying an iterative distributed machine learning model training process between the server network function and at least one local network function by a server network function, the server network function initially providing an initial global machine learning model to at least one local network function, the initial global machine learning model to be trained by at least one local network function through corresponding local data of at least one local network function; obtaining new parameter data associated with the trained initial global machine learning model from each of at least one local network function, the trained initial global machine learning model being trained by each local network function in its most recent iteration; updating the initial global machine learning model based at least in part on the new parameter data obtained from at least one local network function; generating feedback data for at least one local network function based at least in part on the updated initial global machine learning model, the feedback data of the local network function reflecting the individual performance of the local network function in its most recent iteration; and sending the feedback data to at least one local network function.

[0036] According to an eighth aspect, a computer-readable medium may include a program, the program including instructions, which, when executed by a device, cause the device to perform at least the following items: apply an iterative distributed machine learning model training process between a server network function and a local network function by a local network function, wherein an initial global machine learning model is initially obtained from the server network function; obtain local data; obtain feedback data from the server network function, wherein the feedback data reflects the performance of the local network function in its most recent iteration; and train the initial global machine learning model based at least in part on the feedback data and the local data.

[0037] According to the ninth aspect, a device may include components for the following items: applying an iterative distributed machine learning model training process between a server network function and at least one local network function by a server network function, the server network function initially providing an initial global machine learning model to at least one local network function, the initial global machine learning model to be trained by at least one local network function through the corresponding local data of at least one local network function; obtaining new parameter data associated with the trained initial global machine learning model from each of the at least one local network function, the trained initial global machine learning model being trained by each local network function in its most recent iteration; updating the initial global machine learning model based at least in part on the new parameter data obtained from at least one local network function; generating feedback data for at least one local network function based at least in part on the updated initial global machine learning model, the feedback data of the local network function reflecting the individual performance of the local network function in its most recent iteration; and sending the feedback data to at least one local network function.

[0038] According to the tenth aspect, an apparatus may include components for: applying an iterative distributed machine learning model training process between a server network function and a local network function by a local network function, wherein an initial global machine learning model is initially obtained from the server network function; obtaining local data; obtaining feedback data from the server network function, wherein the feedback data reflects the performance of the local network function in its most recent iteration; and training the initial global machine learning model based at least in part on the feedback data and the local data.

[0039] Many of the attendant features will be more readily appreciated as the same become better understood by reference to the following detailed description considered in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are included to provide a further understanding of example embodiments and constitute a part of this specification, illustrate example embodiments and together with the description assist in understanding the example embodiments. In the drawings:

[0041] Figure 1A An example of a method according to an example embodiment is illustrated.

[0042] Figure 1B An example of a method according to an example embodiment is illustrated.

[0043] Figure 2 A flow chart according to an example embodiment is illustrated.

[0044] Figure 3 A flow chart according to an example embodiment is illustrated.

[0045] Figure 4A flow chart according to an example embodiment is illustrated.

[0046] Figure 5A A sequence diagram according to an example embodiment is illustrated.

[0047] Figure 5B A sequence diagram according to an example embodiment is illustrated.

[0048] Figure 6 An example of an apparatus configured to practice one or more example embodiments is illustrated.

[0049] In the drawings, the same reference numerals are used to refer to the same components. DETAILED DESCRIPTION

[0050] Reference will now be made in detail to example embodiments, examples of which are illustrated in the accompanying drawings. The detailed description provided below in connection with the accompanying drawings is intended as a description of the present embodiment and is not intended to represent the only form in which the present embodiment may be constructed or utilized. The description illustrates the functions of the example and the sequence of steps for constructing and operating the example. However, the same or equivalent functions and sequences may be implemented by different examples.

[0051] Figure 1A An example of a method according to an example embodiment is illustrated. Figure 1A The example illustrated in the figure may provide a solution for implementing a distributed network function architecture (eg, NWDAF architecture). The architecture may apply federated reinforcement learning based on distributed NWDAF model training.

[0052] At 100, a server network function, such as a server proxy or an aggregator NWDAF, may apply an iterative distributed machine learning model training process between the server network function and at least one local network function. In the iterative distributed machine learning model training process, the server network function initially provides an initial global machine learning model to the at least one local network function, which is to be trained by the at least one local network function through the corresponding local data of the at least one local network function.

[0053] At 102, the server network function may obtain new parameter data associated with a trained initial global machine learning model from each of the at least one local network function, the trained initial global machine learning model being trained by each of the local network functions in its most recent iteration.

[0054] At 104, the server network function may update the initial global machine learning model based at least in part on the new parameter data obtained from the at least one local network function.

[0055] At 106, the server network function may generate feedback data for at least one local network function based at least in part on the updated initial global machine learning model, the feedback data for the local network function reflecting the individual performance of the local network function in its most recent iteration.

[0056] At 108, the server network function may send the feedback data to at least one local network function.

[0057] Figure 1B A flow chart according to an example embodiment is illustrated. Figure 1B The example shown in may provide a solution for implementing a distributed network function architecture (e.g., NWDAF architecture). The architecture may apply federated reinforcement learning based on distributed NWDAF model training.

[0058] At 110, a local network function (e.g., a local proxy or a local NWDAF) may apply an iterative distributed machine learning model training process between a server network function and a local network function, with an initial global machine learning model initially obtained from the server network function.

[0059] At 112 , the local network function may retrieve local data.

[0060] At 114 , the local network function may obtain feedback data from the server network function, the feedback data reflecting the performance of the local network function in its most recent iteration.

[0061] At 114, the local network function may train an initial global machine learning model based at least in part on the feedback data and the local data.

[0062] Figure 2 A flow chart according to an example embodiment is illustrated. Figure 2 The general solution illustrated in

[0045] uses multi-agent federated reinforcement learning to coordinate the distributed NWDAF in the core network. The solution can use federated learning to process data collected at the edge.

[0063] The system can include two types of network functions that coordinate with each other to meet defined goals:

[0064] - A server network function 200 , such as a server NWDAF or an aggregator NWDAF, may be responsible for providing 204 a global machine learning model and managing other (local) agents using a feedback mechanism 210 .

[0065] -A local network function 202, such as a local NWDAF, may perform local training of a global machine learning model and provide 206 local model parameters to a server network function based on its own local data set. For increased security, the local model parameters may be encrypted before transmission to the server network function. In an example embodiment, the local network function may apply local differential privacy (LDP) to the local model parameters. The local network function may also send 208 a loss factor to the server network function.

[0066] Figure 3 A flow chart according to an example embodiment is illustrated. Figure 3 The flowchart illustrated in shows actions that may be performed by a server network function (eg, a server agent or aggregator NWDAF).

[0067] The process starts at 300 when the aggregator NWDAF initiates the first iteration by collecting test data from its own data providers and receiving 304 an initial global model from a database 302 (e.g., an analytical data repository function (ADRF)). This operation can be performed only once in the first iteration. The initial global model is then distributed 306 to the entire local network functions, such as the local NWDAFs in the core network. In each iteration, the aggregator NWDAF can generate a global model using the model parameters of the local NWDAFs during the aggregation phase. The generated global model can be distributed in the local NWDAFs at the beginning of the next iteration.

[0068] At 308, the aggregator NWDAF obtains new parameters 310 from the local NWDAFs and aggregates the obtained parameters. In an example embodiment, the new parameters may have been encrypted by the local NWDAFs. In order to prepare the updated local parameters (i.e., new parameters) for aggregation, the aggregator NWDAF may extract the actual local parameters by eliminating the noise imposed by the local NWDAFs using, for example, a local differential privacy (LDP) method. The new parameters are then aggregated by using, for example, a weighted federated averaging method. In each iteration, the aggregator NWDAF may generate a list of weights based on the behavior of each local NWDAF. The result of the aggregation is the new parameters of the new global model, and the previous global model may be replaced by the new global model at 314. Weighted federated averaging can help the multi-agent federated reinforcement learning (FRL) framework to provide a reliable global model by associating weights to each local NWDAF based on the behavior of each local NWDAF. Aggregation can be performed, for example, based on the following formula:

[0069]

[0070] where w and w iare new aggregation parameters and transmission parameters of the local NWDAF. This mechanism can increase the robustness of the multi-agent FRL to unpredictable unexpected events. Typically, three unexpected events may occur after a considerable period of time: security attacks such as false data injection attacks (FDIA), unbalanced data sets on the edge, and the local NWDAF is killed or fails. To avoid these threats from affecting the learning process, the aggregator NWDAF can associate weights to the local NWDAF based on how close its behavior is to the reference expected behavior. Then at 322, the aggregator NWDAF can update the weight list at each iteration.

[0071] The aggregator NWDAF may also obtain loss factors from the local NWDAF at 316. The loss factors may inform the aggregator NWDAF about the performance of the local NWDAF on its local data set.

[0072] At 318, the new global model can be tested using a test data set collected, for example, from a data provider of the aggregator NWDAF. This supervision mechanism helps the system learn faster and train more efficiently. In addition, having a reference to compare the results of the global model achieved helps the operator judge the credibility and performance of the current global model.

[0073] At 320 , the aggregator NWDAF may provide analytical information about the next state of the core network and predict network behavior in the next state.

[0074] At 322, the aggregator NWDAF may update the weight list and store the weight list to internal memory 340 at 338. One benefit of updating the weight list at the end of each iteration is that this approach makes the entire training process more robust to, for example, unbalanced local datasets or false data injection attacks (FDIA).

[0075] At 324, the aggregator NWDAF may create feedback data to be sent to the local NWDAFs. The local NWDAFs may thus receive feedback data from the aggregator NWDAF based on their individual behavior in the last iteration. Thus, the illustrated solution may use feedback (rewards) to converge the entire learning network to a defined goal. The goal may be to achieve an accuracy above a defined threshold by evaluating the global model on a test dataset. Furthermore, the feedback data helps the local NWDAFs improve their accuracy in their next iteration.

[0076] The feedback data may be sent to the local NWDAF at 326. The feedback data may be separate for each local NWDAF.

[0077] At 328, the global model accuracy may be checked. If the accuracy exceeds a threshold accuracy, the aggregator NWDAF may allow the local NWDAD to make a decision at 336. If the accuracy is below the threshold accuracy, at 330, the aggregator NWDAF may deny permission for the local NWDAF to make a decision and send the new global model parameters to the local NWDAF at 322.

[0078] At 334, the aggregator NWDAF may send instructions for the local NWDAF and update the state number at 336. At 344, the aggregator NWDAF may check the state number and if the state number I exceeds the defined state number N, the process ends at 346. If the state number I is less than the defined state number N, the global model is selected at 348 and a new iteration step is started at 350.

[0079] Figure 4 A flow chart according to an example embodiment is illustrated. Figure 4 The flowchart illustrated in shows actions that may be performed by a local network function (eg, a local proxy or a local NWDAF).

[0080] The process starts at 400 when the local NWDAF receives at 402 an initial global model from the aggregator NWDAF.

[0081] At 404 , the local NWDAF may obtain local data from a data provider on the edge to train its received model with its local data.

[0082] At 408 , the local NWDAF may observe the region of interest based on the information retrieved from the internal memory 410 .

[0083] At 412, the local NWDAF may calculate a feedback function using feedback data collected from the aggregator NWDAF regarding the previous state of the local NWDAF. The feedback data, i.e., rewards, may help the local NWDAF modify its behavior in the current iteration. The rewards may be calculated based on both the rewards received from the server NWDAF and the delayed rewards calculated by the local NWDAF itself for obtaining from its region of interest.

[0084] At 416 , the local NWDAF may train an initial global model based at least in part on the feedback data and the local data 406 .

[0085] At 418, the local NWDAF may predict a new state for the region of interest and, at 420, check whether permission has been received from the aggregator NWDAF at 336. If permission has not been received, the analysis may be sent at 422 and stored in the internal memory 410, e.g., to record the behavior of the local NWDAF over time.

[0086] At 428, when permission has been received from the aggregator NWDAF, the local NWDAF may make a decision based on the provided analysis.

[0087] At 424 , the local NWDAF may collect new parameter data obtained based on training the initial global model at 416 , and send the new parameter data to the aggregator NWDAF at 426 .

[0088] The local NWDAF may send the evaluation results, such as loss factors, to the aggregator NWDAF at 430. The loss factors inform the aggregator NWDAF about the performance of the local NWDAF on its local data set.

[0089] At 432, the local NWDAF may collect feedback from the aggregator NWDAF relating to its most recent iteration sent by the aggregator NWDAF at 326, and store the collected feedback to the internal memory 410 at 434 for use in its next iteration, e.g., to record the behavior of the local NWDAF over time.

[0090] At 436, the local NWDAF may update state number I, and at 438, check state number I, if state number I exceeds the defined state number N, then the process ends at 440. If state number I is less than the defined state number N, then at 442, a new iteration step is started.

[0091] Figure 5A A sequence diagram of an example embodiment of a multi-proxy FRL framework in the context of a distributed NWDAF architecture in a core network (e.g., a 5G core network) is illustrated.

[0092] The architecture includes: a NWDAF service consumer 500 , a network repository function (NRF) 502 , an aggregator NWDAF 504 , and two local NWDAFs 506 , 508 .

[0093] At step 1a , the NWDAF service consumer 500 initiates its request by sending a NWDAF discovery request to the NRF 502 .

[0094] At step 1 b , the NRF 502 responds to the discovery request with the selected aggregator NWDAF 504 .

[0095] At step 2 , the NWDAF service consumer 500 sends a subscription request to the aggregator NWDAF 504 .

[0096] At step 3, discovery of NWDAF is performed. Figure 5B The discovery process is illustrated in more detail.

[0097] At 510, an aggregator NWDAF is selected and the aggregator NWDAF 504 begins the discovery process.

[0098] At step 3a, the aggregator NWDAF 504 sends a Nnrf_NFDiscovery request to the NRF 502 for discovering the local NWDAF in the network.

[0099] At step 3b, a discovery request is responded to by the NRF 502, which includes the IDs of the local NWDAFs 506, 508 participating in the operation.

[0100] At step 3c, the aggregator NWDAF 504 sends a NWDAF_initiatesFRL_framework message in order to force the local NWDAFs 506, 508 to start the learning process. The message may contain defined encrypted data, such as noise. By receiving these initial messages, the local NWDAFs 506, 508 may apply the defined noise to protect their local global model parameters from security attacks (e.g., based on a local differential privacy (LDP) approach). Each local NWDAF 506, 508 may receive a uniquely defined noise to apply it as protection to its local global model parameters in subsequent iteration steps.

[0101] At 512 , a training loop begins using the aggregator NWDAF 504 and the local NWDAFs 506 , 508 .

[0102] At steps 4 and 5, the local NWDAF 506, 508 sends a ML_Model_ProvisionSubscribe() to the aggregator NWDAF 504 in order to receive global model parameters and information related to the training process.

[0103] At steps 6 and 7, the aggregator NWDAF 504 responds to the ML_Model_Provision_Subscription Request() with a ML_Model_Provision Response(). The response message may contain global model parameters and information about the federated reinforcement learning framework, such as the determination of horizontal or vertical federated learning.

[0104] At steps 8 and 9, the aggregator NWDAF 504 first sends Nnwdaf_AnalyticsInfo / Nnwdaf_AnalyticsSubscription_Subscribe requests () to the local NWDAFs 506, 508 to subscribe to their analytics services. After local training at the local NWDAF 506, 508, this message can be sent to retrieve the predictive analytics provided by the local NWDAF 506, 508.

[0105] At steps 10 to 13, after local training at the local NWDAF 506, 508, the local NWDAF 506, 508 sends a Nnwdaf_AnalyticsInfo response to the aggregator NWDAF 504. The response may include predictive analytics about its area of ​​interest. The response message is followed by an AnalyticsSubscritpion_Notify message.

[0106] Local parameters may be collected from the local NWDAFs 506, 508 on the edge. Therefore, at steps 14 to 15, the aggregator NWDAF 504 sends a Nnwdaf_ModelInfo_Request() to the local NWDAFs 506, 508, with the purpose of collecting parameters of the local global model.

[0107] At steps 16-17 after receiving Nnwdaf_ModelInfo_Request(), the local NWDAF 506, 508 applies a local differential privacy (LDP) method to its parameters using the defined noise in the initial FRL framework message received earlier from the aggregator NWDAF 504. This mechanism can protect the local parameters from security attacks such as phishing attacks.

[0108] At step 18, the aggregator NWDAF 504 starts the global model aggregation procedure. The aggregator NWDAF 504 may generate a new global model using the local parameters received by the local NWDAFs 506, 508. The aggregator NWDAF 504 may also utilize weighted federated averaging for model aggregation.

[0109] At step 19, the aggregator NWDAF 504 sends the aggregation result and Nnwdaf_AnalyticsInfoResponse() to the service consumer 500. This step may be performed in each iteration of the learning process.

[0110] At steps 20 to 21 after local training, the local NWDF 506, 508 evaluates its predictive analysis using its local data set. The result of this evaluation can then be sent to the aggregator NWDAF 504 in the form of a loss factor. This information is implemented by the Nnwdaf_SendGradients / LossFactor message. The loss factor informs the aggregator NWDAF 504 about the performance of the local NWDAF 506, 508 on its local data set.

[0111] At step 22, when having all the required information and the new global model, the aggregator NWDAF 504 can test the new aggregated global model on its own test dataset and provide analysis on its area of ​​interest. After training the global model, the aggregator NWDAF 504 can provide a list of feedback (rewards) and a list of weights based on the individual behavior of the local NWDAFs 506, 508 in training the global model and the evaluation results of its local model on its local dataset.

[0112] At steps 23-24, feedback may be sent to the local NWDAFs 506, 508 via ML_ModelProvisionSubscription_Notify messages. This feedback helps the local NWDAFs 506, 508 improve their accuracy in their next iterations. The aggregator NWDAF 504 may use this feedback to help each local NWDAF 506, 508 converge to the globally defined target.

[0113] At step 25 , the aggregator NWDAF 504 may send a Nnwdaf_AnalyticsSubscription_Notify message to the NWDAF service consumer 500 .

[0114] Figure 5A and 5B Some examples of signaling messages between elements are illustrated. These signaling messages are only examples of possible signaling messages that can be used.

[0115] Figure 6An example of an apparatus 600 configured to practice one or more example embodiments is illustrated. The apparatus 600 may include at least one processor 602. The at least one processor 602 may include, for example, one or more of a variety of processing devices or processor circuitry, such as, for example, a coprocessor, a microprocessor, a controller, a digital signal processor (DSP), with or without an accompanying DSP processing circuitry; or various other processing devices including integrated circuits, such as, for example, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a microcontroller unit (MCU), a hardware accelerator, a dedicated computer chip, etc.

[0116] The apparatus 600 may further include at least one memory 604. The at least one memory 604 may be configured to store, for example, computer program code, such as operating system software and application software. The at least one memory 604 may include one or more volatile memory devices, one or more non-volatile memory devices, and / or a combination thereof. For example, the at least one memory 604 may be implemented as a magnetic storage device (such as a hard disk drive, a floppy disk, a magnetic tape, etc.), an optical magnetic storage device, or a semiconductor memory (such as a mask ROM, a PROM (programmable ROM), an EPROM (erasable PROM), a flash ROM, a RAM (random access memory), etc.).

[0117] The device 600 may also include a communication interface 608, which is configured to enable the device 600 to send information to other devices and / or receive information from other devices. In one example, the device 600 may use the communication interface 608 to send or receive signaling information and data according to at least one data communication or cellular communication protocol. The communication interface 608 may be configured to provide at least one wireless radio connection, such as, for example, a 3GPP mobile broadband connection (e.g., 3G, 4G, 5G, 6G, etc.). In another example embodiment, the communication interface 608 may be configured to provide one or more other types of connections, such as, for example, a wireless local area network (WLAN) connection standardized by the IEEE 802.11 series or the Wi-Fi Alliance; a short-range wireless network connection (such as, for example, Bluetooth, NFC (near field communication) or RFID connection); a wired connection (e.g., a local area network (LAN) connection, a universal serial bus (USB) connection or an optical network connection, etc.); or a wired Internet connection. The communication interface 608 may include or be configured to be coupled to at least one antenna to send and / or receive radio frequency signals. One or more of the various types of connections may also be implemented as a separate communication interface, which may be coupled or configured to be coupled to one or more of the multiple antennas.

[0118] When the device 600 is configured to implement some functions, some components and / or multiple components (e.g., at least one processor 602 and / or at least one memory 604) of the device 600 may be configured to implement the functions. In addition, when the at least one processor 602 is configured to implement certain functions, the program code 606 included in the at least one memory 604 may be used to implement the functions.

[0119] The functions described herein may be performed at least in part by one or more computer program product components such as software components. According to an embodiment, the device may include a processor or processor circuit system, such as a microcontroller, which is configured by program code when executed to perform the described operations and embodiments of the functions. Alternatively or additionally, the functions described herein may be performed at least in part by one or more hardware logic components. For example, but not limiting, illustrative types of hardware logic components that can be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), program-specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and graphics processing units (GPUs).

[0120] The apparatus 600 may include means for performing at least one method described herein. In an example embodiment, the means may include the at least one processor 602, the at least one memory 604 including program code 606, the program code 606 being configured to, when executed by the at least one processor, cause the apparatus 600 to perform the method.

[0121] The apparatus 600 may include, for example, a computing device, such as a network server, a network function, a network node, a cloud node, etc. Although the apparatus 600 is illustrated as a single device, it should be understood that the functionality of the apparatus 600 may be distributed to multiple devices, for example, to implement the example embodiments as a cloud computing service, where applicable. The apparatus 600 may be configured to perform or cause the performance of any aspect of the method described herein. In addition, the computer program may include instructions for causing the apparatus to perform any aspect of the method described herein when executed. The computer program may be stored on a computer-readable medium.

[0122] In an example embodiment, the apparatus 600 may be implemented in more detail in Figure 1A , 2, 3, 5A, and 5B and any of their descriptions discussed in relation to the server network function or aggregator NWDAF. In another example embodiment, the apparatus 600 may be implemented in more detail in Figure 1B , 2, 4, 5A and 5B and any of their descriptions discussed regarding the features of the local network function or local NWDAF.

[0123] Compared to traditional algorithms, one or more of the above examples and embodiments may enable solutions for making the system more robust in the event of any security attack or local node failure. In addition, one or more of the examples and embodiments discussed above may reduce latency in the network. In addition, one or more of the examples and embodiments discussed above may enable solutions in which data does not need to be shared between the local NWDAF and the aggregator NWDAF, resulting in improved confidentiality for each of the local agents. The illustrated solution does not send local data to the aggregator NWDAF, but rather the main part of the learning process is performed on the edge by the local NWDAF. In order to improve the security of the framework, the local NWDAF may apply a local differential privacy (LDP) mechanism to the resulting new parameters of the global model at the end of the training phase. As a result of applying the LDP mechanism to the new parameters, the network becomes significantly resistant to security attacks (such as FDIA, phishing attacks, and malicious node attacks). In addition, one or more of the examples and embodiments discussed above may enable solutions in which the local NWDAF is responsible for processing data received from the network functions on the edge and sending the new parameters to the aggregator NWDAF. The aggregator NWDAF then tests the global model based on the new parameters and supervises the local NWDAF. Therefore, there is no need for a central server with high computing power and for sending these large amounts of data to the central server in each cycle. As a result, the illustrated solution is more energy efficient and cheaper in terms of implementation cost than conventional approaches.

[0124] Any range or device value given herein may be extended or altered without losing the effect sought. Any embodiment may also be combined with another embodiment, unless explicitly not permitted.

[0125] Although the subject matter has been described with language specific to structural features and / or actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Instead, the specific features and actions described above are disclosed as examples of implementing the claims, and other equivalent features and actions are intended to be within the scope of the claims.

[0126] It should be understood that the above benefits and advantages may relate to one embodiment or may relate to several embodiments. The embodiments are not limited to embodiments that solve any or all problems or embodiments that have any or all benefits and advantages. It will also be understood that reference to "an" item may refer to one or more of these items.

[0127] The steps or operations of the methods herein may be implemented in any suitable order, or may be implemented simultaneously where appropriate. In addition, individual blocks may be deleted from any method without departing from the scope of the subject matter described herein. Aspects of any embodiment described above may be combined with aspects of any other embodiment described to form additional embodiments without losing the effects sought.

[0128] The term "comprising" is used herein to mean including the identified method blocks or elements, but these blocks or elements do not comprise an exclusive list and the method or apparatus may contain additional blocks or elements.

[0129] As used in this application, the term "circuitry" may refer to one or more or all of the following: (a) hardware circuit implementations only (such as implementations in analog and / or digital circuitry only), and (b) combinations of hardware circuitry and software, such as, as applicable: (i) a combination of analog and / or digital hardware circuits with software / firmware, and (ii) any portion of hardware processor(s) with software (including digital signal processor(s)), software, and memory(s) that work together to enable a device such as a mobile phone or server to perform various functions, and (c) hardware circuit(s) and / or processor(s), such as microprocessor(s) or portions of microprocessor(s), that require software (e.g., firmware) to operate, but that software may be absent when not required for operation. This definition of circuitry applies to all uses of the term in this application, including in any claims.

[0130] It should be understood that the above description is given as an example only, and various modifications may be made by those skilled in the art. The above description, examples and data provide a complete description of the structure and use of exemplary embodiments. Although various embodiments have been described above with a certain degree of particularity or with reference to one or more separate embodiments, those skilled in the art may make various changes to the disclosed embodiments without departing from the scope of this specification.

Claims

1. A device comprising: at least one processor; as well as at least one memory storing instructions, which, when executed by the at least one processor, cause the apparatus to at least perform: applying, by a server network function, an iterative distributed machine learning model training process between the server network function and at least one local network function, the server network function initially providing an initial global machine learning model to the at least one local network function, the initial global machine learning model to be trained by the at least one local network function with corresponding local data of the at least one local network function; obtaining, from each of the at least one local network function, new parameter data associated with a trained initial global machine learning model, the trained initial global machine learning model being trained by each local network function in a most recent iteration thereof; updating the initial global machine learning model based at least in part on the new parameter data obtained from the at least one local network function; generating feedback data for the at least one local network function based at least in part on the updated initial global machine learning model, the feedback data for the local network function reflecting the individual performance of the local network function in its most recent iteration; as well as The feedback data is sent to the at least one local network function.

2. The apparatus of claim 1 , wherein the instructions, when executed by the at least one processor, cause the apparatus to perform: sending an initialization message to each local network function of the at least one network function to start the iterative distributed machine learning model training process, the initialization message sent to the local network function comprising encrypted parameter data defined for the local network function, the encrypted parameter data being used to encrypt the parameter data sent by the local network function; and The new parameter data obtained from the local network function is decrypted using the encrypted parameter data associated with the local network function.

3. The apparatus according to claim 1 or 2, wherein the instructions, when executed by the at least one processor, cause the apparatus to execute: After each iteration, weight data associated with the at least one local network function is generated based at least in part on the individual behavior of the at least one local network function, the weight data providing a weight value for each of the at least one local network function for weighting the new parameter data associated with the local network function.

4. The apparatus of claim 3, wherein the instructions, when executed by the at least one processor, cause the apparatus to perform: The new parameter data obtained from the at least one local network function is aggregated using a weighted federated average and the current weight data associated with the at least one local network function.

5. The apparatus according to any one of claims 1 to 4, wherein the instructions, when executed by the at least one processor, cause the apparatus to execute: receiving an evaluation result from each of the at least one local network function, the evaluation result indicating a performance of the local network function on its local data; and The feedback data is generated using the evaluation results.

6. The apparatus according to any one of claims 1 to 5, wherein the server network function comprises: Network data analysis function.

7. A device comprising: at least one processor; as well as at least one memory storing instructions, which, when executed by the at least one processor, cause the apparatus to at least perform: applying, by a local network function, an iterative distributed machine learning model training process between a server network function and the local network function, an initial global machine learning model being initially obtained from the server network function; Get local data; obtaining feedback data from the server network function, the feedback data reflecting the performance of the local network function in its most recent iteration; and The initial global machine learning model is trained based at least in part on the feedback data and the local data.

8. The apparatus of claim 7, wherein the instructions, when executed by the at least one processor, cause the apparatus to perform: receiving an initialization message from the server network function to begin the iterative distributed machine learning model training process, the initialization message including encrypted parameter data defined for the local network function; obtaining new parameter data associated with the trained initial global machine learning model of the local network function; encrypting the new parameter data using the encrypted parameter data; and The new parameter data is sent encrypted to the server network function.

9. The apparatus according to claim 7 or 8, wherein the instructions, when executed by the at least one processor, cause the apparatus to execute: evaluating the predictive analysis using the local data to provide an evaluation result; and The evaluation result is sent to the server network function.

10. The apparatus according to any one of claims 7 to 9, wherein the instructions, when executed by the at least one processor, cause the apparatus to execute: Based on the trained initial global machine learning model, obtaining new parameter data; and The new parameter data is sent to the server network function.

11. The apparatus according to any one of claims 7 to 9, wherein the local network function comprises: Network data analysis function.

12. A method comprising: applying, by a server network function, an iterative distributed machine learning model training process between the server network function and at least one local network function, the server network function initially providing an initial global machine learning model to the at least one local network function, the initial global machine learning model to be trained by the at least one local network function with corresponding local data of the at least one local network function; obtaining, from each of the at least one local network function, new parameter data associated with a trained initial global machine learning model, the trained initial global machine learning model being trained by each local network function in a most recent iteration thereof; updating the initial global machine learning model based at least in part on the new parameter data obtained from the at least one local network function; generating feedback data for the at least one local network function based at least in part on the updated initial global machine learning model, the feedback data for the local network function reflecting the individual performance of the local network function in its most recent iteration; as well as The feedback data is sent to the at least one local network function.

13. The method according to claim 12, further comprising: sending an initialization message to each local network function of the at least one network function to start the iterative distributed machine learning model training process, the initialization message sent to the local network function comprising encrypted parameter data defined for the local network function, the encrypted parameter data being used to encrypt the parameter data sent by the local network function; and The new parameter data obtained from the local network function is decrypted using the encrypted parameter data associated with the local network function.

14. The method according to claim 12 or 13, further comprising: Based at least in part on the individual behavior of the at least one local network function, weight data associated with the at least one local network function is generated, the weight data providing a weight value for each of the at least one local network function for weighting the new parameter data associated with the local network function.

15. The method according to claim 14, further comprising: The new parameter data obtained from the at least one local network function is aggregated using a weighted federated average and the current weight data associated with the at least one local network function.

16. The method according to any one of claims 12 to 15, further comprising: receiving an evaluation result from each of the at least one local network function, the evaluation result indicating a performance of the local network function on its local data; as well as The feedback data is generated using the evaluation results.

17. The method according to any one of claims 12 to 16, wherein the server network function comprises: Network data analysis function.

18. A method comprising: applying, by a local network function, an iterative distributed machine learning model training process between a server network function and the local network function, an initial global machine learning model being initially obtained from the server network function; Get local data; obtaining feedback data from the server network function, the feedback data reflecting the performance of the local network function in its most recent iteration; and The initial global machine learning model is trained based at least in part on the feedback data and the local data.

19. The method according to claim 18, further comprising: receiving an initialization message from the server network function to begin the iterative distributed machine learning model training process, the initialization message including encrypted parameter data defined for the local network function; obtaining new parameter data associated with the trained initial global machine learning model of the local network function; encrypting the new parameter data using the encrypted parameter data; and The new parameter data is sent encrypted to the server network function.

20. The method according to claim 18 or 19, further comprising: evaluating the predictive analysis using the local data to provide an evaluation result; as well as The evaluation result is sent to the server network function.

21. The method according to any one of claims 18 to 20, further comprising: Based on the trained initial global machine learning model, obtaining new parameter data; as well as The new parameter data is sent to the server network function.

22. The method according to any one of claims 18 to 21, wherein the local network function comprises: Network data analysis function.

23. A computer program comprising instructions, which, when executed by a device, cause the device to at least perform the following: applying, by a server network function, an iterative distributed machine learning model training process between the server network function and at least one local network function, the server network function initially providing an initial global machine learning model to the at least one local network function, the initial global machine learning model to be trained by the at least one local network function using local data of the at least one local network function; obtaining, from each of the at least one local network function, new parameter data associated with a trained initial global machine learning model, the trained initial global machine learning model being trained by each local network function in a most recent iteration thereof; updating the initial global machine learning model based at least in part on the new parameter data obtained from the at least one local network function; generating feedback data for the at least one local network function based at least in part on the updated initial global machine learning model, the feedback data for the local network function reflecting the individual performance of the local network function in its most recent iteration; as well as The feedback data is sent to the at least one local network function.

24. A computer program comprising instructions, which, when executed by a device, cause the device to at least perform the following: applying, by a local network function, an iterative distributed machine learning model training process between a server network function and the local network function, an initial global machine learning model being initially obtained from the server network function; Get local data; obtaining feedback data from the server network function, the feedback data reflecting the performance of the local network function in its most recent iteration; and The initial global machine learning model is trained based at least in part on the feedback data and the local data.

Images