Consumer-participated machine learning (ML) model training in 5G core networks

Abstract

New parameters and processes are provided by which a server network data analysis function (NWDAF) (400) with a model training logic function (MTLF) can select and approve one or more consumer network functions (NF) (500) with an analysis logic function (AnLF) to participate in an upcoming and / or ongoing machine learning (ML) model training process. The ML model training may be, for example, any arbitrary ML format / architecture supported by the NWDAF, such as distributed machine learning (DML), joint learning (FL), and conventional ML.

Description

Technical Field

[0001] This application generally relates to machine learning (ML) processes, and more particularly, to processes for providing training for an ML model. Background Art

[0002] Section 6.2A of 3GPP Technical Specification TS 23.288 (V.17.6.0), titled "Architecture enhancements for 5G System (5GS) to support network data analytics services", which is incorporated herein by reference in its entirety, describes a process for providing a machine learning (ML) model. In this version of the specification, a network data analytics function (NWDAF) including an analytics logic function (AnLF) is locally configured with a set of identifiers (IDs). This set of IDs identifies the NWDAF containing the model training logic function (MTLF), and the (one or more) analytics IDs supported by each NWDAF containing the MTLF, to retrieve the trained ML model. If needed, the NWDAF containing the AnLF can utilize the NWDAF discovery process to discover the NWDAF containing the MTLF identified by the IDs included in the locally configured set of IDs. The NWDAF containing the MTLF can determine that further training is needed for an existing ML model in response to receiving an ML model subscription or an ML model request. Summary of the Invention

[0003] Embodiments of the present disclosure provide new parameters and processes by which a server network data analytics function (NWDAF) having a model training logic function (MTLF) (hereinafter, "server NWDAF") can select and approve one or more consumer network functions (NFs) having an analytics logic function AnLF to participate in an upcoming and / or ongoing machine learning (ML) model training process.

[0004] In a first aspect, for example, the present disclosure provides a method for determining whether a consumer network function (NF) is approved to participate in training a machine learning (ML) model. The method is implemented by a network node acting as a consumer NF and includes sending a discovery request to a registration center to discover a server network data analytics function (NWDAF). The discovery request indicates the ability of the consumer NF to support participation in training the ML model. The method further includes subscribing to the server NWDAF and receiving a subscription response message from the server NWDAF. The subscription response message indicates whether the consumer NF is permitted to participate in the training of the ML model.

[0005] In a second aspect, the present disclosure provides a method for determining whether a consumer network function (NF) is approved to participate in training a machine learning (ML) model. In this aspect, the method is implemented by a network node acting as a server network data analytics function (NWDAF), and includes registering a configuration file (profile) of the server NWDAF with a registration center. The profile indicates the ability of the server NWDAF to support the consumer NF in participating in the training of the ML model. The method further includes receiving a subscription request from the consumer NF. The subscription request includes information related to one or both of the availability of data for training the ML model at the consumer NF and the ability of the consumer NF to evaluate the ML model. The method further includes: based on the information received in the subscription request, determining whether to allow the consumer NF to participate in the training of the ML model, and then sending a subscription response message to the consumer NF, the subscription response message indicating whether the consumer NF is allowed to participate in the training of the ML model.

[0006] In a third aspect, the present disclosure provides a network node configured to act as a consumer network function (NF). In this aspect, the network node includes a processing circuit and a memory. The memory contains instructions executable by the processing circuit, whereby the network node is configured to: send a discovery request to a registration center to discover a server network data analytics function (NWDAF), wherein the discovery request indicates the ability of the consumer NF to support participating in the training of the ML model, subscribe to the server NWDAF, and receive a subscription response message from the server NWDAF, wherein the subscription response message indicates whether the consumer NF is allowed to participate in the training of the ML model.

[0007] In a fourth aspect, the present disclosure provides a network node configured to act as a server network data analytics function (NWDAF). In this aspect, the network node includes a processing circuit and a memory. The memory contains instructions executable by the processing circuit, whereby the network node is configured to: register a configuration file of the server NWDAF with a registration center, wherein the profile indicates the ability of the server NWDAF to support the consumer NF in participating in the training of the ML model, receive a subscription request from the consumer NF, wherein the subscription request includes information related to one or both of the availability of data for training the ML model at the consumer NF and the ability of the consumer NF to evaluate the ML model, and based on the information received in the subscription request, determine whether to allow the consumer NF to participate in the training of the ML model. If so determined, the instructions further configure the network node to send a subscription response message to the consumer NF, the subscription response message indicating whether the consumer NF is allowed to participate in the training of the ML model.

[0008] In a fifth aspect, the present disclosure provides a method for selecting a consumer network function (NF) to evaluate a machine learning (ML) model before training the ML model. The method is implemented by a network node acting as the consumer NF and includes receiving a participation request message from a server network data analytics function (NWDAF). The participation request message includes one or more parameters associated with the consumer NF participating in training the ML model. The method further includes: based on the one or more parameters received in the participation request message, determining to participate in training the ML model, and then sending a participation response message to the server NWDAF, the participation response message indicating that the consumer NF is capable of participating in the training of the ML model.

[0009] In a sixth aspect, the present disclosure provides a method for selecting a consumer network function (NF) to evaluate a machine learning (ML) model before training the ML model. In this aspect, the method is implemented by a network node acting as a server network data analytics function (NWDAF) and includes: sending a participation request message to the consumer NF, wherein the participation request message includes one or more parameters associated with the consumer NF participating in training the ML model, receiving a participation response message from the consumer NF, the participation response message indicating that the consumer NF is capable of participating in the training of the ML model, wherein the participation response message includes at least one of the one or more parameters sent to the consumer NF in the participation request message, and based on the at least one parameter received in the participation response message, determining that the consumer NF is allowed to participate in training the ML model. Having so determined, the method further includes: sending a participation confirmation message to the consumer NF, the participation confirmation message indicating that the consumer NF is allowed to participate in the training of the MF model.

[0010] In a seventh aspect, the present disclosure provides a network node configured to act as a consumer network function (NF). The network node includes a processing circuit and a memory. The memory contains instructions executable by the processing circuit, whereby the network node is configured to: receive a participation request message from a server network data analytics function (NWDAF), wherein the participation request message includes one or more parameters associated with the consumer NF participating in training the ML model, based on the one or more parameters received in the participation request message, determine to participate in training the ML model, and send a participation response message to the server NWDAF, the participation response message indicating that the consumer NF is capable of participating in the training of the ML model.

[0011] In an eighth aspect, the present disclosure provides a network node configured to function as a network data analytics function (NWDAF). In this aspect, the network node includes a processing circuit and a memory. The memory contains instructions executable by the processing circuit, whereby the network node is configured to: send a participation request message to a consumer NF, wherein the participation request message includes one or more parameters associated with the consumer NF participating in training an ML model, receive a participation response message from the consumer NF, the participation response message indicating that the consumer NF is capable of participating in the training of the ML model, wherein the participation response message includes at least one of the one or more parameters sent to the consumer NF in the participation request message, based on the at least one parameter received in the participation response message, determine that the consumer NF is permitted to participate in the training of the ML model, and send a participation confirmation message to the consumer NF, the participation confirmation message indicating that the consumer NF is permitted to participate in the training of the MF model.

[0012] In a ninth aspect, the present disclosure provides a method for selecting a consumer network function (NF) to participate in the training of a machine learning (ML) model. The method is implemented by a network node functioning as a consumer NF and includes: sending a participation announcement message to a server network data analytics function (NWDAF), the participation announcement message indicating that the consumer NF is capable of participating in the training of the ML model, and receiving a participation confirmation message from the server NWDAF, the participation confirmation message indicating that the consumer NF is permitted to participate in the training of the MF model.

[0013] In a tenth aspect, the present disclosure provides a method for selecting a consumer network function (NF) to participate in the training of a machine learning (ML) model. The method is implemented by a network node functioning as a server network data analytics function (NWDAF) and includes: receiving a participation announcement message from the consumer NF that includes one or more parameters, wherein the one or more parameters indicate that the consumer NF is capable of participating in the training of the ML model, based on the one or more parameters received in the participation announcement message, determine that the consumer NF is capable of participating in the training of the ML model, and send a participation confirmation message to the consumer NF, the participation confirmation message indicating that the consumer NF is permitted to participate in the training of the MF model.

[0014] In an eleventh aspect, the present disclosure provides a network node configured to function as a consumer network function (NF). The network node includes a processing circuit and a memory. The memory contains instructions executable by the processing circuit, whereby the network node is configured to: send a participation announcement message to a server network data analytics function (NWDAF), the participation announcement message indicating that the consumer NF is capable of participating in the training of the ML model, and receive a participation confirmation message from the server NWDAF, the participation confirmation message indicating that the consumer NF is permitted to participate in the training of the MF model.

[0015] In a twelfth aspect, the present disclosure provides a network node configured to function as a Network Data Analytics Function (NWDAF). The network node includes processing circuitry and a memory. The memory contains instructions executable by the processing circuitry whereby the network node is configured to: receive, from a consuming NF, a participation announcement message including one or more parameters, wherein the one or more parameters indicate that the consuming NF is capable of participating in training an ML model; determine, based on the one or more parameters received in the participation announcement message, that the consuming NF is capable of participating in the training of the ML model; and send to the consuming NF a participation confirmation message indicating that the consuming NF is permitted to participate in the training of the MF model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a signaling flow diagram illustrating an example message passing for subscribing to and unsubscribing from machine learning (ML) analytics between a Network Data Analytics Function (NWDAF) service consumer having an Analytics Logic Function (AnLF) and a server NWDAF including a Model Training Logic Function (MTLF) according to aspects of the present disclosure.

[0017] Figure 2 is a signaling flow diagram illustrating an example message passing for requesting and obtaining information about an ML model between a NWDAF service consumer and a server NWDAF including an MTLF according to aspects of the present disclosure.

[0018] Figures 3A to 3B is a signaling diagram illustrating an example message passing for Federated Learning (FL) between multiple instances of a NWDAF according to aspects of the present disclosure.

[0019] Figure 4 is a signaling diagram illustrating an example message passing for providing FL training updates from a server NWDAF to a NWDAF service consumer according to aspects of the present disclosure.

[0020] Figures 5A to 5B is a signaling diagram illustrating an example message passing associated with a process for model performance assurance during FL according to aspects of the present disclosure.

[0021] Figure 6 and Figures 7A to 7C are block diagrams of corresponding systems in which one or more NWDAF service consumers interact with a server NWDAF according to aspects of the present disclosure.

[0022] Figure 8 is a signaling diagram illustrating an example message passing for approving a NWDAF service consumer to participate in the training of an ML model according to aspects of the present disclosure.

[0023] Figure 9is a signaling diagram showing an example messaging for searching for and selecting one or more NWDAF service consumers to participate in the training of an ML model according to aspects of the present disclosure.

[0024] Figure 10 is a signaling diagram showing an example messaging for a NWDAF service consumer to notify its ability to participate in the training of an ML model and for a server NWDAF to select one or more of these NWDAF service consumers to participate in the training of an ML model according to aspects of the present disclosure.

[0025] Figure 11 is a flowchart showing a method implemented by a network node acting as a consumer network function (NF) for determining whether the consumer NF is approved to participate in training an ML model according to an aspect of the present disclosure.

[0026] Figure 12 is a flowchart showing a method implemented by a network node acting as a server NWDAF for determining whether a consumer NF is approved to participate in training an ML model according to an aspect of the present disclosure.

[0027] Figure 13 is a flowchart showing a method implemented by a network node acting as a consumer NF for selecting a consumer NF to evaluate an ML model before training the ML model according to an aspect of the present disclosure.

[0028] Figure 14 is a flowchart showing a method implemented by a network node acting as a server NWDAF for selecting a consumer NF to evaluate an ML model before training the ML model according to an aspect of the present disclosure.

[0029] Figure 15 is a flowchart showing a method implemented by a network node acting as a consumer NF for selecting a consumer NF to participate in training an ML model according to an aspect of the present disclosure.

[0030] Figure 16 is a flowchart showing a method implemented by a network node acting as a server NWDAF for selecting a consumer NF to participate in training an ML model according to an aspect of the present disclosure.

[0031] Figure 17A is a block diagram showing some components of a network node acting as a server NWDAF according to an aspect of the present disclosure.

[0032] Figure 17B is a block diagram showing some functional components of a computer program product executed on a processing circuit of a server NWDAF according to an aspect of the present disclosure.

[0033] Figure 18Ais a block diagram showing some components of a network node acting as an NWDAF service consumer according to an aspect of the present disclosure.

[0034] Figure 18B is a block diagram showing some functional components of a computer program product executed on a processing circuit of an NWDAF service consumer according to an aspect of the present disclosure. Detailed Description

[0035] Embodiments of the present disclosure provide new parameters and processes by which a server network data analytics function (NWDAF) having a model training logic function (MTLF) (hereinafter, "server NWDAF") can select and approve one or more consumer network functions (NFs) having an analysis logic function AnLF to participate in an upcoming and / or ongoing machine learning (ML) model training process. By way of example only, the ML model training can be any arbitrary ML format / architecture supported by the NWDAF, such as distributed machine learning (DML), federated learning (FL), and conventional ML.

[0036] Subscribe / Unsubscribe ML Model Analysis

[0037] Turning now to the drawings, Figure 1 illustrates an example messaging 10 for subscribing to and unsubscribing from machine learning (ML) analysis between an NWDAF service consumer 12 having an analysis logic function (AnLF) and a server NWDAF 14 including a model training logic function (MTLF) according to aspects of the present disclosure. The process can be used, for example, by the NWDAF service consumer (i.e., NWDAF 12 having AnLF) to subscribe to and unsubscribe from the server NWDAF 14 using the Nnwdaf_MLModelProvision service (as defined in clause 7.5 of TS 23.288 V.17.6.0, which TS is incorporated herein by reference in its entirety) in order to be notified when ML model information associated with the analysis of a model becomes available. The ML model information is used by the NWDAF service consumer to derive an analysis. The service is also used by the NWDAF to modify an existing (one or more) ML model subscription. It should be understood that for the purposes of the present disclosure, the NWDAF can be a consumer of services provided by one or more other NWDAFs and a provider of services to one or more other NWDAFs.

[0038] As Figure 1As shown, the NWDAF service consumer 12 subscribes to, modifies, or unsubscribes from a (set of) trained ML models associated with (a set of) analysis IDs by invoking the Nnwdaf_MLModelProvision_Subscribe or Nnwdaf_MLModelProvision_Unsubscribe service operations. In Figure 1 the embodiment of Figure 1 , the NWDAF service consumer 12 invokes the Nnwdaf_MLModelProvision_Subscribe service operation to subscribe to the ML model provision from the server NWDAF 14, or invokes the Nnwdaf_MLModelProvision_Unsubscribe service operation to unsubscribe from the ML model provision from the server NWDAF 14 (line 16). The parameters that can be provided by the NWDAF service consumer 12 during these operations are described in more detail in Table 1A and Table 1B below. In any case, when receiving a subscription for a trained ML model associated with an analysis ID, the server NWDAF 14 can determine one or more aspects including but not limited to the following:

[0039] · Whether an existing trained ML model is available for the subscription; and

[0040] · Whether it is necessary to trigger further training of the existing trained ML model for the subscription.

[0041] If the server NWDAF 14 determines that further training is necessary, the server NWDAF 14 can initiate collecting data from one or more NFs to generate an ML model. For example, as described in Section 6.2 of TS23.288 V.17.6.0, such NFs can include the access and mobility management function (AMF), the data collection coordination function (DCCF), the analytics data repository function (ADRF), the user equipment (UE) application (via the application function (AF)), and / or the operation, administration, and maintenance (OAM) function.

[0042] If the service call is for subscription modification or subscription cancellation, the NWDAF service consumer 12 includes the identifier to be modified (i.e., the subscription-related ID) when invoking the Nnwdaf_MLModelProvision_Subscribe service operation.

[0043] If NWDAF service consumer 12 subscribes to a (set of) trained ML models associated with an (set of) analysis IDs, the server NWDAF 14 notifies the NWDAF service consumer 12 of information for the trained ML models (e.g., a set of file addresses of the trained ML models) by invoking the Nnwdaf_MLModelProvision_Notify service operation (line 20). The content of the trained ML model information that can be provided by the server NWDAF is described in more detail in Table 2 below.

[0044] In some aspects, in response to invoking the Nnwdaf_MLModelProvision_Subscribe service operation at line 16, the server NWDAF 14 may determine that a previously provided trained ML model needs to be retrained. In such an instance, the server NWDAF may also invoke the Nnwdaf_MLModelProvision_Notify service operation (line 20) to notify the subscribed NWDAF service consumers of the availability of the retrained ML model.

[0045] Additionally or alternatively, the NWDAF service consumer 12 may invoke the Nnwdaf_MLModelProvision_Subscribe service operation at line 16 to modify the subscription (i.e., by including subscription-related IDs). In this case, the server NWDAF 14 may be configured to provide a new trained ML model different from the previously provided trained ML model or provide a retrained ML model by invoking the Nnwdaf_MLModelProvision_Notify service operation at line 20.

[0046] Content Provided by the ML Model (Section 6.2A.2 of TS 23.288 v.17.6.0)

[0047] As described above, a consumer NF of the ML model provision service (e.g., NWDAF service consumer) may provide various input parameters when invoking the Nnwdaf_MLModelProvision_Subscribe and / or Nnwdaf_MLModelProvision_UnSubscribe service operations. These parameters are listed in Table 1A below and are described in more detail in Sections 7.5 and 7.6 of TS23.288 V.17.6.0. Not all parameters are required; rather, some parameters are optional.

[0048]

[0049] Table 1A: Information on the analysis for which the requested ML model is to be used

[0050] Table 1B lists the ML reporting information parameters according to the event reporting information parameter defined in Table 4.15.1-1 of 3GPP Technical Specification TS 23.502 V.17.6.0, titled "Procedures for the 5G System (5GS); Stage 2", which is incorporated herein by reference in its entirety. Note that the parameters listed in Table 1B are only for the Nnwdaf_MLModelProvision_Subscribe service operation, and like Table 1A, some parameters are optional.

[0051]

[0052] Table 1B: ML Model Reporting Information Parameters

[0053] In addition, the server NWDAF can provide output information to the NWDAF service consumers of the service operation of the ML model, as described in Sections 7.5 and 7.6 of TS 23.288 V.17.6.0. Such information includes notification-related information and is provided by the server NWDAF 14, for example, only in the Nnwdaf_MLModelProvision_Notify service operation. The validity period and spatial validity parameters are determined by the internal logic of the MTLF. Further, they are subsets of the Age of Information (AoI) (when provided in the ML model filter information) and the ML model target period parameter, respectively (see Tables 1A - 1B).

[0054]

[0055] Table 2: ML Model Provision Service Operations Provided by Server NWDAF

[0056] ML Model Request (Section 6.2A.3 of TS23.288 v.17.6.0)

[0057] Figure 2 Illustrates an example messaging 30 between a consumer NF 12 (e.g., NWDAF service consumer) and a server NWDAF 14 for requesting and obtaining information about an ML model according to aspects of the present disclosure, for example, using the Nnwdaf_MLModelInfo service operation defined in Section 7.6 of TS 23.288 v.17.6.0. The ML model information parameters are used by the consumer NF to derive an analysis. As described above, a given NWDAF can be both a consumer of services provided by one or more other NWDAFs and a provider of services to one or more other NWDAFs.

[0058] As Figure 2As shown, the NWDAF service consumer 12 (i.e., the consumer NF) requests one or more ML models associated with (a set of) analysis IDs by invoking the Nnwdaf_MLModelInfo_Request service operation (line 32). The parameters that can be provided by the NWDAF service consumer 12 during this service operation are listed in Table 1A - Table 1B above. In any case, in response to receiving the request, the server NWDAF can determine various aspects. By way of example only, the server NWDAF 14 can determine:

[0059] · Whether existing trained ML models are available for the request; and

[0060] · Whether triggering further training of the existing trained ML models is required for the request.

[0061] If the server NWDAF 14 determines that further training is required, the server NWDAF 14 can initiate collecting data from one or more various NFs to generate an ML model. As described above and as described in Section 6.2 of TS 23.288 V.17.6.0, one or more NFs can include but are not limited to AMF, DCCF, ADRF, UE applications (via AF), and / or OAM functions.

[0062] After receiving the Nnwdaf_MLModelInfo_Request service operation on line 32, the server NWDAF 14 responds to the NWDAF service consumer 12 by invoking the Nnwdaf_MLModelInfo_Request response service operation (line 34). This operation provides the NWDAF service consumer 12 with ML model information, which includes (a set of) file addresses of the trained ML models. The content of the ML model information that can be provided by the server NWDAF 14 in the Nnwdaf_MLModelInfo_Request response service operation is listed and described in Table 2 above.

[0063] NWDAF Service Consumer Participates in FL Among Multiple NWDAF Instances in 5GC

[0064] The 3GPP Technical Specification TR 23.700-81 (V.1.1.0) entitled "Study of Enablers for Network Automation for 5G System (5GS); Phase 3", which is incorporated herein by reference in its entirety, considers the role of the NWDAF service consumer in the FL process among multiple NWDAF instances in the 5GC. By way of example only, these considerations are discussed in connection with Solutions #24, #52, and #69 in TR 23.700-81 (V.1.1.0).

[0065] NWDAF Service Consumer in Solution #24 Participates in FL

[0066] In TR 23.700-81 (V.1.1.0), Solution #24 addresses Key Issue #8 (i.e., "Support for Federated Learning in the 5GC"). This solution specifically proposes that during the FL training process and based on a request received from the consumer NF, the server NWDAF notifies the consumer NF (e.g., the NWDAF service consumer) of the training status of the ML model. So notified, the client NF can modify its subscription to the server NWDAF for a new model request. The server NWDAF will then update or terminate the FL training process accordingly.

[0067] Figures 3A to 3B An example message passing 40 for the FL process among multiple instances of such NWDAF in accordance with aspects of the present disclosure is shown. As Figure 3A shown, in the NWDAF registration and discovery phase 60, one or more consumer NF instances 42a, 42b, 42c (e.g., client NWDAFs such as NWDAF service consumer 12) invoke the Nnrf_NFManagement_NFRegister_request service operation to register their respective client NWDAF profiles (lines 62, 64, 66) with the Network Repository Function (NRF). As described in Section 5.2.7.2.2 of TS 23.502, the client NWDAF profile has a client NWDAF type and includes, as parameters, the address of the consumer NF, information related to its ability to support FL, one or more analysis IDs, and a service area. After receiving each request, the NRF stores the profiles of each consumer NF instance 46a, 46b, 46c (block 68) and responds to the consumer NF instances 46a, 46b, 46c by invoking the corresponding Nnrf_NFManagement_NFRegister_response service operation (lines 70, 72, 74).

[0068] In addition, the server NWDAF 44 may invoke the Nnrf_NFDiscovery_Request service operation to the NRF 48 to discover one or more consumer NF instances 46a, 46b, 46c (line 76) available for FL. This service operation allows the server NWDAF 44 to obtain the IP addresses of the consumer NF instances 46a, 46b, 46c and includes one or more parameters, which include one or more analysis IDs, capability information regarding the ability of the consumer NF instances 46a, 46b, and 46c to support FL, and a service area. Upon receipt, the NRF 48 authorizes NF service discovery (block 78) and responds by invoking the Nnrf_NFDiscovery_RequestResponse service operation, which provides the instances (i.e., IP addresses) of the consumer NFs that support the (one or more) analysis IDs provided by the server NWDAF (line 80).

[0069] In some embodiments, it may be assumed that the analysis ID provided by the server NWDAF 44 is preconfigured for the FL type. Thus, based on this preconfiguration, the NRF 48 is able to determine that the server NWDAF 44 sending the request will perform federated learning. As described above, the NRF 48 responds to the server NWDAF 44 with the (one or more) IP addresses of one or more consumer NF instances 46a, 46b, 46c that support the provided (one or more) analysis IDs. Note that in some cases, the (one or more) analysis IDs that support FL are configured by the network operator. In any case, the server NWDAF 44 selects which consumer NF instances 46a, 46b, 46c will participate in the FL training of a given ML model based on the response from the NRF 48.

[0070] In the federated learning training phase 90, the server NWDAF 44 then sends requests (initial FL parameter provision requests) to the selected consumer NF instances 46a, 46b, 46c participating in FL (lines 92, 94, 96). The request may include parameters such as information identifying the initial ML model, a list of data types, a maximum response time window, etc., to assist in local model training for FL. In one embodiment, this step is consistent with the result of key issue #8 addressed in solution #24 described in TR 23.700-81 (V.1.1.0).

[0071] Then, each consumer NF instance 46a, 46b, 46c collects its local data (box 98) by using the current mechanism described in Section 6.2 of TS 23.288 v.17.6.0. Then, during FL training, each consumer NF instance 46a, 46b, 46c trains the ML model retrieved from the server NWDAF 44 based on its own data and reports the results of the ML model training (e.g., gradients) to the server NWDAF 44 (lines 100, 102, 104). During the FL training process, the trained models / parameters are shared / exchanged among the multiple consumer NF instances 46a, 46b, 46c by using the Nnwdaf_MLAggregation service operation defined in Section 6.24.3 of TR 23.700-81 (V.1.1.0) or the extended Nnwdaf_MLModelProvision service operation. For the specification phase, only one option should be selected.

[0072] After receiving the results of the ML model training, the server NWDAF 44 aggregates all the local ML model training results retrieved in lines 100, 102, 104, such as gradients, to update the global ML model (box 106). Then, based on the requests from the consumer NF instances 46a, 46b, 46c, the server NWDAF 44 uses the Nnwdaf_MLModelProvision_Notify service operation to update the training status (e.g., accuracy level) to the consumer NF (line 108). The update can occur periodically (e.g., per round of training, multiple rounds of training, every 10 minutes, etc.) or dynamically (such as when a certain predefined state (e.g., accuracy level) is reached). Optionally, the consumer NF instances 46a, 46b, 46c can determine whether the current ML model can meet the given requirements (e.g., accuracy and time). If so, the consumer NF instances 46a, 46b, 46c can modify its subscription (line 110). In any case, according to the requests from the consumer NF instances 46a, 46b, 46c, the server NWDAF 44 updates or terminates the current FL training process by invoking the Nnwdaf_MLAggregation_Modify or Nnwdaf_MLAggregation_Terminate service operation (box 112).

[0073] However, if the FL process continues, the server NWDAF 44 sends the aggregated ML model information (e.g., updated ML model) to each consumer NF instance 46a, 46b, 46c for the next round of ML model training (lines 114, 116, 118). In one aspect, this is achieved by the server NWDAF 44 invoking the Nnwdaf_MLAggregration_Notify / Nnwdaf_MLModelProvision_Notify service operation. Upon receipt, each consumer NF instance 46a, 46b, 46c updates its own ML model based on the aggregated model information (e.g., updated ML model) distributed by the server NWDAF (block 120). In at least one aspect of the present disclosure, a portion of the collaborative learning training phase 90 (i.e., the portion of the process from line 100 to block 120) is repeated until a training termination condition is reached (e.g., when a maximum number of iterations is reached or the result of the loss function is below a threshold). In any case, once the ML training process is complete, the globally optimal ML model or ML model parameters can be distributed to the consumer NFs for inference.

[0074] NWDAF Service Consumer in Solution #52 Participates in FL

[0075] Figure 4 An example message passing 130 for providing FL training updates from a server NWDAF to a consumer NF (e.g., NWDAF service consumer) in accordance with aspects of the present disclosure is shown. TR 23.700-81 (V.1.1.0) presents solution #52 for key issue #8 to support the FL process between different server NWDAFs and provide FL training updates from a server NWDAF (e.g., NWDAF having FL aggregation capabilities / executing the role of an FL server) to one or more consumer NFs (e.g., one or more NWDAFs, each containing a corresponding AnLF). Section 6.52.2 in TR 23.700-81 (V.1.1.0) describes an example process for FL training updates from a server NWDAF (e.g., NWDAF including MTLF) to a consumer NF (e.g., NWDAF including AnLF). It should be noted here that aspects related to the sharing of the (one or more) trained ML models should be consistent with key issue #5.

[0076] As Figure 4As shown, the consumer NF 132 (e.g., NWDAF including AnLF) discovers one or more instances (lines 140, 142, 144) of the server NWDAF 134 (e.g., NWDAF including MTLF) via the NRF 136. In particular, the consumer NF 132 invokes the Nnrf_Discovery_Request service operation on the NRF 136, which provides one or more analytics IDs and ML model filter information as parameters (line 140). Then, the NRF 136 responds by invoking the Nnrf_Discovery_Response service operation, which provides the IP addresses of one or more server NWDAF 134 instances as parameters (line 142). Additionally or alternatively, the consumer NF 132 may invoke the Nnwdaf_MLModelProvision_Subscribe service operation on the server NWDAF 134, which provides one or more analytics IDs and notification correlation IDs as parameters (line 144).

[0077] Once discovered, the FL training between the server NWDAF with joint aggregation capabilities and the server NW DAF with joint participation capabilities is performed as described in Solution #21 of TR 23.700-81 (V.1.1.0) (see, for example Figure 6 .21.2.3-1) or in Solution #23 of TR 23.70081 (V.1.1.0) (see, for example Figure 6 .23.2-1) (box 146). In addition, the server NWDAF 134 with joint participation capabilities sends its local training accuracy metric via the Nnwdaf_MLModelTraining_Notify service operation (see, for example, Figure 6 .23.2-1 of TR 23.700-81 (V.1.1.0)) or the process of exchanging ML model parameters (e.g., step 11 in Figure 6 .23.2-1 of TR 23.700-81 (V.1.1.0)).

[0078] Then, the server NWDAF 134 sends an Nnwdaf_MLmodelProvision_Notify message that has (one or more) analysis IDs, (one or more) ML model IDs, (one or more) ML model file addresses, (one or more) ML model serialization formats, and a training accuracy metric per ML model ID (line 148). The training accuracy metric indicates the ML model accuracy when the server NWDAF performs training using a training data set. In this embodiment, the training accuracy metric is calculated by the server NWDAF 134 with joint aggregation capabilities by aggregating local training accuracy metrics received by the box 146 Figure 4 combined with.

[0079] Then, the consumer NF 132 (e.g., NWDAF including an AnLF) conditionally sends an Nnwdaf_MLModelTrainingUpdate_Subscribe message to the server NWDAF 134 that has parameters including one or more analysis IDs, (one or more) ML model IDs, a basic accuracy metric, and one or more notification-related IDs (line 150). In this embodiment, the same (one or more) ML model IDs provided in Figure 4 line 148 are also included in the Nnwdaf_MLModelTrainingUpdate_Subscribe message sent to the server NWDAF 134. The basic accuracy metric is an accuracy metric determined by the consumer NF 132 using a data set from the live network and provided by the consumer NF132 to notify the server NWDAF 134. Specifically, the server NWDAF 134 is notified when the same ML model or a new ML model (e.g., for a given analysis ID) in Figure 4 line 148 is available and the training accuracy is higher than the basic accuracy metric. In at least one embodiment, the accuracy metric used in the embodiment depends on the conclusion of Key Issue #1 in TR 23.700-81 (V.1.1.0), which is similar to accuracy or multi-access edge (MAE).

[0080] Then, the server NWDAF 134 conditionally responds to the consumer NF 132 (line 152) by sending an Nnwdaf_MLModelTrainingUpdate_Notify message to the consumer NF with one or more analysis IDs, (one or more) ML model IDs, and (one or more) training accuracy metrics. The (one or more) ML model IDs included in the message can be the same retrained ML model as provided in line 148, which has a new training accuracy higher than the base accuracy metric provided in line 150. Additionally or alternatively, the (one or more) ML model IDs can be the IDs of new trained ML models available for the (one or more) analysis IDs provided in line 150, and the new trained ML models have an accuracy level higher than the base accuracy metric.

[0081] Then, the consumer NF 132 conditionally decides whether it wants to use the ML model ID provided in step 4 (block 154). If so, the consumer NF 132 conditionally sends an Nnwdaf_MLModelProvision_Request message with the (one or more) analysis IDs and (one or more) ML model IDs provided in line 152 (line 156). Then, the server NWDAF 134 conditionally sends an Nnwdaf_MLModelProvision_Response message to the consumer NF with the (one or more) analysis IDs, (one or more) ML model IDs, (one or more) ML model file addresses, and training accuracy metrics (line 158).

[0082] NWDAF Service Consumer in Solution #69 Participates in FL

[0083] Solution #69 of TR 23.700-81 (V.1.1.0) is proposed for Key Issue #8 regarding FL among multiple NWDAF instances in TR 23.700-81 (V.1.1.0). More specifically, Solution #69 proposes that the consumer NF (e.g., the NWDAF service consumer with AnLF) calculates a "usage accuracy" metric during the FL training process and sends the calculated result to the FL server with MTLF. Therefore, Figures 5A to 5B An example message passing 160 associated with a process for model performance guarantee during FL in accordance with aspects of the present disclosure is shown.

[0084] It should be noted that as a prerequisite, the server NWDAF 168 (e.g., FL server or FL client) registers "FL capability" support information (box 172) with the NRF 166. In particular, the server NWDAF 168 can register information about available ML models in the NRF 166, such as the analysis ID of the available ML models, model filter information, and model accuracy levels.

[0085] Then, a consuming NF (such as analytics consumer 162) requests an analytics subscription from the NWDAF (line 174). For example, in one embodiment, the analytics consumer NF 162 receives an Nnwdaf_AnalyticsSubscription_Subscribe request from the AnLF 164 (line 176). The request message in this embodiment can indicate the preferred accuracy level for the analytics requested by the analytics consumer NF 162. After receiving it, the AnLF 164 accepts the subscription and sends an Nnwdaf_AnalyticsSubscription_Subscribe response to the analytics consumer NF 162.

[0086] Then, the AnLF 164 derives the analysis ID, model filter information, and model accuracy level information from the analysis ID, analysis filter information, and preferred accuracy level of the analysis received from the analytics consumer NF 162 (box 178). If the AnLF 164 does not have a model that meets the derived analysis ID, model filter information, and model accuracy level, the AnLF will attempt to discover an MTLF with the required model (box 180). If the discovered MTLF can provide or train a model that meets the model accuracy level, the AnLF 164 can obtain the model for analysis and proceed directly to line 198 ( Figure 5B ). In this case, FL is not required. If there is no MTLF that can provide a model with the required model accuracy level, the AnLF 134 discovers an MTLF that supports the FL server (i.e., the MTLF has registered the "FL capability" of the FL server in the NRF). In this case, FL is required.

[0087] The AnLF 164 sends an Nnwdaf_MLModelInfo_Request to the FL server MTLF 168 and provides one or more analysis IDs, model filter information, and model accuracy level information as parameters in the message (line 182). Then, the FL server MTLF 168 discovers one or more candidate FL client MTLFs 170 from the NRF 166 and adds them to the federated learning group.

[0088] The FL server MTLF 168 delivers the initial / common model to the AnLF 164 for accuracy evaluation before each iteration of FL by invoking the Nnwdaf_MLModelEvaluation_Request service operation (line 184). Additionally, the AnLF 164 uses the historically collected data as a validation dataset to evaluate the accuracy level of the initial / common model. In at least one embodiment, the AnLF 164 provides the usage accuracy of the initial / common model to the FL server MTLF by invoking the Nnwdaf_MLModelEvaluation_Request Response operation (line 186).

[0089] Then, the FL server MTLF 168 delivers the initial / common model to each FL client MTLF 170 for accuracy evaluation before each iteration of FL by invoking the Nnwdaf_MLModelEvaluation_Request operation (line 188). The FL client MTLF 170 utilizes the local training data as a validation dataset to evaluate the accuracy level of the initial / common model and provides the training accuracy value of the initial / common model to the FL server MTLF by invoking the Nnwdaf_MLModelEvaluation_Request Response operation (line 190).

[0090] Then, the FL server MTLF 168 compares the training accuracy of the initial / common model from the FL client MTLF 170 with the usage accuracy of the initial / common model from the AnLF (block 192). If the training accuracy calculated by the FL client MTLF 170 is very different from the usage accuracy calculated by the AnLF 166, it can be assumed that the characteristics of the local dataset of the MTLF will be different from the characteristics of the data used by the AnLF 164. Therefore, the FL server MTLF 168 can remove the FL client MTLF 170 from the FL group.

[0091] Then, the FL server MTLF 168 performs the current iteration of FL with the FL client MTLF 170 in the FL group and generates the total training accuracy by aggregating the received training accuracies (block 194). Note that, in one aspect, for each iteration of FL, the FL server MTLF 168 repeats the part of method 160 from line 184 to block 194 until it receives a model with a satisfactory accuracy level. The FL server MTLF 168 can stop starting a new iteration of FL in response to determining that there is no improvement in accuracy.

[0092] The FL server MTLF 168 provides the model obtained from the FL to the AnLF 164 (line 196) by invoking the Nnwdaf_MLModelInfo_Response service operation. Then, using the model received from the FL, the AnLF 166 provides the analysis output to the analytics consumer NF 162 by invoking the Nnwdaf_AnalyticsSubscriptionNotify message (line 198).

[0093] In the solutions given in TR 23.700-81 (V.1.1.0), the participation of the consumer NF (e.g., the NWDAF service consumer having the AnLF) in the FL (i.e., solutions #24, #52, and #69) is considered to ensure and improve the performance of the trained ML model. However, it is still unclear how the server NWDAF obtains information from the consumers who are to participate in the ML model training process. Further, these solutions do not indicate how the server NWDAF selects a consumer NF (e.g., the AnLF) to participate in the ML model training process in the case where more than one consumer NF (e.g., multiple AnLFs) is available. Additionally, these solutions do not address the corresponding interactions between the consumer NF and the server NWDAF before and / or during the training process for obtaining information, approving, and selecting one or more appropriate consumer NFs.

[0094] Embodiments of the present disclosure address these and other drawbacks. In particular, the present embodiment provides new parameters and procedures for use by the server NWDAF in approving and selecting appropriate consumer NFs to participate in an upcoming / ongoing ML model training process. The new parameters include parameters that define the following items:

[0095] · The ability to support the participation of the consumer NF in the ML model training;

[0096] · An indication of the availability of real usage data, test data, and validation data;

[0097] · An indication of the ability to evaluate the initial ML model, intermediate ML model, and / or final ML model (e.g., testing / validating the accuracy, precision, etc. of the ML model); and

[0098] · The participation mode. In at least one embodiment, the possible participation modes for the consumer NF during the ML model training process include:

[0099] · Mode A: Indicates the status of the ML model being evaluated. According to this embodiment, the ML model status evaluation is different from the ML model evaluation. That is, the server NWDAF performs an evaluation on the ML model and provides the status of the updated ML model to the consuming NF, as introduced in Solution #24 of TR 23.700-81 (V.1.1.0);

[0100] · Mode B: Participates substantially continuously in each round of training for ML model evaluation;

[0101] · Mode C: Participates periodically in training for ML model evaluation (e.g., every n rounds, where n > 1);

[0102] · Mode D: Participates in training once for ML model evaluation when triggered by the server NWDAF; and

[0103] · Mode E: Participates in the final ML model evaluation.

[0104] In addition, this embodiment provides new interactions and corresponding procedures that occur between the server NWDAF and the consuming NF. For example, consider the following scenarios for the server NWDAF to approve and select appropriate consuming NFs to participate in the ML model training process.

[0105] · In the consuming NF request, the server NWDAF approves the consuming NF to participate in the upcoming / ongoing ML model training process;

[0106] · The consuming NF is also selected by the server NWDAF to participate in the upcoming / ongoing ML model training process. This process can occur, for example, in the context of the following scenarios:

[0107] · Case 1: When the server NWDAF searches for and selects one or more appropriate consuming NFs; and

[0108] · Case 2: When the server NWDAF monitors and receives announcements from the consuming NFs and then selects one or more consuming NFs.

[0109] As explained in more detail below, by providing these new parameters and procedures, this embodiment provides benefits and advantages that traditional solutions cannot or do not provide.

[0110] Figure 6 and Figures 7A to 7C are block diagrams showing corresponding systems 200, 210, 220, 230 in which one or more NWDAF service consumers interact with the server NWDAF according to aspects of the present disclosure. More specifically, Figure 6Shows the communication interaction between the NWDAF service consumer 202 and the server NWDAF 204. Figures 7A to 7C Shows the communication interaction between one or more consumer NFs 206 and the server NWDAF 204 and, in some cases (e.g., Figure 7B ) between one or more consumer NFs 206, the server NWDAF 204, and the NRF 208.

[0111] This embodiment considers two cases where the NWDAF service consumer (i.e., the consumer NF) participates in the ML model training.

[0112] These cases are:

[0113] 1. When the server NWDAF approves the consumer NF to participate in the upcoming / ongoing ML model training process after receiving a request from the consumer NF; and

[0114] 2. When the server NWDAF selects the consumer NF to participate in the upcoming / ongoing ML model training process.

[0115] More specifically, Figure 7A Corresponding to the case where the consumer NF 206 sends a request to the server NWDAF 204 to participate in the upcoming / ongoing ML model training process, and in response, the server NWDAF 204 approves the consumer NF 206 to participate in the process. As described in more detail later, the server NWDAF 204 decides whether to allow the consumer NF 206 to participate in the upcoming / ongoing ML model training process in the agreed ML training participation mode.

[0116] Figure 7B And Figure 7C Corresponding to the case where the server NWDAF 204 selects the approved consumer NFs 206a, 206b to participate in the upcoming / ongoing ML model training process. In this context, there are two cases where the server NWDAF 204 selects a given consumer NF 206a, 206b to participate in the upcoming / ongoing ML model training process.

[0117] Specifically:

[0118] · Case 1: The server NWDAF 204 searches for and selects one or more consumer NFs 206a, 206b, as Figure 7BAs shown. In this case, the server NWDAF 204 searches for available consumer NFs 206a, 206b. After receiving responses from one or more consumer NFs 206a, 206b, 206c, the server NWDAF 204 determines which of the consumer NFs 206a, 206b can participate in the ML model training process in the agreed participation mode and indicates this selection to the consumer NFs 206a, 206b.

[0119] · Case 2: The server NWDAF 204 monitors the announcements made by the consumer NFs 206a, 206b and selects the consumer NFs 206a, 206b, as Figure 7C shown. In this case, the server NWDAF 204 monitors the announcements sent by one or more consumer NFs 206a, 206b indicating their ability to participate in the upcoming / ongoing ML model training process. After receiving the announcements, the server NWDAF 204 determines which consumer NFs

[0120] 206a, 206b are able to participate in the ML model training process in the agreed participation mode and indicates these selections to the consumer NFs 206a, 206b.

[0121] Figure 8 illustrates an example messaging 240 in which the server NWDAF 204 according to one or more embodiments of the present disclosure approves a consumer NF206 (e.g., a NWDAF service consumer with an AnLF) to participate in an upcoming / ongoing ML model training process in an agreed ML training participation mode.

[0122] As Figure 8 shown, the server NWDAF 204 first registers its NWDAF profile (block 242) in a registry (such as, for example, the NRF). In addition to the regular NRF registration elements of the NWDAF profile, this embodiment configures the server NWDAF204 to also provide the following elements when registering its profile.

[0123] · Information indicating the server NWDAF 204's ability to support the participation of the consumer NF 206 in the ML model training process;

[0124] and

[0125] · The supported participation modes, if available.

[0126] Then, the consuming NF 206 discovers the server NWDAF 204 from the registry (block 244). For example, in this embodiment, the consuming NF 206 invokes the Nnrf_NFDiscovery_Request service operation to discover the server NWDAF 204. In addition to the parameters typically sent in the Nnrf_NFDiscovery_Request service operation, this embodiment configures the consuming NF 206 to also provide the following parameters in the discovery request.

[0127] · Information indicating the ability of the consuming NF 206 to participate in the ML model training process; and

[0128] · The supported participation mode.

[0129] Then, the consuming NF 206 initiates a subscription to the server NWDAF 204 by invoking, for example, the Nnwdaf_MLModelProvision_Subscribe request service operation (line 246). In addition to the parameters typically provided with this request, this embodiment configures the consuming NF 206 to also provide:

[0130] · An indication of the availability of one or more data that has been actually used, data for testing, and data for verification;

[0131] · An indication of the ability of the consuming NF to evaluate an initial ML model, an intermediate ML model, and / or a final ML model. Such evaluation includes, but is not limited to, tests and / or verifications for evaluating the accuracy and / or precision of the ML model; and

[0132] · The supported participation mode. As described above, the possible participation modes for the consuming NF 206 in the ML model training process include:

[0133] · Mode A: Indicates participation in evaluating the state of the ML model. According to this embodiment, the ML model state evaluation is different from the ML model evaluation. That is, the server NWDAF 204 performs an evaluation on the ML model and provides the updated state of the ML model to the consuming NF 206, as introduced in Solution #24 of TR 23.700-81

[0134] (V.1.1.0);

[0135] · Mode B: Participates substantially continuously in each round of training for ML model evaluation;

[0136] · Mode C: Participates periodically in training for ML model evaluation (e.g., every n rounds, where n > 1);

[0137] · Mode D: Participate in training once when triggered by the server NWDAF 204 for ML model evaluation; and

[0138] · Mode E: Participate in the final ML model evaluation.

[0139] In addition to operations for preparing and / or executing the ML model training process, this embodiment further configures the server NWDAF 204 to determine whether to allow the consumer NF 206 to participate in an upcoming / ongoing ML model training process (block 248). In at least one embodiment, the decision is made based on the training logic at the server NWDAF 204, one or more local policies at the server NWDAF 204, and the information provided in the request from the consumer NF 206.

[0140] Then, the server NWDAF 204 responds to the consumer NF 206, indicating whether the consumer NF 206 is allowed to participate in the ML model training process (line 250). As Figure 8 shown, for example, the server NWDAF 204 invokes the Nnwdaf_MLModelProvision_Subscribe response service operation. According to this embodiment, the server NWDAF 204 is configured to provide the following information to the requesting consumer NF 206 in addition to the information normally sent with this message.

[0141] · An indication of whether the consumer NF 206 is allowed to participate in the ML model training process; and

[0142] · The supported participation modes for the consumer NF 206 to use when participating in an upcoming / ongoing ML model training process.

[0143] Figure 9 and Figure 10 illustrates an example messaging where appropriate consumer NFs 206a - 206n are identified and selected to participate in an upcoming / ongoing ML model training process. In particular, there are two cases where the server NWDAF 204 selects a given consumer NF 206a - 206n to participate in an upcoming / ongoing ML model training process. These cases are:

[0144] · Case 1: When the server NWDAF 204 searches for and selects one or more consumer NFs 206a–206n (e.g., as Figure 7B and Figure 9 shown); and

[0145] · Case 2: When the server NWDAF 204 monitors the announcements made by the consumer NFs 206a - 206n and selects the consumer NFs 206a - 206n (e.g., as shown in Figure 7C and Figure 10 ).

[0146] Specifically, Figure 9 shows an example messaging 260 in which the server NWDAF 204 searches for and selects one or more consumer NFs 206a - 206n (e.g., NWDAF service consumers with AnLF) to participate in an upcoming / ongoing ML model training process. As shown in Figure 9 , each consumer NF 206a - 206n and the server NWDAF 204 first register their respective profiles (block 262) in a registry (such as, for example, the NRF). In addition to the regular NRF registration elements, this embodiment configures the server NWDAF 204 to also provide the following elements.

[0147] · Information indicating the server NWDAF 204's ability to support the consumer NFs 206a - 206n in participating in the ML model training process; and

[0148] · The supported participation mode, if available.

[0149] Furthermore, this embodiment configures each consumer NF 206a - 206n to also provide the following registration elements.

[0150] · Information indicating the consumer NF 206a - 206n's ability to participate in the ML model training process; and

[0151] · The supported participation mode.

[0152] In some cases, the server NWDAF 204 may need to discover the consumer NFs 206a - 206n from the registry (e.g., the NRF) (block 264). In such a case, the server NWDAF 204 is configured according to this embodiment to invoke the Nnrf_NFDiscovery_Request service operation, as previously described. According to the present disclosure, the server NWDAF 204 is configured to provide the following parameters in the discovery request in addition to the parameters that are normally present.

[0153] · Information indicating the server NWDAF 204's ability to support the consumer NFs 206a - 206n in participating in the ML model training process; and

[0154] · The participation mode.

[0155] Anyhow, then, the server NWDAF 204 is configured to send search messages (lines 266, 268) to the consumer NFs 206a - 206n. According to the present disclosure, the following parameters are included in the search message.

[0156] · One or more analysis IDs;

[0157] · An ML - related ID or an FL - related ID; and

[0158] · A participation mode. The possible participation modes for the consumer NFs 206a–206n are listed above as Mode A–Mode E.

[0159] After receiving the search message from the server NWDAF 204, the consumer NFs 206a - 206n decide whether to participate in the ML model training process announced by the server NWDAF 204. The (one or more) consumer NFs 206a - 206n that decide to participate in the training process then send a response (lines 270, 272) to the server NWDAF 204 with response information requesting to participate in the ML model training process. As Figure 9 shown, for example, the consumer NFs 206a - 206n can respond by providing the following parameters to the server NWDAF 204.

[0160] · One or more analysis IDs;

[0161] · An ML - related ID or an FL - related ID; and

[0162] · A participation mode (identified above as Mode A - Mode E).

[0163] After receiving the (one or more) responses from the (one or more) consumer NFs 206a - 206n, the server NWDAF 204 decides whether to allow one or more of the consumer NFs 206a - 206n to participate in the upcoming / ongoing ML model training process (block 274). As described above, the server NWDAF 204 configured according to this embodiment makes this determination based on its own training logic, one or more local policies accessible to the server NWDAF 204, and the information provided in the response information from the (one or more) consumer NFs 206a - 206n.

[0164] The server NWDAF 204 responds to the (one or more) consumer NFs 206a - 206n, indicating whether it has been selected to participate in the upcoming / ongoing ML model training process (lines 276, 278). According to this embodiment, the server NWDAF 204 is configured to provide the following information to the consumer NFs 206a - 206n.

[0165] · An indication of whether the consumer NFs 206a–206n are allowed to participate in the ML model training process; and

[0166] · The supported participation modes for the consumer NFs 206a–206n to use when participating in an upcoming / ongoing ML model training process.

[0167] It should be noted here that the (one or more) consumer NFs that receive a response from the server NWDAF indicating that they are approved to participate in the ML model training process will participate in the ML model training process according to the supported participation modes indicated in the response.

[0168] As previously mentioned, Figure 10 illustrates some example messaging 280 for the consumer NFs 206a-206n (e.g., NWDAF service consumers with corresponding AnLFs) to announce their ability to participate in the training of an ML model and for the server NWDAF 204 to select one or more of these consumer NFs 206a-206n to participate in the training of the ML model, according to one or more embodiments of the present disclosure.

[0169] As Figure 10 shown, each consumer NF 206a-206n and the server NWDAF 204 first register their respective profiles (block 282) in a registry (e.g., the NRF). As mentioned above, both the server NWDAF 204 and the consumer NFs 206a-206n are configured to provide new registration elements in addition to the regular NRF registration elements. Specifically, the server NWDAF 204 is configured to also provide:

[0170] · Information indicating the server NWDAF 204's ability to support the consumer NFs 206a-206n in participating in the ML model training process; and

[0171] · The supported participation modes, if available.

[0172] The consumer NFs 206a-206n are configured to also provide:

[0173] · Information indicating the consumer NFs 206a-206n's ability to participate in the ML model training process; and

[0174] · The supported participation modes.

[0175] Optionally, one or more consumer NFs 206a - 206n may discover the server NWDAF 204 (block 284) from a registry (e.g., the NRF) by invoking the Nnrf_NFDiscovery_Request service operation. In this case, in addition to the parameters conventionally included, the consumer NFs 206a - 206n may also provide the following parameters in the discovery request.

[0176] · An indication of the server NWDAF 204's ability to support the consumer NFs 206a - 206n during the ML model training process; and

[0177] · Participation mode.

[0178] Then, the consumer NFs 206a - 206n send a message (lines 286, 288) to the server NWDAF 204 notifying that it can participate in the ML model training process. According to the present disclosure, the following parameters may be included in the notification message.

[0179] · One or more analysis IDs;

[0180] · An ML - related ID or an FL - related ID; and

[0181] · Participation mode. The possible participation modes for the consumer NFs 206a–206n are listed above as Mode A - Mode E.

[0182] After receiving the notification message from the consumer NFs 206a - 206n, the server NWDAF 204 decides whether to allow one or more of the consumer NFs 206a - 206n to participate in the ML model training process (block 290). As described above, the server NWDAF 204 makes this determination based on its own training logic, one or more local policies accessible to the server NWDAF 204, and the information provided in the notification message from the consumer NFs 206a - 206n.

[0183] Then, the server NWDAF 204 responds to each of the (one or more) consumer NFs 206a - 206n, indicating whether it has been selected to participate in the upcoming / ongoing ML model training process (lines 292, 294). According to this embodiment, the server NWDAF 204 is configured to provide the following information to the consumer NFs 206a - 206n.

[0184] · An indication of whether the consumer NF 206a - 206n is allowed to participate in the ML model training process; and

[0185] · Supported participation modes for the consumer NF 206a - 206n to use when participating in an upcoming / ongoing ML model training process.

[0186] As described above, the (one or more) consumer NF 206a - 206n that receive a response from the server NWDAF 204 indicating that they are approved to participate in the ML model training process will participate in the ML model training process according to the supported participation mode indicated in the response.

[0187] Alternatively, Figure 10 the functions shown in box 282 to lines 292, 294 in Figure 8 can be directly replaced by the functions of box 244 to lines 250 in Figure 10 or can be replaced by replacing the single consumer NF 206 shown in Figure 8 with multiple consumer NFs (e.g., consumer NFs 206a - 206n as shown in Figure 10 ). Then, in the context of Figure 10 the embodiments of Figure 8 can be regarded as specific cases of the embodiments shown in

[0188] Figure 11 is a flowchart showing a method 300 implemented by a network node acting as a consumer NF 206 for determining whether the consumer NF 206 is approved to participate in training an ML model according to an aspect of the present disclosure. As shown in Figure 11 the method 300 requires the consumer NF 206 to send a discovery request to the registration center to discover the server NWDAF 204 (box 302). The discovery request indicates the ability of the consumer NF 206 to support participating in training the ML model. Then, the method 300 requires the consumer NF 206 to subscribe to the server NWDAF 204 (box 304). Then, the consumer NF 206 receives a subscription response message from the server NWDAF 204 (box 306). In this embodiment, the subscription response message indicates whether the consumer NF 206 is allowed to participate in the training of the ML model.

[0189] In one embodiment, sending a discovery request to the registration center to discover the server NWDAF includes: the consumer NF sending an Nnrf_NFDiscovery_Request service message to the registration center.

[0190] In one embodiment, the discovery request further identifies one or more ML model training participation modes supported by the consumer NF 206.

[0191] In one embodiment, the discovery request further indicates one or more of the following: the availability of data for use by the consumer NF 206 in the training of the ML model, the ability of the consumer NF 206 to evaluate the training of the ML model, and one or more ML model training participation modes supported by the consumer NF 206.

[0192] In one embodiment, the data for use by the consumer NF 206 in the training of the ML model includes one or more of the following: the actual data used by the ML model, the test data for testing the ML model, and the validation data for validating the ML model.

[0193] In one embodiment, the indication of the ability of the consumer NF 206 to evaluate the training of the ML model specifies whether the consumer NF 206 is able to test or validate the accuracy of the ML model.

[0194] In one embodiment, the ML model is one of an initially trained ML model, an intermediate trained ML model, and a final ML model.

[0195] In one embodiment, one or more ML model training participation modes supported by the consumer NF 206 include: a first participation mode, in which the consumer NF 206 participates in evaluating the state of the ML model; a second participation mode, in which the consumer NF 206 substantially continuously participates in the training of the ML model to evaluate the ML model; a third participation mode, in which the consumer NF 206 periodically participates in the training of the ML model to evaluate the ML model; a fourth participation mode, in which the consumer NF 206 is triggered by the server NWDAF 204 to participate in the training of the ML model to evaluate the ML model; and a fifth participation mode, in which the consumer NF 206 provides the final evaluation of the ML model.

[0196] In one embodiment, in the first participation mode, the consumer NF 206 evaluates the ML model and provides the ML model state to the server NWDAF 204.

[0197] In one embodiment, the subscription response message received from the server NWDAF 204 includes one or both of the following: an indication that the consumer NF 206 is approved to participate in the training of the ML model, and the selected ML model training participation mode for use by the consumer NF 206 in training the ML model. The selected ML model training participation mode is selected by the server NWDAF 204 from one or more ML model training participation modes included in the discovery request.

[0198] Figure 12is a flowchart showing a method 310 implemented by a network node acting as a server NWDAF 204 according to an aspect of the present disclosure for determining whether a consumer NF 206 is approved to participate in training an ML model. As Figure 12 shown, the server NWDAF 204 first registers a configuration file of the server NWDAF 204 with a registry (block 312). The configuration file indicates the ability of the server NWDAF 204 to support the consumer NF 206 in participating in the training of the ML model. Then, the server NWDAF 204 receives a subscription request from the consumer NF 206 (block 314). In this embodiment, the subscription request includes information related to one or both of the availability of data for training the ML model at the consumer NF 206 and the ability of the consumer NF 206 to evaluate the ML model. Then, the server NWDAF 204 determines whether to allow the consumer NF 206 to participate in the training of the ML model based on the information received in the subscription request (block 316), and then sends a subscription response message to the consumer NF 206 indicating whether the consumer NF 206 is allowed to participate in the training of the ML model (block 318).

[0199] In one embodiment, the configuration file of the server NWDAF 204 also indicates one or more ML model training participation modes supported by the server NWDAF 204 for use in the training of the ML model.

[0200] In one embodiment, the subscription request also indicates one or more of the following: the availability of data for use by the consumer NF 206 in the training of the ML model, the ability of the consumer NF 206 to evaluate the training of the ML model, and one or more ML model training participation modes supported by the consumer NF 206.

[0201] In one embodiment, determining whether to allow the consumer NF 206 to participate in the training of the ML model is further based on training logic accessible to the server NWDAF 204 and / or one or more policies of the server NWDAF 204.

[0202] In one embodiment, the subscription response message sent to the consumer NF 206 includes one or both of the following: an indication that the consumer NF 206 is approved to participate in the training of the ML model, and a selected ML model training participation mode for use by the consumer NF 206 in training the ML model. In this embodiment, the selected ML model training participation mode is selected by the server NWDAF 204 from one or more ML model training participation modes included in the discovery request.

[0203] Figure 13is a flowchart showing a method 320 implemented by a network node acting as a consumer NF 206 for selecting the consumer NF 206 to evaluate an ML model before training the ML model according to one aspect of the present disclosure. As Figure 13 shown, method 320 requires the consumer NF 206 to receive a participation request message from the server NWDAF 204 (block 324). In this embodiment, the participation request message includes one or more parameters associated with the consumer NF 206 participating in training the ML model. Then, method 320 requires the consumer NF 206 to decide to participate in training the ML model based on the one or more parameters received in the participation request message (block 326). Then, the consumer NF 206 sends a participation response message to the server NWDAF 204, and the participation response message indicates that the consumer NF 206 can participate in the training of the ML model (block 328).

[0204] In one embodiment, method 320 also requires the consumer NF 206 to register a configuration file of the consumer NF 206 with a registry (block 322).

[0205] In one embodiment, the configuration file of the consumer NF 206 includes information indicating one or both of the following: the ability of the consumer NF 206 to support participating in training the ML model, and one or more ML model training participation modes supported by the consumer NF 206.

[0206] In one embodiment, the one or more parameters received together with the participation request message include one or more of the following: an analysis ID, an ML-related ID or a federated learning (FL)-related ID, and a selected ML model training participation mode for the consumer NF 206 to participate in the training of the ML model.

[0207] In one embodiment, the one or more ML model training participation modes supported by the consumer NF 206 include: a first participation mode, in which the consumer NF 206 participates in evaluating the state of the ML model; a second participation mode, in which the consumer NF 206 participates in training the ML model substantially continuously to evaluate the ML model; a third participation mode, in which the consumer NF 206 participates in training the ML model periodically to evaluate the ML model; a fourth participation mode, in which the consumer NF 206 is triggered by the server NWDAF 204 to participate in training the ML model to evaluate the ML model; and a fifth participation mode, in which the consumer NF 206 provides a final evaluation of the ML model.

[0208] In one embodiment, the participation response message sent to the server NWDAF 204 includes one or more of the following: an analysis ID, an ML-related ID or an FL-related ID, and the selected ML model training participation mode.

[0209] In one embodiment, method 320 further includes: the consumer NF 206 receives a participation confirmation message from the server NWDAF 204, and the participation confirmation message indicates whether the consumer NF 206 is allowed to participate in the training of the MF model.

[0210] In one embodiment, the participation confirmation message further indicates the selected participation mode.

[0211] In one embodiment, method 320 further includes: the consumer NF 206 participates in the training of the ML model according to the selected participation mode (block 330).

[0212] Figure 14 is a flowchart showing method 340 for selecting consumer NF 206 to evaluate an ML model before training the ML model, implemented by a network node acting as server NWDAF 204 according to one aspect of the present disclosure. As Figure 14 shown, method 340 begins with the server NWDAF 204 sending a participation request message to the consumer NF 206 (block 346). In this embodiment, the participation request message includes one or more parameters associated with the consumer NF 204 participating in the training of the ML model. Then, the server NWDAF 204 receives a participation response message from the consumer NF 206, and the participation response message indicates that the consumer NF 206 is capable of participating in the training of the ML model (block 348). The participation response message in this embodiment includes at least one of the one or more parameters sent to the consumer NF 206 in the participation request message. Then, method 340 requires the server NWDAF204 to determine that the consumer NF 206 is allowed to participate in the training of the ML model based on at least one parameter received in the participation response message (block 350), and to send a participation confirmation message to the consumer NF 206, and the participation confirmation message indicates that the consumer NF 206 is allowed to participate in the training of the MF model (block 352).

[0213] In one embodiment, method 340 further includes the server NWDAF 204 sending a registration message including a configuration file of the server NWDAQ to the registration center (block 342).

[0214] In one embodiment, the configuration file of the server NWDAF 204 includes information indicating one or both of the following: the ability of the server NWDAF 204 to support participation in the training of the ML model, and one or more ML model training participation modes supported by the server NWDAF 204.

[0215] In one embodiment, method 340 further includes the server NWDAF 204 sending a discovery request to a registry to discover the consumer NF 206 (block 344). In such an embodiment, the discovery request indicates one or both of the following: the ability of the consumer NF to support participation in the training of the ML model, and one or more ML model training participation modes supported by the consumer NF 204.

[0216] In one embodiment, one or more parameters included in the participation request message include one or more of the following: an analysis ID, an ML-related ID or a federated learning (FL)-related ID, and an ML model training participation mode that the consumer NF 204 should support to participate in the training of the ML model.

[0217] In one embodiment, at least one parameter in the participation response message includes one or more of the following: an analysis ID, an ML correlation D or an FL-related ID, and an ML model training participation mode.

[0218] In one embodiment, determining that the consumer NF 206 is allowed to participate in training the ML model is further based on the training logic at the server NWDAF 204 and / or one or more policies of the server NWDAF 204.

[0219] In one embodiment, the participation confirmation message sent to the consumer NF 206 also indicates the ML training participation mode that the consumer NF will use to participate in the training of the ML model.

[0220] Figure 15 is a flowchart showing a method 360 implemented by a network node acting as a consumer NF 206 for selecting the consumer NF 206 to participate in training an ML model according to an aspect of the present disclosure. As Figure 15 shown, method 360 includes the consumer NF 206 sending a participation announcement message to the server NWDAF 204, the participation announcement message indicating that the consumer NF 206 can participate in the training of the ML model (block 364). Then, the consumer NF 206 receives a participation confirmation message from the server NWDAF 204, the participation confirmation message indicating that the consumer NF 206 is allowed to participate in the training of the MF model (block 366).

[0221] In one embodiment, method 360 further includes the consumer NF 206 sending a registration message including a configuration file of the consumer NF to a registry (block 362).

[0222] In one embodiment, the configuration file of the consumer NF includes information indicating one or both of the following: the ability of the consumer NF to support participation in training the ML model, and one or more ML training participation modes supported by the consumer NF 206.

[0223] In one embodiment, the participation announcement message includes one or more of the following: an analysis ID, an ML-related ID or a federated learning (FL)-related ID, and one or more ML model training participation modes supported by the consumer NF 206.

[0224] In one embodiment, the participation confirmation message received from the server NWDAF 204 indicates one or both of the following: whether the consumer NF 206 is allowed to participate in the training of the MF model, and the selected ML model training participation mode for the consumer NF 206 to use to participate in the training of the ML model. The selected ML model training participation mode is selected from one or more ML model training participation modes supported by the consumer NF 206.

[0225] In one embodiment, one or more ML model training participation modes supported by the consumer NF 206 include: a first participation mode, in which the consumer NF 206 participates in evaluating the state of the ML model; a second participation mode, in which the consumer NF 206 substantially continuously participates in the training of the ML model to evaluate the ML model; a third participation mode, in which the consumer NF 206 periodically participates in the training of the ML model to evaluate the ML model; a fourth participation mode, in which the consumer NF 206 is triggered by the server NWDAF 204 to participate in the training of the ML model to evaluate the ML model; and a fifth participation mode, in which the consumer NF 206 provides a final evaluation of the ML model.

[0226] Figure 16 is a flowchart showing a method 370 implemented by a network node acting as the server NWDAF 204 for selecting a consumer NF 206 to participate in the training of an ML model. As Figure 16 shown, the method 370 requires the server NWDAF 204 to receive a participation announcement message (block 374) including one or more parameters from the consumer NF 206. The one or more parameters indicate that the consumer NF 206 can participate in training the ML model. Then, the server NWDAF 204 determines, based on the one or more parameters received in the participation announcement message, that the consumer NF 206 can participate in the training of the ML model (block 376). Then, the server NWDAF 204 sends a participation confirmation message to the consumer NF 206, the participation confirmation message indicating that the consumer NF 206 is allowed to participate in the training of the MF model (block 378).

[0227] In one embodiment, the method 370 further includes the server NWDAF 204 sending a registration message including a configuration file of the server NWDAQ 204 to a registration center (block 372).

[0228] In one embodiment, the configuration file of the server NWDAF 204 includes information indicating one or both of the following: the ability of the server NWDAF 204 to support participation in the training of the ML model, and one or more ML training participation modes supported by the server NWDAF 204.

[0229] In one embodiment, one or more parameters received in the participation announcement message include one or more of the following: an analysis ID, an ML-related ID or a federated learning (FL)-related ID, and one or more ML model training participation modes supported by the consumer NF 206.

[0230] In one embodiment, the decision that the consumer NF 206 can participate in the training of the ML model is further based on the training logic at the server NWDAF 204 and / or one or more policies of the server NWDAF 204.

[0231] In any embodiment of the present disclosure, the consumer NF 206 includes an analysis logic function (AnLF), and the server NWDAF 204 includes a model training logic function (MTLF).

[0232] In any embodiment of the present disclosure, the registry includes a network repository function (NRF).

[0233] Figure 17A is a functional block diagram showing some components of the network node 400 serving as the server NWDAF 204. As described above, according to one embodiment of the present disclosure, the network node 400 has an MTLF 410 and a set of policies 422, and is configured to select and approve one or more consumer NFs having an AnLF (e.g., Figure 18A the consumer NF 500 shown in Figure 17A to participate in an upcoming and / or ongoing ML model training process. As

[0234] More specifically, the processing circuitry 402 may include one or more microprocessors, hardware, firmware, or a combination thereof. In operation, the processing circuitry 402 controls the overall operation of the network node 400 and processes data and information in accordance with embodiments of the present disclosure. Such processing includes, but is not limited to: registering a profile of the network node 400 (e.g., the server NWDAF 204) with a registration center, where the profile indicates the ability of the network node 400 to support a consumer NF's participation in the training of an ML model, receiving a subscription request from the consumer NF, where the subscription request includes information related to one or both of the availability of data for training the ML model at the consumer NF and the ability of the consumer NF to evaluate the ML model, determining, based on the information received in the subscription request, whether to allow the consumer NF to participate in the training of the ML model, and sending a subscription response message to the consumer NF, the subscription response message indicating whether the consumer NF is allowed to participate in the training of the ML model.

[0235] In addition, in some embodiments, the processing further includes: sending a participation request message to the consumer NF by the network node 400, where the participation request message includes one or more parameters associated with the consumer NF's participation in the training of the ML model, receiving a participation response message from the consumer NF, the participation response message indicating that the consumer NF is capable of participating in the training of the ML model, where the participation response message includes at least one of the one or more parameters sent to the consumer NF in the participation request message, determining, based on the at least one parameter received in the participation response message, that the consumer NF is allowed to participate in the training of the ML model, and sending a participation confirmation message to the consumer NF, the participation confirmation message indicating that the consumer NF is allowed to participate in the training of the MF model.

[0236] In yet another embodiment, the processing further includes: receiving a participation announcement message including one or more parameters from the consumer NF by the network node 400, where the one or more parameters indicate that the consumer NF may participate in the training of the ML model, determining, based on the one or more parameters received in the participation announcement message, that the consumer NF may participate in the training of the ML model, and sending a participation confirmation message to the consumer NF, the participation confirmation message indicating that the consumer NF is allowed to participate in the training of the MF model.

[0237] The memory circuit 404 includes volatile and non-volatile memories for storing computer program code and data required for operation by the processing circuit 402. The memory circuit 404 may include any tangible non-transitory computer-readable storage medium for storing data, including electronic, magnetic, optical, electromagnetic, or semiconductor data storage. As described above, the memory circuit 404 stores a computer program 408 including executable instructions that configure the processing circuit 402 to implement the methods described herein. In this regard, the computer program 408 may include one or more code modules corresponding to the functions described above.

[0238] Typically, computer program instructions (such as the computer program 408) and configuration information are stored in non-volatile memories such as ROM, erasable programmable read-only memory (EPROM), or flash memory. Temporary data generated during operation may be stored in volatile memory such as random access memory (RAM). In some embodiments, the computer program 408 for configuring the processing circuit 402 as described herein may be stored on removable memory such as a portable optical disc, a portable digital video disc, or other removable media. The computer program 408 may also be embodied in a carrier such as an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

[0239] As is known in the art, the communication circuit 406 communicatively connects the network node 400 to one or more consumer NFs via one or more communication networks. In some embodiments, for example, the communication circuit 406 communicatively connects the network node 400 to one or more consumer NFs and / or other nodes and functions (such as core network nodes and functions) via a wired interface. Thus, the communication circuit 406 may include, for example, an Ethernet card or other circuitry configured to communicate via the (one or more) communication networks.

[0240] Any suitable steps, methods, features, functions or benefits disclosed herein may be performed by one or more functional units or modules of one or more virtual devices. Each virtual device may include a plurality of such functional units. These functional units may be implemented via a processing circuit (such as processing circuit 402). Such a processing circuit may include one or more microprocessors or microcontrollers, as well as other digital hardware, which may include a digital signal processor (DSP), dedicated digital logic, etc. The processing circuit may be configured to execute program code (e.g., computer program 408) stored in a memory 404, which may include one or several types of memories, such as read-only memory (ROM), random access memory (RAM), cache memory, flash memory devices, optical storage devices, etc. The program code stored in the memory includes program instructions for executing one or more telecommunication and / or data communication protocols and instructions for executing one or more of the techniques described herein. In some implementations, the processing circuit 402 may be used to cause the corresponding functional unit / module to perform the corresponding function according to one or more embodiments of the present disclosure.

[0241] Figure 17B is a functional block diagram of a computer program product (e.g., computer program 408) that, when executed by the processing circuit 402 of the network node 400, causes the network node 400 to perform the methods described herein. Specifically, as Figure 17B shown, the computer program 408 executed by the processing circuit 402 includes a registration unit / module 420, a consumer NF discovery unit / module 422, a subscription unit / module 424, a training participation unit / module 426, and a training participation determination unit / module 428.

[0242] The registration unit / module 420 includes computer program code that, when executed by the processing circuit 402, configures the network node 400 to register its profile with a registration center (such as the NRF), as previously described.

[0243] The consumer NF discovery unit / module 422 includes computer program code that, when executed by the processing circuit 402, configures the network node 400 to discover one or more consumer NFs to participate in the ML model training process, as previously described.

[0244] The subscription unit / module 424 includes computer program code that, when executed by the processing circuit 402, configures the network node 400 to subscribe / unsubscribe one or more consumer NFs to receive information about the ML model training process and modify the existing subscriptions of one or more consumer NFs, as previously described.

[0245] The training participation unit / component 426 includes computer program code which, when executed by the processing circuitry 402, configures the network node 400 to indicate to one or more consumer NFs whether they are allowed to participate in the ML model training process, as previously described.

[0246] The training participation determination unit / component 428 includes computer program code which, when executed by the processing circuitry 402, configures the network node 400 to determine whether one or more consumer NFs are allowed to participate in the ML model training process, as previously described.

[0247] Figure 18A FIG. is a functional block diagram showing some components of the network node 500 acting as a consumer NF 206. As described above, according to an embodiment of the present disclosure, the network node 500 has an AnLF and is configured to participate in upcoming and / or ongoing ML model training processes. As Figure 18A shown, the network node 500 includes a processing circuitry 502, a memory circuitry 504, and a communication circuitry 506. Further, as described in more detail below, the memory circuitry 504 stores a computer program 508 which, when executed by the processing circuitry 502, configures the network node 500 to implement the methods described herein.

[0248] More specifically, the processing circuitry 502 may include one or more microprocessors, hardware, firmware, or a combination thereof. In operation, the processing circuitry 502 controls the overall operation of the network node 500 and processes data and information according to embodiments of the present disclosure. Such processing includes, but is not limited to: sending a discovery request to a registration center to discover a server network data analytics function (NWDAF), wherein the discovery request indicates the ability of the consumer NF to support participating in training the ML model, subscribing to the server NWDAF, and receiving a subscription response message from the server NWDAF, wherein the subscription response message indicates whether the consumer NF is allowed to participate in the training of the ML model.

[0249] In addition, in some embodiments, the processing further includes: the network node 500 receiving a participation request message from a server network data analytics function (NWDAF), wherein the participation request message includes one or more parameters associated with the consumer NF participating in training the ML model, deciding to participate in training the ML model based on the one or more parameters received in the participation request message, and sending a participation response message to the server NWDAF, the participation response message indicating that the consumer NF can participate in the training of the ML model.

[0250] In yet another embodiment, the processing further includes: the network node 500 sending a participation announcement message to a server network data analytics function (NWDAF), the participation announcement message indicating that a consumer NF can participate in the training of the ML model, and receiving, from the server NWDAF, a participation confirmation message indicating that the consumer NF is allowed to participate in the training of the MF model.

[0251] The memory circuit 504 includes volatile and non-volatile memories for storing computer program code and data required for operation by the processing circuit 500. The memory circuit 504 may include any tangible non-transitory computer-readable storage medium for storing data, including electronic, magnetic, optical, electromagnetic, or semiconductor data storage. As described above, the memory circuit 504 stores a computer program 508 including executable instructions that configure the processing circuit 502 to implement the methods described herein. In this regard, the computer program 508 may include one or more code modules corresponding to the functions described above.

[0252] Generally, computer program instructions (such as the computer program 508) and configuration information are stored in non-volatile memories such as ROM, erasable programmable read-only memory (EPROM), or flash memory. Temporary data generated during operation may be stored in volatile memory such as random access memory (RAM). In some embodiments, the computer program 48 for configuring the processing circuit 502 as described herein may be stored in removable memory such as a portable compact disc, a portable digital video disc, or other removable media. The computer program 508 may also be embodied in a carrier such as an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

[0253] As is known in the art, the communication circuit 506 communicatively connects the network node 500 to one or more server NWDAFs, such as the network node 400, via one or more communication networks. In some embodiments, for example, the communication circuit 506 communicatively connects the network node 500 to one or more server NWDAFs and / or other nodes and functions (such as other consumer NFs, core network nodes and functions) via a wired interface. Thus, the communication circuit 506 may include, for example, an Ethernet card or other circuitry configured to communicate via one or more communication networks.

[0254] As described above, any suitable steps, methods, features, functions, or benefits disclosed herein may be performed by one or more functional units or modules of one or more virtual devices. Each virtual device may include a plurality of such functional units. These functional units may be implemented via a processing circuit (such as processing circuit 502). Such a processing circuit may include one or more microprocessors or microcontrollers, as well as other digital hardware, which may include a digital signal processor (DSP), dedicated digital logic, etc. Processing circuit 502 may be configured to execute program code (e.g., computer program 508) stored in a memory 504, which may include one or several types of memories, such as read-only memory (ROM), random access memory (RAM), cache memory, flash memory devices, optical storage devices, etc. The program code stored in the memory includes program instructions for executing one or more telecommunication and / or data communication protocols and instructions for executing one or more techniques described herein. In some embodiments, processing circuit 502 may be used to cause the corresponding functional unit / module to perform the corresponding function according to one or more embodiments of the present disclosure.

[0255] For this purpose, Figure 18B is a functional block diagram of a computer program product (e.g., computer program 508) that, when executed by the processing circuit 502 of network node 500, causes network node 500 to perform the methods described herein. In particular, as Figure 18B shown, the computer program 508 executed by the processing circuit 502 includes a registration unit / module 510, an NWDAF discovery unit / module 512, a subscription unit / module 514, a training participation unit / module 516, and a training participation determination unit / module 518.

[0256] The registration unit / module 510 includes computer program code that, when executed by the processing circuit 502, configures network node 500 to register its profile with a registration center (such as NRF), as previously described.

[0257] The NWDAF discovery unit / module 512 includes computer program code that, when executed by the processing circuit 502, configures network node 500 to discover one or more server NWDAFs for participating in the ML model training process, as previously described.

[0258] The subscription unit / module 514 includes computer program code that, when executed by the processing circuit 502, configures network node 500 to subscribe / unsubscribe from receiving information about the ML model training process to / from one or more server NWDAFs and modify its existing subscriptions, as previously described.

[0259] The training participation unit / component 516 includes computer program code which, when executed by the processing circuitry 502, configures the network node 500 to indicate to the server NWDAF whether it is capable of participating in the ML model training process, as previously described.

[0260] The training participation determination unit / component 518 includes computer program code which, when executed by the processing circuitry 502, configures the network node 500 to determine whether it is capable of participating in the ML model training process, as previously described.

[0261] Embodiments of the present disclosure also include a carrier containing a computer program, such as computer program 408 and / or computer program 508. The carrier may include one of an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

[0262] Embodiments herein also include a computer program product stored on a non-transitory computer-readable (storage or recording) medium (e.g., memory 404 and / or memory 504) and including instructions which, when executed by a processing circuitry (e.g., processing circuitry 402 and / or processing circuitry 502) of a device (e.g., network node 400 and / or network node 500), cause the device to perform as described above.

[0263] Embodiments also include a computer program product including a program code portion for performing the steps of any one of the embodiments herein when the computer program product is executed by a computing device (such as network node 400 and / or network node 500). The computer program product may be stored on a computer-readable recording medium (e.g., memory 404 and / or memory 504).

[0264] Of course, without departing from the basic characteristics of the present invention, the present embodiments may be implemented in a manner different from that specifically set forth herein. The present embodiments should be considered illustrative rather than restrictive in all respects, and all changes within the meaning and scope of equivalence of the appended claims are intended to be included therein.

Claims

1. A method (300) for determining whether a consumer network function NF is approved to participate in training a machine learning ML model, the method being implemented by a network node (500) serving as the consumer NF, and including: sending (302) a discovery request to a registration center to discover a server network data analytics function NWDAF (400), wherein the discovery request indicates the ability of the consumer NF to support participating in training the ML model; subscribing (304) to the server NWDAF; and receiving (306) a subscription response message from the server NWDAF, wherein the subscription response message indicates whether the consumer NF is allowed to participate in the training of the ML model.

2. The method according to claim 1, wherein, sending the discovery request to the registration center to discover the server NWDAF includes: the consumer NF sending an Nnrf_NFDiscovery_Request service message to the registration center.

3. The method according to claims 1 to 2, wherein, the discovery request further identifies one or more ML model training participation modes supported by the consumer NF.

4. The method according to claims 1 to 3, wherein, the discovery request further indicates one or more of the following: the availability of data used by the consumer NF in the training of the ML model; the ability of the consumer NF to evaluate the training of the ML model; and one or more ML model training participation modes supported by the consumer NF.

5. The method according to claims 1 to 4, wherein, the data used by the consumer in the training of the ML model includes one or more of the following: actual data used by the ML model; test data for testing the ML model; and verification data for validating the ML model.

6. The method according to claim 5, wherein, the indication of the ability of the consumer NF to evaluate the training of the ML model specifies whether the consumer NF is capable of testing or validating the accuracy of the ML model.

7. The method according to claim 6, wherein, the ML model is one of the following: an initially trained ML model; an intermediate trained ML model; and a final ML model.

8. The method according to any one of claims 4 to 7, wherein, the one or more ML model training participation modes supported by the consumer NF include: a first participation mode, wherein the consumer NF participates in evaluating the state of the ML model; a second participation mode, wherein the consumer NF participates substantially continuously in the training of the ML model to evaluate the ML model; a third participation mode, wherein the consumer NF participates periodically in the training of the ML model to evaluate the ML model; a fourth participation mode, wherein the consumer NF is triggered by the server NWDAF to participate in the training of the ML model to evaluate the ML model; and Fifth participation mode, wherein the consumer NF provides the final evaluation of the ML model.

9. The method according to claim 8, wherein, in the first participation mode, the consumer NF evaluates the ML model and provides the ML model status to the server NWDAF.

10. The method according to any one of claims 1 to 9, wherein, the subscription response message received from the server NWDAF includes one or both of the following: an indication that the consumer NF is approved to participate in the training of the ML model; and a selected ML model training participation mode for use by the consumer NF in training the ML model, wherein the selected ML model training participation mode is selected by the server NWDAF from the one or more ML model training participation modes included in the discovery request.

11. A method (310) for determining whether a consumer network function NF is approved to participate in training a machine learning ML model, the method being implemented by a network node (400) acting as a server network data analytics function NWDAF, and comprising: registering (block 312) a configuration file of the server NWDAF with a registration center, wherein the configuration file indicates the ability of the server NWDAF to support the participation of the consumer NF in the training of the ML model; receiving (block 314) a subscription request from the consumer NF, wherein the subscription request includes information related to one or both of the availability of data for training the ML model at the consumer NF and the ability of the consumer NF to evaluate the ML model; determining (316) whether to allow the consumer NF to participate in the training of the ML model based on the information received in the subscription request; and sending (318) a subscription response message to the consumer NF, the subscription response message indicating whether the consumer NF is allowed to participate in the training of the ML model.

12. The method according to claim 11, wherein, the configuration file of the server NWDAF further indicates one or more ML model training participation modes supported by the server NWDAF for use in the training of the ML model.

13. The method according to claims 11 to 12, wherein, the subscription request further indicates one or more of the following: the availability of the data used by the consumer NF in the training of the ML model; the ability of the consumer NF to evaluate the training of the ML model; and one or more ML model training participation modes supported by the consumer NF.

14. The method according to claims 11 to 13, wherein, determining whether to allow the consumer NF to participate in the training of the ML model is further based on the training logic accessible to the server NWDAF and / or one or more policies of the server NWDAF.

15. The method according to any one of claims 11 to 14, wherein, The subscription response message sent to the consumer NF includes one or both of the following: An indication that the consumer NF is approved to participate in the training of the ML model; and A selected ML model training participation mode for the consumer NF to use in training the ML model, where the selected ML model training participation mode is selected by the server NWDAF from the one or more ML model training participation modes included in the discovery request.

16. A network node (500) configured to act as a consumer network function NF, the network node comprising: Processing circuitry (502); and A memory (504) containing instructions (508) executable by the processing circuitry, whereby the network node is configured to: Send (302) a discovery request to a registration center to discover a server network data analytics function NWDAF, where the discovery request indicates the ability of the consumer NF to support participation in training an ML model; Subscribe (304) to the server NWDAF; and Receive (306) a subscription response message from the server NWDAF, where the subscription response message indicates whether the consumer NF is permitted to participate in the training of the ML model.

17. The network node according to claim 16, further configured to perform the method according to any one of claims 2 to 10.

18. A computer program (508) comprising instructions which, when executed on the processing circuitry (502) of a network node (500), cause the processing circuitry to perform the method according to any one of claims 1 to 10.

19. A carrier containing the computer program according to claim 18, wherein The carrier is one of an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

20. A network node (400) configured to act as a server network data analytics function NWDAF, the network node comprising: Processing circuitry (402); and A memory (404) containing instructions (408) executable by the processing circuitry, whereby the network node is configured to: Register (312) a configuration file of the server NWDAF with a registration center, where the configuration file indicates the ability of the server NWDAF to support consumer NF participation in training an ML model; Receive (314) a subscription request from the consumer NF, where the subscription request includes information related to one or both of the availability of data for training the ML model at the consumer NF and the ability of the consumer NF to evaluate the ML model; Based on the information received in the subscription request, determine (316) whether to allow the consumer NF to participate in the training of the ML model; and Send (318) a subscription response message to the consumer NF, the subscription response message indicating whether the consumer NF is permitted to participate in the training of the ML model.

21. The network node according to claim 20 is further configured to perform the method according to any one of claims 12 to 15.

22. A computer program (508) comprising instructions which, when executed on a processing circuit (502) of a network node (500), cause the processing circuit to perform the method according to any one of claims 11 to 15.

23. A carrier containing the computer program according to claim 22, wherein, the carrier is one of an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

24. A method (320) for selecting a consumer network function NF to evaluate a machine learning ML model before training the ML model, the method being implemented by a network node (500) acting as the consumer NF, and comprising: receiving (324) a participation request message from a server network data analytics function NWDAF, wherein the participation request message includes one or more parameters associated with the consumer NF participating in training the ML model; deciding (326) to participate in training the ML model based on the one or more parameters received in the participation request message; and sending (328) a participation response message to the server NWDAF, the participation response message indicating that the consumer NF is capable of participating in the training of the ML model.

25. The method according to claim 24, further comprising: registering (322) a configuration file of the consumer NF with a registration center.

26. The method according to claims 24 to 25, wherein, the configuration file of the consumer NF includes information indicating one or both of the following: the ability of the consumer NF to support participating in the training of the ML model; and one or more ML model training participation modes supported by the consumer NF.

27. The method according to any one of claims 24 to 26, wherein, the one or more parameters received together with the participation request message include one or more of the following: an analysis ID; an ML-related ID or a federated learning FL-related ID; and a selected ML model training participation mode for the consumer NF to participate in the training of the ML model.

28. The method according to any one of claims 24 to 27, wherein, the one or more ML model training participation modes supported by the consumer NF include: a first participation mode, wherein the consumer NF participates in evaluating the state of the ML model; a second participation mode, wherein the consumer NF participates in the training of the ML model substantially continuously to evaluate the ML model; a third participation mode, wherein the consumer NF participates in the training of the ML model periodically to evaluate the ML model; a fourth participation mode, wherein the consumer NF is triggered by the server NWDAF to participate in the training of the ML model to evaluate the ML model; and a fifth participation mode, wherein the consumer NF provides a final evaluation of the ML model.

29. The method according to any one of claims 24 to 28, wherein, the participation response message sent to the server NWDAF includes one or more of the following: the analysis ID; the ML-related ID or the FL-related ID; and the selected ML model training participation mode.

30. The method according to any one of claims 24 to 29, further comprising: receiving a participation confirmation message from the server NWDAF, the participation confirmation message indicating whether the consumer NF is allowed to participate in the training of the MF model.

31. The method according to claim 30, wherein, the participation confirmation message further indicates the selected participation mode.

32. The method according to any one of claims 24 to 31, further comprising: participating (330) in the training of the ML model according to the selected participation mode.

33. A method (340) for selecting a consumer network function NF to evaluate a machine learning ML model before training the ML model, the method being implemented by a network node (400) serving as a server network data analysis function NWDAF, and comprising: sending (346) a participation request message to the consumer NF, wherein the participation request message includes one or more parameters associated with the consumer NF participating in the training of the ML model; receiving (348) a participation response message from the consumer NF, the participation response message indicating that the consumer NF is capable of participating in the training of the ML model, wherein the participation response message includes at least one of the one or more parameters sent to the consumer NF in the participation request message; deciding (350) based on the at least one parameter received in the participation response message that the consumer NF is allowed to participate in the training of the ML model; and sending (352) a participation confirmation message to the consumer NF, the participation confirmation message indicating that the consumer NF is allowed to participate in the training of the MF model.

34. The method according to claim 33, further comprising: sending (342) a registration message including a configuration file of the server NWDAF to a registration center.

35. The method according to any one of claims 33 to 34, wherein, the configuration file of the server NWDAF includes information indicating one or both of the following: the ability of the server NWDAF to support participation in the training of the ML model; one or more ML model training participation modes supported by the server NWDAF.

36. The method according to claim 35, further comprising: sending (344) a discovery request to a registration center to discover the consumer NF, wherein the discovery request indicates one or both of the following: the ability of the consumer NF to support participation in the training of the ML model; and one or more ML model training participation modes supported by the consumer NF.

37. The method according to any one of claims 33 to 36, wherein, the one or more parameters included in the participation request message include one or more of the following: Analysis ID; ML-related ID or Federated Learning (FL)-related ID; and An ML model training participation mode that should be supported by the consumer NF to participate in the training of the ML model.

38. The method according to claims 33 to 37, wherein, The at least one parameter in the participation response message includes one or more of the following: The analysis ID; The ML-related ID or the FL-related ID; and The ML model training participation mode.

39. The method according to any one of claims 33 to 38, wherein, Determining that the consumer NF is allowed to participate in training the ML model is further based on training logic at the server NWDAF and / or one or more policies of the server NWDAF.

40. The method according to any one of claims 33 to 39, wherein, The participation confirmation message sent to the consumer NF also indicates the ML training participation mode that the consumer NF will use to participate in the training of the ML model.

41. A network node (500) configured to act as a consumer network function (NF), the network node comprising: Processing circuitry (502); and A memory (504) containing instructions (508) executable by the processing circuitry, whereby the network node is configured to: Receive (324) a participation request message from a server Network Data Analytics Function (NWDAF), wherein the participation request message includes one or more parameters associated with the consumer NF participating in training an ML model; Based on the one or more parameters received in the participation request message, determine (326) to participate in training the ML model; and Send (328) a participation response message to the server NWDAF, the participation response message indicating that the consumer NF is able to participate in the training of the ML model.

42. The network node according to claim 41, further configured to perform the method according to any one of claims 25 to 32.

43. A computer program (508) comprising instructions that, when executed on the processing circuitry (502) of a network node (500), cause the processing circuitry to perform the method according to any one of claims 24 to 32.

44. A carrier containing the computer program according to claim 43, wherein, The carrier is one of an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

45. A network node configured to act as a server Network Data Analytics Function (NWDAF), the network node comprising: Processing circuitry; and A memory containing instructions executable by the processing circuitry, whereby the network node is configured to: Send a participation request message to a consumer NF, wherein the participation request message includes one or more parameters associated with the consumer NF participating in training an ML model; Receiving a participation response message from the consumer NF, the participation response message indicating that the consumer NF is capable of participating in the training of the ML model, wherein the participation response message includes at least one of the one or more parameters sent to the consumer NF in the participation request message; Determining that the consumer NF is allowed to participate in training the ML model based on the at least one parameter received in the participation response message; and Sending a participation confirmation message to the consumer NF, the participation confirmation message indicating that the consumer NF is allowed to participate in the training of the MF model.

46. The network node according to claim 45, further configured to perform the method according to any one of claims 34 to 40.

47. A computer program (508) comprising instructions which, when executed on a processing circuit (502) of a network node (500), cause the processing circuit to perform the method according to any one of claims 33 to 40.

48. A carrier containing the computer program according to claim 47, wherein, the carrier is one of an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

49. A method (360) for selecting a consumer network function NF to participate in the training of a machine learning ML model, the method being implemented by a network node (500) acting as the consumer NF, and comprising: Sending (364) a participation announcement message to a server network data analytics function NWDAF, the participation announcement message indicating that the consumer NF is capable of participating in the training of the ML model; and Receiving (366) a participation confirmation message from the server NWDAF, the participation confirmation message indicating that the consumer NF is allowed to participate in the training of the MF model.

50. The method according to claim 49, further comprising: Sending (362) a registration message including a configuration file of the consumer NF to a registration center.

51. The method according to any one of claims 49 to 50, wherein, the configuration file of the consumer NF includes information indicating one or both of the following: The ability of the consumer NF to support participation in training the ML model; and One or more ML training participation modes supported by the consumer NF.

52. The method according to any one of claims 49 to 51, wherein, the participation announcement message includes one or more of the following: Analysis ID; ML-related ID or federated learning FL-related ID; and The one or more ML model training participation modes supported by the consumer NF.

53. The method according to any one of claims 49 to 52, wherein, the participation confirmation message received from the server NWDAF indicates one or both of the following: Whether the consumer NF is allowed to participate in the training of the MF model; and The selected ML model training participation mode used by the consumer NF to participate in the training of the ML model, where the selected ML model training participation mode is selected from the one or more ML model training participation modes supported by the consumer NF.

54. The method according to claims 49 to 53, wherein, the one or more ML model training participation modes supported by the consumer NF include: A first participation mode, wherein the consumer NF participates in evaluating the state of the ML model; A second participation mode, wherein the consumer NF participates in the training of the ML model substantially continuously to evaluate the ML model; A third participation mode, wherein the consumer NF participates in the training of the ML model periodically to evaluate the ML model; A fourth participation mode, wherein the consumer NF is triggered by the server NWDAF to participate in the training of the ML model to evaluate the ML model; and A fifth participation mode, wherein the consumer NF provides the final evaluation of the ML model.

55. A method (370) for selecting a consumer network function NF to participate in the training of a machine learning ML model, the method being implemented by a network node (400) acting as a server network data analytics function NWDAF, and comprising: Receiving (374) from the consumer NF a participation announcement message including one or more parameters, wherein the one or more parameters indicate that the consumer NF is capable of participating in the training of the ML model; Based on the one or more parameters received in the participation announcement message, determining (376) that the consumer NF is capable of participating in the training of the ML model; and Sending (378) to the consumer NF a participation confirmation message, the participation confirmation message indicating that the consumer NF is allowed to participate in the training of the MF model.

56. The method according to claim 55, further comprising: Sending (362) to the registration center a registration message including a configuration file of the server NWDAF.

57. The method according to any one of claims 55 to 56, wherein, the configuration file of the server NWDAF includes information indicating one or both of the following: The ability of the server NWDAF to support participation in the training of the ML model; and One or more ML training participation modes supported by the server NWDAF.

58. The method according to any one of claims 55 to 57, wherein, the one or more parameters received in the participation announcement message include one or more of the following: Analysis ID; ML-related ID or federated learning FL-related ID; and One or more ML model training participation modes supported by the consumer NF.

59. The method according to any one of claims 55 to 58, wherein, determining that the consumer NF is capable of participating in the training of the ML model is further based on the training logic at the server NWDAF and / or one or more policies of the server NWDAF.

60. A network node (500) configured to act as a consumer network function NF, the network node comprising: processing circuitry (502); and a memory (504) containing instructions executable by the processing circuitry, whereby the network node is configured to: send (364) a participation announcement message to a server network data analytics function NWDAF, the participation announcement message indicating that the consumer NF is capable of participating in the training of an ML model; and receive (366) a participation confirmation message from the server NWDAF, the participation confirmation message indicating that the consumer NF is permitted to participate in the training of the MF model.

61. The network node according to claim 60, further configured to perform the method according to any one of claims 46 to 48.

62. A computer program (508) comprising instructions which, when executed on the processing circuitry of a network node, cause the processing circuitry to perform the method according to any one of claims 45 to 48.

63. A carrier containing the computer program according to claim 62, wherein the carrier is one of an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

64. A network node (400) configured to act as a server network data analytics function NWDAF, the network node comprising: processing circuitry (402); and a memory (404) containing instructions executable by the processing circuitry, whereby the network node is configured to: receive (374) from a consumer NF a participation announcement message including one or more parameters, wherein the one or more parameters indicate that the consumer NF is capable of participating in the training of an ML model; decide (376) based on the one or more parameters received in the participation announcement message that the consumer NF is capable of participating in the training of the ML model; and send (378) to the consumer NF a participation confirmation message, the participation confirmation message indicating that the consumer NF is permitted to participate in the training of the MF model.

65. The network node according to claim 64, further configured to perform the method according to any one of claims 56 to 59.

66. A computer program (408) comprising instructions which, when executed on the processing circuitry of a network node, cause the processing circuitry to perform the method according to any one of claims 55 to 59.

67. A carrier containing the computer program according to claim 66, wherein the carrier is one of an electronic signal, an optical signal, a radio signal, or a computer-readable storage medium.

68. The method according to any one of the preceding claims, wherein the consumer NF includes an analysis logic function AnLF, and wherein the server NWDAF includes a model training logic function MTLF.

69. The method according to any one of the preceding claims, wherein the registration center includes a network repository function NRF.

Images