Enhanced wireless device (WD) delay performance analysis for assisting application layer artificial intelligence / machine learning (AI / ML) operations

Abstract

Methods, network data analysis functions (NWDAF), and network functions (NF) are disclosed for enhancing wireless device (WD) delay performance analysis to assist application layer artificial intelligence / machine learning (AI / ML) operations. According to one aspect, a method in an NWDAF includes collecting a WD delay performance input from a network function NF, the delay performance input including a plurality of transmission data volume values and a transmission time value. The method includes determining a set of delay performance analyses to assist the joint learning process, the set of delay performance analyses being based at least in part on the delay performance input and including at least one of delay predictions and delay statistics related to uplink and downlink transmissions between the NWDAF and the NF. The method further includes sending the set of delay performance analyses to the NF.

Description

Technical Field

[0001] The present disclosure relates to wireless communications, and in particular to enhanced wireless device (WD) latency performance analysis for assisting application-layer artificial intelligence (AI) / machine learning (ML) operations. Background Art

[0002] The Third Generation Partnership Project (3GPP) has developed and is developing standards for fourth-generation (4G) (also known as Long Term Evolution (LTE)) and fifth-generation (5G) (also known as New Radio (NR)) wireless communication systems. Such systems provide, among other features, broadband communication between network nodes (such as base stations) and mobile wireless devices (WDs) (also known as user equipment (UE)), as well as communication between network nodes and between WDs. 3GPP is also developing standards for sixth-generation (6G) wireless communication systems.

[0003] Conclusions on Delay Performance Analysis in 3GPP Technical Release (TR) 23.700-80

[0004] In 3GPP Technical Release (TR) 23.700-80 (Conclusions for 5GS Assisted KI#7 for Federated Learning Operations), the following description is provided:

[0005] The Application Function (AF) can request the Network Data Analysis Function (NWDAF) (via the Network Exposure Function (NEF) if necessary) to provide existing, enhanced, or new analytics to assist in federated learning operations. The NWDAF analytics service will be enhanced as follows:

[0006] - Introduced new latency profiling for NWDAF.

[0007] Note 1: The specific input and output parameters for latency performance analysis between WD groups to assist in applying AI / ML FL operations will be determined during the specification phase.

[0008] Analysis ID: WD Delay Performance in CR#0616 (to be added to 3GPP TS23.288)

[0009] Overall

[0010] Section 2.1.2.1 of 3GPP Technical Standard (TS) 23.288 describes how the NWDAF can provide WD latency performance analytics to service consumers in the form of statistics, predictions, or both. The NWDAF collects input data related to WD latency performance from 5G core network functions (NFs), operations management and maintenance (OAM), and application functions (AFs). Consumers can subscribe to analytics notifications (i.e., a subscription-notification model) or request individual notifications (i.e., a request-response model).

[0011] WD latency performance refers to the time delay required to complete the transfer of a specific amount of data from a WD to an AF, or vice versa. Given an expected number of repeated data transfers or an expected time interval between data transfers, WD latency performance can be provided as the average transmission latency performance of each data packet within the target analysis period. WD latency performance analysis can be used to assist AFs hosting AI / ML-based services, such as member selection for federated learning.

[0012] The WD delay performance analysis may be provided for an individual WD or for a list of WDs as defined in 3GPP 23.288 clause 2.1.2.3.

[0013] The service consumer may be a NF (eg, AF or Network Exposure Function (NEF)).

[0014] Consumers of these analyses can indicate in their requests:

[0015] -Analysis ID = "WD Latency Performance";

[0016] - Target of analysis report: Single WD (Subscription Permanent Identifier (SUPI) / General Public Subscription Identifier (GPSI))

[0017] or a set of WDs (a list of SUPI / GPSI);

[0018] -Analysis filter information, optionally including:

[0019] -Data Network Name (DNN);

[0020] -S- Network Slice Selection Assistance Information (NSSAI);

[0021] - Application ID;

[0022] - Area of ​​Interest (AOI(s): limits the scope of WD delay performance analysis to the provided area;

[0023] - an optional list of requested analysis subsets (see 3GPP TS 23.288 clause 2.1.2.3);

[0024] - Data volume uplink / downlink (UL / DL): indicates the specific amount of data transmitted once from WD to AF or from AF to WD;

[0025] - Quality of Service (QoS) requirements (e.g., 5G QoS Identifier (5QI), QoS characteristics);

[0026] -optionally, analyzing the expected number of repeated data transmissions within the target period;

[0027] - Optionally, the expected time interval between data transmissions;

[0028] - Optionally, a request for the geographical distribution of WDs, i.e. Area of ​​Interest (AoI);

[0029] - an analysis target period indicating the time period over which statistics or forecasts are requested;

[0030] - In the subscription, the notification correlation Id and notification target address are included;

[0031] -optionally, the preferred level of accuracy of the analysis;

[0032] -Optionally, order of preference for the results of the WD latency performance list:

[0033] - Sort criteria: "WD latency performance";

[0034] - Order: ascending or descending;

[0035] - Optionally, reporting thresholds indicating conditions on the level to be reached for the corresponding analysis subset in order to be notified by the NWDAF (see 3GPP TS 23.288 clause 2.1.2.3); for example, the NWDAF may provide the percentage of WDs that have reached a specific reporting threshold(s);

[0036] - Optionally, the maximum number of WDs.

[0037] Input Data

[0038] The NWDAF that supports analysis of WD delay performance should be able to

[0039] And 5GC NF collects WD performance information.

[0040] More detailed information collected by NWDAF from OAM is defined in Table 2.1.2.2-1, and more detailed information collected from related 5GC NFs (i.e., User Plane Function (UPF), Session Management Function (SMF), Access and Mobility Management Function (AMF)) is defined in Table 2.1.2.2-2.

[0041] Table 2.1.2.2-1: Input data from OAM related to WD delay performance

[0042]

[0043]

[0044] The NWDAF subscribes to the network data in Table 2.1.2.2-1 from the OAM by using the services provided by the OAM described in clause 6.2.3 of 3GPP TS 23.288.

[0045] NOTE 1: Whether the WD(s) support slicing can be checked by retrieving the registered Access and Mobility Management Function (AMF) details from the Unified Data Management (UDM) or by asking the AMF what slices the WD(s) use at the time of current registration. (Alternatively, if the Network Slice Admission Control Function (NSACF) is deployed, the NSACF can provide a report on what slices the WD(s) use).

[0046] NOTE 2: User consent checks from the UDM may be applied to these analyses.

[0047] Table 2.1.2.2-2: Service data from 5GC NF for WD latency performance analysis

[0048]

[0049] Output Analysis

[0050] The NWDAF that supports WD delay performance analysis provides analysis results to the consumer NF (such as AF or NEF). The analysis results provided by the NWDAF can be the WD delay performance statistics defined in Table 2.1.2.3-1 or the predictions defined in Table 2.2.2.3-2.

[0051] Table 2.1.2.3-1: WD delay performance statistics

[0052]

[0053]

[0054]

[0055] Table 2.1.2.3-2: WD delay performance prediction

[0056]

[0057]

[0058]

[0059] An example procedure for WD latency performance analysis is given in Figure 1 Shown in.

[0060] process

[0061] NWDAF can provide WD latency performance analysis to 5GC NFs (such as AF or NEF):

[0062] 1. The consumer NF (e.g., AF or NEF) requests or subscribes to NWDAF for analysis of WD delay performance (possibly via NEF if the consumer NF is AF) and provides the input information specified in 2.1.2.1 to the 5GC;

[0063] 2a-b. NWDAF uses the Namf_EventExposure_Subscribe service to subscribe to the service data in Table 2.1.2.2-2 to AMF for collecting (one or more) WD locations for a WD or a group of WDs;

[0064] NOTE: If NWDAF needs to have WD location information with finer granularity than TA / cell, NWDAF collects location data from GMLC instead of AMF;

[0065] 2c. NWDAF subscribes to the service data in Table 2.1.2.2-2 from SMF by calling Nsmf_EventExposure_Subscribe(event ID, (one or more) SUPIs or application IDs).

[0066] To provide the requested analysis, the NWDAF subscribes to the WD's information and can subscribe to the SMF for N4 session-related input data defined in Table 2.1.2.2-2;

[0067] 2d-e.N4 related input data is provided by UPF to SMF;

[0068] 2f.SMF provides the requested input data to NWDAF;

[0069] 2g-h.NWDAF may subscribe to the input data in Table 2.1.2.2-1 to OAM according to the data collection principles from OAM described in clause 6.2.3 of 3GPP TS 23.288;

[0070] 3. NWDAF exports the requested analysis in the form of latency performance statistics or predictions, or both;

[0071] 4. NWDAF provides the requested latency performance analytics to the NF using Nnwdaf_AnalyticsInfo_RequestResponse or Nnwdaf_AnalyticsSubscription_Notify, depending on the service used in step 1; and

[0072] 5-7. If the NF subscribes to the WD latency performance analysis in step 1, when the NWDAF generates a new analysis, it notifies the consumer of the newly generated analysis.

[0073] However, in the current version of the new analysis ID (WD latency performance), specific features for AI / ML business (for federated learning) are not considered.

[0074] A new analysis ID for WD latency performance has been proposed in the SA2#154-adhoc-e meeting to assist in application layer federated learning (FL) operations. As described in the approved CR#0616 (to be added to 3GPP TS23.288):

[0075] The latency performance of a WD group is the output information in the WD latency performance analysis used to evaluate the performance of a group of WDs to assist in federated learning. The motivation for introducing WD latency performance analysis is to provide the latency-related performance of a group of WDs to assist in AI / ML services, especially the member selection of federated learning in this version. The performance of federated learning is based on the service quality of a group of WDs. Therefore, the output based on the performance of a group of WDs is necessary for NEF and also for AF (in this version) to select WD members and evaluate WD performance during model training. Statistics and predictions of the performance of a group of WDs should be completed by NWDAF and provided to consumers.

[0076] The characteristics of AI / ML services (e.g., for federated learning) are significantly different from those of other general services, such as retransmission of the same AI / ML data or transmission of different AI / ML data in multiple time intervals with and without ultra-low latency requirements, different data volumes of various AI / ML data to be transmitted, and different transmission conditions for AI / ML data transmission between different network entities. However, the new analysis ID does not consider the specific characteristics of AI / ML services to distinguish them from other general services, which may result in the analysis results being unable to effectively assist application-layer AI / ML operations (e.g., FL). Summary of the Invention

[0077] Some embodiments advantageously provide methods and network nodes for enhancing wireless device (WD) delay performance analysis to assist application layer artificial intelligence (AI) / machine learning (ML) operations.

[0078] In some embodiments, specific characteristics of AI / ML traffic for federated learning are considered using latency-related analysis of AI / ML traffic. New inputs and outputs are added to the analysis ID WD latency performance.

[0079] In some embodiments, features of AI / ML services for federated learning are considered. These may include one or more of the following:

[0080] ■ Existing inputs are enhanced, such as the amount of data transferred and the time it takes to transfer;

[0081] ■ New inputs are added to analyze ID WD latency performance, including round trip-related time, (list of) paired timestamps (start and end times), IP filter information, application location, application server instance address; and / or

[0082] ■New outputs have been added, including:

[0083] - estimated number of transmissions (statistics / forecasts);

[0084] - Statistics / predictions about UL delay for target transmission;

[0085] - Statistics / predictions about the DL delay for target transmissions;

[0086] - statistics / predictions regarding round trip delay for target transmissions; and / or

[0087] - Statistics / predictions about the maximum delay for target transmission.

[0088] Some embodiments take into account the specific characteristics of AI / ML services, such as federated learning (FL), and focus on latency-related analysis for AI / ML services. Existing inputs are enhanced, and new inputs and outputs are added to analyze IDWD latency performance to enable analysis to assist application-layer AI / ML operations (e.g., federated learning).

[0089] Analysis IDs for WD latency performance were considered in the SA2#154-adhoc-e meeting to assist with application-layer federated learning operations. However, the new analysis IDs do not consider the specific characteristics of AI / ML services (e.g., for FL) that distinguish them from other services. This may result in analysis results that cannot be used to effectively assist application-layer AI / ML operations (e.g., FL).

[0090] According to one aspect, a method in a core network node configured to include a network data analysis function (NWDAF) is provided. The method includes collecting wireless device (WD) delay performance input from a network function (NF), the delay performance input including a plurality of transmitted data amount values ​​and transmission time values. The method includes determining a set of delay performance analyses to assist in a federated learning process, the set of delay performance analyses being based at least in part on the delay performance input and including at least one of delay predictions and delay statistics related to uplink and downlink transmission delays between the WD and an application function (AF). The method also includes sending the set of delay performance analyses to the NF.

[0091] According to this aspect, in some embodiments, at least one of the delay prediction and the delay statistics includes the time delay required to complete the transmission of a target amount of data from a WD to a NF or from a NF to a WD. In some embodiments, at least one of the delay prediction and the delay statistics includes an estimated number of retransmissions required to complete the transmission of the target amount of data. In some embodiments, at least one of the delay prediction and the delay statistics includes at least one of uplink packet delay, downlink packet delay, and round-trip packet delay. In some embodiments, at least one of the delay prediction and the delay statistics includes a value averaged over a target analysis period. In some embodiments, at least one of the delay prediction and the delay statistics includes a validity period for delay performance analysis. In some embodiments, at least one of the delay prediction and the delay statistics includes a maximum packet delay observed for communications with the NF. In some embodiments, at least one of the delay prediction and the delay statistics includes a spatial validity parameter indicating the region to which the delay performance analysis applies. In some embodiments, at least one of the delay prediction and the delay statistics is grouped into delay categories based on delay performance ranges. In some embodiments, at least one of the delay prediction and the delay statistics includes an indication of the percentage of WDs in each delay category. In some embodiments, the NF is implemented as an application function.

[0092] According to another aspect, a core network node is provided that is configured to include a network data analysis function (NWDAF). The core network node is configured to collect wireless device (WD) delay performance inputs from a network function (NF), the delay performance inputs comprising a plurality of transmitted data volume values ​​and transmission time values. The core network node is configured to determine a set of delay performance analyses to assist in a federated learning process, the set of delay performance analyses being based at least in part on the delay performance inputs and comprising at least one of delay predictions and delay statistics related to uplink and downlink transmission delays between the WD and an application function (AF). The core network node is further configured to send the set of delay performance analyses to the NF.

[0093] According to this aspect, in some embodiments, at least one of the delay prediction and the delay statistics includes the time delay required to complete the transmission of a target amount of data from a WD to a NF or from a NF to a WD. In some embodiments, at least one of the delay prediction and the delay statistics includes an estimated number of retransmissions required to complete the transmission of the target amount of data. In some embodiments, at least one of the delay prediction and the delay statistics includes at least one of uplink packet delay, downlink packet delay, and round-trip packet delay. In some embodiments, at least one of the delay prediction and the delay statistics includes a value averaged over a target analysis period. In some embodiments, at least one of the delay prediction and the delay statistics includes a validity period for delay performance analysis. In some embodiments, at least one of the delay prediction and the delay statistics includes a maximum packet delay observed for communications with the NF. In some embodiments, at least one of the delay prediction and the delay statistics includes a spatial validity parameter indicating the region to which the delay performance analysis applies. In some embodiments, at least one of the delay prediction and the delay statistics is grouped into delay categories based on delay performance ranges. In some embodiments, at least one of the delay prediction and the delay statistics includes an indication of the percentage of WDs in each delay category. In some embodiments, the NF is implemented as an application function.

[0094] According to yet another aspect, a method is provided in a network node configured to include a network function (NF) and to communicate with a network data analysis function (NWDAF). The method includes sending a delay performance input to the NWDAF, the delay performance input including a plurality of transmission data amount values ​​and transmission time values. The method also includes receiving a set of delay performance analyses to assist in a federated learning process, the set of delay performance analyses being based at least in part on the delay performance input and including at least one of delay predictions and delay statistics related to uplink and downlink transmission delays between the NWDAF and the NF. The method also includes hosting an artificial intelligence / machine learning (AI / ML)-based service based at least in part on the federated learning process.

[0095] According to this aspect, in some embodiments, at least one of the delay prediction and the delay statistics includes the time delay required to complete the transmission of a target amount of data from a WD to a NF or from a NF to a WD. In some embodiments, at least one of the delay prediction and the delay statistics includes an estimated number of retransmissions required to complete the transmission of the target amount of data. In some embodiments, at least one of the delay prediction and the delay statistics includes at least one of uplink packet delay, downlink packet delay, and round-trip packet delay. In some embodiments, at least one of the delay prediction and the delay statistics includes a value averaged over a target analysis period. In some embodiments, at least one of the delay prediction and the delay statistics includes a validity period for delay performance analysis. In some embodiments, at least one of the delay prediction and the delay statistics includes a maximum packet delay observed for communications with the NF. In some embodiments, at least one of the delay prediction and the delay statistics includes a spatial validity parameter indicating the region to which the delay performance analysis applies. In some embodiments, at least one of the delay prediction and the delay statistics is grouped into delay categories based on delay performance ranges. In some embodiments, at least one of the delay prediction and the delay statistics includes an indication of the percentage of WDs in each delay category. In some embodiments, the NF is implemented as an application function.

[0096] According to another aspect, a network node is provided that includes a network function (NF) and is configured to communicate with a network data analysis function (NWDAF). The network node is configured to send a delay performance input to the NWDAF, the delay performance input comprising a plurality of transmission data amount values ​​and transmission time values. The network node is further configured to receive a set of delay performance analyses to assist in a federated learning process, the set of delay performance analyses being based at least in part on the delay performance input and comprising at least one of delay predictions and delay statistics related to uplink and downlink transmission delays between the NWDAF and the NF. The network node is further configured to host an artificial intelligence / machine learning (AI / ML)-based service based at least in part on the federated learning process.

[0097] According to this aspect, in some embodiments, at least one of the delay prediction and the delay statistics includes the time delay required to complete the transmission of a target amount of data from a WD to a NF or from a NF to a WD. In some embodiments, at least one of the delay prediction and the delay statistics includes an estimated number of retransmissions required to complete the transmission of the target amount of data. In some embodiments, at least one of the delay prediction and the delay statistics includes at least one of uplink packet delay, downlink packet delay, and round-trip packet delay. In some embodiments, at least one of the delay prediction and the delay statistics includes a value averaged over a target analysis period. In some embodiments, at least one of the delay prediction and the delay statistics includes a validity period for delay performance analysis. In some embodiments, at least one of the delay prediction and the delay statistics includes a maximum packet delay observed for communications with the NF. In some embodiments, at least one of the delay prediction and the delay statistics includes a spatial validity parameter indicating the region to which the delay performance analysis applies. In some embodiments, at least one of the delay prediction and the delay statistics is grouped into delay categories based on delay performance ranges. In some embodiments, at least one of the delay prediction and the delay statistics includes an indication of the percentage of WDs in each delay category. In some embodiments, the NF is implemented as an application function. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] A more complete understanding of the present embodiments and its attendant advantages and features will be more readily appreciated by reference to the following detailed description when considered in conjunction with the accompanying drawings, in which:

[0099] Figure 1 It is the process of WD delay performance analysis;

[0100] Figure 2 is a schematic diagram illustrating an example network architecture of a communication system connected to a host via an intermediate network according to the principles of the present disclosure;

[0101] Figure 3 is a block diagram of a host communicating with a wireless device via a network node over an at least partially wireless connection according to some embodiments of the present disclosure;

[0102] Figure 4 A flow chart illustrating an example method for executing a client application at a wireless device implemented in a communication system including a host, a network node, and a wireless device according to some embodiments of the present disclosure;

[0103] Figure 5 is a flow chart illustrating an example method for receiving user data at a wireless device implemented in a communication system including a host, a network node, and a wireless device according to some embodiments of the present disclosure;

[0104] Figure 6is a flow chart illustrating an example method implemented in a communication system including a host, a network node, and a wireless device for receiving user data from a wireless device at a host according to some embodiments of the present disclosure;

[0105] Figure 7 is a flow chart illustrating an example method for receiving user data at a host implemented in a communication system including a host, a network node, and a wireless device according to some embodiments of the present disclosure;

[0106] Figure 8 is a flow chart of an example process in a network node for enhancing wireless device (WD) latency performance analysis to assist application layer artificial intelligence (AI) / machine learning (ML) operations;

[0107] Figure 9 is a flow diagram of an example process in a network node configured to include a Network Data Analysis Function (NWDAF) according to the principles disclosed herein; and

[0108] Figure 10 is a flow chart of an example process in a network node configured to include a network function NF and to communicate with a network data analysis function NWDAF according to the principles disclosed herein. DETAILED DESCRIPTION

[0109] Before describing the exemplary embodiments in detail, it should be noted that the embodiments reside primarily in a combination of apparatus components and processing steps associated with enhancing wireless device (WD) latency performance analysis to assist in application-layer artificial intelligence (AI) / machine learning (ML) operations. Accordingly, components have been represented by conventional symbols in the drawings where appropriate, with only those specific details relevant to understanding the embodiments being shown so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein. Like numbers represent like elements throughout the description.

[0110] As used herein, relational terms such as "first" and "second," "top" and "bottom," etc., may be used solely to distinguish one entity or element from another entity or element, and do not necessarily require or imply any physical or logical relationship or order between such entities or elements. The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the concepts described herein. As used herein, the singular forms "a," "an," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that the terms "comprises" and / or "comprising," when used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0111] In the embodiments described herein, connection terms such as "in communication with..." and the like may be used to indicate electrical or data communication, which may be achieved through, for example, physical contact, induction, electromagnetic radiation, radio signal transmission, infrared signal transmission, or optical signal transmission. Those skilled in the art will appreciate that various components may interoperate and that modifications and variations in achieving electrical and data communication are possible.

[0112] In some embodiments described herein, the terms "coupled," "connected," and the like may be used herein to indicate a connection, although not necessarily a direct one, and may include wired and / or wireless connections.

[0113] The term "network node" as used herein may be any type of network node included in a radio network, which may further include any of the following: base station (BS), radio base station, base transceiver station (BTS), base station controller (BSC), radio network controller (RNC), gNodeB (gNB), evolved NodeB (eNB or eNodeB), NodeB, multi-standard radio (MSR) radio node (such as MSR BS), multi-cell / multicast coordination entity (MCE), integrated access and backhaul (IAB) node, relay node, donor node controlled relay, radio access point (AP), transmission point, transmission node, remote radio unit (RRU), remote radio head (RRH). Core network node (such as mobility management entity (MME), self-organizing network (SON) node, coordination node, positioning node, MDT node, etc.), external node (such as third-party node, node outside the current network), node in distributed antenna system (DAS), spectrum access system (SAS) node, element management system (EMS), etc. Network nodes may also include test equipment. As used herein, the term "radio node" may also be used to refer to a wireless device (WD), such as a wireless device (WD), or a radio network node.

[0114] In some embodiments, the non-limiting terms wireless device (WD) or user equipment (UE) can be used interchangeably. The WD herein can be any type of wireless device capable of communicating with a network node or another WD via a radio signal, such as a wireless device (WD). The WD can also be a radio communication device, a target device, a device-to-device (D2D) WD, a machine type WD, or a WD capable of machine-to-machine communication (M2M), a low-cost and / or low-complexity WD, a sensor equipped with a WD, a tablet computer, a mobile terminal, a smart phone, a laptop embedded device (LEE), a laptop mounted device (LME), a USB dongle, a customer premises equipment (CPE), an Internet of Things (IoT) device, or a narrowband IoT (NB-IOT) device, etc.

[0115] Furthermore, in some embodiments, the general term "radio network node" is used. It may be any kind of radio network node, which may include any of the following: base station, radio base station, base transceiver station, base station controller, network controller, RNC, evolved Node B (eNB), Node B, gNB, multi-cell / multicast coordination entity (MCE), IAB node, relay node, access point, radio access point, remote radio unit (RRU), remote radio head (RRH).

[0116] Note that although terminology from one particular wireless system, such as 3GPP LTE and / or New Radio (NR), may be used in this disclosure, this should not be considered to limit the scope of this disclosure to only the aforementioned systems. Other wireless systems, including but not limited to Wideband Code Division Multiple Access (WCDMA), Worldwide Interoperability for Microwave Access (WiMax), Ultra Mobile Broadband (UMB), and Global System for Mobile Communications (GSM), may also benefit from utilizing the ideas encompassed within this disclosure.

[0117] It is further noted that the functions described herein as being performed by a wireless device or network node may be distributed across multiple wireless devices and / or network nodes. In other words, it is contemplated that the functions of the network nodes and wireless devices described herein are not limited to being performed by a single physical device and, in fact, may be distributed across several physical devices.

[0118] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that the terms used herein should be interpreted as having a meaning consistent with their meaning in the context of this specification and the relevant art, and will not be interpreted in an idealized or overly formal sense unless explicitly defined as such herein.

[0119] Some embodiments provide enhanced wireless device (WD) latency performance analysis to assist application layer artificial intelligence (AI) / machine learning (ML) operations.

[0120] Referring again to the drawings, in which like elements are designated by like reference numerals, Figure 2 , a schematic diagram of a communication system 10 according to an embodiment is shown, such as a 3GPP-type cellular network that can support standards such as LTE and / or NR (5G), including an access network 12 (such as a radio access network) and a core network 14. The access network 12 includes a plurality of network nodes 16a, 16b, 16c (collectively referred to as network nodes 16), such as NBs, eNBs, gNBs, or other types of wireless access points, each defining a corresponding coverage area 18a, 18b, 18c (collectively referred to as coverage areas 18). Each network node 16a, 16b, 16c can be connected to the core network 14 via a wired or wireless connection 20. A first wireless device (WD) 22a located in the coverage area 18a is configured to wirelessly connect to or be paged by the corresponding network node 16a. A second WD 22b in the coverage area 18b can also wirelessly connect to the corresponding network node 16b. Although multiple WDs 22a, 22b (collectively referred to as wireless devices 22) are shown in this example, the disclosed embodiments are equally applicable to situations where individual WDs are located in the coverage area or where individual WDs are connected to corresponding network nodes 16. Note that although only two WDs 22 and three network nodes 16 are shown for convenience, the communication system may include many more WDs 22 and network nodes 16.

[0121] Furthermore, it is contemplated that the WD 22 may communicate simultaneously with more than one network node 16 and more than one type of network node 16, and / or be configured to separately communicate with more than one network node 16 and more than one type of network node 16. For example, the WD 22 may have dual connectivity with a network node 16 that supports LTE and the same or different network node 16 that supports NR. As an example, the WD 22 may communicate with an eNB for LTE / E-UTRAN and a gNB for NR / NG-RAN.

[0122] The communication system 10 itself can be connected to a host 24, which can be embodied in the hardware and / or software of a stand-alone server, a cloud-implemented server, a distributed server, or as processing resources in a server farm. The host 24 can be under the ownership or control of a service provider, or can be operated by or on behalf of the service provider. The connections 26, 28 between the communication system 10 and the host 24 can extend directly from the core network 14 to the host 24, or can extend via an optional intermediate network 30. The intermediate network 30 can be one or a combination of more than one of a public, private, or managed network. The intermediate network 30, if any, can be a backbone network or the Internet. In some embodiments, the intermediate network 30 can include two or more subnetworks (not shown).

[0123] Figure 2 The communication system as a whole implements a connection between one of the connected WDs 22a, 22b and the host 24. This connection can be described as an over-the-top (OTT) connection. The host 24 and the connected WDs 22a, 22b are configured to communicate data and / or signaling via the OTT connection, using the access network 12, the core network 14, any intermediate networks 30, and possibly additional infrastructure (not shown) as intermediaries. The OTT connection can be transparent in the sense that at least some of the participating communication devices through which the OTT connection passes are unaware of the routing of uplink and downlink communications. For example, the network node 16 may not be informed or need not be informed of the past routing of incoming downlink communications having data originating from the host 24 that will be forwarded (e.g., handed over) to the connected WD 22a. Similarly, the network node 16 does not need to know the future routing of outgoing uplink communications originating from the WD 22a toward the host 24.

[0124] The network node 16 is configured to include an AI / ML unit that can be configured to implement a federated learning process including multiple inputs (artificial intelligence / machine learning AI / ML), the multiple inputs including transmission data information. The network node 16 can be configured to include a NF 32. The NF 32 can be configured to host AI / ML-based services based at least in part on the federated learning process. A core network node 36 in the core network 14 can include an NWDAF 34, and the network node 16 can include the NF 32 that receives delay performance analysis from the core network node 34. The NWDAF 34 can be configured to determine a set of delay performance analysis to assist in the federated learning process, the set of delay performance analysis being based at least in part on the delay performance inputs and including at least one of delay predictions and delay statistics related to uplink and downlink transmission delays between the WD 22 and the application function AF.

[0125] Now refer to Figure 3 An example implementation according to an embodiment of the WD 22, network node 16, host 24, and core network node 36 discussed in the previous paragraphs is described. In the communication system 10, the host 24 includes hardware (HW) 38, which includes a communication interface 40 that is configured to establish and maintain a wired or wireless connection to the interface of different communication devices of the communication system 10. The host 24 also includes a processing circuit 42 that may have storage and / or processing capabilities. The processing circuit 42 may include a processor 44 and a memory 46. In particular, in addition to or as an alternative to a processor (such as a central processing unit) and a memory, the processing circuit 42 may include an integrated circuit for processing and / or control, such as one or more processors and / or processor cores and / or an FPGA (field programmable gate array) and / or an ASIC (application-specific integrated circuit) suitable for executing instructions. The processor 44 may be configured to access (e.g., write to and / or read from) a memory 46, which may include any kind of volatile and / or non-volatile memory, such as cache and / or buffer memory and / or RAM (random access memory) and / or ROM (read-only memory) and / or optical memory and / or EPROM (erasable programmable read-only memory).

[0126] Processing circuitry 42 may be configured to control any of the methods and / or processes described herein, and / or to cause such methods and / or processes to be performed, for example, by host computer 24. Processor 44 corresponds to one or more processors 44 for performing the functions of host computer 24 described herein. Host computer 24 includes memory 46 configured to store data, programming software code, and / or other information described herein. In some embodiments, software 48 and / or host application 50 may include instructions that, when executed by processor 44 and / or processing circuitry 42, cause processor 44 and / or processing circuitry 42 to perform the processes described herein with respect to host computer 24. The instructions may be software associated with host computer 24.

[0127] The software 48 may be executable by the processing circuitry 42. The software 48 includes a host application 50. The host application 50 may be operable to provide services to a remote user, such as a WD 22 connected via an OTT connection 52 terminating at the WD 22 and the host 24. In providing services to the remote user, the host application 50 may provide user data sent using the OTT connection 52. "User data" may be data and information described herein as implementing the described functionality. In one embodiment, the host 24 may be configured to provide control and functionality to a service provider and may be operated by or on behalf of the service provider. The processing circuitry 42 of the host 24 may enable the host 24 to observe, monitor, control, send to, and / or receive from the network node 16 and / or wireless device 22.

[0128] Communication system 10 also includes a network node 16, which is disposed within communication system 10 and includes hardware 58 that enables communication with host 24 and WD 22. Hardware 58 may include a communication interface 60 for establishing and maintaining wired or wireless connections to various communication devices of communication system 10, and a radio interface 62 for establishing and maintaining at least a wireless connection 64 with WD 22 located within coverage area 18 served by network node 16. Radio interface 62 may be formed as or may include, for example, one or more RF transmitters, one or more RF receivers, and / or one or more RF transceivers. Communication interface 60 may be configured to facilitate a connection 66 to host 24. Connection 66 may be direct, or it may traverse core network 14 of communication system 10 and / or one or more intermediate networks 30 external to communication system 10.

[0129] In the illustrated embodiment, the hardware 58 of the network node 16 also includes processing circuitry 68. The processing circuitry 68 may include a processor 70 and a memory 72. In particular, in addition to or as an alternative to a processor (such as a central processing unit) and memory, the processing circuitry 68 may include integrated circuits for processing and / or control, such as one or more processors and / or processor cores and / or FPGAs (field programmable gate arrays) and / or ASICs (application-specific integrated circuits) adapted to execute instructions. The processor 70 may be configured to access (e.g., write to and / or read from) the memory 72, which may include any type of volatile and / or non-volatile memory, such as cache and / or buffer memory and / or RAM (random access memory) and / or ROM (read-only memory) and / or optical memory and / or EPROM (erasable programmable read-only memory).

[0130] Therefore, network node 16 also has software 74, which is stored internally, for example, in memory 72, or in external memory (e.g., a database, storage array, network storage device, etc.) accessible to network node 16 via an external connection. Software 74 can be executed by processing circuitry 68. Processing circuitry 68 can be configured to control any of the methods and / or processes described herein and / or cause such methods and / or processes to be performed, for example, by network node 16. Processor 70 corresponds to one or more processors 70 configured to perform the network node 16 functions described herein. Memory 72 is configured to store data, programming software code, and / or other information described herein. In some embodiments, software 74 can include instructions that, when executed by processor 70 and / or processing circuitry 68, cause processor 70 and / or processing circuitry 68 to perform the processes described herein with respect to network node 16. For example, processing circuitry 68 of network node 16 can include an AI / ML unit that can be configured to implement a federated learning process including multiple inputs, artificial intelligence / machine learning (AI / ML), including transmission data information. The NF 32 may be configured to host AI / ML-based services based at least in part on a federated learning process. In some embodiments, the NWDAF 34 may also be configured in the network node 16.

[0131] The communication system 10 also includes the already mentioned WD 22. The WD 22 may have hardware 80 that may include a radio interface 82 configured to establish and maintain a wireless connection 64 with a network node 16 serving the coverage area 18 in which the WD 22 is currently located. The radio interface 82 may be formed as or may include, for example, one or more RF transmitters, one or more RF receivers, and / or one or more RF transceivers.

[0132] The hardware 80 of the WD 22 also includes processing circuitry 84. The processing circuitry 84 may include a processor 86 and a memory 88. In particular, in addition to or as an alternative to a processor (such as a central processing unit) and memory, the processing circuitry 84 may include an integrated circuit for processing and / or control, for example, one or more processors and / or processor cores and / or an FPGA (field programmable gate array) and / or an ASIC (application-specific integrated circuit) adapted to execute instructions. The processor 86 may be configured to access the memory 88 (e.g., write to and / or read from the memory 88), which may include any type of volatile and / or non-volatile memory, such as a cache and / or buffer memory and / or RAM (random access memory) and / or ROM (read-only memory) and / or optical memory and / or EPROM (erasable programmable read-only memory).

[0133] Therefore, WD 22 may also include software 90, which is stored in, for example, memory 88 at WD 22, or in an external memory (e.g., a database, storage array, network storage device, etc.) accessible by WD 22. The software 90 may be executed by the processing circuit 84. The software 90 may include a client application 92. The client application 92 may be operable to provide services to human or non-human users via WD 22 with the support of the host 24. In the host 24, the executing host application 50 may communicate with the executing client application 92 via the OTT connection 52 terminated at the WD 22 and the host 24. In the process of providing services to the user, the client application 92 may receive request data from the host application 50 and provide user data in response to the request data. The OTT connection 52 may transmit both the request data and the user data. The client application 92 may interact with the user to generate the user data it provides.

[0134] The processing circuit 84 can be configured to control any method and / or process described herein, and / or cause such method and / or process to be performed, for example, by the WD 22. The processor 86 corresponds to one or more processors 86 for performing the WD 22 functions described herein. The WD 22 includes a memory 88 that is configured to store data, programming software code, and / or other information described herein. In some embodiments, the software 90 and / or client application 92 may include instructions that, when executed by the processor 86 and / or the processing circuit 84, cause the processor 86 and / or the processing circuit 84 to perform the processes described herein with respect to the WD 22.

[0135] The core network node 36 may include processing circuitry 94 including a processor and memory (not shown). The processor may include a central processing unit and / or processing circuitry including integrated circuits for processing and / or control, such as one or more processors and / or processor cores and / or FPGAs (field programmable gate arrays) and / or ASICs (application-specific integrated circuits) adapted to execute instructions. The processor may be configured to access (e.g., write to and / or read from) the memory, which may include any type of volatile and / or non-volatile memory, such as cache and / or buffer memory and / or RAM (random access memory) and / or ROM (read-only memory) and / or optical memory and / or EPROM (erasable programmable read-only memory).

[0136] Therefore, the core network node 36 may also include software stored in a memory at the core network node or in an external memory accessible by the core network node (e.g., a database, a storage array, a network storage device, etc.). The software may be executed by the processing circuit 94 to perform the functions of the NWDAF as described herein. The processing circuit 94 may be configured to include the NWDAF 34, which may be configured to determine a set of delay performance analyses to assist in the joint learning process. In some embodiments, the NF 32 may also be configured in the core network node 36.

[0137] The core network node 36 may also include a radio interface 96 configured to establish and maintain a wireless connection with the network node 16 serving the coverage area 18 in which the WD 22 is currently located. The radio interface 96 may be formed as or may include, for example, one or more RF transmitters, one or more RF receivers, and / or one or more RF transceivers. In some embodiments, the internal workings of the network node 16, the WD 22, the host 24, and the core network node 36 may be as follows: Figure 3 shown, and independently, the surrounding network topology can be Figure 2 network topology.

[0138] exist Figure 3 In FIG, OTT connection 52 is abstractly drawn to illustrate communication between host 24 and wireless device 22 via network node 16, without explicitly mentioning any intermediate devices and the precise routing of messages through these devices. The network infrastructure can determine the routing, and the network infrastructure can be configured to hide the routing from WD 22, or from the service provider operating host 24, or both. When OTT connection 52 is active, the network infrastructure can also make decisions by which the network infrastructure dynamically changes the routing (e.g., based on load balancing considerations or network reconfiguration).

[0139] The wireless connection 64 between the WD 22 and the network node 16 is in accordance with the teachings of the embodiments described throughout this disclosure. One or more of the various embodiments improve the performance of OTT services provided to the WD 22 using the OTT connection 52, in which the wireless connection 64 may form the final leg. More specifically, the teachings of some of these embodiments may improve data rates, latency, and / or power consumption, thereby providing benefits such as reduced user wait time, relaxed file size restrictions, better responsiveness, extended battery life, and the like.

[0140] In some embodiments, a measurement process may be provided for the purpose of monitoring data rate, latency, and other factors improved by one or more embodiments. Optional network functionality may also be provided for reconfiguring the OTT connection 52 between the host 24 and the WD 22 in response to changes in measurement results. The measurement process and / or network functionality for reconfiguring the OTT connection 52 may be implemented in the host 24 software 48 or the WD 22 software 90, or both. In embodiments, sensors (not shown) may be deployed in or associated with the communication devices through which the OTT connection 52 traverses. The sensors may participate in the measurement process by supplying values ​​for the monitored quantities exemplified above, or by supplying values ​​for other physical quantities from which the software 48 or 90 may calculate or estimate the monitored quantities. Reconfiguration of the OTT connection 52 may include message formats, retransmission settings, preferred routing, and the like. The reconfiguration need not affect the network node 16 and may be unknown or imperceptible to the network node 16. Some of these processes and functionality may be known and practiced in the art. In some embodiments, the measurements may involve proprietary WD signaling that facilitates the host 24's measurement of throughput, propagation time, latency, etc. In some embodiments, the measurements may be implemented in such a way that the software 48, 90 causes a message (particularly a null message or 'dummy' message) to be sent using the OTT connection 52 while the software 48, 90 monitors propagation time, errors, etc.

[0141] Thus, in some embodiments, host 24 includes processing circuitry 42 configured to provide user data, and communication interface 40 configured to forward the user data to a cellular network for transmission to WD 22. In some embodiments, the cellular network also includes a network node 16 having a radio interface 62. In some embodiments, network node 16 is configured and / or processing circuitry 68 of network node 16 is configured to perform the functions and / or methods described herein for preparing / initiating / maintaining / supporting / terminating transmissions to WD 22 and / or preparing / terminating / maintaining / supporting / terminating receipt of transmissions from WD 22.

[0142] In some embodiments, host 24 includes processing circuitry 42 and communication interface 40 configured to receive user data originating from a transmission from WD 22 to network node 16. In some embodiments, WD 22 is configured to perform and / or includes a radio interface 82 and / or processing circuitry 84 configured to perform the functions and / or methods described herein for preparing / initiating / maintaining / supporting / terminating transmissions to network node 16 and / or preparing / terminating / maintaining / supporting / terminating receipt of transmissions from network node 16.

[0143] although Figure 2and Figure 3 Various "units" such as AI / ML unit 32 are shown as being located within respective processors, but it is contemplated that these units may be implemented such that a portion of the unit is stored in a corresponding memory within the processing circuitry. In other words, the units may be implemented within the processing circuitry in hardware or a combination of hardware and software.

[0144] Figure 4 FIG. 1 is a diagram illustrating a communication system (such as FIG. Figure 2 and Figure 3 The communication system may include a host 24, a network node 16, and a WD 22, which may be a reference Figure 3 Those described herein. In a first step of the method, host 24 provides user data (block S100). In an optional sub-step of the first step, host 24 provides user data by executing a host application (such as host application 50) (block S102). In a second step, host 24 initiates a transmission carrying the user data to WD 22 (block S104). In an optional third step, in accordance with the teachings of the embodiments described throughout this disclosure, network node 16 sends the user data carried in the transmission initiated by host 24 to WD 22 (block S106). In an optional fourth step, WD 22 executes a client application, such as client application 92, associated with host application 50 executed by host 24 (block S108).

[0145] Figure 5 FIG. 1 is a diagram illustrating a communication system (such as FIG. Figure 2 The communication system may include a host 24, a network node 16, and a WD 22, which may be a reference Figure 2 and Figure 3 Those described herein. In a first step of the method, host 24 provides user data (block S110). In an optional sub-step (not shown), host 24 provides the user data by executing a host application (such as host application 50). In a second step, host 24 initiates a transmission carrying the user data to WD 22 (block S112). According to the teachings of the embodiments described throughout this disclosure, the transmission can be delivered via network node 16. In an optional third step, WD 22 receives the user data carried in the transmission (block S114).

[0146] Figure 6 FIG. 1 is a diagram illustrating a communication system (such as FIG. Figure 2 The communication system may include a host 24, a network node 16, and a WD 22, which may be a reference Figure 2 and Figure 3Those described. In an optional first step of the method, WD 22 receives input data provided by host 24 (box S116). In an optional sub-step of the first step, WD 22 executes client application 92, which provides user data in response to the input data received from host 24 (box S118). Additionally or alternatively, in an optional second step, WD 22 provides user data (box S120). In an optional sub-step of the second step, WD 22 provides user data by executing a client application (such as client application 92) (box S122). In the process of providing user data, the executed client application 92 may also consider user input received from the user. Regardless of the specific manner in which the user data is provided, WD 22 may initiate the transmission of the user data to host 24 in an optional third sub-step (box S124). In a fourth step of the method, in accordance with the teachings of the embodiments described throughout this disclosure, host 24 receives the user data sent from WD 22 (box S126).

[0147] Figure 7 FIG. 1 is a diagram illustrating a communication system (such as FIG. Figure 2 The communication system may include a host 24, a network node 16, and a WD 22, which may be a reference Figure 2 and Figure 3 In an optional first step of the method, network node 16 receives user data from WD 22 in accordance with the teachings of the embodiments described throughout this disclosure (block S128). In an optional second step, network node 16 initiates a transmission of the received user data to host 24 (block S130). In a third step, host 24 receives the user data carried in the transmission initiated by network node 16 (block S132).

[0148] Figure 8 1 is a flow chart of an example process for enhancing wireless device (WD) delay performance analysis in a network node 16 to assist in application-layer artificial intelligence (AI) / machine learning (ML) operations. One or more blocks described herein may be executed by one or more elements of the network node 16, such as by one or more of the processing circuitry 68 (including the AI / ML unit 32), the processor 70, the radio interface 62, and / or the communication interface 60. The network node 16 is configured, such as via the processing circuitry 68 and / or the processor 70 and / or the radio interface 62 and / or the communication interface 60, to implement a federated learning process comprising multiple inputs of artificial intelligence / machine learning (AI / ML), the multiple inputs comprising transmission data information (block S134). The federated learning process comprises a WD delay performance analysis configured to produce multiple outputs, the multiple outputs comprising delay statistics (block S136).

[0149] In some embodiments, the transmission data information includes at least one of a round-trip transmission data volume, a start and end time for the data transmission, and filtering information. In some embodiments, the delay statistics include uplink and downlink delay statistics for the target transmission. In some embodiments, the delay statistics include round-trip delay statistics for the target transmission. In some embodiments, the delay statistics include a maximum delay for the target transmission.

[0150] Figure 9 1 is a flow chart of an example process in a core network node 36 configured to include a network data analysis function (NWDAF 34) configured according to the principles disclosed herein. One or more of the blocks described herein may be executed by one or more elements of the core network node 36, such as by one or more of the processing circuitry 94 (including the NWDAF 34) and the radio interface 96. The core network node 36 is configured, such as via the processing circuitry 94 and the radio interface 96, to collect wireless device WD 22 delay performance input from the network function NF 32, the delay performance input including a plurality of transmission data amount values ​​and transmission time values ​​(block S138). The method includes determining a set of delay performance analyses to assist in a joint learning process, the set of delay performance analyses being based at least in part on the delay performance inputs and including at least one of delay predictions and delay statistics related to uplink and downlink transmission delays between the WD 22 and the application function AF (block S140). The method also includes sending the set of delay performance analyses to the NF 32 (block S142).

[0151] In some embodiments, at least one of the delay prediction and the delay statistics includes a time delay for completing transmission of a target amount of data from a WD 22 to a NF 32 or from a NF 32 to a WD 22. In some embodiments, at least one of the delay prediction and the delay statistics includes an estimated number of retransmissions for completing transmission of the target amount of data. In some embodiments, at least one of the delay prediction and the delay statistics includes at least one of an uplink packet delay, a downlink packet delay, and a round-trip packet delay. In some embodiments, at least one of the delay prediction and the delay statistics includes a value averaged over a target analysis period. In some embodiments, at least one of the delay prediction and the delay statistics includes a valid period for delay performance analysis. In some embodiments, at least one of the delay prediction and the delay statistics includes a maximum packet delay observed for communications with the NF 32. In some embodiments, at least one of the delay prediction and the delay statistics includes a spatial validity parameter indicating a region for which the delay performance analysis is applicable. In some embodiments, at least one of the delay prediction and the delay statistics is grouped into delay categories according to delay performance ranges. In some embodiments, at least one of the delay prediction and the delay statistics includes an indication of the percentage of WDs 22 in each delay category. In some embodiments, the NF is implemented as an application function.

[0152] Figure 10 1 is a flow chart of an example process in a network node 16 configured to include a network function (NF 32) configured according to the principles disclosed herein. One or more of the blocks described herein may be executed by one or more elements of the network node 16, such as by one or more of the processing circuitry 68 (including the NWDAF 34), the processor 70, the radio interface 62, and / or the communication interface 60. The network node 16 is configured, such as via the processing circuitry 68 and / or the processor 70 and / or the radio interface 62 and / or the communication interface 60, to send a delay performance input to the NWDAF 34, the delay performance input including a plurality of transmission data amount values ​​and transmission time values ​​(Block S144). The method includes receiving a set of delay performance analyses to assist in a federated learning process, the set of delay performance analyses being based at least in part on the delay performance inputs and including at least one of delay predictions and delay statistics related to uplink and downlink transmission delays between the WD 22 and the NF 32 (Block S146). The method also includes hosting an artificial intelligence / machine learning (AI / ML)-based service based at least in part on the federated learning process (Block S148).

[0153] According to this aspect, in some embodiments, at least one of the delay prediction and the delay statistics includes the time delay required to complete transmission of a target amount of data from a WD 22 to a NF 32 or from a NF 32 to a WD 22. In some embodiments, at least one of the delay prediction and the delay statistics includes an estimated number of retransmissions required to complete transmission of the target amount of data. In some embodiments, at least one of the delay prediction and the delay statistics includes at least one of an uplink packet delay, a downlink packet delay, and a round-trip packet delay. In some embodiments, at least one of the delay prediction and the delay statistics includes a value averaged over a target analysis period. In some embodiments, at least one of the delay prediction and the delay statistics includes a valid period for delay performance analysis. In some embodiments, at least one of the delay prediction and the delay statistics includes a maximum packet delay observed for communications with the NF 32. In some embodiments, at least one of the delay prediction and the delay statistics includes a spatial validity parameter indicating the region for which the delay performance analysis is applicable. In some embodiments, at least one of the delay prediction and the delay statistics is grouped into delay categories according to delay performance ranges. In some embodiments, at least one of the delay prediction and the delay statistics includes an indication of the percentage of WDs 22 in each delay category. In some embodiments, NFs are implemented as application functions.

[0154] Having described the general process of the arrangements of the present disclosure and provided examples of hardware and software arrangements for implementing the processes and functions of the present disclosure, the following section provides details and examples of arrangements for enhancing wireless device (WD) latency performance analysis to assist in application layer artificial intelligence (AI) / machine learning (ML) operations.

[0155] Input for WD latency performance analysis of federated learning

[0156] Referring to Table 1 below, expanding an existing input from a single value to a list of (possible) values ​​may include one or more of the following:

[0157] ■ Changing the input “transferred data amount” from a single value to a list of (possible) values ​​and further expanding it to “round-trip data amount” in order to represent the possibility of different data amounts for different AI / ML services in a federated learning process; and

[0158] ■ In order to represent the possibility of different transmission times for data volumes of different AI / ML services in one federated learning process, the input “transmission time” is changed from a single value to a list of (possible) values.

[0159] Some embodiments may include adding new inputs for WD latency performance analysis. This may include one or more of the following:

[0160] ■ If the round trip data volume is provided, the “time associated with the round trip” (e.g., the time interval of the total round trip time for each data volume) can also be provided by the AF;

[0161] ■ In order to obtain information about the start and end time of the transmission of data with (one or more) transmission data volumes for the AI / ML service used in the federated learning process, a new input "timestamp" can be added to identify the start and end time of the transmission of data with the transmission data volume. This new input can be used for analysis of a specific time interval, such as tomorrow 10:00-10:30, etc.;

[0162] ■In order to get more detailed information about AI / ML traffic to generate statistics and predictions about latency performance at specific time intervals (e.g., using the service flow for AI / ML traffic between WD 22 and AF during 10:00-10:30), new inputs “IP filter information”, “Location of application” and “Application server instance address” can be implemented.

[0163] Table 1: Service data from 5GC NF for WD latency performance analysis

[0164]

[0165]

[0166] Output of WD latency performance analysis for federated learning

[0167] Referring to Tables 2 and 3 below, in order to obtain analysis results of the UL / DL / round-trip delay of the WD 22 for communicating with the application to complete the transmission of a target data volume (for example, the NWDAF 34 can estimate the number of transmissions for a certain data volume), there may be a single transmission for the data volume or multiple retransmissions for the data volume to achieve the QoS requirements. New outputs can be added for the WD delay performance analysis, which may include one or more of the following:

[0168] ■Estimated number of transmissions (statistical / forecast);

[0169] ■ Statistics / predictions about the UL delay for target transmissions;

[0170] ■ Statistics / predictions about the DL delay for target transmissions;

[0171] ■ Statistics / predictions regarding round trip delay for target transmissions; and / or

[0172] ■ Statistics / predictions about the maximum delay for target transmission.

[0173] Table 2: WD latency performance statistics

[0174]

[0175]

[0176]

[0177] Table 3: WD latency performance prediction

[0178]

[0179]

[0180]

[0181]

[0182] Some embodiments may include one or more of the following:

[0183] Embodiment A1. A network node configured to communicate with a wireless device (WD), the network node being configured to and / or comprising a radio interface and / or comprising a processing circuit configured to:

[0184] Implementing a joint learning process involving multiple inputs of artificial intelligence / machine learning AI / ML, including transmission data information; and

[0185] The federated learning process includes a WD latency performance analysis configured to produce a plurality of outputs, the plurality of outputs including latency statistics.

[0186] Embodiment A2. The network node of Embodiment A1, wherein the transmission data information includes at least one of a round-trip transmission data amount, a start and end time for the data transmission, and filter information.

[0187] Embodiment A3. The network node of any of Embodiments A1 and A2, wherein the delay statistics include uplink and downlink delay statistics for a target transmission.

[0188] Embodiment A4. The network node of any one of Embodiments A1-A3, wherein the delay statistics include round-trip delay statistics for target transmissions.

[0189] Embodiment A5. The network node of any one of Embodiments A1-A4, wherein the delay statistics include a maximum delay for a target transmission.

[0190] Embodiment B1. A method implemented in a network node configured to communicate with a wireless device WD, the method comprising:

[0191] Implementing a joint learning process involving multiple inputs of artificial intelligence / machine learning AI / ML, including transmission data information; and

[0192] The federated learning process includes a WD latency performance analysis configured to produce a plurality of outputs, the plurality of outputs including latency statistics.

[0193] Embodiment B2. The method of Embodiment B1, wherein the transfer data information includes at least one of a round-trip transfer data amount, a start and end time for data transfer, and filter information.

[0194] Embodiment B3. The method of any one of Embodiments B1 and B2, wherein the delay statistics include uplink and downlink delay statistics for the target transmission.

[0195] Embodiment B4. The method of any one of Embodiments B1-B3, wherein the delay statistics include round-trip delay statistics for target transmissions.

[0196] Embodiment B5. The method of any one of Embodiments B1-B4, wherein the delay statistics include a maximum delay for a target transmission.

[0197] As will be appreciated by those skilled in the art, the concepts described herein may be embodied as methods, data processing systems, computer program products, and / or computer storage media storing executable computer programs. Accordingly, the concepts described herein may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects, collectively referred to herein as "circuits" or "modules." Any process, step, action, and / or function described herein may be performed by and / or associated with a corresponding module, which may be implemented in software and / or firmware and / or hardware. Furthermore, the present disclosure may take the form of a computer program product on a tangible computer-usable storage medium having computer program code embodied therein that can be executed by a computer. Any suitable tangible computer-readable medium may be utilized, including a hard disk, a CD-ROM, an electronic storage device, an optical storage device, or a magnetic storage device.

[0198] Some embodiments are described herein with reference to flowchart illustrations and / or block diagrams of methods, systems, and computer program products. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer (thereby creating a special-purpose computer), a special-purpose computer, or other programmable data processing device to produce a machine, such that instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0199] These computer program instructions may also be stored in a computer-readable memory or storage medium, which may direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0200] The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be executed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0201] It will be understood that the functions / actions noted in the blocks may be performed in a different order than that noted in the operational diagrams. For example, two blocks shown in succession may actually be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending on the functions / actions involved. Although some figures include arrows on communication paths to illustrate the primary direction of communication, it will be understood that communication may occur in the reverse direction of the arrows depicted.

[0202] Computer program code for carrying out operations of the concepts described herein can be written in an object oriented programming language such as Python, Or C++. However, the computer program code for performing the operations of the present disclosure may also be written in a conventional procedural programming language, such as the "C" programming language. The program code may be executed entirely on the user's computer, partially on the user's computer as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer. In the latter case, the remote computer may be connected to the user's computer via a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0203] Many different embodiments have been disclosed herein in conjunction with the above description and accompanying drawings. It will be understood that it would be unduly repetitive and confusing to literally describe and illustrate every combination and subcombination of these embodiments. Therefore, all embodiments may be combined in any manner and / or combination, and this specification (including the accompanying drawings) should be construed to constitute a complete written description of all combinations and subcombinations of the embodiments described herein, as well as the manner and process of making and using them, and will support the claims to the extent of any such combination or subcombination.

[0204] Abbreviations that may be used in the preceding description include:

[0205] AF application function

[0206] AMF Access and Mobility Management Function

[0207] AI Artificial Intelligence

[0208] FL Federated Learning

[0209] FQDN Fully Qualified Domain Name

[0210] GMLC Gateway Mobile Location Center

[0211] ML Machine Learning

[0212] NEF Network Exposure Function

[0213] NF Network Function

[0214] NWDAF network data analysis function

[0215] OAM Operations, Administration and Maintenance

[0216] SMF session management functions

[0217] UPF User Plane Function

[0218] 5GC 5G core network

[0219] It will be understood by those skilled in the art that the embodiments described herein are not limited to what is specifically shown and described herein above. In addition, unless otherwise indicated above, it should be noted that all drawings are not drawn to scale. Various modifications and variations are possible in light of the above teachings without departing from the scope of the appended claims.

Claims

1. A method in a core network node (36) configured to include a network data analysis function (NWDAF) (34), the method comprising: collecting (S138) a wireless device WD (22) delay performance input from a network function NF (32), the delay performance input including a plurality of transmission data amount values ​​and transmission time values; determining (S140) a set of delay performance analyses to assist a federated learning process, the set of delay performance analyses being based at least in part on the delay performance input and comprising at least one of delay predictions and delay statistics related to uplink and downlink transmission delays between the WD (22) and an application function AF; as well as The set of delay performance analyses is sent (S142) to the NF (32).

2. The method according to claim 1, wherein The at least one of delay prediction and delay statistics includes a time delay for completing transmission of a target amount of data from a WD (22) to the NF (32) or from the NF (32) to the WD (22).

3. The method according to claim 2, wherein: The at least one of delay prediction and delay statistics comprises an estimated number of retransmissions for completing the transmission of the target amount of data.

4. The method according to any one of claims 1 and 2, wherein The at least one of delay prediction and delay statistics includes at least one of uplink packet delay, downlink packet delay, and round-trip packet delay.

5. The method according to any one of claims 1 to 4, wherein The at least one of the delay prediction and the delay statistics includes a value averaged over an analysis target period.

6. The method according to any one of claims 1 to 5, wherein The at least one of delay prediction and delay statistics includes a valid period for the delay performance analysis.

7. The method according to any one of claims 1 to 6, wherein The at least one of a delay prediction and a delay statistic comprises a maximum packet delay observed for communications with the NF (32).

8. The method according to any one of claims 1 to 7, wherein The at least one of delay predictions and delay statistics includes a spatial significance parameter indicating a region to which the delay performance analysis is applicable.

9. The method according to any one of claims 1 to 8, wherein The at least one of delay predictions and delay statistics are grouped into delay categories according to delay performance ranges.

10. The method of claim 9, wherein: The at least one of the delay prediction and the delay statistics includes an indication of a percentage of WDs (22) in each delay category.

11. The method according to claims 1 to 10, wherein: The NF is implemented as the application function.

12. A core network node (36) configured to include a network data analysis function (NWDAF) (34), the core network node (36) being configured to: collecting a wireless device WD (22) delay performance input from a network function NF (32), the delay performance input including a plurality of transmission data amount values ​​and transmission time values; determining a set of delay performance analyses to assist in a federated learning process, the set of delay performance analyses being based at least in part on the delay performance input and comprising at least one of delay predictions and delay statistics related to uplink and downlink transmission delays between the WD (22) and an application function AF; as well as The set of delay performance analyses is sent to the NF (32).

13. The core network node (36) of claim 12, wherein: The at least one of delay prediction and delay statistics includes a time delay for completing transmission of a target amount of data from a WD (22) to the NF (32) or from the NF (32) to the WD (22).

14. The core network node (36) of claim 13, wherein: The at least one of delay prediction and delay statistics comprises an estimated number of retransmissions for completing the transmission of the target amount of data.

15. The core network node (36) according to any one of claims 12 and 13, wherein The at least one of delay prediction and delay statistics includes at least one of uplink packet delay, downlink packet delay, and round-trip packet delay.

16. The core network node (36) according to any one of claims 12 to 15, wherein: The at least one of the delay prediction and the delay statistics includes a value averaged over an analysis target period.

17. The core network node (36) according to any one of claims 12 to 16, wherein: The at least one of delay prediction and delay statistics includes a valid period for the delay performance analysis.

18. The core network node (36) according to any one of claims 12 to 17, wherein: The at least one of a delay prediction and a delay statistic comprises a maximum packet delay observed for communications with the NF (32).

19. The core network node (36) according to any one of claims 12 to 18, wherein: The at least one of delay predictions and delay statistics includes a spatial significance parameter indicating a region to which the delay performance analysis is applicable.

20. The core network node (36) according to any one of claims 12 to 19, wherein: The at least one of delay predictions and delay statistics are grouped into delay categories according to delay performance ranges.

21. The core network node (36) of claim 20, wherein: The at least one of the delay prediction and the delay statistics includes an indication of a percentage of WDs (22) in each delay category.

22. The core network node according to claim 12-21, wherein: The NF is implemented as the application function.

23. A method in a network node configured to include a network function NF (32) and configured to communicate with a network data analysis function NWDAF (34), the method comprising: sending (S144) a delay performance input to the NWDAF (34), the delay performance input including a plurality of transmission data amount values ​​and transmission time values; receiving (S146) a set of delay performance analyses to assist in a federated learning process, the set of delay performance analyses being based at least in part on the delay performance input and comprising at least one of delay predictions and delay statistics related to uplink and downlink transmission delays between the wireless device WD (22) and the application function AF; as well as An artificial intelligence / machine learning AI / ML based service is hosted (S148) based at least in part on the federated learning process.

24. The method of claim 23, wherein: The at least one of delay prediction and delay statistics includes a time delay for completing transmission of a target amount of data from a WD (22) to the NF (32) or from the NF (32) to the WD (22).

25. The method of claim 24, wherein: The at least one of delay prediction and delay statistics comprises an estimated number of retransmissions for completing the transmission of the target amount of data.

26. The method of any one of claims 23 to 25, wherein The at least one of delay prediction and delay statistics includes at least one of uplink packet delay, downlink packet delay, and round-trip packet delay.

27. The method of any one of claims 23 to 26, wherein The at least one of the delay prediction and the delay statistics includes a value averaged over an analysis target period.

28. The method of any one of claims 23 to 27, wherein The at least one of delay prediction and delay statistics includes a valid period for the delay performance analysis.

29. The method of any one of claims 23 to 28, wherein The at least one of a delay prediction and a delay statistic comprises a maximum packet delay observed for communications with the NF (32).

30. The method of any one of claims 23 to 29, wherein The at least one of delay predictions and delay statistics includes a spatial significance parameter indicating a region to which the delay performance analysis is applicable.

31. The method of any one of claims 23 to 30, wherein The at least one of delay predictions and delay statistics are grouped into delay categories according to delay performance ranges.

32. The method of claim 31, wherein The at least one of the delay prediction and the delay statistics includes an indication of a percentage of WDs (22) in each delay category.

33. The method of claims 23-32, wherein: The NF is implemented as the application function.

34. A network node (16) configured to include a network function NF (32) and configured to communicate with a network data analysis function NWDAF (34), the network node (16) being configured to: Sending a delay performance input to the NWDAF (34), the delay performance input including a plurality of transmission data amount values ​​and transmission time values; receiving a set of delay performance analyses to assist in a federated learning process, the set of delay performance analyses being based at least in part on the delay performance input and comprising at least one of delay predictions and delay statistics related to uplink and downlink transmission delays between a wireless device WD (22) and an application function AF; and Hosting artificial intelligence / machine learning AI / ML based services based at least in part on the federated learning process.

35. The network node (16) of claim 34, wherein: The at least one of delay prediction and delay statistics includes a time delay for completing transmission of a target amount of data from a WD (22) to the NF (32) or from the NF (32) to the WD (22).

36. The network node (16) of claim 35, wherein: The at least one of delay prediction and delay statistics comprises an estimated number of retransmissions for completing the transmission of the target amount of data.

37. The network node (16) according to any one of claims 35-36, wherein The at least one of delay prediction and delay statistics includes at least one of uplink packet delay, downlink packet delay, and round-trip packet delay.

38. The network node (16) according to any one of claims 34 to 37, wherein The at least one of the delay prediction and the delay statistics includes a value averaged over an analysis target period.

39. The network node (16) according to any one of claims 34 to 38, wherein The at least one of delay prediction and delay statistics includes a valid period for the delay performance analysis.

40. The network node (16) according to any one of claims 34 to 39, wherein The at least one of a delay prediction and a delay statistic comprises a maximum packet delay observed for communications with the NF (32).

41. The network node (16) according to any one of claims 34 to 40, wherein: The at least one of delay predictions and delay statistics includes a spatial significance parameter indicating a region to which the delay performance analysis is applicable.

42. The network node (16) according to any one of claims 34 to 41, wherein: The at least one of delay predictions and delay statistics are grouped into delay categories according to delay performance ranges.

43. The network node (16) of claim 42, wherein: The at least one of the delay prediction and the delay statistics includes an indication of a percentage of WDs (22) in each delay category.

44. The network node (16) according to claims 34-43, wherein The NF is implemented as the application function.

Images