Adaptive performance monitoring

Abstract

A solution for adaptive performance monitoring is disclosed. The solution includes maintaining (200) the ability to collect network performance data with a collection policy from more than one collection policy; receiving (202) a request from the network element to apply a given collection policy; applying (204) the requested collection policy to collect network performance data; and sending (206) network performance data to the network based on the applied policy.

Description

Technical Field

[0001] The exemplary and non-limiting embodiments of the present invention generally relate to wireless communication systems. In particular, the embodiments of the present invention relate to apparatus and methods in wireless communication networks. Background Art

[0002] Modern wireless communication systems are complex systems. Communication systems and networks are widely used, and protecting the operation of the systems and monitoring the services provided with acceptable quality is an important part of network maintenance.

[0003] Typically, the performance of a network is monitored by collecting data related to the operation of the network. Important performance parameters may be referred to as key performance indicators, KPIs. If the network may operate under varying conditions using several operating algorithms, it is important to collect KPIs in a reliable manner. Summary of the invention

[0004] According to one aspect, the subject matter of the independent claims is provided. Embodiments are defined in the dependent claims.

[0005] One or more examples of implementations are set forth in more detail in the drawings and the description that follows.Other features will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Hereinafter, embodiments will be described in more detail with reference to the accompanying drawings, in which

[0007] Figure 1 An exemplary wireless communication system is shown;

[0008] Figure 2 and Figure 3 is a flow chart illustrating some embodiments;

[0009] Figure 4 , Figure 5 and Figure 6 is a signaling diagram illustrating some embodiments;

[0010] Figure 7 An example of the operation in the embodiment is shown;

[0011] Figure 8 is a flow chart illustrating an embodiment; and

[0012] Fig.9A , Fig. 9B and Fig. 9C An apparatus according to some embodiments is shown. DETAILED DESCRIPTION

[0013] The following embodiments are presented as examples only. Although the specification may refer to "one", "an" or "some" embodiments and / or examples in several places in the text, this does not necessarily mean that the same embodiment or example is referenced each time, or that a particular feature applies only to a single embodiment and / or example. Individual features of different embodiments and / or examples may also be combined to provide other embodiments and / or examples.

[0014] In the following, different exemplary embodiments will be described using a radio access architecture based on Advanced Long Term Evolution (LTE-Advanced, LTE-A) or New Radio (NR, 5G) as an example of an access architecture to which the embodiments may be applied, however, the embodiments are not limited to such architectures. It is obvious to a person skilled in the art that the embodiments may also be applied to other kinds of communication networks with suitable components by appropriately adjusting the parameters and procedures. Some examples of other options for suitable systems are Universal Mobile Telecommunications System (UMTS) Radio Access Network (UTRAN or E-UTRAN), Long Term Evolution (LTE, same as E-UTRA), Wireless Local Area Network (WLAN or WiFi), Worldwide Interoperability for Microwave Access (WiMAX), Personal Communications Service (PCS), Wideband Code Division Multiple Access (WCDMA), systems using Ultra-Wideband (UWB) technology, sensor networks, mobile ad hoc networks (MANETs), and Internet Protocol Multimedia Subsystem (IMS), or any combination thereof.

[0015] Figure 1 An example of a simplified system architecture is depicted showing only some elements and functional entities, all of which are logical units, the implementation of which may differ from what is shown. Figure 1 The connections shown are logical connections; the actual physical connections may be different. It will be apparent to those skilled in the art that the system typically also includes Figure 1 Other functions and structures than those shown.

[0016] However, the embodiments are not limited to the systems given as examples, but a person skilled in the art may apply the solutions to other communication systems provided with the necessary properties.

[0017] Figure 1 The example of FIG. 1 shows a portion of an exemplary wireless access network.

[0018] Figure 1Shown is a user equipment 100 and 102 configured to be wirelessly connected to an access node (such as (e / g) NodeB) 104 providing a cell on one or more communication channels in the cell. The physical link from the user equipment to the (e / g) NodeB is called an uplink or reverse link, and the physical link from the (e / g) NodeB to the user equipment is called a downlink or forward link. It should be understood that the (e / g) NodeB or its functions can be implemented by using any node, host, server or access point entity suitable for such use.

[0019] The communication system usually includes more than one (e / g)NodeB, in which case the (e / g)NodeB can also be configured to communicate with each other through wired or wireless links designed for the purpose. These links can be used for signaling purposes. (e / g)NodeB is a computing device configured to control the radio resources of the communication system to which it is coupled. NodeB can also be referred to as a base station, an access point, or any other type of interface device including a relay station capable of operating in a wireless environment. (e / g)NodeB includes or is coupled to a transceiver. From the transceiver of the (e / g)NodeB, a connection is provided to an antenna unit that establishes a bidirectional radio link to a user equipment. The antenna unit may include multiple antennas or antenna elements. (e / g)NodeB is also connected to the core network 110 (CN or next generation core NGC). Depending on the system, the counterpart on the CN side may be: a serving gateway (S-GW, routing and forwarding user data packets); a packet data network gateway (P-GW), for providing a connection of a user equipment (UE) to an external packet data network; or a mobile management entity (MME), etc.

[0020] User equipment (also referred to as UE, user device, user terminal, terminal device, etc.) shows a type of device to which resources on the air interface are allocated and assigned, so any features described herein with user equipment can be implemented using corresponding devices (such as relay nodes). An example of such a relay node is a layer 3 relay (self-backhaul relay) towards a base station.

[0021] User equipment generally refers to portable computing devices including wireless mobile communication devices operating with or without a subscriber identity module (SIM), including but not limited to the following types of devices: mobile stations (mobile phones), smart phones, personal digital assistants (PDAs), mobile phones, devices using wireless modems (alarm or measurement devices, etc.), laptop and / or touch screen computers, tablet computers, game consoles, notebooks and multimedia devices. It should be understood that user equipment can also be almost exclusively uplink-only devices, examples of which are cameras or video cameras that load images or video clips to the network. User equipment can also be a device with the ability to operate in an Internet of Things (IoT) network, which is a scenario in which objects are provided with the ability to transmit data over a network without the need for human-to-human or human-to-computer interaction. A user device (or in some embodiments, a layer 3 relay node) is configured to perform one or more user equipment functions. A user device may also be referred to as a subscriber unit, a mobile station, a remote terminal, an access terminal, a user terminal or a user equipment (UE), only some of the names or devices are mentioned here.

[0022] The various techniques described herein can also be applied to cyber-physical systems (CPS) (systems of cooperating computing elements that control physical entities). CPS can implement and utilize a large number of interconnected ICT devices (sensors, actuators, processor microcontrollers, etc.) embedded in physical objects at different locations. Mobile cyber-physical systems, where the physical systems in question have inherent mobility, are a subcategory of cyber-physical systems. Examples of mobile physical systems include mobile robots and electronic devices transported by humans or animals.

[0023] It should be understood that Figure 1 In the figure, for the sake of clarity only, the user equipment is depicted as comprising 2 antennas. The number of receiving and / or transmitting antennas may naturally vary according to the current implementation.

[0024] Additionally, although the device has been depicted as a single entity, different units, processors and / or memory units may be implemented. Figure 1 Not all are shown).

[0025] 5G enables the use of multiple-input multiple-output (MIMO) antennas, many more base stations or nodes than LTE (the so-called small cell concept), including macro sites that operate in collaboration with smaller stations and adopt various radio technologies according to service requirements, use cases and / or available spectrum. 5G mobile communications support a wide range of use cases and related applications, including video streaming, augmented reality, different data sharing methods, and various forms of machine-type applications, including vehicle safety, different sensors, and real-time control. 5G is expected to have multiple radio interfaces, i.e., below 6GHz, cmWave, and mmWave, and can also be integrated with existing traditional radio access technologies such as LTE. Integration with LTE can be implemented as a system, at least in the early stages, in which macro coverage is provided by LTE, and 5G radio interface access comes from small cells by aggregation to LTE. In other words, 5G is planned to support inter-RAT operability (such as LTE-5G) and inter-RI operability (inter-radio interface operability, such as below 6GHz-cmWave, below 6GHz-cmWave-mmWave). One of the concepts considered for use in 5G networks is network slicing, in which multiple independent and dedicated virtual subnetworks (network instances) can be created within the same infrastructure to run services with different requirements for latency, reliability, throughput, and mobility.

[0026] The current architecture in LTE networks is completely distributed in the radio and completely concentrated in the core network. Low-latency applications and services in 5G require content to be close to the radio, which leads to local interruptions and multi-access edge computing (MEC). 5G enables analysis and knowledge generation to occur at the data source. This approach requires the use of resources that may not be continuously connected to the network, such as laptops, smart phones, tablet computers and sensors. MEC provides a distributed computing environment for application and service hosting. It also has the ability to store and process content near cellular subscribers to obtain faster response times. Edge computing covers a variety of technologies, such as wireless sensor networks, mobile data acquisition, mobile signature analysis, collaborative distributed peer-to-peer self-organizing networking and processing, and can also be classified as local cloud / fog computing and grid / mesh computing, dew computing, mobile edge computing, micro-cloud, distributed data storage and retrieval, autonomous self-healing networks, remote cloud services, augmented and virtual reality, data caching, Internet of Things (large-scale connectivity and / or latency critical), critical communications (autonomous vehicles, traffic safety, real-time analysis, time-critical control, healthcare applications).

[0027] The communication system can also communicate with other networks, such as the public switched telephone network or the Internet 112, or utilize services provided by them. The communication network can also support the use of cloud services, for example, at least a portion of the core network operations can be performed as a cloud service (this is in the Figure 1114). The communication system may also include a central control entity or the like which provides facilities for networks of different operators to collaborate, for example in spectrum sharing.

[0028] The edge cloud can be brought into the radio access network (RAN) by leveraging network function virtualization (NVF) and software defined networking (SDN). Using edge cloud can mean access node operations to be performed at least partially in a server, host or node operably coupled to a remote radio head or base station including radio components. The node operations may also be distributed among multiple servers, nodes or hosts. The application of cloud RAN architecture enables RAN real-time functions to be performed on the RAN side (in distributed units DU 104), while non-real-time functions are performed in a centralized manner (in centralized units CU 108).

[0029] It should also be understood that the labor distribution between core network operations and base station operations may be different from that of LTE or even non-existent. Some other technological advances that may be used are big data and all-IP, which may change the way networks are built and managed. 5G (or New Radio, NR) networks are designed to support multiple layers, where MEC servers can be placed between the core and the base station or nodeB (gNB). It should be understood that MEC can also be applied to 4G networks.

[0030] 5G can also make use of satellite communications to enhance or supplement the coverage of 5G services, for example, by providing backhaul. Possible use cases are to provide service continuity for machine-to-machine (M2M) or Internet of Things (IoT) devices or for passengers in vehicles, or to ensure service availability for critical communications and future rail / maritime / aeronautical communications. Satellite communications can make use of geostationary orbit (GEO) satellite systems, but also low earth orbit (LEO) satellite systems, in particular mega constellations (systems in which hundreds of (nano) satellites are deployed). Each satellite 106 in a mega constellation can cover several satellite-enabled network entities creating a terrestrial cell. The terrestrial cell can be created by a terrestrial relay node 104 or by a gNB located on the ground or in a satellite.

[0031] It is obvious to a person skilled in the art that the depicted system is only an example of a part of a radio access system, and in practice, the system may include multiple (e / g)NodeBs, user equipment may access multiple radio cells, and the system may also include other devices, such as physical layer relay nodes or other network elements, etc. In addition, in the geographical area of ​​the radio communication system, multiple different kinds of radio cells and multiple radio cells may be provided. A radio cell may be: a macro cell (or umbrella cell), which is a large cell typically having a diameter of up to tens of kilometers; or a small cell such as a micro cell, a femto cell or a pico cell. Figure 1 The (e / g)NodeB can provide any kind of these cells. A cellular radio system can be implemented as a multi-layer network including several types of cells. Usually, in a multi-layer network, one access node provides one or more types of cells, so multiple (e / g)NodeBs are needed to provide this network structure.

[0032] In order to meet the needs of improving the deployment and performance of communication systems, the concept of "plug and play" (e / g) NodeB is introduced. Generally, in addition to the home (e / g) NodeB (H(e / g) NodeB), the network that can use "plug and play" (e / g) NodeB also includes a home node B gateway or HNB-GW ( Figure 1 (not shown). An HNB gateway (HNB-GW) usually installed in an operator's network can aggregate services from a large number of HNBs back to the core network.

[0033] 6G networks are expected to adopt flexible decentralized and / or distributed computing systems and architectures and ubiquitous computing, where local spectrum licensing, spectrum sharing, infrastructure sharing and intelligent automation management are determined by mobile edge computing, artificial intelligence, short packet communication and blockchain technology. The key features of 6G will include management and control functions for intelligent connections, programmability, integrated sensing and communication, reduced energy usage, trusted infrastructure, scalability and affordability. In addition to these, 6G also targets new use cases that cover the integration of positioning and sensing capabilities into system definitions to unify user experience across the physical and digital worlds.

[0034] In wireless communication networks, operators of the network constantly monitor the operation of the system so that possible problems in system performance can be detected and operation can be evaluated. As mentioned above, the data collected is typically represented as key performance indicators (KPIs).

[0035] In wireless networks, measurement reporting is important to implement many processes such as scheduling and handover. Recent machine learning (ML) has been introduced to enhance the operation of the network. With the integration of ML-based functionalities in the network, measurement reporting is also crucial to monitor the performance of the deployed ML and implement related operations accordingly, such as ML model update or switching to another model or fallback to non-ML operation.

[0036] Typically, the collection and reporting of KPIs is done at an aggregate level. For example, data may be collected and reported averaged over a predefined time window, or averaged over several UEs / cells. Such aggregation is beneficial to facilitate processing and analysis of the collected information, which would otherwise be overwhelming, given the volume of traffic in the network and the number of UEs served.

[0037] However, in some cases where the performance of the network is degrading, such as when ML is utilized, there is a need to obtain more information about the operation of the network, report measurements more deeply, and have deeper analysis to better understand the situation and identify possible solutions.

[0038] Embodiments of the present invention enable adaptive KPI collection in a network. For example, KPIs for ML-based functions of a network may be adaptively collected. Some embodiments relate to processes and associated signaling enhancements for enabling adaptive KPI collection in a wireless access network.

[0039] Figure 2 The flowchart of the embodiment shows an embodiment. The flowchart shows an example of an embodiment applied in an apparatus such as a network element. The apparatus can be, for example, a network element or node that communicates with a terminal device such as an (e / g)NodeB of a communication system. The apparatus can be part of the (e / g)NodeB. For example, the apparatus can be located at the (e / g)NodeB or in a cloud server connected to the (e / g)NodeB.

[0040] In step 200, the apparatus is configured to maintain the ability to collect network performance data using collection strategies from more than one collection strategy. The network performance data may be a KPI of the network. The performance data monitored may be, for example, average UE throughput or average cell spectral efficiency.

[0041] In an embodiment, the network performance data collection policy includes a time granularity for data collection and / or the type of data to be collected.

[0042] In step 202, the apparatus is configured to receive a request to apply a given collection policy from a network element. For example, the network element may be a network element responsible for operation and maintenance OAM in the network or a network data analysis function NWDAF or a management data analysis function MDAF.

[0043] In an embodiment, a request to apply a given policy includes a duration for which the policy is to be applied.

[0044] In step 204, the device is configured to apply the requested collection policy to collect network performance data.

[0045] In step 206, the device is configured to send network performance data to the network based on the applied policy. In an embodiment, not only the collection of KPIs (what and when to collect), but also the transmission of network performance data (e.g., when and how to transmit, such as periodicity) is defined by the applied KPI collection policy. However, in another embodiment, the applied KPI collection policy defines the collection of KPIs, while the transmission of network performance data may be the same for all policies.

[0046] Figure 3 The flowchart of the embodiment shows an embodiment. The flowchart shows an example of an embodiment applied in an apparatus such as a network element. The apparatus may be, for example, a network element communicating with a (e / g)NodeB of a communication network. For example, the network element may be a network element responsible for operation and maintenance OAM or a network data analysis function NWDAF in the network.

[0047] In step 300, the apparatus is configured to send a request to a network node (eg to nodeB) to report network performance data collection policies supported by the network node.

[0048] In step 302, the apparatus is configured to receive, from a network node, a report of policies supported by the network node.

[0049] In step 304, the apparatus is configured to send a request to the network node to apply a given collection policy supported by the network node. In an embodiment, the request to apply the given policy comprises a duration for which the policy is to be applied.

[0050] In step 306, the apparatus is configured to receive network performance data from the network nodes based on the applied policy.

[0051] Therefore, in a network, there may be different network performance data or KPI collection strategies, including configurations of KPI collection. The configuration may include time granularity information and / or the type of KPI to be collected.

[0052] Policies can include different configurations for KPI collection.

[0053] Default monitoring strategy for continuous validation of functionality: for example, KPIs aggregated and reported at a high level of time granularity (such as every 1 hour).

[0054] A "deep" monitoring strategy may include a fine time granularity, such as every 15 minutes. In an embodiment, applying the deep monitoring option may be based on different criteria.

[0055] For example, if a gNB, as an example of a network node, experiences a high load, the gNB may be instructed not to apply deep monitoring. In an embodiment, the request to apply a given collection policy includes an indication that the gNB does not apply the policy or reverts to a previous policy if the load experienced by the gNB is above a given threshold.

[0056] In-depth monitoring options depend on the capabilities of the gNB for KPI / measurement collection.

[0057] The request to apply a given policy may include a fallback to a previously used monitoring option, a default monitoring option, or a time window or trigger condition to switch to in-depth monitoring. Note that although the KPI collection entity (e.g., OAM) may request that a specific policy be applied, the gNB may also or alternatively adjust the policy based on the internal state of the gNB.

[0058] In an embodiment, network elements or entities (eg, core network such as NWDAF and OAM such as MDAF) may be responsible for setting different configurations and enabling advanced analytics using enhanced KPI reporting.

[0059] Figure 4 An embodiment of signaling in conjunction with adaptive KPI collection is shown.As mentioned above, the collection strategy switch may be triggered from the network, for example by a network element responsible for OAM or NWDAF. Figure 4 A generic signaling example is shown and depicts the network element 400 responsible for KPICollectionEntity and the gNB 100 (as Figure 2 signaling between network nodes (example) to enable advanced KPI collection and monitoring.

[0060] The network element 400 sends a request 402 to the gNB 100 to report the KPI collection capabilities from the gNB. For example, these capabilities may relate to KPI type (e.g., which KPIs the gNB can monitor), KPI collection granularity (e.g., how often the KPI can be monitored), duration (e.g., how long the KPI can be monitored), raw measurements (e.g., which measurements the gNB 100 can perform in order to collect the KPI).

[0061] The gNB 100 sends a response 404 to the request. The response may be an indication of the KPI collection capability of the gNB.

[0062] The network element 400 may then perform 406 model monitoring and performance checking. This may include monitoring the performance of the ML algorithms executed in the network.

[0063] The network element 400 may then check 408 which KPI policy among multiple KPI collection policies supported by the gNB should be used to collect the KPIs, and send a request to apply the policy in step 410 .

[0064] Alternatively, if the gNB 100 is already using a KPI collection strategy, the network element 400 may then check 408 whether the KPI collection strategy needs to be switched to another strategy supported by the gNB. In an embodiment, the criteria for switching strategies may be time dependent or dependent on the load of the gNB.

[0065] For example, if you need to get data quickly to troubleshoot whether there is performance degradation in the network, you can switch the monitoring mode to in-depth mode to continuously collect KPIs or change the time interval to speed up the collection of KPIs.

[0066] For example, if the current collection strategy is the default strategy of collecting data at 15 minute intervals, for example, and a performance degradation is detected, it may be necessary to switch to a deep strategy to collect data to determine what action to take. For example, the action could be to switch to a non-ML operation or to perform a retraining of the ML algorithm.

[0067] If the collection strategy needs to be switched, the network element 400 may send a request to switch the strategy in step 410. In an embodiment, the request includes attributes related to the switch. For example, the attributes may be KPI monitoring mode (normal, deep, relaxed), periodicity (coarse, continuous, intermittent), and gNBInteralState (LoadHigh, LoadLow).

[0068] KPI monitoring modes and periodicity can be linked together. If the monitoring mode is "Normal", the periodicity can be coarse, such as once in 15 minutes or 1 hour. For example, if the monitoring mode is "Slack", the periodicity can be intermittent for a given range, such as 5 minutes in an hour if the range is between 8am and 8pm. If the monitoring mode is "Deep", the periodicity can be continuous at a more granular level and intermittent for a given range, such as once in 1 minute or 30 seconds.

[0069] In an embodiment, as part of the request, an indication of the load factor of the gNB 100 may also be a parameter considered in the selection of the KPI monitoring mode. This parameter may be part of a policy. For example, if the gNB has a high load, the indicated deep collection mode may be discarded, or the period for which it is applied may be shortened. Thus, a gNB 100 experiencing a high load may choose to accept or discard the proposed configuration change.

[0070] The gNB acknowledges 412 the request to the network element.

[0071] Next, the gNB sends 414 data with a given period according to the requested mode.

[0072] Network element 400 may perform 416 a KPI analysis based on the received data. This will be explained later.

[0073] When the requested duration has elapsed, the gNB may switch to a normal or default collection strategy and notify 418 the network element.

[0074] Figure 5 An embodiment of signaling in conjunction with adaptive KPI collection is shown, where the network element 400 is the network element responsible for operating and maintaining OAM.

[0075] In this example, steps 402 and 404 are optional because OAM already knows the gNB characteristics. Otherwise, the steps follow Figure 4 steps.

[0076] Figure 6 An embodiment of signaling in conjunction with adaptive KPI collection is shown, wherein the network element 400 is a network element responsible for a network data analysis function NWDAF.

[0077] In this example, the KPI collection policy is triggered from the network element handling the NWDAF, which is a core network element interacting with the NG-RAN via a subscription mechanism.

[0078] NWDAF subscribes 600 to gNB to receive KPI collection capability. Otherwise, the steps follow Figure 4 steps.

[0079] Figure 7 The operations in the model monitoring and performance checking step 406 are shown. It shows network nodes such as gNB 100, a network element 400 responsible for OAM and a network management system NMS 700.

[0080] In an embodiment, in this step, statistical distance calculation is performed on the collected KPIs.

[0081] The gNB 100 is configured to collect KPIs for each KPI collection strategy used, and the data is sent 710 to the data collection module 702 where the data is isolated based on whether the ML algorithm (e.g., KPI collection strategy) is enabled or disabled.

[0082] The analysis toolkit 704 is a platform for data science and machine learning. The RAN optimization algorithm component 706 uses the analysis toolkit and the collected and isolated KPI data. The advanced KPI analysis 708 is periodically triggered by the performance monitoring entity 712 in the NMS 700. The advanced KPI analysis 708 can be part of the RAN algorithm component 706. In an embodiment, box 708 utilizes boxes 704 and 702. In an embodiment, the advanced KPI analysis can be triggered by box 700.

[0083] Figure 8 An example of the operations in the high-level KPI analysis step 416 when performing statistical distance calculations is shown.

[0084] In an embodiment, a statistical distance calculation is used on the collected KPI data to determine whether there are any changes in the KPI when the machine learning function is enabled and when the machine learning function is disabled.

[0085] There are various methods to calculate the statistical distance between two distributions. For example, Hellinger distance is one of the methods known in the art. The distance type is specified as part of the strategy depending on the use case.

[0086] As part of the Hellinger distance determination, changes in the KPI can be determined by comparing moments associated with the KPI. Here, the term "moment" can be defined as the mean, deviation or change in the KPI or KPI distribution when the machine learning algorithm is disabled vs. enabled, possibly specified as part of the policy depending on the use case. On the other hand, "seasons" can be defined by the environment (UE distribution, load, mobility, etc.). These seasons can be recurring trends. How to detect these seasons is outside the scope of this document.

[0087] KPIs are collected continuously for repeating seasons: in one season, machine learning is disabled, and in another season, machine learning is enabled.

[0088] Therefore, in step 800, the KPIs collected with the machine learning functionality disabled are read.

[0089] In step 802, the data is decomposed into seasons, trend residuals and their time ranges.

[0090] In step 804, moments are calculated for each season.

[0091] In step 806, KPIs collected with the machine learning function enabled are read.

[0092] In step 808, the data is decomposed into seasons, trend residuals and their time ranges.

[0093] In step 810, moments are calculated for each season.

[0094] In step 812, a matching (time range) season is searched for from the data obtained with the machine learning functionality disabled. Non-limiting example: the performance of the machine learning model is measured between 2-3 pm on a given day (with machine learning enabled), and again between 2-3 pm on the next day with the machine learning functionality disabled.

[0095] In step 814, the found seasons are compared.

[0096] In step 816, the statistical distance between seasons is searched. For example, to calculate the statistical distance, moments (eg, mean, median) among matching seasons may be utilized.

[0097] In step 818, if the difference between the distance and the statistical distance exceeds a given threshold, a corresponding action may be triggered. The action may be, for example, triggering training of a machine learning algorithm, returning to an operation of disabling the machine learning function, or continuing the current state.

[0098] So, for example, if the statistical distance between KPI distributions is used to analyze the performance of repeated seasons, then moments (mean, median) are collected in the same / matching seasons and these moments between KPI distributions are compared when machine learning features are enabled vs. disabled, and the statistical distance between KPI distributions in similar seasons is calculated. If the difference in the moment and statistical distance exceeds a given specified threshold, which may depend on the use case, then the corresponding action is triggered, which may be to initiate retraining or fall back to traditional, for example.

[0099] Fig.9A An embodiment is shown. The figure shows a simplified example of an apparatus 400 to which an embodiment of the present disclosure is applied. In some embodiments, the apparatus may be a network element. For example, the network element may be a network element responsible for operation and maintenance OAM in the network or a network data analysis function NWDAF or a management data analysis function MDAF.

[0100] It should be understood that the apparatus is described herein as an example illustrating some embodiments. It will be apparent to those skilled in the art that the network element apparatus may also include other functions and / or structures, and not all described functions and structures are required. Although the apparatus has been described as one entity, the different modules and memories may be implemented in one or more physical or logical entities.

[0101] The apparatus 400 of this example includes a control circuit 900 configured to control at least part of the operation of the apparatus.

[0102] The apparatus may comprise a memory 902 for storing data. Furthermore, the memory may store software 904 executable by the control circuit 900. The memory may be integrated in the control circuit.

[0103] The apparatus may include one or more interface circuits 906. The interface circuits are operatively connected to the control circuit 900. The one or more interface circuits 906 may connect the apparatus to other network elements, such as to a (e / g)NodeB of a communication system, in a wired or wireless manner.

[0104] In an embodiment, the software 904 may include a computer program including program code means configured to cause the control circuit 900 of the apparatus to implement at least some of the above-described embodiments.

[0105] Fig. 9B An embodiment is shown. The figure shows a simplified example of a network element / node to which embodiments of the present disclosure are applied. In some embodiments, the network element / node may be a (e / g)NodeB or a part of a (e / g)NodeB of a communication system.

[0106] It should be understood that network elements are described herein as examples illustrating some embodiments. It will be apparent to those skilled in the art that network elements may also include other functions and / or structures, and not all described functions and structures are required. Although network elements have been described as one entity, different modules and memories may be implemented in one or more physical or logical entities.

[0107] The network element 100 of this example includes control circuitry 910 configured to control at least part of the operation of the network element.

[0108] The network element may include a memory 912 for storing data. Furthermore, the memory may store software 914 executable by the control circuit 910. The memory may be integrated in the control circuit.

[0109] The network element may include one or more interface circuits 916, 918. The interface circuits may be operably connected to the control circuit 910. The interface circuits 916 may be a set of transceivers configured to communicate with a UE of a wireless communication network. The interface circuits may be connected to an antenna device (not shown). The network element may also include a connection to a transmitter instead of a transceiver.

[0110] The interface circuit 918 can connect the device to other network elements, such as network elements connected to the communication system, in a wired or wireless manner. The network element can be, for example, a network element responsible for OAM, NWDAF or MDAF in the system.

[0111] In an embodiment, the software 914 may comprise a computer program including program code means configured to cause the control circuitry 910 of the network element to implement at least some of the embodiments described above.

[0112] In an embodiment, Fig. 9C As shown, Fig. 9B At least some of the functions of the apparatus may be shared between two physically separate devices to form an operational entity. Thus, it can be seen that the apparatus depicts an operational entity comprising one or more physically separate devices for performing at least some of the processes described. Therefore, the use of such a shared architecture Fig. 9C The apparatus may include a remote control unit RCU 920, such as a host computer or a server computer, which is operatively coupled (e.g., via a wireless or wired network) to a remote distributed unit RDU 922 located in a base station. In an embodiment, at least some of the described processes may be performed by the RCU 920. In an embodiment, the execution of at least some of the described processes may be shared between the RDU 922 and the RCU 920.

[0113] In an embodiment, the RCU 920 may generate a virtual network through which the RCU 920 communicates with the RDU 922. In general, virtual networking may involve the process of combining hardware and software network resources and network functions into a single software-based management entity (virtual network). Network virtualization may involve platform virtualization, usually combined with resource virtualization. Network virtualization can be classified as external virtual networking, which combines many networks or parts of networks into a server computer or host computer (e.g., RCU). External network virtualization is for optimized network sharing. Another category is internal virtual networking, which provides network-like functions to software containers on a single system. Virtual networking can also be used to test terminal devices.

[0114] In an embodiment, the virtual network can provide a flexible distribution of operations between the RDU and the RCU. In fact, any digital signal processing task can be performed in the RDU or the RCU, and the boundaries of transferring responsibilities between the RDU and the RCU can be selected according to the implementation.

[0115] The steps and related functions described above and in the accompanying drawings are not in absolute chronological order, and some steps may be performed simultaneously or in an order different from the given order. Other functions may also be performed between steps or within steps. Some steps may also be omitted or replaced with corresponding steps.

[0116] The device or controller capable of performing the above steps can be implemented as an electronic digital computer, a processing system or a circuit that can include a working memory (random access memory RAM), a central processing unit (CPU) and a system clock. The CPU may include a set of registers, an arithmetic logic unit and a controller. The processing system, controller or circuit is controlled by a sequence of program instructions transmitted from the RAM to the CPU. The controller may contain a plurality of microinstructions for basic operations. The implementation of the microinstructions may vary according to the CPU design. The program instructions may be encoded in a programming language, which may be a high-level programming language such as C, Java, etc., or a low-level programming language such as a machine language or an assembler. The electronic digital computer may also have an operating system that may provide system services to a computer program written with program instructions.

[0117] As used in this application, the term "circuitry" refers to all of the following: (a) hardware-only circuit implementations, such as implementations in analog and / or digital circuits only, and (b) combinations of circuits and software (and / or firmware), such as (where applicable): (i) a combination of a processor or (ii) portions of a processor / software including a digital signal processor, software, and memory, which work together to enable the device to perform various functions, and (c) circuits, such as a microprocessor or portion of a microprocessor, that require software or firmware to operate, even if the software or firmware is not physically present.

[0118] This definition of "circuitry" applies to all uses of this term in this application. As another example, as used in this application, the term "circuitry" would also cover an implementation of merely a processor (or multiple processors) or portion of a processor and its (or their) accompanying software and / or firmware. The term "circuitry" would also cover, for example and if applicable to the particular element, a baseband integrated circuit or an application processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, or another network device.

[0119] The described embodiments may also be implemented in the form of a computer process defined by a computer program or part thereof. Figure 8Embodiments of the described methods can be implemented by executing at least a portion of a computer program including corresponding instructions. The computer program can be provided as a computer-readable medium including program instructions stored thereon, or provided as a non-transient computer-readable medium including program instructions stored thereon. The computer program can be in source code form, object code form, or some intermediate form, and it can be stored in a carrier, which can be any entity or device capable of carrying the program. For example, the computer program can be stored on a computer program distribution medium readable by a computer or processor. The computer program medium can be, for example, but not limited to, a recording medium, a computer memory, a read-only memory, an electrical carrier signal, a telecommunication signal, and a software distribution packet. The computer program medium can be a non-transient medium. The decoding of the software for executing the illustrated and described embodiments is completely within the scope of ordinary technicians in the art.

[0120] Although the embodiments have been described above with reference to the examples according to the accompanying drawings, it is clear that the embodiments are not limited thereto, but can be modified in several ways within the scope of the appended claims. Therefore, all words and expressions should be interpreted broadly, and they are intended to illustrate rather than limit the embodiments. It is obvious to those skilled in the art that, as technology advances, the inventive concept can be implemented in various ways. In addition, it is clear to those skilled in the art that the described embodiments can, but are not required to, be combined with other embodiments in various ways.

[0121] Some examples of this application are provided below.

[0122] Example 1. A device for communication, comprising: at least one processor; and at least one memory, including computer program code; the at least one memory stores instructions, which, when executed by the at least one processor, enable the device to at least: maintain the ability to collect network performance data using a collection strategy from more than one collection strategy; receive a request to apply a given collection strategy from a network element; apply the requested collection strategy to collect network performance data; and send network performance data to the network based on the applied strategy.

[0123] Example 2. An apparatus according to Example 1, wherein the instructions, when executed by the at least one processor, further cause the apparatus to: receive a request from a network element to report the collection policy supported by the apparatus; and report the supported policy to the network element.

[0124] Example 3. An apparatus according to any of the preceding examples, wherein a network performance data collection policy defines a time granularity for data collection of the network performance data.

[0125] Example 4. An apparatus according to any of the preceding examples, wherein a network performance data collection policy defines the type of data collected for the network performance data.

[0126] Example 5. An apparatus according to any of the preceding examples, wherein the request to apply a given policy defines a duration for which the policy is to be applied.

[0127] Example 6. An apparatus according to Example 5, wherein the instructions, when executed by the at least one processor, further cause the apparatus to: switch to a default network performance data collection policy when the duration of the policy requested by the network element has elapsed.

[0128] Example 7. The apparatus of Example 6, wherein the instructions, when executed by the at least one processor, further cause the apparatus to: send information to the network element that the policy is switched.

[0129] Example 8. An apparatus according to any of the preceding examples, wherein the request to apply a given collection policy includes an instruction for the apparatus to return to a previous policy.

[0130] Example 9. An apparatus comprising: a component for maintaining the ability to collect network performance data utilizing a collection strategy from more than one collection strategy; a component for receiving a request to apply a given collection strategy from a network element; a component for applying the requested collection strategy to collect network performance data; and a component for sending network performance data to a network based on the applied strategy.

[0131] Example 10. A device comprising: at least one processor; and at least one memory storing instructions, which, when executed by the at least one processor, cause the device to at least: send a request to a network node of a network to report a network performance data collection policy supported by the network node; receive a report of the policy supported by the network node from the network node; send a request to the network node to apply a given collection policy supported by the network node; and receive network performance data from the network node based on the applied policy.

[0132] Example 11. An apparatus according to Example 10, wherein the instructions, when executed by the at least one processor, further cause the apparatus to: monitor the performance of the network; determine the given network performance data collection strategy based on the monitored performance, and send the request to the network node to apply the given collection strategy.

[0133] Example 12. An apparatus according to Example 10 or 11, wherein a network performance data collection policy defines a time granularity for data collection of the network performance data.

[0134] Example 13. An apparatus according to Examples 10 to 12, wherein a network performance data collection policy defines the type of data collected for the network performance data.

[0135] Example 14. An apparatus according to any of the preceding Examples 10 to 13, wherein the request to apply the given policy defines a duration for which the policy is to be applied.

[0136] Example 15. An apparatus according to any one of the aforementioned Examples 10 to 14, wherein the instructions, when executed by the at least one processor, further cause the apparatus to: determine, based on the monitored performance, that the performance or load of the network has changed; and, based on the determination, send a request to the network node to apply another collection strategy supported by the network node.

[0137] Example 16. An apparatus according to any one of the preceding Examples 11 to 15, wherein monitoring the performance of the network comprises at least the following items: obtaining key performance indicators of the network while disabling a machine learning function of the network or a part of the network; obtaining key performance indicators of the network while enabling a machine learning function of the network or a part of the network; comparing the key performance indicators and determining a statistical distance between the key performance indicators; and triggering an action in the network if the statistical distance is greater than a given threshold.

[0138] Example 17. A device comprising: a component for sending a request to a network node to report a network performance data collection policy supported by the device; a component for receiving a report of the policy supported by the network node from the network node; a component for sending a request to the network node to apply a given collection policy supported by the network node; and a component for receiving network performance data from the network node based on the applied policy.

[0139] Example 18. A method comprising: maintaining the ability to collect network performance data utilizing a collection strategy from more than one collection strategy; receiving a request to apply a given collection strategy from a network element; applying the requested collection strategy to collect network performance data; and sending network performance data to a network based on the applied strategy.

[0140] Example 19. A method comprising: sending a request to a network node to report a network performance data collection policy supported by the network node; receiving a report of the policy supported by the network node from the network node; sending a request to the network node to apply a given collection policy supported by the NodeB; and receiving network performance data from the network node based on the applied policy.

[0141] Example 20. A computer program comprising instructions which, when executed by an apparatus, cause the apparatus to perform the method according to claim 18 or the method according to claim 19.

Claims

1. A device for communication, comprising: at least one processor; as well as at least one memory including computer program code; The at least one memory stores instructions that, when executed by the at least one processor, cause the apparatus to at least: maintaining the ability to collect network performance data using collection strategies from more than one collection strategy; receiving a request from a network element to apply a given collection policy; applying the requested collection policy to collect network performance data; as well as Network performance data is sent to the network based on the applied policy.

2. The apparatus of claim 1 , wherein the instructions, when executed by the at least one processor, further cause the apparatus to: receiving a request from a network element to report the collection policies supported by the device; The supported policies are reported to the network element.

3. An apparatus according to any one of the preceding claims, wherein a network performance data collection policy defines a time granularity for data collection of the network performance data.

4. An apparatus according to claim 1 or 2, wherein a network performance data collection policy defines the type of data collected for the network performance data.

5. An apparatus according to claim 1 or 2, wherein the request to apply a given policy defines a duration for which the policy is to be applied.

6. The apparatus of claim 5, wherein the instructions, when executed by the at least one processor, further cause the apparatus to: When the duration of the policy requested by the network element has elapsed, switching to a default network performance data collection policy.

7. The apparatus of claim 6, wherein the instructions, when executed by the at least one processor, further cause the apparatus to: A message that the policy is switched is sent to the network element.

8. The apparatus of claim 1 or 2, wherein the request to apply a given collection policy includes an instruction for the apparatus to return to a previous policy.

9. An apparatus comprising: means for maintaining the ability to collect network performance data utilizing collection strategies from more than one collection strategy; means for receiving a request from a network element to apply a given collection policy; for applying the requested collection strategy to a component for collecting network performance data; as well as Means for sending network performance data to the network based on the applied policy.

10. An apparatus comprising: at least one processor; as well as at least one memory storing instructions, which, when executed by the at least one processor, cause the apparatus to at least: sending a request to a network node of a network to report a network performance data collection policy supported by the network node; receiving, from the network node, a report of the policies supported by the network node; sending a request to the network node to apply a given collection strategy supported by the network node; as well as Network performance data is received from the network nodes based on the applied policy.

Images