Operation anomaly detection and isolation in multi-domain communication networks

Abstract

Embodiments include a computer-implemented method for detecting operational anomalies in a multi-domain communication network. Such a method includes obtaining a plurality of time sequences of performance data from a plurality of domains of a communication network and determining one or more models of non-anomalous network behavior based on the plurality of time sequences. Such a method includes classifying each time sequence into a plurality of types based on the presence or absence of at least two types of components in each time sequence. Such a method includes detecting operational anomalies in a plurality of time series or in additional performance data obtained from a plurality of domains of the communication network based on one or more models and classified types. Other embodiments include a network analytics system configured to perform such a method.

Description

Technical Field

[0001] The present disclosure relates generally to communication networks, and more particularly to techniques for detecting operational anomalies (eg, failures, etc.) manifesting across multiple domains of a communication network. Background Art

[0002] The fifth generation (“5G”) cellular system, also known as New Radio (NR), was initially standardized as 3GPP Rel-15 and continues to evolve in subsequent releases. NR was developed for maximum flexibility to support a variety of different use cases, including enhanced mobile broadband (eMBB), machine type communications (MTC), ultra-reliable low latency communications (URLLC), sidelink device-to-device (D2D), and several other use cases. 5G / NR technology has many similarities to the fourth generation LTE.

[0003] At a high level, the 5G system (5GS) consists of an access network (AN) and a core network (CN). The AN provides UE connectivity to the CN via base stations such as gNB or NG-eNB, for example. As described in more detail below, the CN includes various network functions (NFs) that provide a range of different functions such as session management, connection management, billing, authentication, etc.

[0004] The increasing complexity of communication networks, including 5G networks, is driving the development of analytical systems that support the operation, optimization, and planning of these networks. This includes detecting and resolving sudden, unexpected changes (e.g., failures) in network operation and / or performance. These analytical systems, in turn, require the collection and processing of large amounts of data, particularly time series data.

[0005] In general, a time series is a sequence of data or information values, each with an associated time instance (e.g., the time at which the data or information value was generated and / or collected). Data or information can be anything measurable that depends on time in some way, such as price, humidity, or number of people. An important characteristic of a time series is frequency, which is how often the data values ​​of a data set are recorded. Frequency is also inversely proportional to the period (or duration) between consecutive data values.

[0006] Time series analysis includes techniques that attempt to understand or contextualize time series data, such as using models built from past time series data to make forecasts or predictions about future data (or events). To best facilitate such analysis, it is preferred that the time series consist of data values ​​measured and / or recorded at a constant frequency or period.

[0007] Time series data sets may be collected from geographic locations, such as from nodes of a communication network located in one or more geographic regions (e.g., countries, regions, provinces, cities, etc.). For example, the values ​​of performance measurement (PM) counters may be collected from various network nodes at specific time intervals. Time series data collected in this manner may be used to analyze, predict, and / or understand user behavior patterns and network performance trends. Summary of the invention

[0008] However, even with large amounts of available time series data, detecting and resolving sudden, unexpected changes (eg, failures or anomalies) in network operation and / or performance can be very difficult.

[0009] For example, advanced communication networks are robust and distributed, so failures have a relatively limited impact on a subset of users, sessions, and / or network elements, which makes them more difficult to detect. In addition, normal network behavior varies with the time of day, day of the week, month, and / or season. The presence or absence of these trends needs to be considered when detecting abnormal network behavior. In addition, each available time series of data is typically one-dimensional, so that it is collected from a single network node and is not correlated with other data sources. As a result, it is more difficult to detect failures that manifest in multiple network nodes.

[0010] Embodiments of the present disclosure address these and other problems, issues and / or difficulties by providing techniques for detecting and isolating communication network operation anomalies based on relevant data sources from multiple network domains, and corresponding network analysis systems that perform the techniques.

[0011] Some embodiments include methods (eg, processes) for detecting operational anomalies in a multi-domain communication network.

[0012] The exemplary methods may include obtaining a plurality of time series of performance data from a plurality of domains of a communication network. The exemplary methods may also include determining one or more models of non-anomalous network behavior based on the plurality of time series. The exemplary methods may also include classifying each time series into a plurality of types based on the presence or absence of at least two types of components in each time series. The exemplary methods may also include detecting an operational anomaly in the plurality of time series or in additional performance data obtained from the plurality of domains of the communication network based on the one or more models and the classification types.

[0013] In some embodiments, these exemplary methods may further include: based on detecting a plurality of operational anomalies in the additional performance data, determining an order of importance of the detected operational anomalies based on respective deviations from corresponding non-anomalous network behavior. In some of these embodiments, these exemplary methods may further include: in response to one or more detected anomalies being determined to be most important, initiating one or more corrective actions in multiple domains of the communication network. In some of these embodiments, these exemplary methods may further include: in response to one or more detected anomalies being determined to be less important, refraining from initiating one or more further corrective actions in one or more domains of the communication network.

[0014] In some embodiments, classifying each time series based on the presence or absence of at least two types of components comprises the following operations:

[0015] Detecting whether each of the time series includes a seasonal component and / or a non-constant trend component;

[0016] When the time series includes a seasonal component, the time series is classified as type 1;

[0017] Classifying the time series as type II when the time series includes a non-constant trend component but does not include a seasonal component; and

[0018] When a time series includes neither a non-constant trend component nor a seasonal component, the time series is classified as type III.

[0019] Various examples of obtained performance data, determined non-anomalous behavior models, and detection of operational anomalies are disclosed herein.

[0020] Other embodiments include a network analysis system (e.g., NWDAF, SMO node, NM node, cloud system, etc.) configured to perform operations corresponding to any of the exemplary methods described herein. Other embodiments include a non-transitory, computer-readable medium storing program instructions that, when executed by a processing circuit, configure the network analysis system to perform operations corresponding to any of the exemplary methods described herein.

[0021] These and other embodiments described herein can provide a wide range of possibilities for investigating various known network failures as well as fast, automatic detection of still unknown network failures. In this way, embodiments can capture new anomalies early while they are still developing, thereby minimizing their impact on user experience and network performance. In addition, anomaly detection based on learning normal network behavior has significant advantages over conventional, threshold-based alarm systems, because many KPIs depend on factors such as time of day, day of the week, network load, etc. In addition, by monitoring and correlating network-wide key performance indicators (KPIs), embodiments can isolate UEs, data sessions, etc. that are affected by unidentified failures or intercommunication problems. In addition to the more visible network failures typically identified by conventional techniques, embodiments can identify more potential failures and intercommunication problems that are typically missed by conventional techniques.

[0022] These and other objects, features and advantages of embodiments of the present disclosure will become apparent after reading the following detailed description taken in conjunction with the accompanying drawings which are briefly described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a high-level block diagram of an exemplary 5G / NR network architecture.

[0024] Figure 2 An exemplary 5G reference architecture with service-based interfaces and various 3GPP-defined NFs is shown.

[0025] Figure 3 An exemplary multi-domain network including a RAN, a packet-based core network (CN) and an IP Multimedia Subsystem (IMS) is shown.

[0026] Figure 4-7 Various exemplary time series of network performance data collected over a period of approximately four (4) weeks are shown.

[0027] Figure 8 A functional diagram of a network analysis system according to an embodiment of the present disclosure is shown.

[0028] Figure 9-12 An exemplary time series of network performance data and three components extracted from the time series using an embodiment of the present disclosure are shown.

[0029] Fig.13 An exemplary time series including a trend component detected according to an embodiment of the present disclosure is shown.

[0030] Fig.14 Shows Fig.13 The residual component of the time series in after removing the trend component.

[0031] Figure 15-17An exemplary arrangement of upper and lower bounds for anomaly detection for multiple composite time series and two separate time series according to an embodiment of the present disclosure is shown.

[0032] Fig.18 An exemplary implementation of a network analysis system according to an embodiment of the present disclosure is shown.

[0033] Fig.19 A high-level diagram of the Open RAN (O-RAN) architecture is shown.

[0034] Figure 20-21 Two implementation options are shown for integrating embodiments of the present disclosure with the O-RAN architecture.

[0035] FIG. 22 illustrates an exemplary method (eg, process) for detecting operational anomalies in a multi-domain communication network according to various embodiments of the present disclosure.

[0036] Fig.23 A communication system according to various embodiments of the present disclosure is shown.

[0037] Fig.24 A network node according to various embodiments of the present disclosure is shown.

[0038] Fig.25 A host computing system according to various embodiments of the present disclosure is shown.

[0039] Fig.26 is a block diagram of a virtualization environment in which functionality implemented by some embodiments of the present disclosure may be virtualized. DETAILED DESCRIPTION

[0040] Some embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. However, other embodiments are included within the scope of the subject matter disclosed herein, and the disclosed subject matter should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art.

[0041] Generally, all terms used in this article will be interpreted according to their common meaning in the relevant technical field, unless different meanings are clearly given and / or implied from the context of their use. Unless otherwise clearly stated, all references to elements, equipment, components, devices, steps, etc. should be publicly interpreted as referring to at least one instance of elements, equipment, components, devices, steps, etc. The steps of any method disclosed herein do not have to be performed in the exact order disclosed, unless a step is explicitly described as after or before another step and / or implicitly a step must be after or before another step. Any feature of any embodiment disclosed herein can be applied to any other embodiment in appropriate circumstances. Similarly, any advantage of any embodiment can be applied to any other embodiment, and vice versa. Other purposes, features and advantages of the attached embodiments will be apparent from the following description.

[0042] Note that the description herein focuses on 3GPP cellular communication systems and, therefore, 3GPP terminology or terminology similar to 3GPP terminology is often used. However, the concepts disclosed herein are not limited to 3GPP systems. Furthermore, although the term "cell" is used herein, it should be understood (particularly with respect to 5G NR) that a beam may be used in place of a cell and, thus, the concepts described herein are equally applicable to both cells and beams.

[0043] Figure 1 A high-level view of an exemplary 5G network 100 is shown, including a Next Generation RAN (NG-RAN) 199 and a 5G Core (5GC) 198. The NG-RAN 199 can include a set of gNodeBs (gNBs) connected to the 5GC via one or more NG interfaces, such as gNBs 100, 150 connected via interfaces 102, 152, respectively. In addition, the gNBs can be connected to each other via one or more Xn interfaces, such as an Xn interface 140 between gNBs 100 and 150. With respect to the NR interface to the UE, each of the gNBs can support frequency division duplex (FDD), time division duplex (TDD), or a combination thereof. Each of the gNBs can serve a geographic coverage area including one or more cells, and in some cases, various directional beams can also be used to provide coverage in each cell.

[0044] NG-RAN 199 is layered into Radio Network Layer (RNL) and Transport Network Layer (TNL). The NG-RAN architecture, i.e., NG-RAN logical nodes and the interfaces between them, is defined as part of the RNL. For each NG-RAN interface (NG, Xn, F1), the relevant TNL protocols and functions are specified. TNL provides services for user plane transport and signaling transport.

[0045] Figure 1 The NG RAN logical nodes shown include a central unit (CU or gNB-CU) and one or more distributed units (DU or gNB-DU). For example, gNB 100 includes gNB-CU 120 and gNB-DUs 120 and 130. The CU (e.g., gNB-CU 120) is a logical node that hosts higher layer protocols and performs various gNB functions such as controlling the operation of the DU. The DU (e.g., gNB-DU 120, 230) is a decentralized logical node that hosts lower layer protocols and, depending on the functional split option, may include various subsets of gNB functions. The gNB-CU is connected to one or more gNB-DUs via respective Fl logical interfaces (e.g., 122 and 132).

[0046] One change in 5G networks (e.g., in 5GC) is that the traditional peer-to-peer interfaces and protocols established in earlier generation networks are modified and / or replaced by a service-based architecture (SBA), in which a network function (NF) provides one or more services to one or more service consumers. This can be done, for example, through a Hyper Text Transfer Protocol / Representational State Transfer (HTTP / REST) ​​application programming interface (API). Typically, various services are self-contained functions that can be changed and modified in an isolated manner without affecting other services.

[0047] In addition, services are composed of various "service operations", which are finer divisions of the overall service functionality. The interaction between service consumers and producers can be of "request / response" or "subscribe / notify" type. In 5G SBA, the network repository function (NRF) allows each network function to discover services provided by other network functions, and the data storage function (DSF) allows each network function to store its context. This 5G SBA model is based on principles including modularity, reusability, and NF self-containment, which can enable network deployments to take advantage of the latest virtualization and software technologies.

[0048] Figure 2 An exemplary non-roaming architecture of a 5G network (200) with service-based interfaces and various 3GPP-defined NFs is shown. These include the following NFs, with additional details provided for those most relevant to the present disclosure:

[0049] The Application Function (AF, with Naf interface) interacts with the 5GC to provide information to the network operator and subscribe to specific events occurring in the operator's network. The AF provides applications that deliver services in a layer different from the layer that requested the service (i.e., the transport layer), controlling flow resources based on what is negotiated with the network. The AF transmits dynamic session information to the PCF (via the N5 interface), including a description of the media to be delivered by the transport layer.

[0050] Policy Control Function (PCF, with Npcf interface) supports unified policy framework to manage network behavior by providing PCC rules (e.g., about the processing of each service data flow under PCC control) to SMF via N7 reference point. PCF provides policy control decisions and flow-based charging control, including service data flow detection, gating, QoS and flow-based charging (except credit management) to SMF.

[0051] The PCF receives session and media related information from the AF and notifies the AF of the traffic (or user)

[0052] Plane events.

[0053] The User Plane Function (UPF) supports the processing of user plane traffic based on the rules received from the SMF, including packet inspection and different implementation actions (e.g., event detection and reporting). The UPF communicates with the RAN (e.g., NG-RNA) via the N3 reference point, with the SMF (discussed below) via the N4 reference point, and with the external packet data network (PDN) via the N6 reference point. The N9 reference point is used for communication between two UPFs.

[0054] Session Management Function (SMF, with Nsmf interface) interacts with the decoupled traffic (or user) plane, including creating, updating and removing protocol data unit (PDU) sessions and managing session context with the user plane function (UPF), for example, for event reporting. For example, SMF performs data flow detection (based on filter definitions included in PCC rules), online and offline charging interactions, and policy enforcement.

[0055] The Charging Function (CHF, with Nchf interface) is responsible for integrating online and offline charging functions. It provides quota management (for online charging), re-authorization triggers, rating conditions, etc., and receives notifications about usage reports from the SMF.

[0056] Quota management involves granting a specific number of units (e.g., bytes, seconds) to a service.

[0057] Also interacts with billing systems.

[0058] Access and Mobility Management Function (AMF, with Namf interface) terminates the RAN CP interface and handles all mobility and connection management of the UE (similar to the MME in EPC). AMF communicates with the UE via the N1 reference point, with the SMF via the N11 reference point, and with the RAN (e.g., NG-RAN) via the N2 reference point.

[0059] Network Exposure Function (NEF), with Nnef interface

[0060] NEF - Acts as an entry point into the operator's network by securely exposing network capabilities and events provided by 3GPP NFs to the AF and by providing a way for the AF to securely provide information to the 3GPP network. For example, the NEF provides services that allow the AF to provide specific subscription data (e.g., expected UE behavior) for various UEs. In general, the services provided by the NEF are similar to those provided by the SCEF in the EPC.

[0061] Network Repository Function (NRF), with Nnrf interface - provides service registration and discovery, enabling NFs to identify appropriate services available from other NFs.

[0062] Network Slice Selection Function (NSSF) with Nnssf interface - A "network slice" is a logical partition of a 5G network that provides specific network capabilities and features, e.g., to support a specific service. A network slice instance is a set of NF instances and required network resources (e.g., compute, storage, communication) that provide the capabilities and features of a network slice. NSSF enables other NFs (e.g., AMF) to identify a network slice instance that is suitable for the service desired by the UE.

[0063] Authentication Server Function (AUSF) with Nausf interface - Based in the user's home network (HPLMN), it performs user authentication and computes security key material for various purposes.

[0064] Network Data Analytics Function (NWDAF),

[0065] Has Nnwdaf interface - interacts with other NFs to collect relevant data and provides network analysis information (e.g., statistics and / or prediction information of past events) to other NFs.

[0066] Location Management Function (LMF) with Nlmf interface - supports various functions related to the determination of UE location, including location determination for UE and obtaining any of the following: DL location measurements or location estimate from UE; UL location measurements from NG RAN; and non-UE associated assistance data from NG RAN.

[0067] The Unified Data Management (UDM) function supports the generation of 3GPP authentication credentials, user identity processing, access authorization based on subscription data, and other subscriber-related functions. To provide this functionality, UDM uses subscription data (including authentication data) stored in the 5GC Unified Data Repository (UDR). In addition to UDM, UDR also supports the storage and retrieval of policy data by PCF and the storage and retrieval of application data by NEF. The terms "UDM" and "UDM function" are used interchangeably in this article.

[0068] The IP Multimedia Subsystem (IMS) is an architectural framework for delivering multimedia services to wireless devices based on these Internet-centric protocols. IMS was originally specified by the 3rd Generation Partnership Project (3GPP) in Release 5 (Rel-5) as a technology for evolving mobile networks beyond GSM, for example, for delivering Internet services over GPRS. IMS has evolved in subsequent releases to support other access networks and a wide range of services and applications.

[0069] At a high level, the functions of an IMS network can be broken down into two types: control and media, as well as application enablers. Control functions include the Call Session Control Function (CSCF) and the Home Subscriber Server (HSS). CSCF is used for session control of devices and applications using the IMS network. Session control includes secure routing of Session Initiation Protocol (SIP) messages, subsequent monitoring of SIP sessions, and communication with the policy architecture to support media authorization. CSCF functions can also be divided into Proxy CSCF (P-CSCF), Serving CSCF (S-CSCF), and Inquiring CSCF (I-CSCF).

[0070] The CSCF also interacts with the HSS to support network entities that handle calls and sessions. The HSS is a master database containing user and subscriber information. For example, the HSS provides functions such as identity handling, access authorization, authentication, mobility management (e.g., which session control entity is serving the user), session establishment support, service provisioning support, and service authorization support.

[0071] The Media Resource Function (MRF) can provide media services in the user's home network and can manage and process media streams such as voice, video, voice to text, and real-time transcoding of multimedia data. Typically, a WebRTC gateway allows local and browser-based devices to securely access services in the network.

[0072] As briefly mentioned above, the increasing complexity of communication networks, including 5G networks, has driven the evolution of analysis systems that support the operation, optimization and planning of these networks. This includes detecting and resolving sudden, unexpected changes (e.g., failures) in network operation and / or performance. Advanced analysis systems are required to collect and correlate basic network events from different network domains such as CN, RAN, and transport networks. Such analysis systems calculate user-level and session-level E2E service quality metrics (S-KPIs) as well as radio and network resource metrics (R-KPIs) that characterize the radio environment or network operation at the user-level and session-level.

[0073] Figure 3 An exemplary multi-domain network (300) including UE, RAN, packet-based CN and IMS is shown. Figure 3 As shown, the RAN includes an eNB that provides an LTE-Uu radio interface to the UE and a gNB that provides an NR-Uu interface. The CN includes the SMF, AMF, and UPF in the 5GC discussed above, as well as the Mobility Management Entity (MME), Serving Gateway (SGW), and Packet Gateway (PGW) that are part of the Evolved Packet Core (EPC) associated with the LTE network. The UPF is connected to the IMS via the N6 interface, so that Figure 3 IMS in Figure 2 An example of a PDN is shown in FIG.

[0074] Figure 3 Also shown are various "tapping points" from which data can be collected from the three domains of the network. For example, node events (e.g., PM counters) can be collected from eNB, gNB, AMF, SMF, UPF, MME, and PGW. Similarly, interface events can be collected from the S5-U (user), S5-C (control), Sl-U, and S5-U interfaces in the CN, and the Mw interface between the P-CSCF and IS-CSCSF in the IMS. In addition to detecting events and / or conditions at independent nodes and / or interfaces, some more advanced analysis systems combine information collected from multiple domains to determine a "user experience" analysis, which represents the performance experienced by the end user for a particular service.

[0075] Time series data sets can be collected from various nodes and various interfaces in multiple domains of a communication network. Time series data collected in this manner can be used to analyze, predict, and / or understand user behavior patterns and network performance trends. However, even with a large amount of available time series data, it can be very difficult to detect and resolve sudden, unexpected changes (e.g., failures or anomalies) in network operation and / or performance. For example, advanced communication networks such as Figure 3 The exemplary network shown) is robust and distributed, whereby failures have a relatively limited impact on a subset of users, sessions, and / or network elements, making them more difficult to detect.

[0076] A conventional approach to fault detection and troubleshooting involves manually searching for faulty elements using the various types of filtering options provided by network monitoring and analysis tools. In general, these advanced tools provide the ability to investigate a variety of network problems. However, finding unknown problems is nearly impossible using only random searches in the available data.

[0077] Another approach is fixed alert thresholds set for various network KPIs or metrics. This can be used for problematic conditions and / or to avoid manual searches. However, there is a trade-off between sensitivity and false alarms. If the threshold is set too low, the system is overloaded with a large number of alarms; if it is set too high, only highly severe problems are detected, and often later than desired.

[0078] Another commonly used approach is anomaly detection, which sets alerts based on the observed distribution of network KPIs or metrics. In this way, events that are outliers (in some statistical sense) relative to typical or normal values ​​will be detected.

[0079] Even so, these techniques are not always successful. What is considered "normal" network behavior (based on a KPI or metric) varies with the time of day, day of the week, month, and / or season, as well as due to network load and many other variables. The presence or absence of these trends needs to be considered when detecting anomalous network behavior. Furthermore, different KPIs and metrics may have different variability or dependencies on these factors.

[0080] Furthermore, each time series of data collected from a multi-domain network (e.g. Figure 3 ) is typically one-dimensional, such that it is collected from a single network element (e.g., node, interface, etc.) and is not correlated with other data sources. While this supports the detection of faults that have a measurable impact on a single network element, it is difficult to detect faults that manifest themselves in multiple network elements.

[0081] Other existing technologies also have similar shortcomings. U.S. Patent 8200193 describes a UE-based technology for identifying abnormal traffic generated by a unique UE, but does not detect network-level problems. U.S. Patent Publications 2021 / 0058424 and 2020 / 0106795 disclose technologies for anomaly detection in communication networks, which focus on performance metrics of individual elements (e.g., microservices or nodes) without considering multi-dimensional network structures or behavioral differences between time series data (e.g., periodicity, trends, etc.). U.S. Patent 7460498 describes a technology for detecting problems with fixed telecommunication lines based on measurements of individual network elements, also without considering multi-dimensional network structures.

[0082] Embodiments of the present disclosure address these and other problems, issues and / or difficulties through novel, flexible and efficient techniques for detecting and isolating communication network operation anomalies based on relevant time series data sources from various network domains, and corresponding network analysis systems that perform such techniques. Some aspects include:

[0083] Robust learning of time series behavior, exploiting the relationships of various time series describing multiple marginal distributions of KPIs from complex networks to correct errors and anomalies in the training data;

[0084] Classify time series with respect to their seasonality, presence and / or trend change points;

[0085] Targeted anomaly detection applied to time series of various categories; and

[0086] Isolate network problems and / or anomalies within the multi-dimensional space represented by the time series by finding the filters that highlight the greatest impact on the monitored KPIs.

[0087] Correlation of data from multiple sources per user session enables filtering by multiple dimensions and combinations thereof. For example, embodiments may support calculation of dropped call rate for UEs from vendor A on cells from RAN vendor B, or video quality for users of service provider C in region D.

[0088] Each time series of collected data can be considered as a marginal distribution of network performance or user experience within a specific dimension, where the complete network performance is represented by a multidimensional set of time series with unknown relationships between them (i.e., between marginal distributions).

[0089] An embodiment applies anomaly detection to a multi-dimensional set of time series to automatically detect problems during network operations.Monitored network performance metrics and user experience KPI time series are first classified according to the presence of seasonal and trend components, and anomaly detection first learns normal network behavior.

[0090] The relationship between the time series of multi-dimensional systems is used to ensure robustness when learning the normal behavior of the network in an unsupervised system. In some embodiments, an underfitting machine learning (ML) model trained based on L1 regularization can be used to suppress the impact of anomalies in the training data. This approach provides intelligent noise filtering capabilities and allows the ML model to learn periods of normal behavior without capturing minor anomalies that only exist in a subset of other related time series.

[0091] Network problems or abnormal behavior correspond to detected anomalies. Embodiments may apply filtering and sorting to these network anomalies to distinguish, for example, abnormal network operations and abnormal network loads. The dependency of observed metrics and KPIs on underlying traffic may also be considered.

[0092] In some embodiments, marginal distributions of KPIs for multiple dimensions (and combinations of dimensions) are used to isolate problematic network elements on the end-to-end data path by identifying their contribution to the observed performance degradation. Network failures typically affect multiple identifiable groups of subscribers (i.e., marginal distributions of a certain KPI) as "side effects" beyond the actual trigger or root cause of the problem. For example, a severe YouTube service outage (root cause) may affect video QoE metrics (side effects) for all Apple terminals.

[0093] Embodiments may provide various benefits and / or advantages. For example, embodiments provide almost unlimited possibilities to investigate various known network failures, but also provide fast, automatic anomaly detection for still unknown network failures. In this way, embodiments may catch new anomalies early while they are still developing, thereby minimizing their impact on user experience and network performance.

[0094] Furthermore, anomaly detection based on learning normal network behavior has significant advantages over conventional, threshold-based alarm systems, since many KPIs depend on factors such as time of day, day of the week, network load, etc. Having thresholds that adapt to these factors significantly increases the reliability of fault detection. Furthermore, embodiments utilize a learning system that reduces and / or eliminates the impact of training time errors on fault detection during operation.

[0095] Furthermore, by monitoring and correlating KPIs for network-wide QoS / QoE, embodiments can accurately isolate UEs, data sessions, etc. that are affected by unidentified faults or interworking issues. In addition to the more visible network element faults typically identified by conventional FM / PM techniques, embodiments can also identify more potential faults and interworking issues that are typically missed by these conventional techniques.

[0096] Time series data collected in communication networks (e.g. Figure 3 ) can have various formats, characteristics and / or modes. Figure 4-7 Various exemplary time series collected over a period of approximately four (4) weeks are shown. Figure 4 The time series in has a daily pattern with peak times and minimum turnover points, while Figure 5 The time series in has a more random pattern but includes single events represented by peaks. Figure 6 The time series in also has a random pattern, but also includes a non-constant trend component. Finally, Figure 7 The time series in has a similar Figure 4 daily pattern, but also includes similar Figure 6 The non-constant trend component of .

[0097] Embodiments of the present disclosure may detect anomalies in time series data having these and other formats, features, and / or patterns. Figure 8 FIG. 4 is a functional diagram of a network analysis system according to an embodiment of the present disclosure. This exemplary system includes various modules or functions for filtering anomalies (indicating network problems) and ranking them based on their importance.

[0098] The input time series generator function (810) delivers formatted time series data. This module can correlate and / or aggregate data of any given granularity (e.g., minutes, hours, days, etc.) and any dimension (e.g., UE vendor, network region, user subscription type, carrier frequency, etc.) and their combinations (vendor-model-operating system-IMEI software version number, function-service provider, tracking area-service provider, etc.). Each correlated and / or aggregated time series generated by this function can be considered as a "marginal distribution" of the behavior of the multi-dimensional system on one or more dimensions. The output of this function is the data source for the rest of the system.

[0099] The time series behavior module (820) classifies each time series output by the input time series generator function according to its behavior. The system then performs different processing on each time series based on this classification.

[0100] For example, time series can be classified into four categories based on the presence of a non-constant trend component and / or a seasonal component. The distinction between seasonal and non-seasonal data is necessary because treating seasonal data as non-seasonal data results in failure to detect anomalies during non-busy hours. In the case of seasonal data, the presence or absence of a trend component does not affect subsequent analysis because the detector can process trend and seasonality together. However, in the case of non-seasonal data, the presence or absence of a trend component results in different processing, as described below.

[0101] The classification performed in this module can be implemented in a variety of ways. Some statistical tests for seasonality include the Welch test (a two-sample location test for testing the assumption that two populations have equal means) and the QS test (a variant of the Ljung-Box text calculated from seasonal lags, considering only positive autocorrelations). Some statistical tests for the trend component include stationarity tests and the Kolmogorov-Smirnov test. ML techniques such as autoencoders can also be used, which are artificial neural networks (NNs) that can learn patterns in data in an unsupervised manner.

[0102] The robust filtering module (830) applies unsupervised learning techniques to suppress anomalous behavior in each time series during training. This facilitates accurate prediction of "normal" behavior even when the training data includes anomalies.

[0103] A major weakness of (unsupervised) anomaly detection techniques is their sensitivity to anomalies in the training data. In such cases, conventional analysis systems fail to learn “normal” network behavior, and training-time anomalies will shift predictions.

[0104] Embodiments of the present disclosure provide robustness against training time errors by exploiting the fact that the system is not fed by a set of independent time series, but rather by time series of different marginal distributions (e.g., of KPIs) as a complex multi-dimensional system. Since the actual relationship between these marginal distributions is not known in advance, embodiments apply ML models to learn the "normal behavior" of all time series data in the context of a large multi-dimensional system.

[0105] In some embodiments, this can be accomplished by using an intentionally underfit ML model (or system) based on L1 regularization applied to the weights of the NN comprising the ML model. L1 regularization minimizes the combined loss function of the NN weights and the norm of the NN weights, and promotes sparsity, where some weights have an optimal value of zero. This can be thought of as intelligent "noise filtering", where predictions are made based on the main features of the time series through "typical" values ​​of the time series, which represent "normal" (i.e., non-abnormal) behavior of the network. Such features include dimensions, combinations of dimensions, and time-domain features, such as time of day, day of week, etc.

[0106] If the time series was previously classified as seasonal (with or without trend), it is input to the seasonal time series decomposer module (840), which identifies any included seasonal behavior and trends, removes these effects from the time series, and makes a forecast. By removing seasonal behavior and trends, any forecast errors are independent and identically distributed (iid).

[0107] The behavior of the trend component can change over time, and there may be complex seasonal patterns in the data. Therefore, it is necessary to identify trend change points and multiple seasonal patterns, such as daily / weekly seasonality. Some embodiments may apply Bayesian inference methods to extract this information from such complex structures.

[0108] An exemplary Bayesian inference method is the Facebook Prophet algorithm, which uses a Markov Chain Monte Carlo (MCMC) sampling algorithm to fit and predict time series data. In this method, the time series is represented as y(t)=s(t)+g(t)+h(t)+ε(t), where t represents time, s(t) represents the daily / weekly seasonal component (assumed to be a Fourier series), g(t) represents the trend component (assumed to be piecewise linear), h(t) represents the holiday component, and ε(t) represents the error component. It is assumed that the model parameters follow a predefined distribution. In a variant, the trend component can be multiplicative instead of additive, and the time series can be represented as y(t)=g(t)*(1+s(t)+h(t))+ε(t).

[0109] Prophet uses a Bayesian model to find the best parameters for the data (e.g., intercept, current initial slope, delta between slopes). The process starts with a "prior" that represents the assumed value of the parameter before seeing the data. Given this prior and the data, the Bayesian model returns a "posterior," an updated confidence in the parameter value (i.e., with maximum probability). More specifically, Prophet uses a prior with a Laplace statistical distribution. It is known that the maximum a posteriori (MAP) estimate of a Bayesian model with a Laplace prior is equivalent to linear regression with LI regularization.

[0110] Fig. 9 An exemplary time series collected over a period of approximately four (4) weeks is shown. Figure 10-12 The following are respectively shown according to the embodiments of the present disclosure. Fig. 9 The trend component g(t), seasonal component s(t) and error component ε(t) extracted from the time series shown.

[0111] If the time series was previously classified as non-seasonal with trend, it is input to a detrending module (850) which identifies and removes the included trend to obtain an error term for the time series.

[0112] These time series do not possess any kind of seasonal behavior, or at least not behavior that is too complex to be learned by computationally efficient statistical or ML techniques. Therefore, these time series cannot be treated with seasonal models such as those mentioned above; otherwise, the noise will be treated as a missing seasonal component s(t), which will produce an incorrect decomposition of the error component ε(t). On the other hand, treating these time series as pure noise and learning population-wide behavior will prevent anomaly detection for any specific period. For example, rescaling in the presence of a trend will compress the entire time series, bypassing time windows around the median or mean for a given period of time.

[0113] Embodiments of the present disclosure overcome these difficulties by decomposing a non-seasonal time series with a trend into trend and error components. Embodiments model the trend component as piecewise linear to accommodate trend changes over time. Fig.13 An exemplary time series is shown in which trends (detected according to embodiments of the present disclosure) are superimposed as piecewise linear functions.

[0114] Some embodiments may apply Bayesian inference methods to extract trend information from time series. For example, a time series may be represented as y(t)=g(t)+h(t)+ε(t) or y(t)=g(t)*(1+h(t))+ε(t), where t represents time, g(t) represents a trend component (which is assumed to be piecewise linear), h(t) represents a holiday component, and ε(t) represents an error component.

[0115] In these embodiments, the detrending module estimates the trend and holiday components of the time series and removes them using Bayesian inference, for example:

[0116]

[0117] These residual components are estimates of the error function ε(t), and are typically noise with some statistical distribution (e.g., white). Fig.14 Shown in the removal of the same Fig.13 The trend component shown in Fig.13 The residual component of the time series in .

[0118] These time series can be processed by the anomaly detector module 2 (860). If the time series was previously classified as non-seasonal with no trend, it is directly input to this module.

[0119] Anomaly Detector Module 1 (870) and Anomaly Detector Module 2 (860) can run in parallel based on their respective time series inputs from the other modules described above. These modules learn the normal behavior in their respective time series and detect anomalies based on comparing the deviation of any time series with the actual predictability of other similar time series. The result of both anomaly detector modules is to trigger or flag any abnormal time period on the respective time series.

[0120] The anomaly detector module 1 (870) utilizes various metrics for each input time series, such as the rescaled error or trend component. To make anomaly detection more robust, the module operates on all input time series simultaneously, rather than analyzing them individually. The specific anomaly detector algorithm used depends on the metric chosen.

[0121] For example, when the error component is the chosen metric, any detector that assumes a white noise process can be used. As a more specific example, methods based on ML or extreme value theory can be used. More specifically, the second of these methods is based on the Fisher-Tippett-Gnedenko theorem, which states that the maximum value of an iid random variable has the same type of ("extreme value") distribution, regardless of the distribution of the original random variable. In addition, the original data can be used to estimate the parameters of the maximum value distribution. Since the maximum value corresponds to extreme events, this method is convenient for estimating the extreme event distribution. Note that the Fisher-Tippett-Gnedenko theorem is similar to the central limit theorem for the sum of iid random variables.

[0122] Fig.15 shows an exemplary arrangement in which anomaly detector module 1 has created upper and lower bounds for multiple time series of data being analyzed simultaneously, while Figure 16-17 The upper and lower bounds created for the two separate time series are shown, along with the actual values ​​of those individual time series. Fig.16 One data point in the time series that may be detected as an anomaly is shown.

[0123] Anomaly Detector Module 2 (860) processes time series data with white noise characteristics without trend or seasonality, whose distribution is similar to normal distribution. This module groups time series by KPI and filters each KPI in multiple dimensions (i.e. marginal distribution). Clustering algorithm is used to identify outliers.

[0124] For example, this module can be implemented using Density-Based Spatial Clustering of Applications with Noise (DBSCAN), a density-based nonparametric clustering algorithm. Given a set of points in some space, DBSCAN clusters points that have many nearby neighbors together, while marking points that are in low-density areas and whose nearest neighbors are too far away by some metric as outliers.

[0125] Clustering with two dimensions (KPI value, timestamp) may be feasible; since there is no significant autocorrelation in the data, these can be treated as independent data points. The KPI value and timestamp dimensions can be rescaled to approximately the same scale before input to the clustering algorithm. Different scaling techniques can be used, including Z-scores based on the distance from the data mean divided by the data variance.

[0126] While clustering algorithms such as DBSCAN can identify any number of clusters, this module has the goal of identifying only outliers and non-outliers. In this way, the module can identify a "main" cluster at the center (e.g., near the mean) and one or more other clusters farther from the mean. The other clusters are post-processed to identify whether they are outliers or belong to the main cluster. This module can be implemented as a streaming algorithm so that new data points can be labeled as outliers or non-outliers shortly after they arrive.

[0127] Subsequently, the anomaly ranking module (880) ranks any detected anomalies based on their exposure, frequency, and importance in the network. In other words, the anomalies are ranked and filtered by attributes such as their deviation from normal values, the duration of the deviation, and / or the impact of the deviation on the network (e.g., the number of affected subscribers, the affected traffic, the value of the affected service, etc.). In this way, the module attempts to identify the most relevant and / or significant anomalies, including possible "root cause" anomalies and possible side effects. For example, among the anomalies detected at the same time, the anomalies with the highest deviation from normal behavior are generally the anomalies that have the greatest impact on sessions and / or subscribers. These anomalies are indicated as root causes, while other detected anomalies indicate side effects of these root causes.

[0128] In other words, the anomaly ranking module identifies and ranks the most significant anomalies with the most specific filtering dimensions. This corresponds to the marginal distribution with the highest deviation from its respective normal behavior. This ranking can be utilized by the user interface (UI 890) to filter and / or classify anomalies to focus on the most relevant network issues.

[0129] Fig.18 An exemplary implementation of a network analysis system (1800) according to an embodiment of the present disclosure is shown. In particular, the implementation is directed to a cloud computing (or more simply "cloud") environment.

[0130] Fig.18 The cloud implementation shown includes a time series generator module (1810), a time series classification module (1820), a robust filtering module (1830), one or more anomaly detector modules (1840), and an anomaly ranking module (1850). These modules can perform the same Figure 8 The functions / operations are similar to those of the corresponding modules in the example, but are implemented using interfaces and parallel processes that can be adjusted for the cloud environment. The training and calling methods are described below for this embodiment.

[0131] The system receives input data through a streaming service module and collects the input data over a given time period. A time series generator module can trigger streaming aggregation to generate M different single-dimensional or multi-dimensional time series. These are input into a time series behavior detector module, which classifies each time series according to behavior and sends K time series and classification metadata to a persistent database.

[0132] When the time series behavior detector module is complete, it triggers a robust filtering module, which identifies frequent patterns and commonalities in the M time series and purges undesired behavior. This can be done, for example, based on unsupervised learning techniques.

[0133] These M more robust time series are used, for example, for regression and training of different model types that can be predefined in the system. If there are M time series, there can be at most M model types, but Fig.18 the arrangement in

[0134] assumes K < M model types. The number of model types also depends on the available resources of the cloud computing platform. Regression and training for K different model types can be performed in parallel.

[0135] After regression and training, the M time series and their learned behavior (i.e., the K model types) are handed over to anomaly detection modules, shown here as one for each model type. These modules detect anomalies based on the K models in any of the ways described above.

[0136] A scoring proxy encapsulates the K models as service models, which can be used for batch-based and streaming-based prediction processing via a REST API. The scoring proxy REST API also writes the detected anomalies into a persistent database associated with the original time series data collected from the network. This persistent database is then queried by an anomaly ranking module, which will trigger a UI for ranking for the end user. All results at this point can be transmitted to the persistent database for use by the final UI.

[0136] The Open RAN (O-RAN) Alliance is a community of mobile operators and RAN vendors working towards an open, intelligent, virtualized, operationally efficient, and fully interoperable RAN. To achieve these goals, the community has defined an O-RAN architecture with key functions and interfaces. The O-RAN Working Group (WG) publishes various specifications. For example, O-RAN WG1 deals with use cases and the overall architecture. A general principle is that the O-RAN architecture and interface specifications should be as consistent as possible with 3GPP architecture and interface specifications.

[0137] Fig.19The high-level O-RAN architecture and four key interfaces Al, O1, Open Fronthaul M-Plane and O2 are shown. These interfaces connect the Service Management and Orchestration (SMO) framework to the O-RAN Network Function (NF) and the Open Cloud (O-Cloud). In addition, there is an interface between the SMO and external systems to provide enriched data. The NG interface between the O-RAN NF and the NG-Core is also shown, which is similar to the Figure 1 The NG interface of the 5GC shown is consistent.

[0138] The O-RAN architecture description defines the following three control loops with their own latency:

[0139] Real-time (RT) control loop (<10ms);

[0140] Near RT RIC control loop (10-1000ms); and

[0141] Non-RT RIC control loop (>1000ms).

[0142] The use cases for non-RT RIC and near-RT RIC control loops are fully defined by O-RAN, but for the RT control loop (which performs radio scheduling, HARQ, beamforming, etc.) O-RAN only defines the relevant interactions with other O-RAN nodes or functions.

[0143] The non-RT RIC provides an A1 interface to the near-RT RIC. One of the tasks of the non-RT RIC is to provide policy-based guidance, machine learning (ML) model management, and enriched information to support intelligent RAN optimization (e.g., for radio resource management, RRM) of the near-RT RIC. The non-RT RIC can also perform intelligent RRM in longer non-RT intervals (e.g., greater than 1 second).

[0144] The non-RT RIC can use data analytics and artificial intelligence (AI) / ML training and inference to determine RAN optimization, for which it can leverage SMO services such as collecting data from and providing data to O-RAN nodes. These actions are performed by non-RTRIC applications (rAPPs). The non-RT RIC also includes a non-RT RIC framework, which, inside the SMO framework, logically terminates the A1 interface and exposes all required functions and services to the rAPPs.

[0145] As currently described, the O-RAN architecture does not include any components and / or interfaces that enable input data streams from existing data collection components to be used for cross-domain association. For example, the SMO non-RT RIC component does not have any data interface towards domains outside the RAN. More generally, input data from non-RAN domains (e.g., CN, applications, etc.) is outside the scope of the O-RAN. Even so, different possible implementation options for integrating embodiments of the present disclosure into the O-RAN architecture are described below.

[0146] Fig. 20 A first implementation option for integrating an embodiment of the present disclosure in a multi-domain network (2000) including an O-RAN architecture is shown. In this option, an anomaly detector (2010) with cross-domain data correlation runs on an AI server outside the SMO (e.g., on a public or private cloud computing environment) and has an external interface to the SMO. The anomaly detector also has an external interface that facilitates the collection of data from other domains such as CN (e.g., 5GC), IMS, etc.

[0147] Fig.21 A second implementation option for integrating embodiments of the present disclosure in a multi-domain network (2100) including an O-RAN architecture is shown. In this option, an anomaly detector (2110) with cross-domain data correlation runs in a non-RT RIC and optionally runs partially in a near-RT RIC. For example, "training" components such as time series behavior detection, decomposers, and detrending can run in a non-RT RIC, with anomaly detection logic running in either the non-RT RIC or the near-RT RIC, depending on latency requirements.

[0148] Both implementation options apply to the case where data collection and correlation is performed only within the RAN, as well as to the case where data collection and correlation is also performed across other network domains (eg, CN and IMS).

[0149] The various features of the embodiments described above correspond to the various operations illustrated in FIG. 22 (including parts A and B) according to various embodiments of the present disclosure, which depicts an exemplary method (e.g., process) for detecting operational anomalies in a multi-domain communication network. In other words, the various features of the operations described below correspond to the various embodiments described above. Although FIG. 22 shows specific blocks in a particular order, the operations of the exemplary method can be performed in a different order than shown, and can be combined and / or divided into blocks having different functions than shown. Optional blocks or operations are indicated by dotted lines.

[0150] The following description is based on an exemplary method performed by a network analysis system associated with a communication network. For example, the network analysis system may be implemented in (or as): a service management and orchestration (SMO) system for the RAN, an analysis-related CN node such as an NWDAF, a network management node in an OAM system, or an application running in a host computing system external to the network (e.g., a public or private cloud environment).

[0151] The exemplary method may include the operation of block 2210, wherein the network analysis system may obtain multiple time series of performance data from multiple domains of the communication network. The exemplary method may also include the operation of block 2220, wherein the network analysis system may determine one or more models of non-anomalous network behavior based on the multiple time series. The exemplary method may also include the operation of block 2230, wherein the network analysis system may classify each time series into multiple types based on the presence or absence of at least two types of components in each time series. The exemplary method may also include the operation of block 2240, wherein the network analysis system may detect operational anomalies in the multiple time series or in additional performance data obtained from multiple domains of the communication network based on the one or more models and the classified types.

[0152] In some embodiments, the exemplary method may also include the operation of block 2250, wherein based on detecting a plurality of operational anomalies in additional performance data, the network analysis system may determine an order of importance of the detected operational anomalies based on respective deviations from corresponding non-anomalous network behavior. In some of these embodiments, the exemplary method may also include the operation of block 2260, wherein in response to one or more detected anomalies being determined to be most important, the network analysis system may initiate one or more corrective actions in multiple domains of the communication network. In some of these embodiments, the exemplary method may also include the operation of block 2270, wherein in response to one or more detected anomalies being determined to be less important, the network analysis system may refrain from initiating one or more further corrective actions in one or more domains of the communication network.

[0153] In some embodiments, one or more of the following applies:

[0154] Each time series includes data samples from one of the following in a single domain: a network element, or an interface between network elements; and

[0155] Detect operational anomalies in multiple time series collected from multiple domains.

[0156] In some embodiments, classifying the individual time series based on the presence or absence of at least two types of components in block 2230 includes the following operations, labeled with corresponding sub-block numbers:

[0157] (2231) detecting whether each of the time series includes a seasonal component and / or a non-constant trend component;

[0158] (2232) When the time series includes a seasonal component, classify the time series as type 1;

[0159] (2233) When the time series includes a non-constant trend component but does not include a seasonal component,

[0160] Classifying the time series as type 2; and

[0161] (2234) When the time series includes neither a non-constant trend component nor a seasonal component, the time series is classified as the third type.

[0162] In some of these embodiments, one or more of the following apply:

[0163] · detecting in sub-block 2231 whether each of the time series includes a seasonal component is based on one of the following statistical tests: Welch's test, or QS test; and

[0164] · Detecting in sub-block 2231 whether each of the time series includes a non-constant trend component is based on one of the following: a stationarity test, a Kolmogorov-Smirnov test, or a neural network autoencoder.

[0165] In some of these embodiments, detecting an operational anomaly in the plurality of time series in block 2240 includes the following operations, labeled with corresponding sub-block numbers:

[0166] (2241) decomposing each time series classified as the first type into a seasonal component, a non-constant trend component, and a noise component; and

[0167] (2242) calculating upper and lower bounds applicable to all time series classified as the first type; and

[0168] (2243) Detecting an operational anomaly in each time series classified as a first type based on comparing one of the following to an upper bound and a lower bound: a respective non-constant trend component and a respective noise component.

[0169] In some of these embodiments, each time series classified as the third type includes a noise component. In such embodiments, detecting an operational anomaly in the plurality of time series in block 2240 includes the following operations, labeled with corresponding sub-block numbers:

[0170] (2244) decomposing each time series classified as the second type into a non-constant trend component and a noise component; and

[0171] (2245) Detecting an operational abnormality in each time series classified as the second type or the third type based on the respective noise components.

[0172] In some variations, each noise component includes a series of tuples, where each tuple includes a data value and a corresponding time. In such variations, detecting an operational anomaly in each time series classified as the second type or the third type in sub-block 2245 includes the following operations, labeled with corresponding sub-sub-block numbers:

[0173] (2245a) rescaling the data values ​​and / or times of the tuple including the noise component; and

[0174] (2245b) Detecting operational anomalies based on arranging the tuples into a plurality of clusters, the plurality of clusters comprising non-outlier clusters and at least one outlier cluster.

[0175] In some embodiments, determining one or more models of non-anomalous network behavior based on multiple time series in block 2220 includes the operation of sub-block 2221, where the network analysis system can train one or more machine learning (ML) models based on the multiple time series using LI regularization. For example, each ML model includes a neural network (NN) having multiple weights, and the one or more ML models are trained using LI regularization in sub-block 2221, including the operation of sub-sub-block 2221a, where the network analysis system can minimize the loss function of the weights of the NN and the loss function of the norm of the weights of the NN for each ML model.

[0176] In some of these embodiments, detecting operational anomalies based on the one or more models in block 2240 includes operations in subblock 2246, where using the one or more trained ML models, the network analysis system may predict non-anomalous network behavior in one or more of the following:

[0177] a second portion of the plurality of time series, different from the first portion, for training one or more ML models; and

[0178] • Additional performance data obtained from multiple domains of the communication network.

[0179] For example, detecting an operational anomaly in block 2240 is based on predicting non-anomalous network behavior in subblock 2246 using one or more trained ML models.

[0180] In some embodiments, the number of models of non-anomalous network behavior (eg, determined in block 2220) is less than the number of time series. In some embodiments, the plurality of time series represent a plurality of marginal distributions of performance of the corresponding multi-domain communication system.

[0181] In some embodiments, the plurality of domains include at least two of the following domains: a user equipment (UE) domain; a radio access network (RAN) domain; a core network (CN) domain; and an IP multimedia system (IMS) domain. In such embodiments, the plurality of time series include at least one time series obtained from each of the at least two domains.

[0182] In some of these embodiments, the RAN domain includes an open RAN (O-RAN) architecture. In such embodiments, the acquisition, determination, and classification operations of blocks 2210-2230 are performed by an O-RAN non-real-time RAN intelligent controller (non-RT RIC), and the detection operation of block 2240 is performed by an O-RAN non-RT RIC or an O-RAN near-RT RIC.

[0183] In some of these embodiments, the plurality of time series includes at least two of the following:

[0184] Time series of one or more of the following RAN domain Quality of Service (QoS) metrics:

[0185] RAN resources used, serving cell load, mobility events between serving cells, and serving and neighbor cell radio measurements;

[0186] · Time series of one or more of the following CN domain QoS metrics: packet delay, packet delay jitter, packet loss, and priority level;

[0187] A time series of tracking data for the respective cell provided by the RAN node;

[0188] A time series of performance management (PM) counter values ​​associated with a RAN node;

[0189] A time series of User Plane (UP) event information associated with the CN domain; and

[0190] • Time series of control plane (CP) event information associated with the CN domain.

[0191] Although various embodiments are described above herein in terms of methods, apparatuses, devices, computer-readable media, and receivers, a person of ordinary skill will readily appreciate that such methods may be implemented through various combinations of hardware and software in various systems, communication devices, computing devices, control devices, apparatuses, non-transitory computer-readable media, and the like.

[0192] Fig.23 An example of a communication system 2300 according to some embodiments is shown. In this example, the communication system 2300 includes a telecommunications network 2302 and a core network 2306, the telecommunications network 2302 including an access network 2304 such as a radio access network (RAN), and the core network 2306 including one or more core network nodes 2308. In some embodiments, the telecommunications network 2302 may also include one or more network management (NM) nodes 2318, which may be part of an operations support system (OSS), a business support system (BSS), and / or an OAM system. The NM nodes may monitor and / or control the operation of other nodes of the access network 2304 and the core network 2306. Although Fig.23 Not shown, but the NM node 2318 is configured to communicate with other nodes in the access network 2304 and the core network 2306 for these purposes.

[0193] The access network 2304 includes one or more access network nodes, such as network nodes 2310a and 2310b (one or more of which may be generally referred to as network node 2310), or any other similar 3GPP access node or non-3GPP access point. The network node 2310 facilitates direct or indirect connection of UEs, for example, by connecting UEs 2312a, 2312b, 2312c, and 2312d (one or more of which may be generally referred to as UE 2312) to the core network 2306 over one or more wireless connections.

[0194] Example wireless communications via wireless connections include sending and / or receiving wireless signals using electromagnetic waves, radio waves, infrared waves, and / or other types of signals suitable for transmitting information without the use of wires, cables, or other material conductors. In addition, in various embodiments, the communication system 2300 may include any number of wired or wireless networks, network nodes, UEs, and / or any other components or systems that may facilitate or participate in the communication of data and / or signals, whether via a wired or wireless connection. The communication system 2300 may include and / or interface with any type of communication, telecommunication, data, cellular, radio network, and / or other similar type of system.

[0195] UE 2312 can be any of a variety of communication devices, including wireless devices that are arranged, configured and / or operable to wirelessly communicate with network node 2310 and other communication devices. Similarly, network node 2310 is arranged, capable, configured and / or operable to communicate directly or indirectly with UE 2312 and / or with other network nodes or devices in telecommunication network 2302 to enable and / or provide network access, such as wireless network access, and / or perform other functions, such as management in telecommunication network 2302.

[0196] In the depicted example, the core network 2306 connects the network node 2310 to one or more hosts, such as the host 2316. These connections may be direct or indirect via one or more intermediate networks or devices. In other examples, the network node may be directly coupled to the host. The core network 2306 includes one or more core network nodes (e.g., core network node 2308) composed of hardware and software components. The features of these components may be substantially similar to those described with respect to the UE, network node and / or host, so that the description is generally applicable to the corresponding components of the core network node 2308. The exemplary core network node includes a mobile switching center (MSC), a mobility management entity (MME), a home subscriber server (HSS), an access and mobility management function (AMF), a session management function (SMF), an authentication server function (AUSF), a subscription identifier de-hiding function (SIDF), a unified data management (UDM), a security edge protection agent (SEPP), a network open function (NEF) and / or a user plane function (UPF) One or more functions.

[0197] The host 2316 may be under the ownership or control of a service provider other than the operator or provider of the access network 2304 and / or the telecommunications network 2302, and may be operated by or on behalf of the service provider. The host 2316 may host a variety of applications to provide one or more services. Examples of such applications include live and pre-recorded audio / video content, data collection services such as retrieving and compiling data about various environmental conditions detected by multiple UEs, analysis functions, social media, functions for controlling or otherwise interacting with remote devices, functions for alarm and monitoring centers, or any other such functions performed by a server.

[0198] In some embodiments, the access network 2304 may include a service management and orchestration (SMO) system or node 2320 that may monitor and / or control the operation of the access network node 2310. For example, such an arrangement may be used when the access network 2304 utilizes an open RAN (O-RAN) architecture. The SMO system 2320 may be configured to communicate with the core network 2306 and / or the host 2316, such as Fig.23 shown.

[0199] In some embodiments, one or more of the host 2316, the network management node 2318, and the SMO system 2320 may be configured to perform various operations of an exemplary method (e.g., process) for detecting operational anomalies in a multi-domain communication network, such as described above with respect to FIG. 22.

[0200] As a whole, Fig.23 The communication system 2300 enables connectivity between UEs, network nodes, and hosts. In this sense, the communication system can be configured to operate according to predefined rules or procedures, such as specific standards, including but not limited to: Global System for Mobile Communications (GSM); Universal Mobile Telecommunications System (UMTS); Long Term Evolution (LTE), and / or other suitable 2G, 3G, 4G, 5G standards, or any applicable future generation standards (e.g., 6G); Wireless Local Area Network (WLAN) standards, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard (WiFi); and / or any other appropriate wireless communication standards, such as Worldwide Interoperability for Microwave Access (WiMax), Bluetooth, Z-Wave, Near Field Communication (NFC) ZigBee, LiFi, and / or any Low Power Wide Area Network (LPWAN) standards such as LoRa and Sigfox.

[0201] In some examples, telecommunication network 2302 is a cellular network that implements 3GPP standardized features. Thus, telecommunication network 2302 can support network slicing to provide different logical networks to different devices connected to telecommunication network 2302. For example, telecommunication network 2302 can provide ultra-reliable low-latency communication (URLLC) services to some UEs while providing enhanced mobile broadband (eMBB) services to other UEs, and / or provide massive machine type communication (mMTC) / massive IoT services to yet other UEs.

[0202] In some examples, UE 2312 is configured to send and / or receive information without direct human interaction. For example, the UE may be designed to send information to access network 2304 according to a predetermined schedule when triggered by an internal or external event, or in response to a request from access network 2304. In addition, the UE may be configured to operate in a single RAT or multi-RAT or multi-standard mode. For example, the UE may operate with any one or a combination of Wi-Fi, NR (New Radio), and LTE, i.e., be configured for multi-radio dual connectivity (MR-DC), such as new radio dual connectivity (EN-DC) of E-UTRAN (Evolved-UMTS Terrestrial Radio Access Network).

[0203] In an example, the hub 2314 communicates with the access network 2304 to facilitate indirect communication between one or more UEs (e.g., UE 2312c and / or 2312d) and a network node (e.g., network node 2310b). In some examples, the hub 2314 may be a controller, a router, a content source and analysis, or any other communication device described herein with respect to the UE. For example, the hub 2314 may be a broadband router that enables the UE to access the core network 2306. As another example, the hub 2314 may be a controller that sends commands or instructions to one or more actuators in the UE. The commands or instructions may be received from the UE, the network node 2310, or may be received by an executable code, a script, a process, or other instructions in the hub 2314. As another example, the hub 2314 may be a data collector that acts as a temporary storage for UE data, and in some embodiments, analysis or other processing of the data may be performed. As another example, the hub 2314 may be a content source. For example, for a UE that is a VR headset, display, speaker, or other media delivery device, the hub 2314 can retrieve VR assets, video, audio, or other media or data related to sensory information via a network node, and the hub 2314 then provides it to the UE either directly, after performing local processing, and / or after adding additional local content. In another example, the hub 2314 acts as a proxy server or orchestrator for the UE, particularly when one or more of the UEs are low-energy loT devices.

[0204] Hub 2314 can have constant / persistent or intermittent connection to network node 2310b. Hub 2314 can also allow different communication schemes and / or scheduling between hub 2314 and UE (e.g., UE 2312c and / or 2312d) and hub 2314 and core network 2306. In other examples, hub 2314 is connected to core network 2306 and / or one or more UE via wired connection. In addition, hub 2314 can be configured to be connected to M2M service provider and / or connected to another UE via direct connection through access network 2304. In some scenes, UE can establish wireless connection with network node 2310, while still connected via hub 2314 via wired or wireless connection. In some embodiments, hub 2314 can be a dedicated hub--that is, its main function is to route the communication to / from UE from network node 2310b or route to the hub of network node 2310b. In other embodiments, the hub 2314 may be a non-dedicated hub, i.e., a device operable to route communications between UEs and the network node 2310b, but otherwise capable of operating as a communications origin and / or endpoint for certain data channels.

[0205] Fig.24 A network node 2400 according to some embodiments is shown. As used herein, a network node refers to a device capable of, configured, arranged and / or operable to communicate directly or indirectly with a UE and / or with other network nodes or devices in a telecommunications network. Examples of network nodes include, but are not limited to, access points (APs) (e.g., radio access points), base stations (BSs) (e.g., radio base stations, Node Bs, evolved Node Bs (eNBs), and NR Node Bs (gNBs)).

[0206] Base stations can be classified based on the amount of coverage they provide (or, in other words, the transmit power level of the base station), and therefore, depending on the amount of coverage provided, a base station can be referred to as a femto base station, a pico base station, a micro base station, or a macro base station. A base station can be a relay node or a relay donor node that controls a relay. A network node may also include one or more (or all) parts of a distributed radio base station, such as a centralized digital unit and / or a remote radio unit (RRU), sometimes referred to as a remote radio head (RRH). Such a remote radio unit may or may not be integrated with an antenna as an antenna-integrated radio. Portions of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[0207] Other examples of network nodes include multiple transmission point (multi-TRP) 5G access nodes, multi-standard radio (MSR) equipment such as an MSR BS, a network controller such as a radio network controller (RNC) or a base station controller (BSC), a base transceiver station (BTS), a transmission point, a transmission node, a multi-cell / multicast coordination entity (MCE), an operation and maintenance (O&M) node, an operation support system (OSS) node, a self-organizing network (SON) node, a positioning node (e.g., an evolved serving mobile positioning center (E-SMLC)), and / or minimization of drive tests (MDT).

[0208] In some embodiments, the network node 2400 can be configured to perform various operations of an exemplary method (eg, process) for detecting operational anomalies in a multi-domain communications network, such as described above with respect to FIG. 22 .

[0209] The network node 2400 includes a processing circuit 2402, a memory 2404, a communication interface 2406, and a power supply 2408. The network node 2400 may be composed of multiple physically separated components (e.g., a NodeB component and an RNC component, or a BTS component and a BSC component, etc.), each of which may have its own components. In a specific scenario where the network node 2400 includes multiple separated components (e.g., a BTS and a BSC component), one or more of the separated components may be shared between several network nodes. For example, a single RNC may control multiple NodeBs. In this scenario, each unique NodeB and RNC pair may be considered as a single separated network node in some cases. In some embodiments, the network node 2400 may be configured to support multiple radio access technologies (RATs). In such embodiments, some components may be duplicated (e.g., separate memories 2404 for different RATs), and some components may be reused (e.g., the same antenna 2410 may be shared by different RATs). The network node 2400 may also include multiple sets of various components shown for different wireless technologies integrated into the network node 2400, such as GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, radio frequency identification (RFID), or Bluetooth wireless technologies. These wireless technologies may be integrated into the same or different chips or chipsets and other components within the network node 2400.

[0210] The processing circuit 2402 may include a combination of one or more of a microprocessor, a controller, a microcontroller, a central processing unit, a digital signal processor, an application specific integrated circuit, a field programmable gate array, or any other suitable computing device, resource, or combination of hardware, software and / or encoded logic, operable to provide the functionality of the network node 2400 alone or in combination with other network node 2400 components (such as memory 2404).

[0211] In some embodiments, processing circuitry 2402 includes a system on a chip (SOC). In some embodiments, processing circuitry 2402 includes one or more radio frequency (RF) transceiver circuitry 2412 and baseband processing circuitry 2414. In some embodiments, radio frequency (RF) transceiver circuitry 2412 and baseband processing circuitry 2414 may be on separate chips (or groups of chips), boards, or units, such as a radio unit and a digital unit. In alternative embodiments, some or all of RF transceiver circuitry 2412 and baseband processing circuitry 2414 may be on the same chip or chipset, board, or unit.

[0212] The memory 2404 may include any form of volatile or non-volatile computer-readable memory, including, but not limited to, persistent memory, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (e.g., hard disk), removable storage media (e.g., flash drive, compact disk (CD) or digital video disk (DVD)), and / or any other volatile or non-volatile, non-transitory device-readable and / or computer-executable memory device that stores information, data and / or instructions that can be used by the processing circuit 2402. The memory 2404 may store any suitable instructions, data or information, including computer programs, software, applications including one or more of logic, rules, codes, tables and / or other instructions that can be executed by the processing circuit 2402 and utilized by the network node 2400 (collectively referred to as computer program product 2404a). The memory 2404 may be used to store any calculations performed by the processing circuit 2402 and / or any data received via the communication interface 2406. In some embodiments, processing circuit 2402 and memory 2404 are integrated.

[0213] The communication interface 2406 is used for wired or wireless communication of signaling and / or data between network nodes, access networks and / or UEs. As shown, the communication interface 2406 includes (one or more) ports / (one or more) terminals 2416, for example, to send data to the network and receive data from the network via a wired connection. The communication interface 2406 also includes a radio front-end circuit 2418, which can be coupled to the antenna 2410, or in some embodiments, coupled to a part of the antenna 2410. The radio front-end circuit 2418 includes a filter 2420 and an amplifier 2422. The radio front-end circuit 2418 can be connected to the antenna 2410 and the processing circuit 2402. The radio front-end circuit can be configured to adjust the signal communicated between the antenna 2410 and the processing circuit 2402. The radio front-end circuit 2418 can receive digital data to be sent to other network nodes or UEs via a wireless connection. The radio front-end circuit 2418 can use a combination of a filter 2420 and / or an amplifier 2422 to convert digital data into a radio signal with appropriate channel and bandwidth parameters. The radio signal may then be transmitted via antenna 2410. Similarly, when receiving data, antenna 2410 may collect the radio signal, which may then be converted into digital data by radio front end circuit 2418. The digital data may be passed to processing circuit 2402. In other embodiments, the communication interface may include different components and / or different combinations of components.

[0214] In certain optional embodiments, the network node 2400 does not include a separate radio front end circuit 2418, instead the processing circuit 2402 includes the radio front end circuit and is connected to the antenna 2410. Similarly, in some embodiments, all or some of the RF transceiver circuit 2412 is part of the communication interface 2406. In other embodiments, the communication interface 2406 includes one or more ports or terminals 2416, the radio front end circuit 2418, and the RF transceiver circuit 2412 as part of a radio unit (not shown), and the communication interface 2406 communicates with the baseband processing circuit 2414 as part of a digital unit (not shown).

[0215] Antenna 2410 may include one or more antennas or antenna arrays configured to send and / or receive wireless signals. Antenna 2410 may be coupled to radio front end circuit 2418 and may be any type of antenna capable of wirelessly sending and receiving data and / or signals. In some embodiments, antenna 2410 is separate from network node 2400 and may be connected to network node 2400 via an interface or port.

[0216] Antenna 2410, communication interface 2406 and / or processing circuit 2402 may be configured to perform any receiving operation and / or certain obtaining operations described herein as being performed by a network node. Any information, data and / or signal may be received from a UE, another network node and / or any other network device. Similarly, antenna 2410, communication interface 2406 and / or processing circuit 2402 may be configured to perform any transmitting operation described herein as being performed by a network node. Any information, data and / or signal may be transmitted to a UE, another network node and / or any other network device.

[0217] The power supply 2408 provides power to the various components of the network node 2400 in a form suitable for the respective components (e.g., at the voltage and current levels required by each respective component). The power supply 2408 may further include or be coupled to a power management circuit to supply power to the components of the network node 2400 for performing the functions described herein. For example, the network node 2400 may be connected to an external power source (e.g., a power grid, a power outlet) via an input circuit or interface such as a cable, whereby the external power source supplies power to the power circuit of the power supply 2408. As another example, the power supply 2408 may include a power source in the form of a battery or battery pack, which is connected to the power circuit or integrated in the power circuit. If the external power source fails, the battery can provide backup power.

[0218] Embodiments of network node 2400 may include, in addition to Fig.24 , to provide certain aspects of the network node functionality, including any functionality described herein and / or any functionality necessary to support the subject matter described herein. For example, the network node 2400 may include a user interface device to allow information to be input into the network node 2400 and to allow information to be output from the network node 2400. This may allow a user to perform diagnostics, maintenance, repair, and other management functions on the network node 2400.

[0219] Fig.25 is a block diagram of a host 2500 according to various aspects described herein, which may be Fig.23 2316 of the embodiment of the host. As used herein, the host 2500 can be or include various combinations of hardware and / or software, including processing resources in a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, a container, or a server farm. The host 2500 can provide one or more services to one or more UEs.

[0220] The host 2500 includes a processing circuit 2502, which is operably coupled to an input / output interface 2506, a network interface 2508, a power supply 2510, and a memory 2512 via a bus 2504. Other components may be included in other embodiments. The features of these components may be substantially similar to the features described with respect to the devices of the previous figures, so that their description is generally applicable to the corresponding components of the host 2500.

[0221] Memory 2512 may include one or more computer programs, including one or more host applications 2514 and data 2516, which may include user data, for example, data generated by a UE for the host 2500 or data generated by the host 2500 for the UE. An embodiment of the host 2500 may utilize only a subset or all of the components shown. The host application 2514 may be implemented in a container-based architecture and may provide support for video codecs (e.g., generic video coding (VVC), high efficiency video coding (HEVC), advanced video coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, advanced audio coding (AAC), MPEG, G.711), including transcoding for multiple different categories, types, or implementations of UE (e.g., handsets, desktop computers, wearable display systems, head-up display systems). The host application 2514 may also provide user authentication and license checks, and may periodically report health, routing, and content availability to a central node, such as a device in or on the edge of a core network. Thus, the host 2500 can select and / or indicate different hosts for cross-top services for the UE. The host application 2514 can support various protocols, such as HTTP Live Streaming (HLS) protocol, Real-time Messaging Protocol (RTMP), Real-time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.

[0222] In some embodiments, the host 2500 may be configured to perform various operations of an exemplary method (eg, process) for detecting operational anomalies in a multi-domain communications network, such as described above with respect to FIG. 22 .

[0223] Fig.262600, in which the functions implemented by some embodiments can be virtualized. In this context, virtualization means creating a virtual version of a device or equipment, which may include a virtualized hardware platform, storage device, and networking resources. As used herein, virtualization may be applied to any device or component thereof described herein, and relates to an embodiment in which at least a portion of a function is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 2600, the virtual environment being hosted by one or more of the hardware nodes, such as a hardware computing device operated as a network node, UE, core network node, or host. In addition, in embodiments in which the virtual node does not require radio connectivity (e.g., a core network node or host), the node may be fully virtualized.

[0224] Applications 2602 (which may alternatively be referred to as software instances, virtual appliances, network functions, virtual nodes, virtual network functions, etc.) run in virtualized environment 2600 to implement some features, functions, and / or benefits of some embodiments disclosed herein. In some embodiments, one or more applications 2602 may be configured to perform various operations of an exemplary method (e.g., process) for detecting operational anomalies in a multi-domain communication network, such as described above with respect to FIG. 22 .

[0225] The hardware 2604 includes processing circuitry, memory storing software and / or instructions (collectively referred to as computer program product 2604a) executable by the hardware processing circuitry, and / or other hardware devices as described herein, such as network interfaces, input / output interfaces, etc. The software may be executed by the processing circuitry to instantiate one or more virtualization layers 2606 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 2608a and 2608b (one or more of which may be generally referred to as VMs 2608), and / or perform any of the functions, features, and / or benefits described in connection with some embodiments described herein. The virtualization layer 2606 may present a virtual operating platform that appears to be networked hardware to the VMs 2608.

[0226] VM 2608 includes virtual processing, virtual memory, virtual networking or interfaces, and virtual storage, and may be run by a corresponding virtualization layer 2606. Different embodiments of instances of virtual devices 2602 may be implemented on one or more of VM 2608, and may be implemented in different ways. Virtualization of hardware is referred to in some contexts as network function virtualization (NFV). NFV may be used to merge many network device types onto industry standard high-capacity server hardware, physical switches, and physical storage, which may be located in data centers as well as customer premises equipment.

[0227] In the context of NFV, VMs 2608 can be software implementations of physical machines that run programs as if they were executed on physical, non-virtualized machines. Each of VMs 2608 and the portion of hardware 2604 on which the VM executes, whether hardware dedicated to the VM and / or hardware shared by the VM with other VMs, forms a separate virtual network element. Still in the context of NFV, a virtual network function is responsible for handling a specific network function running in one or more VMs 2608 on hardware 2604 and corresponding to an application 2602.

[0228] Hardware 2604 can be implemented in a standalone network node with general or specific components. Hardware 2604 can implement some functions via virtualization. Optionally, hardware 2604 can be part of a larger hardware cluster (e.g., such as in a data center or CPE), where many hardware nodes work together and are managed via management and orchestration 2610, which in particular oversees the life cycle management of application 2602. In some embodiments, hardware 2604 is coupled to one or more radio units, each of which includes one or more transmitters and one or more receivers that can be coupled to one or more antennas. The radio unit can communicate directly with other hardware nodes via one or more appropriate network interfaces, and can be used in combination with virtual components to provide radio capabilities to virtual nodes, such as radio access nodes or base stations. In some embodiments, some signaling can be provided by using a control system 2612, which is optionally used for communication between hardware nodes and radio units.

[0229] The foregoing merely illustrates the principles of the present disclosure. In view of the teachings herein, various modifications and changes to the described embodiments will be apparent to those skilled in the art. It will therefore be appreciated that those skilled in the art will be able to design many systems, arrangements, and processes that, although not explicitly shown or described herein, embody the principles of the present disclosure and therefore may be within the spirit and scope of the present disclosure. It will be appreciated by those of ordinary skill in the art that the various embodiments may be used together or interchangeably with each other.

[0230] As used herein, the term "unit" may have a conventional meaning in the field of electronics, electrical devices and / or electronic devices, and may include, for example, electrical and / or electronic circuits, devices, modules, processors, memories, logical solid-state and / or discrete devices, computer programs or instructions for performing corresponding tasks, processes, calculations, output and / or display functions, etc., such as those described herein.

[0231] Any suitable steps, methods, features, functions or benefits disclosed herein may be performed by one or more functional units or modules of one or more virtual devices. Each virtual device may include a plurality of these functional units. These functional units may be implemented via processing circuits and other digital hardware, and the processing circuits may include one or more microprocessors or microcontrollers, and the other digital hardware may include digital signal processors (DSPs), dedicated digital logic, etc. The processing circuit may be configured to execute program codes stored in a memory, and the memory may include one or more types of memory such as read-only memory (ROM), random access memory (RAM), cache memory, flash memory device, optical storage device, etc. The program code stored in the memory includes program instructions for executing one or more telecommunications and / or data communication protocols and instructions for executing one or more of the techniques described herein. In some embodiments, the processing circuit may be used to enable each functional unit to perform the corresponding functions according to one or more embodiments of the present disclosure.

[0232] As described herein, the device and / or apparatus may be represented by a semiconductor chip, a chipset, or a (hardware) module including such a chip or chipset; however, this does not exclude the possibility that the function of the device or apparatus is not implemented by hardware, but is implemented as a software module, such as a computer program or computer program product including an executable software code portion for execution or operation on a processor. In addition, the functions of the device or apparatus may be implemented by any combination of hardware and software. A device or apparatus may also be considered as a collection of multiple devices and / or apparatuses, whether functionally cooperating with each other or independent of each other. In addition, the devices and apparatus may be implemented in a distributed manner throughout the system, as long as the functions of the device or apparatus are retained. These and similar principles are considered to be known to the technician.

[0233] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those commonly understood by those of ordinary skill in the art to which the present disclosure belongs. It will be further understood that the terms used herein should be interpreted as having a meaning consistent with their meaning in the context of this specification and the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined as such herein.

[0234] In addition, certain terms used in this disclosure (including the specification and the drawings) may be used synonymously in some cases (e.g., "data" and "information"). It should be understood that although these terms (and / or other terms that may be synonymous with each other) may be used synonymously herein, there are instances where these words are not intended to be used synonymously.

Claims

1. A computer-implemented method for detecting an operational anomaly in a multi-domain communication network, the method comprising: obtaining (2210) a plurality of time series of performance data from a plurality of domains of a communication network; determining (2220) one or more models of non-anomalous network behavior based on the plurality of time series; classifying (2230) each time series into a plurality of types based on the presence or absence of at least two types of components in each time series; as well as Based on the one or more models and the classified types, operational anomalies are detected (2240) in the plurality of time series or in additional performance data obtained from a plurality of domains of the communication network.

2. The method according to claim 1, further comprising: Based on detecting a plurality of operational anomalies in the additional performance data, determining (2250) an order of importance of the detected operational anomalies based on respective deviations from corresponding non-anomalous network behavior.

3. The method according to claim 2, further comprising one or more of the following: initiating (2260) one or more corrective actions in a plurality of domains of the communication network in response to the one or more detected anomalies determined to be most significant; and In response to the one or more detected anomalies being determined to be less significant, refraining from initiating (2270) one or more further corrective actions in one or more domains of the communication network.

4. The method according to any one of claims 1 to 3, wherein one or more of the following apply: Each time series includes data samples from one of the following in a single domain: a network element or an interface between network elements; and Detect operational anomalies in multiple time series collected from multiple domains.

5. The method according to any one of claims 1 to 4, wherein: Classifying the respective time series based on the presence or absence of at least two types of components (2230) includes: detecting (2231) whether each of the time series includes a seasonal component and / or a non-constant trend component; When the time series includes a seasonal component, classifying (2232) the time series as a first type; When the time series includes a non-constant trend component but does not include a seasonal component, classifying (2233) the time series as a second type; and When a time series includes neither a non-constant trend component nor a seasonal component, the time series is classified (2234) as a third type.

6. The method of claim 5, wherein one or more of the following applies: detecting (2231) whether each of the time series includes a seasonal component based on one of the following statistical tests: a Welch test or a QS test; and Whether each of the time series includes a non-constant trend component is detected (2231) based on one of the following: a stationarity test, a Kolmogorov-Smirnov test, or a neural network autoencoder.

7. The method according to any one of claims 5 to 6, wherein: Detecting (2240) operational anomalies in the plurality of time series includes: decomposing (2241) each time series classified as the first type into a seasonal component, a non-constant trend component, and a noise component; and Calculating (2242) upper and lower bounds applicable to all time series classified as the first type; and An operational anomaly in each time series classified as a first type is detected (2243) based on comparing one of the following to the upper and lower bounds: a respective non-constant trend component and a respective noise component.

8. The method according to any one of claims 5 to 7, wherein: Each time series classified as the third type includes a noise component; as well as Detecting (2240) operational anomalies in multiple time series includes: decomposing (2244) each time series classified as the second type into a non-constant trend component and a noise component; as well as An operational anomaly in each time series classified as the second type or the third type is detected ( 2245 ) based on the respective noise components.

9. The method according to claim 8, wherein: Each noise component includes a series of tuples, each tuple includes a data value and a corresponding time; as well as Detecting (2245) an operational anomaly in each time series classified as the second type or the third type includes: rescaling (2245a) data values ​​and / or times of the tuple including the noise component; as well as Operational anomalies are detected (2245b) based on arranging the tuples into a plurality of clusters, the plurality of clusters including non-outlier clusters and at least one outlier cluster.

10. The method according to any one of claims 1-9, wherein determining (2220) one or more models of non-anomalous network behavior based on the multiple time series includes training (2221) one or more machine learning (ML) models based on the multiple time series using LI regularization.

11. The method according to claim 10, wherein: Each ML model includes a neural network NN with multiple weights; as well as Training (2221) the one or more ML models using LI regularization includes: For each ML model, minimize (2221a) the loss function of the NN's weights and the loss function of the norm of the NN's weights.

12. The method according to any one of claims 10 to 11, wherein: Detecting (2240) operational anomalies based on the one or more models includes: using the one or more trained ML models, predicting (2246) non-anomalous network behavior in one or more of the following: a second portion of the plurality of time series that is different from the first portion used to train the one or more ML models; and The further performance data is obtained from a plurality of domains of the communications network.

13. The method according to claim 12, wherein: Detecting (2240) operational anomalies is based on non-anomalous network behavior predicted using the one or more trained ML models.

14. A method according to any one of claims 1-13, wherein the multiple time series include one or more multidimensional time series, and obtaining (2210) the multiple time series includes aggregating (2211) at least two obtained single-dimensional time series to form each multidimensional time series.

15. The method according to any one of claims 1 to 14, wherein: The number of models of non-abnormal network behavior is less than the number of time series.

16. The method according to any one of claims 1 to 15, wherein: The plurality of time series represent a plurality of marginal distributions of performance of the corresponding multi-domain communication system.

17. The method according to any one of claims 1 to 16, wherein: The multiple domains include at least two of the following domains: a user equipment UE domain; a radio access network RAN ​​domain; a core network CN domain; and IP Multimedia System IMS domain; as well as The plurality of time series include at least one time series obtained from each of the at least two domains.

18. The method of claim 17, wherein: The RAN domain includes an open RAN (O-RAN) architecture; The obtaining, determining and classifying operations are performed by an O-RAN non-real-time RAN intelligent controller (non-RT RIC); and The detection operation is performed by the O-RAN non-RT RIC or by the O-RAN near-RT RIC.

19. The method according to any one of claims 17-18, wherein: The plurality of time series include at least two of the following: time series of one or more of the following RAN domain Quality of Service (QoS) metrics: RAN resources used, serving cell load, mobility events between serving cells, and serving and neighbor cell radio measurements; time series of one or more of the following CN domain QoS metrics: packet delay, packet delay jitter, packet loss, and priority level; Time series of tracking data for each cell provided by the RAN node; a time series of performance management (PM) counter values ​​associated with a RAN node; A time series of User Plane (UP) event information associated with the CN domain; and Time series of control plane (CP) event information associated with the CN domain.

20. A network analysis system (800, 1800, 2010, 2110, 2316, 2318, 2320, 2400, 2500, 2600) configured to detect operational anomalies in a multi-domain communication network (198, 199, 200, 300, 2000, 2100, 2302), the network analysis system comprising: Communications interface circuitry (2406, 2508, 2604) configured to communicate with a plurality of domains of a communications network; as well as a processing circuit (2402, 2502, 2604) operably coupled to the communication interface circuit, whereby the processing circuit and the communication interface circuit are configured to: obtaining a plurality of time series of performance data from the plurality of domains of the communications network; determining one or more models of non-anomalous network behavior based on the plurality of time series; classifying each time series into a plurality of types based on the presence or absence of at least two types of components in each time series; as well as Based on the one or more models and the classified types, operational anomalies are detected in the plurality of time series or in additional performance data obtained from a plurality of domains of the communication network.

21. The network analysis system according to claim 20, wherein: The processing circuit and the communication interface circuit are further configured to perform operations corresponding to any one of claims 2-19.

22. A network analysis system (800, 1800, 2010, 2110, 2316, 2318, 2320, 2400, 2500, 2600) configured to detect operational anomalies in a multi-domain communication network (198, 199, 200, 300, 2000, 2100, 2302), the network analysis system comprising: a time series generator module (810, 1810) configured to obtain a plurality of time series of performance data from a plurality of domains of a communication network; a robust filtering module (830, 1830) configured to determine one or more models of non-anomalous network behavior based on the plurality of time series; A time series classification module (820, 1820) configured to classify each time series into a plurality of types based on the presence or absence of at least two types of components in each time series; as well as One or more anomaly detection modules (860, 870, 1840) are configured to detect operational anomalies in the plurality of time series or in additional performance data obtained from a plurality of domains of the communication network based on the one or more models and the classified types.

23. The network analysis system of claim 22, further comprising an anomaly ranking module (880, 1850) configured to determine an order of importance of multiple operational anomalies detected by the one or more anomaly detection modules in additional performance data based on respective deviations of the detected multiple operational anomalies from corresponding non-anomalous network behaviors.

24. The network analysis system according to claim 22, further configured to perform operations corresponding to any one of the methods according to claims 3-19.

25. A non-transitory computer-readable medium (2404, 2604) storing computer-executable instructions that, when executed by a processing circuit (2402, 2502, 2604), configure a network analysis system (800, 1800, 2010, 2110, 2316, 2318, 2320, 2400, 2500, 2600) to detect operational anomalies in a multi-domain communication network (198, 199, 200, 300, 2000, 2100, 2302) based on performing operations corresponding to any of the methods of claims 1-19.

26. A computer program product (2404a, 2604a) comprising computer executable instructions which, when executed by a processing circuit (2402, 2502, 2604), configure a network analysis system (800, 1800, 2010, 2110, 2316, 2318, 2320, 2400, 2500, 2600) to detect operational anomalies in a multi-domain communication network (198, 199, 200, 300, 2000, 2100, 2302) based on performing operations corresponding to any of the methods of claims 1-19.

Images