Systems and methods for unified anomaly detection and isolation for 5g

Abstract

Systems, methods, techniques, and other embodiments are disclosed with regard to techniques for detecting anomalies in a system associated with a wireless network. A method for detecting anomalies in the system associated with the wireless network includes; determining one or more features based on time series values detected by a plurality of monitoring nodes located in the system and network information of the wireless network, determining a global status representing normal operation or abnormal operation of the system based on the one or more features and one or more global decision boundaries, in response to determining that the global status represents the abnormal operation of the system, selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof, detecting anomalies of one or more subsystems of the selected subsystems based on the one or more features and one or more decision boundaries for the selected subsystems, selecting one or more monitoring nodes belonging to the one or more subsystems and detecting anomalies of at least one monitoring node of the selected one or more monitoring nodes based on the one or more features and one or more local decision boundaries for the selected one or more monitoring nodes. Each of the subsystems includes a subset of the plurality of monitoring nodes.

Description

System and Methods for Unified Anomaly Detection and Isolation for 5GTECHNICAL FIELD

[0001] The disclosed implementations relate generally to cyber-physical systems and more specifically to Al-based anomaly detection systems, devices and methods for safety, cybersecurity and resilience for cyber-physical systems and devices.BACKGROUND

[0002] Industrial control systems that operate cyber-physical systems are increasingly connected through a communication network (such a wireless network or a network compatible with a 3GPP standard, such as a 5G network). As a result, these control systems have been increasingly vulnerable to threats, such as cyber-attacks (e.g., associated with a computer virus, malicious software, etc.). However, wireless networks, such as 5G, 6G, often comprise large- scale systems with many monitoring nodes. Thus, monitoring every node in these cyber-physical systems to detect anomalies requires considerable time and resources.SUMMARY

[0003] Accordingly, there is a need for systems and methods for detection of anomalies (e.g., cyber-attacks) with scalable and efficient ways to investigate monitoring nodes in cyberphysical systems connected wireless networks. In one aspect, some implementations include a computer-implemented method for implementing a hierarchal top-down approach to detect anomalies in the system. The hierarchal top-down approach may provide selecting subsystems based on subsystem selection algorithm. The subsystem selection algorithm may result in the cyber-physical systems to have a hierarchical structure with a set of connected nodes so that the system can detect anomalies of plurality nodes by tracking abnormal nodes along the hierarchical structure.

[0004] In one aspect, some implementations include a computer-implemented method for detecting anomalies in a system associated with a wireless network. The method may include determining one or more features based on time series values detected by a plurality of monitoring nodes located in the system and network information of the wireless network, determining a global status representing normal operation or abnormal operation of the system based on the one or more features and one or more global decision boundaries, in response to1SUBSTITUTE SHEET (RULE 26)determining that the global status represents the abnormal operation of the system, selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof, detecting anomalies of one or more subsystems of the selected subsystems based on the one or more features and one or more decision boundaries for the selected subsystems, selecting one or more monitoring nodes belonging to the one or more subsystems and detecting anomalies of at least one monitoring node of the selected one or more monitoring nodes based on the one or more features and one or more local decision boundaries for the selected one or more monitoring nodes. In some implementations, each of the subsystems includes a subset of the plurality of monitoring nodes. In some implementations, the plurality of monitoring nodes includes at least one of a sensor, an actuator and a user device associated with the wireless network. In some implementations, the one or more features include flow-based features and QoS features of the wireless network, the QoS features include QoS Identifier, IP data flow, flow or bearer identifier, reflective QoS identifier and data session information. In some implementations, the selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof includes selecting the subsystems following network architecture and hierarchy based on the knowledge-based subsystem selection method. In some implementations, the data-driven subsystem selection method includes a data clustering method, the selecting subsystems based on one of a knowledge-based subsystem selection method, a data- driven subsystem selection method and a combination thereof includes clustering the plurality of monitoring nodes into one or more clusters based on the data clustering method, a cluster corresponding a subsystem. In some implementations, each monitoring node of the plurality of monitoring nodes is assigned to a cluster based on a distance from each monitoring node to a cluster centroid of the cluster based on Euclidian or Mahalanobis distance so that each monitoring node belongs to a single cluster. In some implementations, the data-driven subsystem selection method includes a graph-based method, the selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof includes selecting the subsystems based on the graph-based method by partitioning the system based on a graph map of the network indicating volume of network traffic. In some implementations, the system has a hierarchical structure including a root node corresponding to the system, subsystem nodes corresponding the subsystems and a plurality of2SUBSTITUTE SHEET (RULE 26)leaf nodes corresponding to the plurality of monitoring nodes. In some implementations, the method comprises using an ensemble of distributed digital twin models associated with the system, providing anomaly decision for the system, the one or more global decision boundaries for the system and a global score representing a likelihood of the anomalies for the system. In some implementations, the method comprises using a digital twin model associated with each of the plurality of monitoring nodes, providing anomaly decision for the one or more subsystems and the at least one monitoring node, the one or more decision boundaries for the selected subsystems, the one or more local decision boundaries for the one or more monitoring nodes and one or more scores representing a likelihood of the anomalies for the one or more subsystems and the at least one monitoring node, each of the digital twin models associated with each of the plurality of monitoring nodes. In some implementations, the wireless network includes a network compatible with a 3GPP standard.

[0005] In another aspect, an apparatus for detecting anomalies in a system associated with a wireless network comprises a processor configured to determine one or more features based on time series values detected by a plurality of monitoring nodes located in the system and network information of the wireless network, determine a global status representing normal operation or abnormal operation of the system based on the one or more features and one or more global decision boundaries, in response to determining that the global status represents the abnormal operation of the system, selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof, detect anomalies of one or more subsystems of the selected subsystems based on the one or more features and one or more decision boundaries for the selected subsystems, select one or more monitoring nodes belonging to the one or more subsystems and detect anomalies of at least one monitoring node of the selected one or more monitoring nodes based on the one or more features and one or more local decision boundaries for the selected one or more monitoring nodes. In some implementations, each of the subsystems includes a subset of the plurality of monitoring nodes. In some implementations, the plurality of monitoring nodes includes at least one of a sensor, an actuator and a user device associated with the wireless network. In some implementations, the one or more features include flow- based features and QoS features of the wireless network, the QoS features include QoS Identifier, IP data flow, flow or bearer identifier, reflective QoS identifier and data session information. In some implementations, the system has3SUBSTITUTE SHEET (RULE 26)a hierarchical structure including a root node corresponding to the system, subsystem nodes corresponding the subsystems and a plurality of leaf nodes corresponding to the plurality of monitoring nodes. In some implementations, the wireless network includes two-layer structures including a 5G Core (5GC) network layer and a Radio Access Network (RAN) layer. The 5GC network layer is higher than the RAN layer. In some implementations, the apparatus includes a first module is configured to collet RAN level information including radio level information, modulation and demodulation information, channel estimation information, aggregated information between multiple network management and antenna port information at a RAN layer to monitor signal at the RAN layer to detect anomalies of the RAN layer and a second module is configured to receive the collected RAN level information and detect anomalies of the 5GC network layer based on the received RAN level information. In some implementations, the wireless network includes a network compatible with a 3 GPP standard. In some implementations, the apparatus includes an ensemble of distributed digital twin models. The processor executes an ensemble of distributed digital twin models, each of the digital twin models associated with each of the plurality of monitoring nodes. In some implementations, the processor uses an ensemble of distributed digital twin models associated with the system and provide anomaly decision for the system, the one or more global decision boundaries for the system and a global score representing a likelihood of the anomalies for the system. In some implementations, the processor uses a digital twin model associated with each of the plurality of monitoring nodes and provide anomaly decision for the one or more subsystems and the at least one monitoring node, the one or more decision boundaries for the selected subsystems, the one or more local decision boundaries for the one or more monitoring nodes and one or more scores representing a likelihood of the anomalies for the one or more subsystems and the at least one monitoring node, each of the digital twin models associated with each of the plurality of monitoring nodes.

[0006] In another aspect, a non-transitory computer-readable storage medium stores one or more programs for execution by one or more processors of an electronic device. The one or more programs include instructions for determining one or more features based on time series values detected by a plurality of monitoring nodes located in the system and network information of the wireless network, determining a global status representing normal operation or abnormal operation of the system based on the one or more features and one or more global decision4SUBSTITUTE SHEET (RULE 26)boundaries, in response to determining that the global status represents the abnormal operation of the system, selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof, detecting anomalies of one or more subsystems of the selected subsystems based on the one or more features and one or more decision boundaries for the selected subsystems, selecting one or more monitoring nodes belonging to the one or more subsystems and detecting anomalies of at least one monitoring node of the selected one or more monitoring nodes based on the one or more features and one or more local decision boundaries for the selected one or more monitoring nodes. In some implementations, each of the subsystems includes a subset of the plurality of monitoring nodes.

[0007] In another aspect, an apparatus for detecting anomalies in a system associated with a wireless network comprises a processor configured to determine one or more features based on time series values detected by a plurality of monitoring nodes located in the system and network information of the wireless network, select subsystems associated with abnormal operation of the system based on one of a knowledge-based subsystem selection method, a data- driven subsystem selection method and a combination thereof, detect anomalies of one or more subsystems of the selected subsystems based on the one or more features and one or more decision boundaries for the selected subsystems, select one or more monitoring nodes belonging to the one or more subsystems and detect anomalies of at least one monitoring node of the selected one or more monitoring nodes based on the one or more features and one or more local decision boundaries for the selected one or more monitoring nodes. In some implementations, the system has a hierarchical structure including a root node corresponding to the system, subsystem nodes corresponding the subsystems and a plurality of leaf nodes corresponding to the plurality of monitoring nodes. In some implementations, each of the subsystems includes a subset of the plurality of monitoring nodes.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] For a better understanding of the various described implementations, reference should be made to the Description of Implementations below, in conjunction with the following drawings in which like reference numerals refer to corresponding parts throughout the Figures.5SUBSTITUTE SHEET (RULE 26)

[0009] Figure 1 shows a block diagram of an example anomaly detection system.

[0010] Figure 2 shows a high-level block diagram of an example anomaly detection system.

[0011] Figure 3 is an example of a table including flow-based features.

[0012] Figure 4 illustrates an example of a hierarchical tree structure for hierarchal top- down approach.

[0013] Figure 5 illustrates an example of a two-layer anomaly detection architecture of wireless system.

[0014] Figure 6 illustrates a block diagram of the threat detection computer platform.

[0015] Figure 7 shows a flowchart of an example method for detecting anomalies in a system.DESCRIPTION OF IMPLEMENTATIONS

[0016] Reference will now be made in detail to implementations, examples of which are illustrated in the accompanying drawings. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the various described implementations. However, it will be apparent to one of ordinary skill in the art that the various described implementations may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the implementations.

[0017] It will also be understood that, although the terms first, second, etc. are, in some instances, used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another.

[0018] For example, a first electronic device could be termed a second electronic device, and, similarly, a second electronic device could be termed a first electronic device, without departing from the scope of the various described implementations. The first electronic device and the second electronic device are both electronic devices, but they are not necessarily the same electronic device.

[0019] The terminology used in the description of the various described implementations herein is for the purpose of describing particular implementations only and is not intended to be6SUBSTITUTE SHEET (RULE 26)limiting. As used in the description of the various described implementations and the appended claims, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and / or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “includes,” “including,” “comprises,” and / or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0020] As used herein, the term “if’ is, optionally, construed to mean “when” or “upon” or “in response to determining” or “in response to detecting” or “in accordance with a determination that,” depending on the context. Similarly, the phrase “if it is determined” or “if [a stated condition or event] is detected” is, optionally, construed to mean “upon determining” or “in response to determining” or “upon detecting [the stated condition or event]” or “in response to detecting [the stated condition or event]” or “in accordance with a determination that [a stated condition or event] is detected,” depending on the context.

[0021] Figure 1 shows a block diagram of an example anomaly detection system.

[0022] The block diagram shows an overall architecture of an Al-based anomaly detection system (or an anomaly detection system) 100 for security of a cyber-physical system (e.g., industrial asset) and resilience architecture. The Al-based anomaly detection system 100 is provided for safety, cybersecurity and resilience for cyber-physical systems and devices physically co-located or connected through a communication network such as a wireless network or a network compatible with a 3GPP standard, such as a 5G network, 6G network, etc.

[0023] The anomaly detection system 100 may process real-time monitoring nodes raw values using anomaly detection and localization algorithms and then computes features which can then be compared to a decision boundaries.

[0024] The anomaly detection system 100 includes monitoring nodes 110 and a threat detection computer platform 114. The monitoring nodes 110 generate a series of current monitoring node values 112 over time. In some embodiments, the monitoring nodes 110 include, but are not limited, sensors, actuators, controller parameters / gains, a component, a subsystem located in the cyber-physical system. The series of current monitoring mode values 112 represent7SUBSTITUTE SHEET (RULE 26)a current operation of the cyber-physical system. The threat detection computer platform 114 receives the monitoring node raw values 112 and network data (e.g., state data and configuration data of a network connecting the one or more monitoring nodes).

[0025] The threat detection computer platform 114 may perform off-line training process to train an Al-based model via a training data set to detecting anomalies in the system and online operation process. During the off-line training process, time-series data (or the monitoring node data sets) are used for feature engineering and decision boundary generation. The online operation process is run in real-time to compare the node measurements (converted into the feature space) against the decision boundary and provide system status (normal, abnormal). In some implementations, the threat detection computer platform 114 is implemented as an apparatus including one or more processors and one or more memories.

[0026] A data pre-processing element 116 on the threat detection computer platform 114 filters and / or smooths noisy data (e.g., to address gaps in the data, bad data, outliers, etc.) in the received monitoring nodes raw values 112 and / or network data to output preprocessed data. The pre-processed data is provided to a feature extraction module 118. The feature extraction module 118 generates one or more feature vectors 119 (or feature information) for the current monitoring node using the current monitoring node signal values (or the current monitoring node raw values) in the received monitoring nodes raw values 112. The feature extraction module 118 generates the one or more feature vectors 119 via one of feature transforms, identity transforms, and feature-based dynamic models. In some embodiments, at least one of the current monitoring node feature vectors is associated with principal components, statistical features, deep learning features, frequency domain features, time series analysis features, logical features, geographic or position-based locations, and / or interaction features.

[0027] The one or more feature vectors 119 are provided to a global status determination module 120 (or a detection module 120) and a local status determination module 130 (a localization module 130). The global status determination module 120 may be referred to as a detection module and the local status determination module 130 may be referred to as a localization module. The global status determination module 120 generates anomaly decisions at a system level (e.g., the cyber-physical system) based on the one or more feature vectors and global decision boundary(ies) 122. The anomaly detection system 100 may process normal data and abnormal data while off-line using decision boundary algorithms and generate an anomaly8SUBSTITUTE SHEET (RULE 26)model including global decision boundary(ies) 122 for the global system. The global decision boundary(ies) 122 may be automatically calculated for an anomaly detection model based on the set of normal feature vectors and the set of abnormal feature vectors. In some embodiments, the global decision boundary(ies) 122 might be associated with a line, a hyperplane, a non-linear boundary separating normal space from abnormal space, and / or a plurality of decision boundaries. In some embodiments, the global decision boundary(ies) 122 might be a combined decision boundary based on a combination of monitoring nodes. The global status determination module 120 generates a global anomaly status 126 indicating whether the cyber-physical system is experiencing “normal” or “abnormal” operation. In some implementations, the global status determination module 120 may receive a policy related to the wireless network such as 5G policy to generate the global anomaly status 126. In some embodiments, the global status determination module 120 further generates a global confidence score 128 that represents how- confident the global status determination module 120 is about the anomaly status (risk of anomaly). For example, higher values of the global confidence score 128 indicates a greater likelihood of anomaly while lower values of the global confidence score 128 indicates lower likelihood of anomaly.

[0028] The local status determination module 130 generates anomaly decision at a local level (e.g., a monitoring node level) based on the one or more feature vectors and local decision boundaries 124. In some implementations, the local status determination module 120 may receive a policy related to the wireless network such as 5G policy to generate the global anomaly status 126. The local decision boundaries 124 may be automatically calculated for an anomaly detection model based on the set of normal feature vectors and the set of abnormal feature vectors. In some embodiments, the local decision boundaries 124 might be associated with a line, a hyperplane, a non-linear boundary separating normal space from abnormal space, and / or a plurality of decision boundaries for each monitoring node. For each monitoring node, the local determination module 130 generates a local anomaly status 132 indicating whether each monitoring node is experiencing “normal” or “abnormal” operation. The local status determination module 130 generates a local confidence score 134 that represents how confident the local determination module 130 is about the anomaly status (risk of anomaly) for each monitoring node. For example, higher values of the local confidence score 134 indicates a9SUBSTITUTE SHEET (RULE 26)greater likelihood of anomaly while lower values of the local confidence score 134 indicates lower likelihood of anomaly.

[0029] The global status determination module 120 and the local status determination module 130 may generate a relatively small number of “false positive” decisions and “false negative” decisions. For example, the false positive decisions indicate abnormal operation of a corresponding system or a corresponding monitoring node when the corresponding system or the corresponding monitoring node operates normal. The false negative decisions indicate normal operation of a corresponding system or a corresponding monitoring node when the corresponding system or the corresponding monitoring node operates abnormal. As a result, the global and local statuses could provide contradictory information. For example, the global anomaly status might indicate normal operation even when one or more local anomaly statuses indicate abnormal operation of one or more monitoring nodes. To address these situations, a decision fusion computer platform 136 receives the global anomaly status 126 and the local anomaly status 132, as well as the global score 128 and local score 134. In some embodiments, the local anomaly status 132 may itself be a combination of the status of multiple nodes. The decision fusion computer platform 136 generates a fused global status 129 and, for each monitoring node, a fused local status 133, in addition to a fused confidence score for the status 131 / 133, respectively. In some embodiments, the global score 128 maybe a single number, while the local score 134 may be a vector (multiple numbers) the same size as the local anomaly status 132. These statuses are “fused” in the sense that information is merged, and contradictory situations (e.g., the global anomaly status represents normal operation while one or more local anomaly status represent abnormal operation) may be avoided.

[0030] The anomaly detection system 100 includes a conformance test module 138. The conformance test module 138 receives the local anomaly status 132 and the local confidence score 134 (per a monitoring node) and determines on a node-by-node basis whether an abnormal local status is “independent” or “dependent” based on a casual dependency matrix, propagation paths, control loops time constraints, etc. The independent abnormal local status indicates that a corresponding abnormal local status is not caused by anomaly existing at another monitoring node. The dependent abnormal local status indicates that the corresponding abnormal local status is caused by anomaly existing at another monitoring node. The conformance test module 13810SUBSTITUTE SHEET (RULE 26)also receives output data from the decision fusion module 136 and use the output data in the conformance determination (independent or dependent).

[0031] The anomaly detection system 100 also includes a system / subsystem / node level anomaly forecasting and early warning system 140, which may receive the global anomaly status 126, the global confidence score 128 from the global status determination module 120, and the output data from the decision fusion computer 136 as input, and, in turn, output a forecasted status for early warning generation. The early warning may be communicated to an operator as an indication of emerging anomalies (cyber-threats, faults) for situational awareness, or may be used for early engagement of mitigation or neutralization strategies.

[0032] The anomaly detection system 100 also includes a system / subsystem / node level attack vs. fault separation module 142 which may also receive the global anomaly status 126 and the global confidence score 128 from the global status determination module 120, and output from the decision fusion computer 136 as input, and, in turn, outputs an attack / fault status 137 and / or a failure mode 139.

[0033] The anomaly detection system 100 also includes a neutralization module 144 which may receive the output (local status / confidence score) from the decision fusion computer 136 and the output (dependent / independent local statuses) of the conformance test module 138. The neutralization module 144 is to cancel the effect of the anomaly by calculating normal values for the affected nodes using values from monitoring nodes that have a normal anomaly status. These calculated normal values may be sent to a control system (virtual sensors / back up controller, controller) to neutralize (retum / system curtailment, shut down, etc.) the anomaly.

[0034] It is not described in Figure 1, the anomaly detection system 100 may include a postprocessing (such as alarm persistency check) module. The postprocessing module also applies to the outputs of Decision Fusion and Conformance. Furthermore, early engagement of neutralization may be triggered by the forecasting module (also not described in Figure 1).

[0035] Figure 2 shows a high-level block diagram of an example anomaly detection system.

[0036] The anomaly detection system 200 depicted in Figure 2 is a high-level block diagram of the anomaly detection system 100 which is depicted in Figure 1. The anomaly detection system 200 may be implemented as an apparats including one or more processors and one or more memories.11SUBSTITUTE SHEET (RULE 26)

[0037] The anomaly detection system 200 includes off-line modules 210 to perform offline training process and online modules 220 to perform on-line operation process. During the off-line training, time-series data 201 (or the monitoring node data sets) are used for feature engineering and decision boundary generation. The off-line modules 210 may access the normal data and abnormal data and process the normal data and abnormal data by decision boundary algorithm while off-line, e.g., for a two-class training. The off-line modules 210 may access the normal data only and process the normal data by decision boundary algorithm while off-line, e g., for a one-class training. The decision boundaries may be computed for each individual monitoring node using machine learning techniques. For example, some embodiments utilize Extreme Learning Machine (“ELM”) as a binary classification decision boundary. ELM is a special type of flashforward neural network recently developed for fast training. Numerous empirical studies, and recently some analytical studies as well, have shown that ELM has better generalization performance than other machine learning algorithms including Support Vector Machines (“SVM”) and is efficient and effective for both classification and regression. The online modules 220 are run in real-time to compare the node measurements (converted into the feature space) against the decision boundary and provide system status (normal operation or abnormal operation). In some embodiments, the off-line modules 210 may include modules for feature engineering and decision boundary generation. In some embodiments, the on-line module 220 may include elements or components (e.g., the data pre-processing element 116, the feature extraction module 118, a global status determination module 120, a local status determination module 130, etc.) included in the threat detection computer platform 114 depicted in Figure. 1.

[0038] The anomaly detection system 200 (or the online modules 220) receives timeseries data 201 (or monitoring nodes raw values 112 in Figure. 1) from a collection of monitoring nodes over the network devices and assets (sensor / actuators / controller nodes) and wireless network data 202 from the network connecting one or more systems, devices and assets. The wireless network data 202 may include YANG data, state information and configuration information of the network, but is not limited thereto. The anomaly detection system 200 process time-series data 201 and wireless network data 202 using anomaly detection and localization algorithm.

[0039] The anomaly detection system 200 (or the online modules 220) extracts features from the time series data 201 and wireless network data 202 for each monitoring node. In some12SUBSTITUTE SHEET (RULE 26)embodiments, the features may refer to, for example, mathematical characterizations of data. In some embodiments the features may include one of knowledge-based features, data-driven features and a combination thereof. The knowledge-based features are hand-crafted using domain knowledge and the data-driven features are computed using statistical or signal-based relations and / or deep-learning. Examples of features as applied to data might include the maximum and minimum, mean, standard deviation, variance, settling time, Fast Fourier Transform (“FFT”) spectral components, linear and non-linear principal components, independent components, sparse coding, deep learning, etc. The type and number of features for each monitoring node, might be optimized using domain-knowledge, feature engineering, or ROC statistics. The features are calculated over a sliding window of the signal time series. The length of the window and the duration of slide are determined from domain knowledge and inspection of the data or using batch processing. The features are computed at the local level associated with each particular monitoring node and the global level associated with the system or a part of the network. The time-domain values of the monitoring nodes or their extracted features may be normalized for better numerical conditioning.

[0040] Examples of input features include flow-based features, QoS features, collected via YANG data models available in the network, or any physical measurement from the network equipment (such a power profile or temperature), or the location of the node. In some embodiments, QoS features include QoS Identifier, IP Data Flow, Flow / Bearer Identifier, Reflective QoS and Data Session parameters. For 5G networks, the QoS features include 5G QoS parameters as follows: 5G QoS Identifier (5GQI), QoS Flow, QFI (QoS Flow Identifier), RQI (Reflective QoS ID) and PDU Session. For 4G networks, the QoS features include 4G QoS parameters as follows: QCI (QoS Class Identifier), EPC (Evolved Packet Core) Bearer, EBI (EPS Bearer ID) and PDN Connection. In some embodiments, the input features may include potential input features as follows:1 Flow duration Duration of the flow in Microsecond2 Total Fwd Packet Total packets in the forward direction3 Total Bwd Packets Total packets in the backward direction4 Total Length of Fwd Packet Total size of packet in forward direction5 Total Length of Bwd Packet Total size of packet in backward direction6 Fwd Packet Length Min Minimum size of packet in forward direction13SUBSTITUTE SHEET (RULE 26)7 Fwd Packet Length Max Maximum size of packet in forward direction8 Fwd Packet Length Mean Mean size of packet in forward direction9 Fwd Packet Length Std Standard deviation size of packet in forward direction10 Bwd Packet Length Min Minimum size of packet in backward direction11 Bwd Packet Length Max Maximum size of packet in backward direction12 Bwd Packet Length Mean Mean size of packet in backward direction13 Bwd Packet Length Std Standard deviation size of packet in backward direction14 Flow Bytes / s Number of flow bytes per second15 Flow Packets / s Number of flow packets per second16 Flow IAT Mean Mean time between two packets sent in the flow17 Flow IAT Std Standard deviation time between two packets sent in the flow18 Flow IAT Max Maximum time between two packets sent in the flow19 Flow IAT Min Minimum time between two packets sent in the flow20 Fwd IAT Min Minimum time between two packets sent in the forward direction21 Fwd IAT Max Maximum time between two packets sent in the forward direction22 Fwd IAT Mean Mean time between two packets sent in the forward direction23 Fwd IAT Std Standard deviation time between two packets sent in the forward direction24 Fwd IAT Total Total time between two packets sent in the forward direction25 Bwd IAT Min Minimum time between two packets sent in the backward direction26 Bwd IAT Max Maximum time between two packets sent in the backward direction27 Bwd IAT Mean Mean time between two packets sent in the backward direction28 Bwd IAT Std Standard deviation time between two packets sent in the backward direction29 Bwd IAT Total Total time between two packets sent in the backward direction30 Fwd PSH flags Number of times the PSH flag was set in packets travelling in the forward direction (0 for UDP)31 Bwd PSH Flags Number of times the PSH flag was set in packets travelling in the backward direction (0 for UDP)32 Fwd URG Flags Number of times the URG flag was set in packets travelling in the forward direction (0 for UDP)33 Bwd URG Flags Number of times the URG flag was set in packets travelling in the backward direction (0 for UDP)14SUBSTITUTE SHEET (RULE 26)Fwd Header Length Total bytes used for headers in the forward directionBwd Header Length Total bytes used for headers in the backward directionFWD Packets / s Number of forward packets per secondBwd Packets / s Number of backward packets per secondPacket Length Min Minimum length of a packetPacket Length Max Maximum length of a packetPacket Length Mean Mean length of a packetPacket Length Std Standard deviation length of a packetPacket Length Variance Variance length of a packetFIN Flag Count Number of packets with FINSYN Flag Count Number of packets with SYNRST Flag Count Number of packets with RSTPSH Flag Count Number of packets with PUSHACK Flag Count Number of packets with ACKURG Flag Count Number of packets with URGCWR Flag Count Number of packets with CWRECE Flag Count Number of packets with ECE down / Up Ratio Download and upload ratioAverage Packet Size Average size of packetFwd Segment Size Avg Average size observed in the forward directionBwd Segment Size Avg Average number of bytes bulk rate in the backward directionFwd Bytes / Bulk Avg Average number of bytes bulk rate in the forward directionFwd Packet / Bulk Avg Average number of packets bulk rate in the forward directionFwd Bulk Rate Avg Average number of bulk rate in the forward directionBwd Bytes / Bulk Avg Average number of bytes bulk rate in the backward directionBwd Packet / Bulk Avg Average number of packets bulk rate in the backward directionBwd Bulk Rate Avg Average number of packets bulk rate in the backward directionBwd Bulk Rate Avg Average number of bulk rate in the backward directionSubflow Fwd Packets The average number of packets in a sub flow in the forward directionSubflow Fwd Bytes The average number of bytes in a sub flow in the forward directionSubflow Bwd Packets The average number of packets in a sub flow in the backward direction15SUBSTITUTE SHEET (RULE 26)65 Subflow Bwd Bytes The average number of bytes in a sub flow in the backward direction66 Fwd Init Win bytes The total number of bytes sent in initial window in the forward direction67 Bwd Init Win bytes The total number of bytes sent in initial window in the backward direction68 Fwd Act Data Pkts Count of packets with at least 1 byte of TCP data payload in the forward direction69 Fwd Seg Size Min Minimum segment size observed in the forward direction70 Active Min Minimum time a flow was active before becoming idle71 Active Mean Mean time a flow was active before becoming idle72 Active Max Maximum time a flow was active before becoming idle73 Active Std Standard deviation time a flow was active before becoming idle74 Idle Min Minimum time a flow was idle before becoming active75 Idle Mean Mean time a flow was idle before becoming active76 Idle Max Maximum time a flow was idle before becoming active77 Idle Std Standard deviation time a flow was idle before becoming active

[0041] The anomaly detection system 200 (or the off-line modules 210) includes one or more high-fidelity physics-based model associated with the one or more monitoring nodes to create normal data and / or abnormal data which is processed by decision boundary algorithms. The decision boundary algorithms may generate an anomaly model including decision boundaries for the cyber-physical systems and monitoring nodes. In some embodiments, the decision boundary may be computed by a classifier which is trained off-line using both two-class and one-class supervised training approached with a combination of field data and / or simulated data as training dataset.

[0042] The anomaly detection system 200 may include an individual model (individual digital twins) for each monitoring node (e.g., asset / device) and a collective model used for anomaly detection of each monitoring node. The individual model generates training datasets for each monitoring node, the collective model is an online module (e.g., the online module 220) which runs in real-time to compare the node measurements (converted into the feature space) against the decision boundary and provide system status (normal, abnormal).

[0043] Since some connected assets might be very complex or have too many variants, the anomaly detection system 200 (or the off-line modules 210) may utilize data-driven digital16SUBSTITUTE SHEET (RULE 26)twins to generate normal / abnormal training datasets. Furthermore, if any domain-knowledge if available (e.g., from physics, network topology, etc.), it can be combined into the digital twin as a hybrid model.

[0044] The two-class supervised training includes two-class supervised learning using physics-based digital twins and two-class supervised learning using data-driven digital twins, two-class supervised learning using physics-based digital twins is the most accurate approach which consists of dataset generation for both normal and abnormal space and supervised machine learning (extensive simulations of both normal operations and attacks). The classifier is trained using this approach when the high-fidelity physics-based digital twin is readily available and very high accuracies are required. The two-class supervised learning using data-driven digital twins uses the data-driven digital twins which are developed with medium to low fidelity. The data-driven digital twins are used for synthetic dataset generation of the abnormal space (and potentially the normal space) in conjunction with available normal-space data from the field.

[0045] Detection of cyber-attacks with high accuracy in wireless networks is critical to maintain safe operation and / or take necessary corrective action. The anomaly detection system 200 with high-definition simulation models are hence trained on extensive normal and attack simulation data to achieve the high detection accuracy. However, the anomaly detection system 200 with access to no simulation models and no / limited attack data cannot be developed using the same paradigm due to imbalance of classes. Thus, the one-class supervised training is employed in to detect cyberattacks which is trained only using normal simulation data, normal historical field data or a combination of both. The one-class supervised training includes one- class supervised learning without digital twins and one-class supervised learning with digital twins. The one-class supervised learning without digital twins may be used to compute decision boundaries using normal data only from the field. In this approach, the detection system is trained only on historical field data thereby eliminating dependence on availability of high- definition simulation model and / or substantial amount of attack data. The one-class supervised learning with digital twins uses the digital twins (e.g., physics-based or data-driven digital twin) to simulate a normal operation. The simulated data may be used solely or conjunction with normal fields data as training dataset. No attack simulations are performed.

[0046] The anomaly detection system 200 (or the off-line modules 210) includes an ensemble of distributed detection agent models, in which the agents will work together and share17SUBSTITUTE SHEET (RULE 26)information in the machine learning feature extraction (i.e., the latent input space to the anomaly classifier), or the anomaly decision (i.e., decision fusion) to have a robust, rapid and vetted detection against coordinate attacks with a law false positive rate. The ensemble model relies on an underlying machine / deep learning based multi-class classifier for change detection and classification. The classifier, in addition to event classification, provides associated confidence score based on conformal prediction methods.

[0047] The anomaly detection system 200 (or the off-line modules 210) provides an anomaly decision (event status) as detection and classification (isolation and type identification), anomaly score (e.g., a global confidence score 128, a local confidence score 134), confidence / trust score (e.g., fused confidence score) about its decision using conformal prediction methods based on an ensemble model. The ensemble model also provides an overall anomaly decision / classification, anomaly score, and confidence score as the fused information among the agents for the overall ensemble.

[0048] The anomaly detection system 200 monitors the plurality of monitoring nodes (e.g., 110) connected in the wireless network, and within each node, a plurality of critical sensors, actuators and parameters, to detect and isolate anomalies and generate alarms in the presence of anomalies and hazards. Furthermore, the anomaly detection system 200 includes Al models which is trained based on machine learning and deep learning methods to learn the physical behavior of cyber-physical systems using simulation data and / or historical data from network operations. Thus, the anomaly detection system 200 is attack-type agnostic with a unified structure receiving a common input base from the wireless network and being able to detect multiple attack types as system anomalies, and ideally identify the type of the attack as well.

[0049] The anomaly detection system 200 also performs distributed and partially distributed anomaly detection and localization for safety, cybersecurity and resilience for cyberphysical systems and devices connected through the wireless network. The wireless network (e.g., 5G, 6G, etc.) includes a large-scale system (or a system) with a plurality of monitoring nodes. Here, the large-scale system may include industrial asset. In some embodiments, the monitoring nodes include, but are not limited, sensors, actuators, controller parameters / gains, a component, a subsystem (e.g., cyber-physical systems connected through the wireless network) located in the large-scale system (e.g., the large-scale system including cyber-physical systems18SUBSTITUTE SHEET (RULE 26)connected through the wireless network). The subsystem may include one or more components. The subsystem may be a subset of monitoring nodes. The subsystem may correspond to a monitoring node. The anomaly detection decision at a subsystem level may use a global decision boundary which may be calculated for the large-scale system and decision boundaries for each subsystem may be computed based on features including local features for each component located in a corresponding subsystem and interacting features of two or more of components. Thus, the anomaly detection system 200 may select a subsystem based on a subsystem selection algorithm (or a subsystem selection method) to optimize the number and boundaries of the subsystems. The subsystem selection algorithm includes a knowledge-based subsystem selection method and a data-driven subsystem selection.

[0050] According to the knowledge-based subsystem selection method, subsystems may be selected using the natural layout of the system and the network architecture. Thus, the anomaly detection system 200 may follow the wireless network architecture and hierarchy of the wireless network through which subsystems are connected and select one or more subsystems.

[0051] The data-driven subsystem selection method includes a data clustering method and a graph-based method. The data clustering method is to cluster the plurality of monitoring nodes located in the system into one or more cluster. The anomaly detection system 200 may cluster the plurality of monitoring nodes located in the system into one or more cluster using the data clustering method. Monitoring nodes that belong to the same cluster may form a subsystem. During clustering, data associated with a specific node might scatter among two or more clusters depending on the training dataset. In order to make the localization decisions, the anomaly detection system 200 allocates monitoring nodes to mutually exclusive clusters. Thus, the anomaly detection system 200 may assign each monitoring node to one subsystem based on different criteria, such as a distance of data of the monitoring node (e.g., a particular device in an loT network, such as a sensor or a robotic manipulator, or a component of the network infrastructure hardware) to the cluster centroids of clusters, based on a Euclidian or Mahalanobis distance. In some embodiments, the date of the monitoring node is the training data for each node (simulated or field) and consists of the input features as described above. This cluster data- driven clustering is an off-line process to come up with a hierarchical tree structure. Each monitoring node has time-series associated with its normal or both normal and abnormal behavior acquired as historical field data or generated offline for training. Thus, the anomaly19SUBSTITUTE SHEET (RULE 26)detection system 200 may use time-series clustering method to cluster the monitoring nodes. For example, correlation heat map may be computed for all monitoring nodes. The correlation coefficient -1 < r < 1 , can be used as a metric for distance among the monitoring nodes in a clustering algorithm. For hierarchical clustering, the distance may be defined as d = 1 - |r|, and for k-means clustering, the Euclidian distance may be defined as d =.

[0052] The Graph-based method is to partition the system based on the map of network (or wireless n network) using graph theory. The graph theory is the study of graphs which are mathematical structures used to model pairwise relations between monitoring nodes in the network. The graph theory provides mathematical object that naturally encodes relationships and hence provides a robust framework to build such applications. For example, with the data cast as a graph, the graph identifies a small subset of monitoring nodes with much higher volume of network traffic, than is typical for those monitoring nodes which may indicate the onset of some malicious activity (abnormal operation of those monitoring nodes).

[0053] The anomaly detection system 200 also adopt a hybrid approach combining two methods above to optimize the number and boundaries of the subsystems. Although the knowledge-based subsystem selection method and the data-driven subsystem selection are described herein, note that other methods might be used instead in any of the embodiments.

[0054] The subsystem selection algorithm may result in the system to have a hierarchical tree structure (or hierarchical structure) with a set of connected nodes. In the hierarchical tress structure, the leaf node (e.g., the bottommost node of the hierarchical tree structure) corresponding to a monitoring node at component level (e.g., UE, or a component of the subsystem or system) represents a lowest level. The root node (e.g., the topmost node of the hierarchical tree structure) corresponding to the system (e.g., the large-scale system) represents a highest level. The lowest level may be referred to as a node level and the highest level may be referred to as a global level.

[0055] Figure 3 is an example of a table including flow-based features.

[0056] The wireless network data 202 may include flow-based features collected via YANG data models available in the network and power-flow features communicated by an UE. As described with respect to Figure. 2, examples of input features include flow-based features collected via YANG data models available in the network, power, and location of the nodes in accordance with 3GPP standards (3GPP TS 133.501 V17.8.0). For example, the anomaly20SUBSTITUTE SHEET (RULE 26)detection system (e.g. the anomaly detection system 200) may use the power strength measurement and location transmitted by the UE as part of its input features, to locate the anomaly as well as identify a specific type of an attack such as a false base station attack. 3GPP TS 133.501 V17.8.0 is incorporated by reference herein in its entirely.

[0057] Figure 4 illustrates an example of a hierarchical tree structure for hierarchal top- down approach.

[0058] The anomaly detection system (e.g., the anomaly detection system 100 or the anomaly detection system 200) selects a subsystem based on a subsystem selection algorithm to optimize the number and boundaries of the subsystems. The subsystem selection algorithm may result in the system to have a hierarchical tree structure 400 with a set of connected nodes as show in Figure 4. In the hierarchical tress structure 400, a box 410 represents a root node and boxes 420 represents leaf nodes. The root node (e.g., the topmost node of the hierarchical tree structure) corresponding to the system (e.g., the large-scale system) represents a highest level and the leaf node (e.g., the bottommost node of the hierarchical tree structure) corresponding to a component or a monitoring node (e.g., 110) at a component level (e.g., UE, or an endpoint component of the system) represents a lowest level. The highest level may be referred to as a system level and the lowest level may be referred to as a node level (or a component level).

[0059] The anomaly detection system monitors the system from the highest level to the lowest level of the hierarchical tree structure 400 based on a hierarchal top-down approach. The hierarchal top-down approach is used to localize attacks in the system. According to the subsystem selection algorithm, the system has two or more layers of subsystems before the anomaly detection system reaches the node level. Each subset may include a subset of a plurality of monitoring nodes. In Figure 4, boxes 430 represent first nodes corresponding to first subsystems in a first level of the hierarchical tree structure 400. The first subsystem may include first nodes (e.g., 5G Core layer of wireless network). Boxes 440 represent second nodes corresponding to second subsystems in a second level of the hierarchical tree structure 400. The second subsystem may include second layer nodes (e.g., RAN layer of wireless network) which corresponds to a lower layer than a layer of the first nodes.

[0060] The anomaly detection system may compute separate classification decision boundaries (or subsystem decision boundaries or a local decision boundaries) for each sub systems. In the hierarchal top-down approach, each classifier to compute decision boundary for21SUBSTITUTE SHEET (RULE 26)each subsystem is run only if the classifier at the higher level to compute a decision boundary for the system is reporting an abnormal status.

[0061] Thus, the anomaly detection system selects the first subsystems based on subsystem selection algorithm including a knowledge-based subsystem selection method and a data-driven subsystem selection when the classifier of the root node reports that the system operates abnormal. If one or more first subsystems among the selected first subsystems operate abnormal (e.g., first node SL and first node SL) then the anomaly detection system selects second subsystems as child nodes of the one or more first nodes corresponding to abnormal subsystems based on subsystem selection algorithm. The anomaly detection system may select and detect subsystems until reaching to the monitoring nodes and detecting one or more abnormal monitoring nodes (e.g., leaf node M7 and leaf node M2).

[0062] Note that, in multi-prong attacks, or as a result of propagation of the anomaly from one node to others thought the feedback control system, multiple subsystems and monitoring nodes might report anomaly simultaneously.

[0063] Each agent in the ensemble model, could be an entire security system as described in Figure 1 or any simplified version of it contains all or a subset of its modules and functionality in accordance with 3GPP specifications (5G system support for AI / ML-based Service (Release 18) and 3GPP TR 23.700-80 VI.1.0 (2022-10)). The 3GPP specifications including 5G system support for AI / ML-based Service (Release 18) and 3GPP TR 23.700-80 VI.1.0 (2022-10) are incorporated by reference herein their entirety. Each agent is associated with a monitoring node. In some embodiments, the monitoring node may be a single device, or a set of devices connected to each other. Each agent may share it anomaly decisions, anomaly score, confidence score, or computed features with the upper-level agent. Furthermore, at each level, the agents may share information for collaborative and federated learning. The data sharing among agents may be based on any mechanism available at that sector / layer of the network or through homographic encryption. The distributed anomaly detection system may use 5G Core assistance for federated learning to ensure proper allocation of network resources.

[0064] Figure 5 illustrates an example of a two-layer anomaly detection architecture of wireless system.

[0065] The network may have two-layer structure 500 including a 5G Core (5GC) network layer and a Radio Access Network (RAN) layer. 5GC network layer may correspond to22SUBSTITUTE SHEET (RULE 26)first subsystems in a first level of the hierarchical tree structure 400 and RAN layer may correspond to second subsystems in a second level of the hierarchical tree structure 400 in Figure 4. Thus, the RAN layer is in a lower level than a level to where the 5GC network layer belongs in the hierarchical tree structure 400. Figure 5 shows an example of how RAN layer level information is shared with the 5GC layer.

[0066] The anomaly detection system (e.g., the anomaly detection system 100 or the anomaly detection system 200) may include an anomaly symptom detector at RAN level (RAN level agent) and a network anomaly detector at 5G Core level (5GC level agent). The anomaly symptom detector at RAN level may collect information available RAN level and monitor RAN signals using the collected information to detect anomaly at the RAN level. The network anomaly detector at 5GC layer may receive RAN level information from the anomaly symptom detector at RAN level and collects data from the core network to detect the network anomaly. In some embodiments, the RAN level information from the anomaly symptom detector at RAN level may include Radio level RF (I / Q digital streams raw data), power consumption if available through RAN interface, modulation, demodulation, channel estimation, utilization of channel (network management), aggregated information between multiple network management and antenna port (physical access through the radio hardware).

[0067] Figure 6 illustrates a block diagram of the threat detection computer platform.

[0068] Note that the embodiments described herein may be implemented using any number of different hardware configurations. Figure 6 illustrates a block diagram of threat detection computer platform 114 that may be, for example, associated with the anomaly detection system 100 and the anomaly detection system 200 of Figure 1 and Figure 2 and / or any other system described herein.

[0069] The threat detection computer platform 600 may be an apparatus, a computing device or a computer including one or more processors 620, such as one or more commercially available Central Processing Units (“CPUs”) in the form of one-chip microprocessors, coupled to a communication device 610 configured to communicate via a communication network (not shown in Figure 6). The communication device 610 may be used to communicate, for example, with one or more remote monitoring nodes (e.g., 110) (industrial assets or cyber-physical system), user platforms, digital twins, etc. The threat detection computer platform 600 further includes an input device (e.g., a computer mouse and / or keyboard to input industrial asset and / or23SUBSTITUTE SHEET (RULE 26)predictive modeling information) and / an output device (e.g., a computer monitor to render a display, provide alerts, transmit recommendations, and / or create reports). According to some embodiments, a mobile device, monitoring physical system, and / or PC may be used to exchange information with the threat detection computer platform 600,

[0070] The one or more processors 620 also communicate with a storage device 630. The one or more processors 620 execute modules, programs and / or instructions stored in the memory 308 and thereby perform processing operations, including the methods described herein. The storage device 630 may comprise any appropriate information storage device, including combinations of magnetic storage devices (e.g., a hard disk drive), optical storage devices, mobile telephones, and / or semiconductor memory devices. The storage device 630 stores one or more programs 631 and / or threat detection engine 632(or module), and / or data structures, collectively referred to as “modules” herein for controlling the one or more processors 620. The one or more processors 620 perform instructions of the programs 631, 632, and thereby operates in accordance with any of the embodiments described herein. For example, the one or more processors 620 are configured to determine one or more features based on time series values detected by a plurality of monitoring nodes located in the system and network information of the wireless network, determine a global status representing normal operation or abnormal operation of the system based on the one or more features and one or more global decision boundaries, in response to determining that the global status represents the abnormal operation of the system, selecting subsystems based on one of a knowledge-based subsystem selection method, a data- driven subsystem selection method and a combination thereof, detect anomalies of one or more subsystems of the selected subsystems based on the one or more features and one or more decision boundaries for the selected subsystems, select one or more monitoring nodes belonging to the one or more subsystems and detect anomalies of at least one monitoring node of the selected one or more monitoring nodes based on the one or more features and one or more local decision boundaries for the selected one or more monitoring nodes. In some implementations, each of the subsystems includes a subset of the plurality of monitoring nodes.

[0071] In some implementations, the one or more processors 620 are configured to determine one or more features based on time series values detected by a plurality of monitoring nodes located in the system and network information of the wireless network, select subsystems associated with abnormal operation of the system (e.g., first nodes S1, SX4 in Figure 4, second24SUBSTITUTE SHEET (RULE 26)nodes S22i, S24i in Figure 4) based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof, detect anomalies of one or more subsystems of the selected subsystems based on the one or more features and one or more decision boundaries for the selected subsystems, select one or more monitoring nodes belonging to the one or more subsystems and detect anomalies of at least one monitoring node (e.g., monitoring nodes M3, M7, M2, Muin Figure 4) of the selected one or more monitoring nodes based on the one or more features and one or more local decision boundaries for the selected one or more monitoring nodes. In some implementations, the system has a hierarchical structure including a root node corresponding to the system, subsystem nodes corresponding the subsystems and a plurality of leaf nodes corresponding to the plurality of monitoring nodes and each of the subsystems includes a subset of the plurality of monitoring nodes.

[0072] For example, the one or more processors 620 may receive from a plurality of monitoring nodes that each generate a time-series of current monitoring node values representing current operation of a cyber-physical system (or a system). The one or more processors 620 may receive the time-series of and generate a set of current feature vectors using feature discovery techniques. The feature behavior for each monitoring node may be characterized by one or more processors 620 in the form of decision boundaries that separate normal and abnormal space based on operating data of the system. In some implementations, the one or more features include flow- based features and QoS features of the wireless network, the QoS features include QoS Identifier, IP data flow, flow or bearer identifier, reflective QoS identifier and data session information. A set of ensemble state-space models may be constructed by one or more processors 620 to represent feature evolution in the time-domain, the forecasted outputs from the set of ensemble state-space models comprise anticipated time evolution of features. The one or more processors 620 may then obtain overall features forecast through dynamic ensemble averaging and compare the overall features forecast to a threshold to generate an estimate associated with at least one feature vector crossing an associated decision boundary. In some implementations, the system has a hierarchical structure including a root node corresponding to the system, subsystem nodes corresponding the subsystems and a plurality of leaf nodes corresponding to the plurality of monitoring nodes. The one or more processors 620 may monitors the system from the highest level to the lowest level of the hierarchical tree structure based on a hierarchal top-down approach. The hierarchal top-down approach is used to localize25SUBSTITUTE SHEET (RULE 26)attacks in the system. According to the subsystem selection algorithm, the system has two or more layers of subsystems before the anomaly detection system reaches the node level. The one or more processors 620 select subsystems based on subsystem selection algorithm including a knowledge-based subsystem selection method and a data-driven subsystem selection when the classifier of the root node reports that the system operates abnormal. For example, one or more processors 620 select the subsystems following network architecture and hierarchy based on the knowledge-based subsystem selection method. For example, the one or more processors 620 cluster the plurality of monitoring nodes into one or more clusters based on the data clustering method, a cluster corresponding a subsystem. In some implementations, each monitoring node of the plurality of monitoring nodes is assigned to a cluster based on a distance from each monitoring node to a cluster centroid of the cluster based on Euclidian or Mahalanobis distance so that each monitoring node belongs to a single cluster. For example, the one or more processors 620 select the subsystems based on the graph-based method by partitioning the system based on a graph map of the network indicating volume of network traffic. The one or more processors 620 may compute separate classification decision boundaries for each sub systems. The one or more processors 620 may select and detect subsystems until reaching to the monitoring nodes and detecting one or more abnormal monitoring nodes. For example, the one or more processors 620 execute an ensemble of distributed digital twin models. In some implementations, each of the digital twin models associated with each of the plurality of monitoring nodes. The one or more processors 620 may use the ensemble of distributed digital twin models associated with the system and provide anomaly decision for the system, the one or more global decision boundaries for the system and a global score representing a likelihood of the anomalies for the system. In some implementations, the one or more processors 620 may use a digital twin model associated with each of the plurality of monitoring nodes and provide anomaly decision for the one or more subsystems and the at least one monitoring node, the one or more decision boundaries for the selected subsystems, the one or more local decision boundaries for the one or more monitoring nodes and one or more scores representing a likelihood of the anomalies for the one or more subsystems and the at least one monitoring node, each of the digital twin models associated with each of the plurality of monitoring nodes. In some implementations, the wireless network includes a network compatible with a 3 GPP standard. In some implementations, the wireless network includes two-layer structures including26SUBSTITUTE SHEET (RULE 26)a 5G Core (5GC) network layer and a Radio Access Network (RAN) layer as discussed above with respect to Figure 5. The 5GC network layer is higher than the RAN layer. In some implementations, the apparatus 600 includes, but is not limited to, a first module (at RAN level) and a second module (at 5GC level). The first module is configured to collet RAN level information including radio level information, modulation and demodulation information, channel estimation information, aggregated information between multiple network management and antenna port information at a RAN layer to monitor signal at the RAN layer to detect anomalies of the RAN layer and a second module is configured to receive the collected RAN level information and detect anomalies of the 5GC network layer based on the received RAN level information.

[0073] The programs 631, 632 may be stored in a compressed, uncompiled and / or encrypted format. The programs 631, 632 may furthermore include other program elements, such as an operating system, clipboard application, a database management system, and / or device drivers used by the one or more processors 620 to interface with peripheral devices. In some implementations, the programs 631, 632 may include instructions for determining one or more features based on time series values detected by a plurality of monitoring nodes located in the system and network information of the wireless network, determining a global status representing normal operation or abnormal operation of the system based on the one or more features and one or more global decision boundaries, in response to determining that the global status represents the abnormal operation of the system, selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof, detecting anomalies of one or more subsystems of the selected subsystems based on the one or more features and one or more decision boundaries for the selected subsystems, selecting one or more monitoring nodes belonging to the one or more subsystems and detecting anomalies of at least one monitoring node of the selected one or more monitoring nodes based on the one or more features and one or more local decision boundaries for the selected one or more monitoring nodes. In some implementations, each of the subsystems includes a subset of the plurality of monitoring nodes.

[0074] In some implementations, the plurality of monitoring nodes includes at least one of a sensor, an actuator and a user device associated with the wireless network. In some implementations, the one or more features include flow-based features and QoS features of the 1SUBSTITUTE SHEET (RULE 26)wireless network, the QoS features include QoS Identifier, IP data flow, flow or bearer identifier, reflective QoS identifier and data session information. In some implementations, the selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof includes selecting the subsystems following network architecture and hierarchy based on the knowledge-based subsystem selection method. In some implementations, the data-driven subsystem selection method includes a data clustering method, the selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof includes clustering the plurality of monitoring nodes into one or more clusters based on the data clustering method, a cluster corresponding a subsystem. In some implementations, each monitoring node of the plurality of monitoring nodes is assigned to a cluster based on a distance from each monitoring node to a cluster centroid of the cluster based on Euclidian or Mahalanobis distance so that each monitoring node belongs to a single cluster. In some implementations, the data- driven subsystem selection method includes a graph-based method, the selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof includes selecting the subsystems based on the graph-based method by partitioning the system based on a graph map of the network indicating volume of network traffic. In some implementations, the system has a hierarchical structure including a root node corresponding to the system, subsystem nodes corresponding the subsystems and a plurality of leaf nodes corresponding to the plurality of monitoring nodes. In some implementations, the programs 631, 632 may include instructions for using an ensemble of distributed digital twin models associated with the system and providing anomaly decision for the system, the one or more global decision boundaries for the system and a global score representing a likelihood of the anomalies for the system. In some implementations the programs 631, 632 may include instructions for using a digital twin model associated with each of the plurality of monitoring nodes and providing anomaly decision for the one or more subsystems and the at least one monitoring node, the one or more decision boundaries for the selected subsystems, the one or more local decision boundaries for the one or more monitoring nodes and one or more scores representing a likelihood of the anomalies for the one or more subsystems and the at least one monitoring node, each of the digital twin models associated with each of the plurality of monitoring nodes. In some implementations, the wireless network includes a network28SUBSTITUTE SHEET (RULE 26)compatible with a 3GPP standard. In some implementations, the wireless network includes two- layer structures including a 5G Core (5GC) network layer and a Radio Access Network (RAN) layer as discussed above with respect to Figure 5. The 5GC network layer is higher than the RAN layer. In some implementations, the programs 631, 632 may include instructions for collecting RAN level information including radio level information, modulation and demodulation information, channel estimation information, aggregated information between multiple network management and antenna port information at a RAN layer to monitor signal at the RAN layer to detect anomalies of the RAN layer, receiving receive the collected RAN level information and detecting anomalies of the 5GC network layer based on the received RAN level information.

[0075] As used herein, information may be “received” by or “transmitted” to, for example: (i) the threat detection computer platform 600 from another device; or (ii) a software application or module within the threat detection computer platform 600 from another software application, module, or any other source. In some embodiments, the storage device 630 further stores a database 633 to store anomaly detection result from the or update the anomaly detection result.

[0076] Figure 7 shows a flowchart of an example method for detecting anomalies in a system.

[0077] The method 700 can be executed on an apparatus for detecting anomalies in a system associated with a wireless network (e.g., a computing device, a computer, etc.). The system may includes industrial assets (e.g., the assets 302). The method (700) includes determining (710) one or more features based on time series values detected by a plurality of monitoring nodes located in the system and network information of the wireless network. In some implementations, the method includes pre-processing time series values detected by a plurality of monitoring nodes located in the system and network information of a network connecting the plurality of monitoring nodes. In some embodiments, the monitoring nodes 110 include, but are not limited, sensors, actuators, controller parameters / gains, a component, a subsystem located in the cyber-physical system. The monitoring nodes 110 may be physically co-located or connected through a wired or wireless network (in the context of loT over 5G, 6G or Wi-Fi 6)) of industrial assets. The method also includes extracting the one or more features from the time series values and the network information. In some embodiments, the one or more29SUBSTITUTE SHEET (RULE 26)features include flow- based features and QoS features of the network, the QoS features include QoS Identifier, IP data flow, flow or bearer identifier, reflective QoS identifier and data session information. In some embodiments, the one or more features include one of knowledge-base features, data-driven features and a combination thereof, the knowledge-base features are handcrafted using domain knowledge, and the data driven features are computed using statistical or signal- based relations and deep learning. The method also includes determining (720) a global status representing whether normal operation or abnormal operation of the system based on the one or more features and one or more global decision boundaries. In response to determining that the global status represents the abnormal operation of the system, the method includes selecting (730) subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof. In some implementations, each of the subsystems includes a subset of the plurality of monitoring nodes. The method also includes detecting (740) anomalies of one or more subsystems of the selected subsystems based on the one or more features and one or more decision boundaries for the selected subsystems. The method also includes selecting (750) one or more monitoring nodes belonging to the one or more subsystems. The method also includes detecting (760) anomalies of at least one monitoring node of the selected one or more monitoring nodes based on the one or more features and one or more local decision boundaries for the selected one or more monitoring nodes.

[0078] In some embodiments, the method further includes selecting subsystems following network architecture and hierarchy based on the knowledge-based subsystem selection method. In some embodiments, the method further includes clustering the plurality of monitoring nodes into one or more clusters based on the data clustering method, a cluster corresponding a subsystem, wherein a monitoring node of the plurality of monitoring nodes is assigned to a cluster based on a distance from the monitoring node to a cluster centroid of the cluster based on Euclidian or Mahalanobis distance so that each monitoring node belongs to a single cluster. In some embodiments, the method further includes selecting subsystems based on the graph-based method by partitioning the system based on a graph map of the network indicating volume of network traffic. In some embodiments, the system has a hierarchical structure including a root node corresponding to the system, subsystem nodes corresponding the subsystems and a plurality of leaf nodes corresponding to the plurality of monitoring nodes. In some implementations, the method comprises using an ensemble of distributed digital twin models associated with the30SUBSTITUTE SHEET (RULE 26)system, providing anomaly decision for the system, the one or more global decision boundaries for the system and a global score representing a likelihood of the anomalies for the system. In some implementations, the method comprises using a digital twin model associated with each of the plurality of monitoring nodes, providing anomaly decision for the one or more subsystems and the at least one monitoring node, the one or more decision boundaries for the selected subsystems, the one or more local decision boundaries for the one or more monitoring nodes and one or more scores representing a likelihood of the anomalies for the one or more subsystems and the at least one monitoring node, each of the digital twin models associated with each of the plurality of monitoring nodes. In some embodiments, the wireless network includes a network compatible with a 3GPP standard.

[0079] In some embodiments, the wireless network includes two-layer structures including a 5G Core (5GC) network layer and a Radio Access Network (RAN) layer as shown in Figure 5. The 5GC network layer is higher than the RAN layer. In some embodiments, the method further includes colleting RAN level information including radio level information, modulation and demodulation information, channel estimation information, aggregated information between multiple network management and antenna port information at the second layer to monitor signal at the second layer to detect anomalies at the RAN layer. In some embodiments, the method further includes receiving the collected RAN level information and detecting anomalies of the 5GC network layer based on the collected RAN level information.

[0080] In some embodiments, the method further includes computing the one or more global decision boundaries for the system and a global score representing a likelihood of anomalies. In some embodiments, the method further includes computing the one or more decision boundaries for the selected subsystems and one or more score representing a likelihood of anomalies. In some embodiments, the method further includes generating an alarm that alerts an operator of the system based on the detected anomalies. In some embodiments, the method further includes transmitting the detected anomalies to a cyber fault neutralization system configured to neutralize the detected cyber-faults in the industrial assets. The term cyber-fault may refer to any cyber incident or naturally occurring fault.

[0081] The foregoing description, for purpose of explanation, has been described with reference to specific implementations. However, the illustrative discussions above are not intended to be exhaustive or to limit the scope of the claims to the precise forms disclosed. Many31SUBSTITUTE SHEET (RULE 26)modifications and variations are possible in view of the above teachings. The implementations are chosen in order to best explain the principles underlying the claims and their practical applications, to thereby enable others skilled in the art to best use the implementations with various modifications as are suited to the particular uses contemplated.SUBSTITUTE SHEET (RULE 26)

Claims

What is claimed1. A method for detecting anomalies in a system associated with a wireless network, comprising: determining one or more features based on time series values detected by a plurality of monitoring nodes located in the system and network information of the wireless network; determining a global status representing normal operation or abnormal operation of the system based on the one or more features and one or more global decision boundaries; in response to determining that the global status represents the abnormal operation of the system, selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof, wherein each of the subsystems includes a subset of the plurality of monitoring nodes; detecting anomalies of one or more subsystems of the selected subsystems based on the one or more features and one or more decision boundaries for the selected subsystems; selecting one or more monitoring nodes belonging to the one or more subsystems; and detecting anomalies of at least one monitoring node of the selected one or more monitoring nodes based on the one or more features and one or more local decision boundaries for the selected one or more monitoring nodes.

2. The method according to claim 1, wherein the plurality of monitoring nodes includes at least one of a sensor, an actuator and a user device associated with the wireless network.

3. The method according to claim 1, wherein the one or more features include flow-based features and QoS features of the wireless network, the QoS features include QoS Identifier, IP data flow, flow or bearer identifier, reflective QoS identifier and data session information.

4. The method according to claim 1, wherein the selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof includes:33SUBSTITUTE SHEET (RULE 26)selecting the subsystems following network architecture and hierarchy based on the knowledge-based subsystem selection method.

5. The method according to claim 1, wherein the data-driven subsystem selection method includes a data clustering method, the selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof includes: clustering the plurality of monitoring nodes into one or more clusters based on the data clustering method, a cluster corresponding a subsystem, wherein each monitoring node of the plurality of monitoring nodes is assigned to a cluster based on a distance from each monitoring node to a cluster centroid of the cluster based on Euclidian or Mahalanobis distance so that each monitoring node belongs to a single cluster.

6. The method according to claim 1, wherein the data-driven subsystem selection method includes a graph-based method, the selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof includes: selecting the subsystems based on the graph-based method by partitioning the system based on a graph map of the network indicating volume of network traffic.

7. The method according to claims 1, wherein the system has a hierarchical structure including a root node corresponding to the system, subsystem nodes corresponding the subsystems and a plurality of leaf nodes corresponding to the plurality of monitoring nodes.

8. The method according to claim 1, comprising: using an ensemble of distributed digital twin models associated with the system, providing anomaly decision for the system, the one or more global decision boundaries for the system and a global score representing a likelihood of the anomalies for the system.34SUBSTITUTE SHEET (RULE 26)9. The method according to claim 1, comprising: using a digital twin model associated with each of the plurality of monitoring nodes, providing anomaly decision for the one or more subsystems and the at least one monitoring node, the one or more decision boundaries for the selected subsystems, the one or more local decision boundaries for the one or more monitoring nodes and one or more scores representing a likelihood of the anomalies for the one or more subsystems and the at least one monitoring node, each of the digital twin models associated with each of the plurality of monitoring nodes.

10. The method according to claims 1, wherein the wireless network includes a network compatible with a 3GPP standard.

11. An apparatus for detecting anomalies in a system associated with a wireless network comprising: a processor configured to: determine one or more features based on time series values detected by a plurality of monitoring nodes located in the system and network information of the wireless network; determine a global status representing normal operation or abnormal operation of the system based on the one or more features and one or more global decision boundaries; in response to determining that the global status represents the abnormal operation of the system, selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof, wherein each of the subsystems includes a subset of the plurality of monitoring nodes; detect anomalies of one or more subsystems of the selected subsystems based on the one or more features and one or more decision boundaries for the selected subsystems; select one or more monitoring nodes belonging to the one or more subsystems; and detect anomalies of at least one monitoring node of the selected one or more monitoring nodes based on the one or more features and one or more local decision boundaries for the selected one or more monitoring nodes.35SUBSTITUTE SHEET (RULE 26)12. The apparatus according to claim 11, wherein the plurality of monitoring nodes includes at least one of a sensor, an actuator and a user device associated with the wireless network.

13. The apparatus according to claim 11, wherein the one or more features include flowbased features and QoS features of the wireless network, the QoS features include QoS Identifier, IP data flow, flow or bearer identifier, reflective QoS identifier and data session information.

14. The apparatus according to claim 11, wherein the system has a hierarchical structure including a root node corresponding to the system, subsystem nodes corresponding the subsystems and a plurality of leaf nodes corresponding to the plurality of monitoring nodes.

15. The apparatus according to claim 11, wherein the wireless network includes two-layer structures including a 5G Core (5GC) network layer and a Radio Access Network (RAN) layer, the 5GC network layer is higher than the RAN layer.

16. The apparatus according to claim 15, wherein the apparatus includes: a first module is configured to collet RAN level information including radio level information, modulation and demodulation information, channel estimation information, aggregated information between multiple network management and antenna port information at a RAN layer to monitor signal at the RAN layer to detect anomalies of the RAN layer; and a second module is configured to receive the collected RAN level information and detect anomalies of the 5GC network layer based on the received RAN level information.

17. The apparatus according to claim 11, wherein the wireless network includes a network compatible with a 3GPP standard.36SUBSTITUTE SHEET (RULE 26)18. The apparatus according to claim 11, wherein the processor executes an ensemble of distributed digital twin models, each of the digital twin models associated with each of the plurality of monitoring nodes.

19. A non-transitory computer-readable storage medium storing one or more programs for execution by one or more processors of an electronic device, the one or more programs including instructions for: determining one or more features based on time series values detected by a plurality of monitoring nodes located in the system and network information of the wireless network; determining a global status representing normal operation or abnormal operation of the system based on the one or more features and one or more global decision boundaries; in response to determining that the global status represents the abnormal operation of the system, selecting subsystems based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof, wherein each of the subsystems includes a subset of the plurality of monitoring nodes; detecting anomalies of one or more subsystems of the selected subsystems based on the one or more features and one or more decision boundaries for the selected subsystems; selecting one or more monitoring nodes belonging to the one or more subsystems; and detecting anomalies of at least one monitoring node of the selected one or more monitoring nodes based on the one or more features and one or more local decision boundaries for the selected one or more monitoring nodes.

20. An apparatus for detecting anomalies in a system associated with a wireless network comprising: a processor configured to: determine one or more features based on time series values detected by a plurality of monitoring nodes located in the system and network information of the wireless network; select subsystems associated with abnormal operation of the system based on one of a knowledge-based subsystem selection method, a data-driven subsystem selection method and a combination thereof, wherein the system has a hierarchical structure including a root node37SUBSTITUTE SHEET (RULE 26)corresponding to the system, subsystem nodes corresponding the subsystems and a plurality of leaf nodes corresponding to the plurality of monitoring nodes, each of the subsystems includes a subset of the plurality of monitoring nodes; detect anomalies of one or more subsystems of the selected subsystems based on the one or more features and one or more decision boundaries for the selected subsystems; select one or more monitoring nodes belonging to the one or more subsystems; and detect anomalies of at least one monitoring node of the selected one or more monitoring nodes based on the one or more features and one or more local decision boundaries for the selected one or more monitoring nodes.38SUBSTITUTE SHEET (RULE 26)

Images