Methods and systems for facilitating self-healing networks

Abstract

The present disclosure relates to methods and systems for facilitating self-healing networks. An event analysis system is provided. During operation, the system can determine an event description associated with a switch from an event log of the switch. The event description can correspond to an entry in a table in a switch configuration database of the switch. The corresponding database in the switch can be a relational database. The system can then obtain an event log segment including the event description based on an entry range, the event log segment being a portion of the event log. Subsequently, the system can apply pattern recognition techniques to the event log segment based on the entries in the switch configuration database to determine one or more patterns corresponding to the event, the event being associated with the event description. The switch can then apply machine learning techniques using the one or more patterns to determine a recovery action for mitigating the event.

Description

Technical Field

[0001] The present disclosure relates to communication networks. More particularly, the present disclosure relates to methods and systems for facilitating self-healing networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Figure 1 An exemplary event analysis system for facilitating self-healing networks according to an embodiment of the present application is shown;

[0003] Figure 2A An exemplary feature group and corresponding pattern definition of an event analysis system according to an embodiment of the present application are shown;

[0004] Figure 2B An exemplary anomaly detection process of an event analysis system according to an embodiment of the present application is shown;

[0005] Figure 3 An exemplary self-healing process facilitated by an event analysis system according to an embodiment of the present application is shown;

[0006] Figure 4A A flowchart illustrating a process of extracting event log segments for analysis by an event analysis system according to an embodiment of the present application is presented;

[0007] Figure 4B A flow chart illustrating a self-healing process of an event analysis system according to an embodiment of the present application is presented;

[0008] Figure 5 A flowchart showing a process of verifying recovery actions for a self-healing process by an event analysis system according to an embodiment of the present application is presented; and

[0009] Figure 6 An exemplary switch equipped with an event analysis system according to an embodiment of the present application is shown.

[0010] In the drawings, like reference numerals refer to the same drawing elements. DETAILED DESCRIPTION

[0011] The following description is provided to enable any person skilled in the art to make and use the invention, and is provided in the context of a specific application and its requirements. Various modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the embodiments shown, but should be given the widest scope consistent with the claims.

[0012] Overview

[0013] The Internet is a delivery medium for a variety of applications running on physical and virtual devices. Such applications bring an ever-increasing demand for bandwidth. As a result, equipment vendors compete to manufacture switches that can perform a variety of functions. However, the resulting complexity of the switch can increase the difficulty of detecting errors in the switch. In addition, a network may include multiple such complex switches. In addition, a network may include different types of switches. Each type of switch may have different capabilities and functions. As a result, if an event (e.g., an anomaly or error) occurs, the administrator may need to troubleshoot each switch in the network individually.

[0014] One common element between different types of switches in a network may be log files maintained by the switches. The log files in each switch may record events associated with the switch. Such events may include configuration changes, operations, and applications of the switch. However, monitoring switches using log files may be challenging because the size of the log files may be large. For example, the size of a log file in a switch may grow to a considerable size (e.g., approximately a gigabyte) within a week. Furthermore, for multiple switches in a network, the combined size of the log files may extend to petabytes of data. As a result, debugging problems in log files may be inefficient and time consuming.

[0015] One embodiment of the present invention provides an event analysis system in a corresponding switch in a network. During operation, the system can determine an event description associated with the switch from an event log of the switch. The event description can correspond to an entry in a table in a switch configuration database of the switch. The switch configuration database in the switch can be a relational database. The system can then obtain an event log segment including the event description based on an entry range, the event log segment being a portion of the event log. Subsequently, the system can apply pattern recognition techniques to the event log segment based on the entries in the switch configuration database to determine one or more patterns corresponding to an event associated with the event description. The switch can then apply machine learning techniques using the one or more patterns to determine a recovery action for mitigating the event.

[0016] In a variation of this embodiment, the system may maintain multiple recovery actions in an action database of the switch. The corresponding recovery actions may correspond to one or more modes.

[0017] In another variation, the system may apply machine learning techniques to the action database using one or more patterns to determine recovery actions from the action database.

[0018] In a variation of this embodiment, the machine learning technology is a pre-trained model loaded on the switch.

[0019] In a variation of this embodiment, the system may evaluate the recovery action in a shadow database and determine, based on the evaluation, whether the recovery action generates a conflict in the switch configuration database. The shadow database may be a copy of the switch configuration database and is not used to determine the configuration of the switch.

[0020] In a variation of this embodiment, the system may determine whether the event is a critical problem and perform recovery actions on the switch if the event is a non-critical problem.

[0021] In another variation, if the incident is a critical issue, the system may obtain confirmation from the user before performing recovery actions.

[0022] In a variation of this embodiment, the system may determine a set of feature groups of the switch and determine a trigger indicating an event based on monitoring the set of feature groups. The corresponding feature group may include one or more relevant features of the switch. The trigger may correspond to the feature group associated with the event.

[0023] In another variation, the system may use pattern recognition techniques to match a set of patterns defined for a feature group with event descriptions in an event log segment.

[0024] In a variation of this embodiment, the range of entries may include a first range of entries before the log entry and a second range of entries after the log entry.The log entry may include a description of an event in the event log.

[0025] The embodiments described herein solve the problem of event detection and recovery in switches in a network by: (i) efficiently identifying events (e.g., anomalies) by determining patterns in relevant log segments, and (ii) using artificial intelligence (AI) models to determine recovery actions corresponding to the events. In this way, the corresponding switches in the network can automatically detect events and corresponding recovery actions. If the event is a non-critical event, the switch can perform a recovery action, thereby facilitating self-healing in the network.

[0026] Using the prior art, the corresponding switch can be equipped with one or more monitoring agents. Each monitoring agent can be a hardware module or a software application capable of monitoring a specific feature of the switch. For example, a monitoring agent can monitor routing operations (e.g., its configuration, operation, and stability), while another monitoring agent can monitor the minimum spanning tree protocol (MSTP). The corresponding monitoring agent can identify events associated with the corresponding features of the switch. However, since a switch can have a large number of features and can be deployed in various situations, the events reported by the monitoring agent of the switch can be diverse and numerous.

[0027] Solving network events (e.g., problems caused by abnormal events) can include identifying the events and quickly solving the problems that caused the events to reduce the impact. It may be error-prone and time-consuming for administrators to manually identify resources and actions to solve events reported by monitoring agents. To further simplify the process, administrators can utilize the event log of the switch. Administrators can use keyword searches and rule matching to manually check entries in event log files to identify events. However, for large-scale deployments (e.g., in large distributed networks), it may not be feasible to identify events by manual inspection. In addition, even if events can be effectively identified, solving problems associated with the events is still challenging. As a result, administrators may not be able to proactively prepare for abnormal events and determine corresponding recovery actions.

[0028] To address this problem, the corresponding switches in the network can be equipped with an event analysis system that can effectively identify problems associated with the event and perform corrective recovery actions. The recovery action can include one or more operations or configuration changes. The system can combine analysis of event log files, pattern recognition in entries (or logs) of event log files, and notifications from monitoring agents to promote self-healing of corresponding switches and their associated links. The corresponding instances of the system on the switches of the network can be synchronized with each other and can operate in conjunction with each other to promote a self-healing network.

[0029] The switch may maintain a switch database to store configuration, state, and statistics associated with the switch in different tables / columns of the database. To efficiently determine events, the system may maintain additional columns in each configuration table of the database. The additional columns may be referred to as feature columns. For each class or category of configuration (e.g., routing, high availability, etc.), the system may include one or more keywords in the feature column. The keywords may correspond to events reported in an event log (i.e., the keywords may appear in an event log entry or log).

[0030] During operation, a monitoring agent on a corresponding switch in the network can monitor the configuration and status of the switch. Monitoring all features of a switch may be inefficient. Therefore, the system can monitor some features instead of monitoring all features of a switch. For example, routing related features and operations (such as Link Layer Discovery Protocol (LLDP), Spanning Tree Protocol (STP), Virtual Local Area Network (VLAN), and Address Resolution Protocol (ARP)) can be located in a routing feature group. When an event for a feature group is detected, the monitoring agent can generate a trigger for the system. In some embodiments, the monitoring agent can change the state of a feature column associated with a feature group. A state change of a feature group can be used as a trigger for the feature group.

[0031] The state of the feature column may include keywords that can be matched to event entries in the event log file. The system may then identify the event entry in the event log file (e.g., based on a keyword or timestamp of the event). The system may then determine a surrounding window of the event entry for the event log file. The system may determine the surrounding window based on an entry range (e.g., a predefined range) indicated by an upper threshold and a lower threshold. The range may be defined for each feature or a set of features. The upper threshold may be a time range before a timestamp associated with the entry, and the lower threshold may be a time range after the timestamp. In addition, the upper threshold may indicate the number of entries before the event entry, and the lower threshold may indicate the number of entries after the event entry. The system may then obtain an event log segment (e.g., a portion of the event log file) corresponding to the surrounding window from the event log file, which may correspond to the upper and lower thresholds.

[0032] After acquiring the event log segments, the system can perform pattern recognition on the event log segments. The system can maintain a set of patterns that can be defined for feature groups. In some embodiments, the system can use Rossi Pattern Language (RPL) based on the set of patterns. The set of patterns can be predefined network-related patterns or templates available in RPL. In addition, the system can also maintain definitions of extended patterns specific to the software and hardware of the corresponding switch. After identifying one or more patterns in the event log segments, the system can determine self-recovery actions based on the identified patterns.

[0033] The system may maintain a mapping between a corresponding pattern in a set of patterns and a corresponding recovery action. The switch may be equipped with an action database that can store the mapping. The action database may be a relational database. The switch database and the action database may be the same type of database (e.g., running on the same database management system (DBMS), but each database may have a unique architecture. These databases may also be different types of databases. Since a single pattern may have multiple recovery actions, the system may use an AI model (e.g., a machine learning-based model) to match the pattern with the recovery action. To ensure lightweight execution on the switch, the AI ​​model may be a pre-trained AI model. The AI ​​model may obtain the determined pattern as input and determine the recovery action. If the event is a non-critical event, the system may perform a recovery action to mitigate the event.

[0034] In some embodiments, the system can validate the recovery action on a shadow database that is distinct from the switch database and the action database before execution. The shadow database can be a copy of the switch database. By merging (or applying) the recovery action into the shadow database, the system can determine whether the recovery action causes a conflict with the configuration of any other element of the switch. If the validation is successful, the system can execute the recovery action.

[0035] The system can also facilitate a client-server model, where the client can be executed on the corresponding switch of the network, and the server can reside on the network manager or cloud platform. The AI ​​model can be trained by the server at the network manager based on the entries in the event log of the switches in the network. After training, the AI ​​model can be loaded on the switches of the network. If a new event occurs, the AI ​​model deployed in the switch may not recognize the corresponding pattern. Therefore, the system on the switch can send the event and pattern to the server as a new data point. The server can retrain the AI ​​model using the new data point. The server can periodically send the updated AI model to the switch. Using the updated AI model on the switch, the system can identify a set of the latest patterns.

[0036] If the event is a critical event, the system can send the event and the recovery action to the network manager. The server instance of the system can present the event and the recovery action to the administrator. The administrator can determine whether the recovery action is an appropriate solution to the event. If so, the administrator can approve the recovery action. Upon receiving approval, the system can execute the recovery action on the switch. It should be noted that the system can also verify the recovery action on the shadow database for critical events. If the verification of the recovery action (e.g., for both critical and non-critical events) is unsuccessful, the system can send the recovery action to the network manager.

[0037] In this disclosure, the term "switch" is used in a general sense, and it may refer to any stand-alone or structural switch operating in any network layer. "Switch" should not be interpreted as limiting embodiments of the present invention to Layer 2 networks. Any device that can forward traffic to external devices or other switches can be referred to as a "switch". Any physical or virtual device (e.g., a virtual machine / switch operating on a computing device) that can forward traffic to a terminal device can be referred to as a "switch". Examples of "switch" include, but are not limited to: a Layer 2 switch, a Layer 3 router, a routing switch, a component of a Gen-Z network, or a structural switch comprising multiple similar or heterogeneous smaller physical and / or virtual switches.

[0038] The term "packet" refers to a group of bits that can be transmitted together on a network. "Packet" should not be interpreted as limiting embodiments of the present invention to layer 3 networks. "Packet" may be replaced with other terms that reference a group of bits, such as "message," "frame," "cell," "datagram," or "transaction."

[0039] Network Architecture

[0040] Figure 1 An exemplary event analysis system for promoting a self-healing network according to an embodiment of the present application is shown. Figure 1 As shown, network 100 includes switches 101, 102, 103, 104, and 105. In some embodiments, network 100 is a Gen-Z network, and the corresponding switches of network 100 (such as switch 102) are Gen-Z components. A Gen-Z network can be a memory semantic structure that can be used to communicate with devices in a computing environment. By unifying communication paths and simplifying software through simple memory semantics, Gen-Z components can facilitate high-performance solutions for complex systems. In this case, the communication between switches in network 100 is based on a memory semantic structure. In some other embodiments, network 100 is an Ethernet and / or IP network, and the corresponding switches of network 100 (such as switch 102) are Ethernet switches and / or IP routers. In this case, the communication between switches in network 100 is based on Ethernet and / or IP.

[0041] Using existing technology, the corresponding switch can be equipped with one or more monitoring agents, each of which can monitor individual features or feature groups of the switch. For example, the monitoring agent 140 can monitor the feature group of the switch 102. The monitoring agent 140 can identify events 150 associated with the corresponding features of the switch 102. However, since the switch 102 can have a large number of features and can be deployed in various scenarios in the network 100 (for example, as a collection switch, an edge switch, or a core switch), the events reported by the monitoring agent of the switch 102 can be diverse and numerous.

[0042] Solving events in the network 100 may include identifying the problem that caused the event and quickly resolving the problem to reduce the impact. It may be error-prone and time-consuming for an administrator to manually identify resources and actions to resolve events reported by the monitoring agent 140. To further simplify the process, the administrator can utilize entries in the event log or event log file 120 of the switch 102. The administrator can manually check the entries in the event log file 120 using keyword searches and rule matching to identify problems associated with the event 150. However, for large-scale deployments in the network 100, it may not be feasible to identify problems by manual inspection. In addition, even if the problem can be effectively identified, it is still challenging to resolve the problem. As a result, the administrator may not be able to proactively prepare for the event and cannot determine the corresponding recovery action for the problem 150 that caused the event.

[0043] To address this problem, the corresponding switches in the network 100 may be equipped with an event analysis system 110 that can effectively identify one or more problems that caused the event 150 and perform corrective recovery actions. In the present disclosure, the system 110 may refer to an instance of the system 110 operating on the switch 102. The system 110 may combine analysis of the event log file 120, pattern recognition in entries of the event log file 120, and notifications from monitoring agents (such as monitoring agents 140) to facilitate self-healing of the switches 102 and related links in the network 100. The instances of the system 110 on the switches of the network 100 may operate in conjunction with each other to facilitate self-healing in the network 100.

[0044] The switch 102 may maintain a switch database 126 to store configuration, state, and statistical information associated with the switch 102 in different tables / columns of the database 126. To efficiently determine events, the system 110 may maintain additional feature columns in each configuration table. The configuration table may store configuration information for the switch 102. For each feature group, the feature column may include one or more keywords that may correspond to events reported in the event log file 120. During operation, upon detecting an event 150 for a corresponding feature group, the monitoring agent 140 may generate a trigger 142 for the system 110. The monitoring agent 140 may change the state of the feature column associated with the feature group. The state change of the feature group may serve as a trigger 142 for the feature group.

[0045] The event engine 112 of the system 110 can manage the trigger 142. The alarm module 122 can determine the trigger 142 based on the database 126. The heartbeat module 124 that can periodically monitor the event log file 120 can identify the event entry corresponding to the trigger 142 and determine the event log segment from the event log file 120 based on the event entry. After obtaining the event log segment, the pattern engine 114 of the system 110 can perform pattern recognition on the event log segment based on the keywords of the feature column entry and determine the pattern associated with the event 150. The system 110 can maintain a mapping between a corresponding pattern in a set of patterns associated with typical network problems / events and a corresponding recovery action. The switch 102 can be equipped with an action database 128 that can store the mapping.

[0046] Since a single pattern can have multiple recovery actions, the action engine 116 of the system 110 can use the AI ​​model to match the determined pattern with the recovery action 144 in the mapping in the action database 128. The recovery action 144 can include one or more operations or configuration changes. If the event 150 is a non-critical event (e.g., related to a non-critical problem), the recovery engine 118 of the system 110 can execute the recovery action 144 to mitigate the impact of the event 150. To ensure that the recovery action 144 does not cause a conflict with the remaining switch configuration, the recovery engine 118 can verify the recovery action 144 on the shadow database 130 before execution.

[0047] In some embodiments, the system 110 may also facilitate a client-server model. The client instance of the system 110 may be executed on a corresponding switch of the network 100, and the server instance may be run on a network manager 160. The network manager 160 may be located in the network 100 or may be deployed in a cloud (e.g., accessible via the Internet), and facilitates network configuration and management for corresponding switches in the network 100. If the event 150 is a critical event (e.g., related to a critical problem), the recovery engine 118 may send the event and the recovery action 144 to the network manager 160. The server instance of the system 110 may present the event and the recovery action 144 to the administrator. The administrator may then determine whether the recovery action 144 is an appropriate solution to the event 150.

[0048] The AI ​​model may be trained by a server instance of system 110 at network manager 160 based on entries in event logs of switches in network 100. After training, the AI ​​model may be loaded on switches of network 100 (such as switch 102). If event 150 is a new event (e.g., an event that has not yet been used to train the AI ​​model), the AI ​​model deployed in switch 102 may not recognize the corresponding pattern. Therefore, the instance of system 110 on switch 102 may send information associated with event 150 to the server instance of system 110 as a new data point. The server instance of system 110 may retrain the AI ​​model using the new data point. The server instance of system 110 may periodically send updated AI models to switches of network 100. As a result, the AI ​​model of the client instance of system 110 on the switch may recognize the latest set of patterns.

[0049] Figure 2A An exemplary feature group and corresponding mode definition of an event analysis system according to an embodiment of the present application are shown. A corresponding switch in the network 100 (such as switch 102) can support multiple features 202. For example, the switch 102 can support routing related features and operations, such as LLDP, STP (and any variants thereof), VLAN, and ARP. The switch 102 can also support proprietary protocols for supporting specialized operations. The features 202 can also include one or more policies and access control lists (ACLs) that can provide security and permissions for corresponding users.

[0050] Monitoring every feature of the switch 102 may be challenging. Therefore, the system 110 may group related features into feature groups and monitor the feature groups. As a result, the features 202 of the switch 102 may be grouped into a set of feature groups 204. The routing-related features and operations in the features 202 may be grouped into a routing feature group 212. The feature groups may include standard and proprietary features of the switch. For example, the high availability feature group 214 may include standard features such as synchronization, and proprietary features. The policies and ACLs defined for the switch 102 may be grouped into a security feature group 216. It should be noted that the switch 102 may include features related to Figure 2A More (or fewer) features and feature groups than shown.

[0051] In order to perform pattern recognition on the event log segmentation, the pattern engine 114 can maintain a set definition 206 of patterns, which can include one or more pattern definitions (represented by dotted and dashed lines) for each feature group 204. The pattern definition can represent one or more events that can occur at the switch 102. For example, a link failure or port flap event can correspond to a pattern definition indicating a packet drop. In this way, the pattern definition can indicate what event may have occurred at the switch 102. The pattern definition 206 can be pre-defined by an administrator or a pattern recognition technology used by the pattern engine 114. If the pattern engine 114 uses RPL, the pattern definition 206 can be a pre-defined network-related pattern available in RPL. In addition, the system can also maintain definitions of extended patterns that are specific to the software and hardware of the corresponding switch, such as proprietary features in the feature 202.

[0052] In some embodiments, the pattern engine 114 can learn "normal" patterns when the switch 102 performs without abnormal events. These patterns can generate baseline pattern definitions in the pattern definitions 206 (for example, in addition to the RPL pattern). The monitoring agent 140 can periodically provide log information to the pattern engine 114. By performing pattern recognition in the log information, the pattern engine 114 can determine a baseline pattern for the switch 102. The baseline pattern can allow the pattern engine 114 to determine abnormal patterns in event log entries. The pattern engine 114 can also use Splunk pattern matching technology using regular expressions. If the features 202 of the switch 102 are defined using regular expressions, the pattern engine 114 can use Splunk to match the features with corresponding log entries.

[0053] Figure 2B An exemplary anomaly detection process of an event analysis system according to an embodiment of the present application is shown. The switch 102 may maintain a switch database 126 to store configuration, status, and statistical information associated with the switch 102 in different tables such as table 240. The database 126 may be a relational database running on a DBMS. The table 240 may include a plurality of columns 242, which may store configurations associated with a feature group 250. In order to efficiently determine an event, the system 110 may maintain an additional column 244 in the table 240. The additional column 244 may be referred to as a feature column. For the feature group 250, the system 110 may include a keyword 232 in the feature column 242. The keyword 232 may correspond to an event reported in the event log file 120 (i.e., the keyword 232 may appear in the event log entry).

[0054] The event log file 120 may include a plurality of entries 221, 222, 223, 224, 225, and 226. A respective entry in the event log file 120 may include one or more of the following: a timestamp (e.g., date and time), a type of action (e.g., an action of the switch 102 or an action of a neighbor), an associated application (e.g., a protocol daemon), event information (e.g., an identifier and indicator as to whether the entry is an information entry or an error entry), and an event description. For example, upon receiving a Network Time Protocol (NTP) packet, the event log file 120 may create the following entry 222:

[0055] 2019-08-19T05:45:65.747267+00:00 switch ntpd

[45584] :

[0056] Event|1102|LOG_INFO|AMM|1 / 5|NTP server 1 package received

[0057] The monitoring agent 140 may monitor the features in the feature group 250 in the switch 102. Upon detecting an event 150 for the feature group 250, the monitoring agent 140 may generate a trigger 142 for the system 110. The monitoring agent 140 may change the state of a feature column associated with the feature group 250. The change in state of the feature group 250 may serve as a trigger 142 for the feature group 250. Since the state of the feature column 244 may include a keyword 232 that may be matched to an event entry in the event log file 120, the system 110 may then identify an event entry 224 (indicated by a dashed line) in the event log file 120. The system 110 may identify the entry 224 based on the keyword 232 and the timestamp of the event 150.

[0058] For example, if switch 102 receives a large number of NTP packets in a short period of time, the NTP daemon of switch 102 may have a high processor utilization. Flooding switch 102 with request traffic may overwhelm switch 102 and may cause blocking of essential traffic. This abnormal security pattern may indicate an NTP amplification attack, which may generate corresponding entries in event log file 120. Thus, entries 223, 224, and 225 may be:

[0059] 2019-08-19T05:46:02.788056+00:00 switch system

[45584] :Event|7601|LOG_ERR|AMM|-|High CPU utilization by daemon 12mac-mgrd

[0060] 2019-08-19T05:46:32.984564+00:00 switch policyd

[26099] :Event|6901|LOG_INFO|AMM|-|Action triggered by monitoring agent

[0061] 2019-08-19T05:46:37.986937+00:00 switch policyd

[26099] :Event|5507|LOG_INFO|AMM|-|Configuration change system 110

[0062] The system 110 may then determine a surrounding window of the event entry 224 in the event log file 120. The system 110 may determine the surrounding window based on a range of entries (e.g., a predefined range) indicated by an upper threshold and a lower threshold. The range may be specific to the feature group 250. The upper threshold may be a time range before a timestamp associated with the entry 224, and the lower threshold may be a time range after the timestamp. In addition, the upper threshold may indicate the number of entries before the entry 224, and the lower threshold may indicate the number of entries after the entry 224. For example, the upper and lower thresholds for the feature group 250 may be 2 and 1, respectively. Alternatively, the upper and lower thresholds for the feature group 250 may be 10 seconds and 5 seconds, respectively. The system 110 may then obtain an event log segment 220 (e.g., a portion of the event log file 120) corresponding to the surrounding window from the event log file 120.

[0063] After obtaining the event log segment 220, the pattern engine 114 can perform pattern recognition on the event log segment 220. The system 110 can maintain a set definition 208 of patterns that can be defined for the feature group 250 and can include it in the pattern definition 208. In some embodiments, the pattern engine 114 can use pattern recognition technology, such as RPL and Splunk, based on the pattern definition 208. The pattern definition 208 can be a predefined network-related pattern available in the pattern recognition technology. The pattern definition 208 can also be a definition of an extended pattern specific to the software and hardware of the switch 102. After identifying the pattern 234 in the event log segment 220, the pattern engine 114 can determine a self-recovery action based on the pattern 234.

[0064] Figure 3An exemplary self-healing process facilitated by an event analysis system according to an embodiment of the present application is shown. The system 110 may maintain a mapping between a corresponding pattern in a set of patterns and a corresponding recovery action. The switch 102 may be equipped with an action database 128 that may store the mapping in a table 260, which may include a pattern column 262 and a corresponding recovery action column 264. Since the pattern 234 may have multiple recovery actions, the action engine 116 may apply an AI model 310 (e.g., a machine learning-based model) to the table 260. The AI ​​model 310 may match the pattern 234 with the recovery action 144. To ensure lightweight execution on the switch 102, the AI ​​model 310 may be a pre-trained AI model.

[0065] The AI ​​model 310 may take the pattern 234 as input and determine the recovery action 144 (e.g., the pattern 234 and the action 144 correspond to the input and output layers of the AI ​​model 310, respectively). When determining the recovery action 144 for the pattern 234, the recovery engine 118 may determine whether the event 150 is a critical event by determining whether the event 150 is related to a critical issue. For example, a problem associated with NTP may be a non-critical issue, while a problem associated with STP may be a critical issue. If the event 150 is a non-critical event, the recovery engine 118 may perform the recovery action 144 to mitigate the event 150. For example, if the pattern 234 corresponds to an NTP amplification attack, the recovery action 144 may indicate that the switch 102 should block the server from sending NTP packets. After the recovery action 144 is performed at the switch 102, the corresponding entry in the event log file 120 may be:

[0066] 2019-08-19T05:46:46.984564+00:00 switch system110

[26099] :

[0067] Event|6901|LOG_INFO|AMM|-|block NTP server

[0068] In some embodiments, the recovery engine 118 may verify the recovery action 144 on the shadow database 130 prior to execution of the recovery action 144. The shadow database 130 may be a shadow copy of the switch database 126. A shadow copy may be a copy that is not directly used to configure or operate the switch 102. By merging (or applying) the recovery action 144 into the shadow database 130, the recovery engine 118 may determine whether the recovery action 144 causes a conflict with the configuration of any other element of the switch 102. If the verification is successful, the recovery engine 118 may execute the recovery action 144. It should be noted that the recovery engine 118 may verify the recovery action 144 on the shadow database 130 even if the event 150 is a critical event. If the verification of the recovery action 144 is unsuccessful (regardless of whether the event 150 is a critical event or a non-critical event), the recovery engine 118 may seek further verification of the recovery action 144 (e.g., from the network manager 160).

[0069] The system 110 can also facilitate a client-server model, where an instance of the system 110 on the switch 102 can be a client instance, and a server instance of the system 110 can operate on the network manager 160. If the event 150 is a critical event or the recovery action 144 may cause a conflict on the switch 102, the recovery engine 118 can send the event 150 and the recovery action 144 to the network manager 160. The server instance of the system 110 can present the event 150 and the recovery action 144 to the administrator. The administrator can determine whether the recovery action 144 is an appropriate solution to the event 150. If so, the administrator can approve the recovery action 144. Upon receiving the approval, the recovery engine 118 can execute the recovery action 144 on the switch 102 and submit the corresponding configuration changes to the switch database 126.

[0070] operate

[0071] Figure 4A A flow chart showing a process of extracting event log segments for analysis by an event analysis system according to an embodiment of the present application is presented. During operation, the system can determine an event trigger for an event (operation 402) and identify an entry corresponding to the event in an event log file (operation 404). The system can then determine the event log segment based on upper and lower thresholds associated with the entry (operation 406) and extract the event log segment from the event log file (operation 408).

[0072] Figure 4BA flow chart illustrating a self-healing process of an event analysis system according to an embodiment of the present application is presented. During operation, the system may obtain keywords from a feature column of a switch database (operation 452). The system may then apply pattern recognition techniques to event log segmentation based on the obtained keywords and one or more pattern definitions (operation 454). Subsequently, the system determines a pattern based on the pattern recognition technique (operation 456). The system may then apply an AI model to a recovery action available in an action database based on the determined pattern (operation 458). Subsequently, the system may determine a recovery action based on the AI ​​model (operation 460).

[0073] Figure 5 A flow chart showing a process of verifying a recovery action for a self-healing process by an event analysis system according to an embodiment of the present application is presented. During operation, the system can apply a recovery action to a shadow database of a switch (operation 502) and check for conflicts with existing configurations (operation 504). In this way, the system can verify before submitting the recovery action to the switch. The switch can then determine whether a conflict is detected (operation 506). If a conflict is detected, the system can notify the server instance of the system on the network manager of the conflict (operation 512).

[0074] On the other hand, if no conflict is detected, the system can check whether the event is a critical event (operation 508). If the event is a critical event, the system can send a recovery action to the server instance of the system on the network manager for further processing (operation 514). If the event is not a critical event, the system can apply the recovery action to the switch, thereby submitting the corresponding changes to the switch database (operation 510).

[0075] Exemplary switch system

[0076] Figure 6 An exemplary switch equipped with an event analysis system according to an embodiment of the present application is shown. In this example, the switch 600 includes multiple communication ports 602, a packet processor 610, an analysis logic block 630, and a storage device 650. The switch 600 may also include switch hardware (e.g., processing hardware of the switch 600, such as its ASIC chip), which includes information based on which switch 600 to process the packet (e.g., determining the output port for the packet). The packet processor 610 extracts and processes header information from the received frame. The packet processor 610 can identify a switch identifier (e.g., a MAC address and / or an IP address) associated with the switch in the header of the packet.

[0077] The communication port 602 may include an inter-switch communication channel for communicating with other switches and / or user equipment. The communication channel may be implemented via a conventional communication port and based on any open or proprietary format. The communication port 602 may include one or more Ethernet ports capable of receiving frames encapsulated in an Ethernet header. The communication port 602 may also include one or more IP ports capable of receiving IP packets. The IP port can receive IP packets and may be configured with an IP address. The packet processor 610 may process Ethernet frames and / or IP packets.

[0078] The switch 600 may maintain a switch database 652, an action database 654, and a shadow database 656 (e.g., in the storage device 650). These databases may be relational databases, each of which may have a unique schema, and may run on one or more DBMS instances. The analysis logic block 630 may include one or more of the following: a mode logic block 632, an action logic block 634, a recovery logic block 636, and a verification logic block 636. During operation, the analysis logic block 630 may receive a trigger based on a state change in a feature column in the switch database 652, the trigger indicating an event associated with the switch 600. The analysis logic block 630 may determine an entry in an event log file of the switch 600 indicating the event, and obtain an event log segment based on the entry.

[0079] The pattern logic block 632 can then perform pattern recognition on the event log segment to determine one or more patterns corresponding to the event. Subsequently, the action logic block 634 can use the AI ​​model to determine a recovery action from a set of recovery actions maintained in the action database 654. The AI ​​model can use one or more patterns and corresponding feature columns to determine the recovery action. The verification logic block 368 can verify the recovery action on the shadow database 656, which can be a shadow copy of the switch database 654. Once successfully verified, if the event is a non-critical event, the recovery logic block 636 can apply the recovery action to the switch 600, thereby submitting the corresponding configuration change to the switch database 652. On the other hand, if the verification is unsuccessful or the event is a critical event, the recovery logic block 636 can send information indicating the recovery action to the network manager.

[0080] The data structures and codes described in this detailed description are typically stored on a computer-readable storage medium, which can be any device or medium that can store code and / or data for use by a computer system. Computer-readable storage media include, but are not limited to, volatile memory, non-volatile memory, magnetic and optical storage devices (such as disks, tapes, CDs (compact disks), DVDs (digital versatile disks or digital video disks)), or other media now known or later developed that are capable of storing computer-readable media.

[0081] The methods and processes described in the detailed description section may be embodied as code and / or data, which may be stored in a computer-readable storage medium as described above. When a computer system reads and executes the code and / or data stored on a computer-readable storage medium, the computer system executes the methods and processes embodied as data structures and code and stored in the computer-readable storage medium.

[0082] The methods and processes described herein may be performed by and / or included in hardware modules or devices. These modules or devices may include, but are not limited to, application specific integrated circuit (ASIC) chips, field programmable gate arrays (FPGAs), dedicated or shared processors that execute a specific software module or piece of code at a specific time, and / or other programmable logic devices now known or later developed. After the hardware modules or devices are activated, they will execute the methods and processes contained therein.

[0083] The foregoing descriptions of the embodiments of the present invention are given for the purpose of illustration and description only. They are not intended to be exhaustive or to limit the present disclosure. Therefore, many modifications and variations will be clear to those skilled in the art. The scope of the present invention is defined by the appended claims.

Claims

1. A method for facilitating an event analysis system in a switch, comprising: determining, from an event log of the switch, an event description associated with the switch, wherein the event description corresponds to an entry in a table in a switch configuration database of the switch, and wherein the switch configuration database in the switch is a relational database; obtaining an event log segment including the event description based on a log entry range associated with a log entry in the event log, the event log segment being a portion of the event log; applying a pattern recognition technique to the event log segment based on the entries in the switch configuration database to determine one or more patterns corresponding to events, the events being associated with the event descriptions; as well as Machine learning techniques are applied to match the one or more patterns with restorative actions for mitigating the incident.

2. The method of claim 1, further comprising maintaining a plurality of recovery actions in an action database in the switch, wherein respective recovery actions correspond to one or more modes.

3. The method of claim 2, further comprising applying the machine learning technique to the action database to match the one or more patterns with the recovery actions from the action database.

4. The method of claim 1, wherein the machine learning technology is a pre-trained model loaded on the switch.

5. The method according to claim 1, further comprising: evaluating the recovery action in a shadow database, wherein the shadow database is a copy of the switch configuration database and is not used to determine the configuration of the switch; as well as A determination is made based on the evaluation whether the recovery action generates a conflict in the switch configuration database.

6. The method according to claim 1, further comprising: Determine whether the incident is a critical issue; as well as In response to the event being a non-critical issue, performing the recovery action on the switch.

7. The method according to claim 6, further comprising: Responsive to the event being a critical issue, obtaining confirmation from a user before performing the recovery action.

8. The method according to claim 1, further comprising: determining a set of feature groups for the switch, wherein a respective feature group includes one or more relevant features of the switch; as well as A trigger indicative of the event is determined based on monitoring the set of feature groups, wherein the trigger corresponds to the feature group associated with the event.

9. The method of claim 8, further comprising matching a set of patterns defined for the feature group with event descriptions in the event log segment using the pattern recognition technique.

10. The method of claim 1, wherein the log entry range comprises: a first range of entries preceding a log entry including the event description in the event log; as well as The second range of entries following the log entry.

11. A non-transitory computer-readable storage medium storing instructions that, when executed by a computer, cause the computer to perform a method for facilitating an event analysis system in a switch, the method comprising: determining, from an event log of the switch, an event description associated with the switch, wherein the event description corresponds to an entry in a table in a switch configuration database of the switch, and wherein the switch configuration database in the switch is a relational database; obtaining an event log segment including the event description based on a log entry range associated with a log entry in the event log, the event log segment being a portion of the event log; applying a pattern recognition technique to the event log segment based on the entries in the switch configuration database to determine one or more patterns corresponding to events, the events being associated with the event descriptions; as well as Machine learning techniques are applied to match the one or more patterns with restorative actions for mitigating the incident.

12. The non-transitory computer-readable storage medium of claim 11, wherein the method further comprises maintaining a plurality of recovery actions in an action database in the switch, wherein respective recovery actions correspond to one or more modes.

13. The non-transitory computer-readable storage medium of claim 12, wherein the method further comprises applying the machine learning technique to the action database to match the one or more patterns with the recovery actions from the action database.

14. The non-transitory computer-readable storage medium of claim 11, wherein the machine learning technique is a pre-trained model loaded on the switch.

15. The non-transitory computer readable storage medium of claim 11, wherein the method further comprises: evaluating the recovery action in a shadow database, wherein the shadow database is a copy of the switch configuration database and is not used to determine the configuration of the switch; as well as A determination is made based on the evaluation whether the recovery action generates a conflict in the switch configuration database.

16. The non-transitory computer-readable storage medium of claim 11, wherein the method further comprises: Determine whether the incident is a critical issue; as well as In response to the event being a non-critical issue, performing the recovery action on the switch.

17. The non-transitory computer readable storage medium of claim 16, wherein the method further comprises: Responsive to the event being a critical issue, obtaining confirmation from a user before performing the recovery action.

18. The non-transitory computer-readable storage medium of claim 11, wherein the method further comprises: determining a set of feature groups for the switch, wherein a respective feature group includes one or more relevant features of the switch; as well as A trigger indicative of the event is determined based on monitoring the set of feature groups, wherein the trigger corresponds to the feature group associated with the event.

19. The non-transitory computer readable storage medium of claim 18, wherein the method further comprises: Using the pattern recognition technique, a set of patterns defined for the feature set is matched to event descriptions in the event log segment.

20. The non-transitory computer-readable storage medium of claim 11, wherein the log entry range comprises: a first range of entries preceding a log entry including the event description in the event log; as well as The second range of entries following the log entry.

Images