Use programmable mathematical models of resource dependencies to perform root cause analysis
By combining model traversal and dependency graph technology in the network management system, the resource-dependent mathematical model is built, which solves the problem of fault diagnosis in complex networks, and quickly identifys the root cause of faults, improving network management efficiency and accuracy.
Patent Information
- Application Number
- CN202011157995.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-08-04
- Filing Date
- 2020-10-26
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2040-10-26
AI Technical Summary
When existing network management systems deal with complex and distributed networks, it is difficult to quickly diagnose dependency events and failures, resulting in an expansion of the impact of network failures. The existing technologies such as model traversal and dependency graph technologies are not enough in frequently changing networks. flexible.
By using a combination of model traversal technology and dependency graph technology by the controller device, a resource-dependent mathematical model is built, combined with a unified network model in mathematical form, to realize dynamic transformation and synchronization of network topology and resources, and support model-driven root cause analysis (RCA) to quickly identify the root cause of the failure.
It realizes rapid diagnosis and failure analysis of complex and distributed networks, improves network management efficiency and accuracy, can effectively identify and solve problems in frequently changing network environments, and reduces the impact of network failures.
Smart Images

Figure CN114070724B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to computer networks, and more particularly, to the management of network devices. Background Art
[0002] A computer network is a collection of interconnected computing devices that can exchange data and share resources. Various devices operate to support communications between computing devices. For example, a computer network may include routers, switches, gateways, firewalls, and various other devices used to provide and support network communications.
[0003] These network devices typically include mechanisms (such as management interfaces) for configuring the device locally or remotely. By interacting with the management interface, the client can perform configuration tasks and execute operational commands to collect and view operational data of the managed device. For example, the client can configure the interface card of the device, adjust parameters for supported network protocols, specify physical components within the device, modify routing information maintained by the router, access software modules and other resources resident on the device, and perform other configuration tasks. In addition, the client can allow the user to view current operating parameters, system logs, information related to network connectivity, network activity or other status information from the device, as well as view and react to event information received from the device.
[0004] Network configuration services can be performed by multiple different devices, such as routers with service cards and / or dedicated service devices. Such services include connectivity services, such as layer 3 virtual private networks (L3VPNs), virtual private LAN services (VPLSs), and peer-to-peer (P2P) services. Other services include network configuration services, such as Dot1q VLAN services. Network management systems (NMSs) and NMS devices (also referred to as controllers or controller devices) can support these services so that administrators can easily create and manage these advanced network configuration services.
[0005] In particular, user configuration of devices can be referred to as "intents". Intent-based networking systems enable administrators to describe the expected network / compute / storage state. User intent can be categorized as business policy or stateless intent. Business policy or stateful intent can be resolved based on the current state of the network. Stateless intent can be a completely declarative way to describe the expected network / compute / storage state without paying attention to the current network state.
[0006] Intents can be represented as intent data models, which can be modeled using a unified graph. Intent data models can be represented as connected graphs so that business policies can be implemented across intent data models. For example, a connected graph in which vertices are connected with has-edges and ref edges can be used to represent a data model. A controller device can model the intent data model as a unified graph so that the intent model can be represented as a connected intent model. In this way, business policies can be implemented across intent data models. When a unified graph model is used to model intents, extending support for new intents requires extending the graph model and compilation logic.
[0007] To configure a device to execute an intent, a user (such as an administrator) can write a conversion program that converts high-level configuration instructions (e.g., instructions according to an intent data model, which can be represented as a unified graph model) into low-level configuration instructions (e.g., instructions according to a device configuration model). As part of the configuration service support, the user / administrator can provide an intent data model and a mapping between the intent data model and the device configuration model.
[0008] In order to simplify the mapping definition for the user, the controller device can be designed to provide the ability to define mappings in a simple manner. For example, some controller devices provide the use of velocity templates and / or extensible stylesheet language transformations (XSLT). Such converters contain the conversion or mapping logic from the intent data model to the low-level device configuration model. Typically, a relatively small number of changes in the intent data model affect a relatively large number of attributes across device configurations. Different converters can be used when creating, updating, and deleting services through the intent data model. Summary of the invention
[0009] In general, the present disclosure describes techniques for managing network devices. A network management system (NMS) device (also referred to herein as a controller or controller device) can be configured to convert a high-level configuration (an intent received from an administrator for multiple managed network devices) to a low-level configuration (to be applied to the managed network devices themselves). In some instances, the controller device can maintain a backup of the configuration (e.g., a low-level configuration) based on the intent so that the controller device can restore the managed network devices to a previous state (e.g., after the network enters a bad state). In some instances, a new intent provided by an administrator to the controller device may create a conflict that can affect the functionality of one or more existing intents, and the controller device can detect the conflict before deploying the intent to one or more managed network devices - to avoid putting the network in a bad state.
[0010] In one example, a method includes: determining, by a controller device that manages multiple network devices, resources provided by a network device; determining, by the controller device, relationships between the resources based on a programmed merging strategy; constructing, by the controller device, a dependency model using the determined relationships, the dependency model representing dependencies between the resources; determining, by the controller device, that at least one of the resources has experienced a failure; and performing a root cause analysis, by the controller device, using the dependency mathematical model to determine a root cause of the failure of the at least one of the resources.
[0011] In another example, a controller device that manages multiple network devices includes: a memory configured to store a dependency model and a programmed merging strategy, the dependency model representing dependencies between resources provided by the network devices; and one or more processors implemented in a circuit device and configured to: determine resources provided by the network devices; determine relationships between the resources based on the programmed merging strategy; construct a dependency model using the determined relationships; determine that at least one of the resources has experienced a failure; and perform a root cause analysis using the dependency mathematical model to determine a root cause of the failure of the at least one of the resources.
[0012] In another example, a computer-readable storage medium has instructions stored thereon that, when executed, cause a processor of a controller device that manages multiple network devices to: determine resources provided by the network devices; determine relationships between the resources based on a programmed consolidation strategy; construct a dependency model using the determined relationships, the dependency model representing dependencies between the resources; determine that at least one of the resources has experienced a failure; and perform a root cause analysis using the dependency mathematical model to determine a root cause of the failure of the at least one of the resources.
[0013] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] Figure 1 is a block diagram illustrating an example of elements that include an enterprise network that is managed using a management device.
[0015] Figure 2 It is a diagram for Figure 1 A block diagram of an example set of components of a management device is shown.
[0016] Figure 3 is a conceptual diagram illustrating an example graph database model.
[0017] Figure 4is a conceptual diagram illustrating an example portion of a resource dependency mathematical model.
[0018] Figure 5 is a conceptual diagram illustrating a network data model discovered by an example.
[0019] Figure 6 is a flow chart illustrating an example method for constructing a mathematical model of resource dependencies and using the model to perform root cause analysis (RCA) in accordance with techniques of this disclosure. DETAILED DESCRIPTION
[0020] Figure 1 1 is a block diagram illustrating an example of elements managed using a controller device 10 including an enterprise network 2. The managed elements 14A to 14G (collectively referred to as "elements 14") of the enterprise network 2 include network devices that are interconnected via communication links to form a communication topology for exchanging resources and information. The elements 14 (also commonly referred to as network devices or remote network devices) may include, for example, routers, switches, gateways, bridges, hubs, servers, firewalls or other intrusion detection systems (IDS) or intrusion prevention systems (IDP), computing devices, computing terminals, printers, other network devices, or combinations of such devices. Although described in the present disclosure as transmitting, conveying, or otherwise supporting packets, the enterprise network 2 may transmit data according to any other discrete data unit defined by any other protocol (such as a cell defined by an asynchronous transfer mode (ATM) protocol, or a datagram defined by a user datagram protocol (UDP)). The communication link that interconnects the elements 14 may be a physical link (e.g., an optical link, copper, etc.), a wireless link, or any combination of the above.
[0021] Enterprise network 2 is shown coupled to public network 18 (e.g., the Internet) via a communication link. Public network 18 may include, for example, one or more client computing devices. Public network 18 may provide access to web servers, application servers, public databases, media servers, end-user devices, and other types of network resource devices and content.
[0022] The controller device 10 is communicatively coupled to the element 14 via the enterprise network 2. Although for the purpose of illustration, Figure 1Only one device of the device management system is illustrated in the figure, but in some examples, the controller device 10 forms part of the device management system. The controller device 10 can be directly or indirectly coupled to various elements 14. Once the elements 14 are deployed and activated, the administrator 12 uses the controller device 10 to manage the network devices using a device management protocol. An example device protocol is the Simple Network Management Protocol (SNMP), which allows the controller device 10 to traverse and modify the management information base (MIB) storing configuration data within each managed element 14. Further details of the SNMP protocol can be found in RFC 3411 "An Architecture for Describing Simple Network Management Protocol (SNMP) Management Frameworks" published by Harrington et al. in the draft of the Internet Engineering Task Force of the Network Working Group in December 2002 (available at http: / / tools.ietf.org / html / rfc3411), the entire contents of which are incorporated herein by reference.
[0023] In common practice, the controller device 10 (also referred to as a network management system (NMS) or NMS device) and the elements 14 are centrally maintained by the enterprise's IT group. The administrator 12 interacts with the controller device 10 to remotely monitor and configure the elements 14. For example, the administrator 12 can receive alerts about any element 14 from the controller device 10, view the configuration data of the element 14, modify the configuration data of the element 14, add new network devices to the enterprise network 2, remove existing network devices from the enterprise network 2, or otherwise manipulate the enterprise network 2 and the network devices therein. Although the technology of the present disclosure is described for an enterprise network, the technology of the present disclosure can be applied to other network types (public and private), including LANs, VLANs, VPNs, etc.
[0024] In some examples, the administrator 12 uses the controller device 10 or a local workstation to interact directly with the element 14, for example, through remote login, secure shell (SSH) or other such communication sessions. That is, the element 14 generally provides an interface for direct interaction (such as, command line interface (CLI), web-based interface, graphical user interface (GUI), etc.), through which the user can interact with the device to directly issue text-based commands. For example, these interfaces generally allow the user to interact directly with the device, for example, through remote login, secure shell (SSH), hypertext transfer protocol (HTTP) or other network sessions, to enter text according to a defined grammar, to submit commands to the managed element. In some examples, the user uses the controller device 10 to initiate an SSH session 15 with one of the elements 14 (e.g., element 14F) to directly configure the element 14F. In this way, the user can directly provide commands to the element 14 in a format to be executed.
[0025] Further, the administrator 12 may also create scripts that may be submitted by the controller device 10 to any or all of the elements 14. For example, in addition to the CLI interface, the element 14 may also provide an interface for receiving a script that specifies commands according to a scripting language. In a sense, the script may be output by the controller device 10 to automatically invoke a corresponding remote procedure call (RPC) on the managed element 14. The script may conform to, for example, the Extensible Markup Language (XML) or another data description language.
[0026] The administrator 12 uses the controller device 10 to configure the element 14 to specify certain operational characteristics that support the administrator's 12 goals. For example, the administrator 12 can specify specific operational policies for the element 14 regarding security, device accessibility, traffic engineering, quality of service (QoS), network address translation (NAT), packet filtering, packet forwarding, rate limiting, or other policies. The controller device 10 performs the configuration using one or more network management protocols designed to manage configuration data within the managed network element 14, such as the SNMP protocol or the Network Configuration Protocol (NETCONF) protocol or its derivatives, such as the Juniper Device Management Interface. In general, NETCONF provides a mechanism for configuring network devices and uses an extensible markup language (XML)-based data encoding for configuration data, which may include policy data. NETCONF is described in the Network Working Group RFC 4741, Enns' "NETCONF Configuration Protocol" in December 2006 (available at tools.ietf.org / html / rfc4741). The controller device 10 can establish a NETCONF session with one or more elements 14.
[0027] The controller device 10 may be configured to accept high-level configuration data or intents from the administrator 12 (e.g., the high-level configuration data or intents may be represented as structured input parameters in accordance with YANG, as described in "YANG—A Data Modeling Language for the Network Configuration Protocol (NETCONF)" by Bjorklund, Internet Engineering Task Force, RFC 6020, October 2010 (available at tools.ietf.org / html / rfc6020)). The controller device 10 may also be configured to output corresponding low-level device configuration data sets, e.g., device configuration additions, modifications, and removals. Additional details regarding an example process for converting high-level configuration information to low-level device configuration information may be found, for example, in U.S. patent application Ser. No. 15 / 198,657, filed on Jun. 30, 2016, by Jiang et al., “TRANSLATING HIGH-LEVEL CONFIGURATION INSTRUCTIONS TO LOW-LEVEL DEVICE CONFIGURATION,” the entire contents of which are incorporated herein by reference.
[0028] In some examples, the controller device 10 may use YANG modeling for an intent data model and a low-level device configuration model. The data may contain relationships across YANG entities such as list items and containers. In some examples, the controller device 10 may convert the YANG data model into a database model and convert the YANG validation into data validation. A technique for managing network devices using a graph model for advanced configuration data is described in U.S. Patent Application No. 15 / 462,465 filed on March 17, 2017, “CONFIGURING AND MANAGING NETWORKDEVICES USING PROGRAM OVERLAY ON YANG-BASED GRAPH DATABASE,” the entire contents of which are incorporated herein by reference.
[0029] The controller device 10 may receive data from the administrator 12 representing any or all of the create, update and / or delete actions for the intended data model. The controller device 10 may be configured to use the same compilation logic for each of the create, update and delete actions applied to the graph model.
[0030] Typically, a controller (such as controller device 10) uses a hierarchical data model for intent, low-level data model, and resources. The hierarchical data model can be based on YANG or YAML. As discussed above, the hierarchical data model can be represented as a graph. Modern systems already support the intention of simplifying the management of the network. The intention is declarative. In order to achieve the intention, the controller device 10 tries to select the best resources. The customer environment can be configured to allow customers (e.g., administrators 12) to control the implementation of the intention and ensure the programmed intention. In some examples, the controller device 10 can construct a graph data model by querying the element 14 to determine the resources provided by the element 14. The controller device 10 can generally be configured with information representing the type of resources provided by each element 14, but the element 14 can be queried to determine the specific resource information of the resource that matches each type of resource provided by the element 14. The type of resource can include, for example, a forwarding table, a routing table, a network access policy (e.g., for access to certain users, a firewall policy, etc.), or other such resources.
[0031] According to the technology of the present disclosure, the controller device 10 can also be configured to transform the topology model into a unified network model, and then use the unified network model to perform root cause analysis (RCA) of various network problems (such as device or resource errors or failures). The unified network model is a model that contains data representing network resources that are partially managed by a domain controller (such as the controller device 10). For example, the controller device 10 can represent an IP / MPLS controller, an optical controller, or an application controller, and the unified network model can be a business engineering model.
[0032] The present disclosure recognizes that the number of network-connected devices, such as Internet of Things (IoT) devices, has increased rapidly recently. The need for large data centers for hosting cloud and web applications has given rise to the need for mechanisms for efficiently controlling complex, heterogeneous, and distributed networks of devices. These rapidly evolving network systems require distributed and fast diagnostic solution techniques to analyze dependent events.
[0033] In any complex network, damage in the base layer of the network may cause a large number of higher layer services to fail, which may or may not be directly related to the failed component. There are various techniques that have been proposed for providing RCA. Techniques for correlating events and identifying the root cause of problems include model traversal techniques and dependency graph techniques.
[0034] Model traversal techniques use object models to determine fault propagation. Networks are represented by various components and relationships. Based on the model, fault dependencies can be inferred and used to identify the root cause of the problem. Unlike fault propagation models (such as dependency graphs), model traversal techniques do not specify fault dependencies directly, but allow fault dependencies to be derived from the model at run time. This is beneficial for networks that change frequently, but does not provide more complex fault propagation scenarios because it is assumed that only one problem occurs at a specific time.
[0035] Dependency graph techniques involve the use of a dependency graph, which is a directed graph that models the dependencies between managed objects. In the case of a network, the nodes in the graph represent network elements (e.g., hosts), and an edge from node A to node B indicates that a failure in node A can cause a failure in node B because node B depends on node A. Dependency graphs are best suited for networks that do not change often. Networks that change frequently require constant updating of dependencies.
[0036] As discussed in more detail below, the controller device 10 can be configured to use a combination of model traversal techniques and dependency graph techniques. In addition, a user (such as an administrator 12) can define resources and resource discovery to customize the controller device 10 based on the customer environment to perform a model-driven RCA solution. Once the controller device 10 discovers the resources, the controller device 10 can build a resource dependency mathematical model that allows the discovery of dependencies across resources. The resource dependency mathematical model provides a relationship between cause and effect dependencies of resources.
[0037] The present disclosure also recognizes that model-driven RCA can use a unified network model in mathematical form to represent dependencies. A domain controller (such as, controller device 10) can discover the network model, and an element management system (EMS) can perform inventory services to discover device-level resources. The present disclosure describes a method for transforming a network topology model and inventory resources into a unified network model in nano-mathematical form. The network model may include transient models that change frequently, such as, dynamic label switching paths (LSPs). The controller device 10 can use the techniques of the present disclosure to resolve these transient states. For example, as discussed in more detail below, the controller device 10 can use a resource mathematical model to discover pipelines.
[0038] In order to support programmable resource mathematical model discovery technology, the controller device 10 can provide an interface through which the administrator 12 can program resource models (including devices and network resources), resource discovery rules, and resource merging strategies. The controller device 10 can also generate a graph database model based on the defined resources. The controller device 10 can also use a discovery pipeline to transform the topology model into a unified network model. The controller device 10 can retain the transformed resources, execute merging strategies to build dependency edges between resources, build dependency mathematical models, handle transient changes, and provide resynchronization support. When the underlying resources are changed, the controller device 10 can merge the changes.
[0039] The controller device 10 may be an existing, previously deployed controller device in the customer's network. The controller device 10 may maintain resources in a local model. According to the technology of the present disclosure, the controller device 10 may be a domain-specific controller configured to synchronize resources to build a common resource database and a corresponding mathematical model.
[0040] Typically, the controller device 10 can provide programmable support for resources that are part of a playbook. Once the playbook is applied, the controller device 10 can transform the data representing the resources and build relationships across resources based on the programmed merge strategy. After discovering the resources, the controller device 10 can build a resource dependency mathematical model (also referred to as a dependency mathematical model) that allows determination of causal dependencies across resources. The controller device 10 can construct the dependency mathematical model as follows. First, the controller device 10 can fill in the relationships / dependency edges between resources, where the dependency edges can be across device resources, across device resources for services, and across services. Based on the model, the controller device 10 can build a dependency mathematical model that provides causal relationships between resources.
[0041] Figure 2 It is a diagram for Figure 1 1 is a block diagram of an example set of components of the controller device 10. In this example, the controller device 10 includes a control unit 22, a network interface 34, and a user interface 36. The network interface 34 represents a method for communicatively coupling the controller device 10 to an external device (e.g., Figure 1 The network interface 34 may represent an example interface of one of the elements 14 shown. The network interface 34 may represent a wireless and / or wired interface, for example, an Ethernet interface or a radio configured to communicate according to a wireless standard, such as one or more IEEE 802.11 wireless networking protocols, such as 802.11a / b / g / n or other such wireless protocols. Although only one network interface is illustrated for purposes of example, in various examples, the controller device 10 may include multiple network interfaces.
[0042] The control unit 22 represents any combination of hardware, software and / or firmware for implementing the functionality attributed to the control unit 22 and its constituent modules and elements. When the control unit 22 includes software or firmware, the control unit 22 also includes any necessary hardware for storing and executing the software or firmware, such as one or more processors or processing units. Typically, the processing unit may include: one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuit devices and any combination of such components. In addition, fixed and / or programmable logic circuit devices are typically used to implement the processing unit.
[0043] The user interface 36 represents one or more interfaces through which a user (such as an administrator 12 ( Figure 1 )) interacts with the controller device 10 through the one or more interfaces, for example, to provide input and receive output. For example, the user interface 36 can represent one or more of a monitor, a keyboard, a mouse, a touch screen, a touch pad, a trackpad, a speaker, a camera, a microphone, etc. In addition, although in this example, the controller device 10 includes a user interface, it should be understood that the administrator 12 need not interact directly with the controller device 10, but can access the controller device 10 remotely, for example, via the network interface 34.
[0044] In this example, the control unit 22 includes a user interface module 38, a network interface module 32, and a management module 24. The control unit 22 executes the user interface module 38 to receive input from the user interface 36 and / or provide output to the user interface 36. The control unit 22 also executes the network interface module 32 to send and receive data (e.g., packets) via the network interface 34. The user interface module 38, the network interface module 32, and the management module 24 may again be implemented as respective hardware units, or in software or firmware, or a combination thereof.
[0045] The control unit 22 executes the management module 24 to manage various network devices, for example, Figure 1 Element 14 is shown. Management includes, for example, based on information from a user (e.g., Figure 1 The management module 24 is a module that receives instructions from the administrator 12 and provides the user with the ability to submit instructions to configure the network device. In this example, the management module 24 also includes a configuration module 26 and a conversion module 28.
[0046] Management module 24 is configured to receive intents (e.g., high-level configuration instructions) for a set of managed network devices from a user (such as administrator 12). Over time, the user may update the configuration instructions, for example, to add new services, remove existing services, or modify existing services performed by the managed devices. Intents may be constructed according to, for example, YANG. In some examples, as discussed below, management module 24 also provides the user with the ability to submit conversion functions, which conversion module 28 performs to transform intents into low-level configuration instructions specific to the devices.
[0047] The controller device 10 also includes a configuration database 40. The configuration database 40 generally includes information describing the managed network devices (e.g., elements 14). For example, the configuration database 40 may include information indicating device identifiers (such as MAC and / or IP addresses), device types, device vendors, device categories (e.g., routers, switches, bridges, hubs, etc.), etc. The configuration database 40 also stores device-level configuration information based on intent (e.g., high-level configuration information, or in some cases, both high-level configuration information and low-level configuration information) for the managed devices (e.g., elements 14).
[0048] The transformation module 28 determines which devices are to be managed using the configuration database 40. The transformation module 28 determines which transformation function 30 to perform for the high-level configuration instructions based on the information in the configuration database 40, e.g., which device will receive the low-level configuration instructions. The transformation module 28 then performs each determined one of the transformation functions 30, thereby providing the high-level configuration instructions to the transformation function as input and receiving the low-level configuration instructions. The transformation module 28 can then provide the low-level configuration instructions to the configuration module 26.
[0049] After receiving the low-level configuration instructions from the conversion module 28, the configuration module 26 sends the low-level configuration instructions to the corresponding managed network device, for which the configuration will be updated via the network interface module 32. The network interface module 32 passes the low-level configuration instructions to the network interface 34. The network interface 34 forwards the low-level configuration instructions to the corresponding network device.
[0050] Although for purposes of example, the user interface 36 is described as allowing the administrator 12 to Figure 1) interacts with the controller device 10, but it should be understood that in other examples, other interfaces may be used. For example, the controller device 10 may include a representative state transfer (REST) client (not shown) that may act as an interface with another device, and the administrator 12 may configure the controller device 10 through the representative state transfer (REST) client. Similarly, the administrator 12 may configure the element 14 by interacting with the controller device 10 through the REST client.
[0051] The controller device 10 also stores in the configuration database 40 a representation of the components 14 ( Figure 1 ) provided by the controller device 10. The resources may include, for example, network services, hardware and / or software units, and tables of the element 14. Thus, the resources may include, for example, network tunnels (such as label switched paths (LSPs)), virtual private networks (VPNs), virtual routing and forwarding (VRF) tables, hardware interfaces, logical interfaces, etc. The controller device 10 may be configured with a script, and the administrator 12 ( Figure 1 ) can use the playbook to program various resources. A playbook can contain a set of resource definitions, resource discovery rules, and resource merging strategies.
[0052] An example dataset for scripts is shown below:
[0053]
[0054]
[0055]
[0056]
[0057] Additionally, an example interface discovery rule for the playbook is shown below:
[0058]
[0059]
[0060] Table 1 below shows a set of example descriptions of resources and their attributes:
[0061] Table 1
[0062]
[0063] Management module 24 can automatically generate a database model (in the form of a graph) based on the resource definitions shown above. The graph for the database model may include a set of nodes, each node corresponding to a resource. The graph may also include edges between nodes as discussed below. The database model may include fields according to Table 2 below:
[0064] Table 2
[0065]
[0066] Management module 24 can form two types of edges between nodes representing resources in a graph for a database model. The two types of edges include dependency edges and parent-child edges. Dependency edges create reference edges between two resources. For dependency edges, the referrer object contains reference attributes. Parent-child edges (or containment edges) represent parent-child relationships between corresponding resources. The parent object contains child attributes.
[0067] Figure 3 is a conceptual diagram illustrating an example graph database model. In this example, the graph database model includes a device node 50, a virtual routing and forwarding (VRF) node 52, an interface node 54, and a logical interface node 56. In this example, the graph database model has an inclusion edge 60 between the device node 50 and the interface node 54, which indicates that the device corresponding to the device node 50 includes (i.e., "has" or contains) an interface, i.e., a physical interface represented by the interface node 54. In this example, the graph database model also has a dependency edge 62 between the VRF node 52 and the interface node, which indicates that the VRF corresponding to the VRF node 52 depends on the physical interface corresponding to the interface node 54. Therefore, when the VRF corresponding to the VRF node 52 is used, a failure of the physical interface corresponding to the interface node 54 may also cause a failure.
[0068] Reference Figure 2 , the controller device 10 and its management module 24 can execute a resource discovery pipeline. After initializing the script on the device / device group, the management module 24 can use the resource discovery pipeline to generally process any resource (e.g., add a node to a graph for a database model). The pipeline may include a resource management phase, a dependency edge management phase between resources, and a resource dependency mathematical model generation phase.
[0069] After discovering the resources, the management module 24 may execute the resource management stage in the pipeline to build and update the parent-child relationship across the resources. The management module 24 may execute the following algorithm to build and update the parent-child relationship:
[0070] ●Check whether the resource is transient.
[0071] ●If not transient:
[0072] ○ Get existing resources in the script.
[0073] ○ Compare resources to existing resources.
[0074] ○ Generate / update resources.
[0075] o An "updated resource ID list" is formed and the ID list is passed to the second stage.
[0076] If transient:
[0077] ○ Arrange parent / child IDs.
[0078] ○ Directly pass resources to the second stage.
[0079] After the resource discovery phase, the management module 24 may perform a dependency edge management phase between resources. During this phase, the management module 24 may add (or remove) resource dependency edges between nodes corresponding to resources in the database model. The management module 24 may construct an association merging strategy according to the following algorithm:
[0080] ●Get resources based on the updated resource ID list.
[0081] ●Acquire dependent resources based on merge strategy rules
[0082] ●Run the merge policy rules
[0083] o Execution of this step includes adding / deleting dependency edges between nodes representing resources in the graph for the database model.
[0084] • An "updated association list" is formed and passed to the next stage.
[0085] ● If the resource is a transient resource, the relationship is built / updated through the input stream.
[0086] The controller device 10 also includes a dependency model 42. The dependency model 42 represents a mathematical model of resource dependencies according to the technology of the present disclosure. The management module 24 can construct the dependency model 42 according to the technology of the present disclosure, and use the combination of the dependency model 42 (i.e., dependency graph) and model traversal to perform root cause analysis (RCA), i.e., identify the causes of problems with various resources. RCA is described in more detail in U.S. application No. 16 / 821,745 filed on March 17, 2020, the entire contents of which are incorporated herein by reference. Similarly, a fault impact analysis technique that the controller device 10 can perform in addition to or as a substitute for RCA (and using the dependency model 42) is described in U.S. application No. 16 / 946,994 filed on July 14, 2020, the entire contents of which are incorporated herein by reference. In addition, multiple layers of model correlation analysis techniques that the controller device 10 can perform in addition to or as a replacement for RCA and / or fault effect analysis techniques are described in U.S. application Ser. No. 16 / 731,372 filed on Dec. 31, 2019, the entire contents of which are incorporated herein by reference.
[0087] Figure 4 is a conceptual diagram illustrating an example portion of a resource dependency mathematical model. In this example, the resource dependency mathematical model includes a resource A node 70, a resource C node 72, and a resource B node 74. According to the resource dependency mathematical model, if resource A uses resource B (as shown by edge 76 from resource A node 70 to resource B node 74), management module 24 determines the dependency in reverse order. The following mathematical model may be provided:
[0088] ●Dependencies (resources): [resource list]
[0089] ●For Figure 4 Example shown: Dependency (Resource B) → [Resource A, Resource C]
[0090] Refer again Figure 2 As discussed above, the management module 24 can construct a resource dependency mathematical model of the dependency model 42 after adding and updating the dependency edges of the resources. The management module 24 can then populate the resource dependency mathematical model as {key, value} pairs. Here, the key element represents the dependent resource, and the value element is a list of reference resources. For each update, the management module 24 can update the reverse dependency map accordingly.
[0091] The management module 24 can also perform resource synchronization. Figure 1), resources may be added, updated, or deleted accordingly. Management module 24 may synchronize these changed resources in the model. Management module 24 may execute pipelines to seamlessly account for updated (e.g., added, updated, or deleted) resources. Management module 24 may also account for resource persistence by acquiring existing resources based on the script and generating differences (increments) relative to the existing resources. During the resource association phase, management module 24 may update associations for updated resources. During the resource dependency mathematical model phase, management module 24 may update reverse dependencies.
[0092] As discussed above, the controller device 10 may be a domain controller previously deployed in a customer network (or interact with the domain controller) and may have maintained resources in a local model. In this scenario, the controller device 10 may be configured to synchronize resources from a domain-specific controller to a common resource database.
[0093] The resources may be modeled as shown in the following example:
[0094]
[0095]
[0096] In the above example, "controller.rule" contains the adapter details. Using the adapter, the management module 24 can obtain the required resources.
[0097] The management module 24 can determine the resources discovered by the external controller and populate the relationships and mathematical models for these resources. The management module 24 can also pass these resources through the pipeline to also build mathematical models for these resources.
[0098] According to IETF RFC 8345, "A YANG Data Model for Network Topologies" by Clemm et al., March 2018 (available at tools.ietf.org / html / rfc8345), networks and network resource models can be described as follows. The network data model enables the definition of network hierarchies or network stacks (i.e., networks stacked on top of each other) and the maintenance of an inventory of nodes contained in the network. The network data model also provides information for describing topological information. Specifically, it can add the concepts of "links" and "termination points" to describe the way nodes in the network are connected to each other. The network data model introduces vertical hierarchical relationships between networks, which can be enhanced to cover network inventories and network / service topologies.
[0099] Figure 5 is a conceptual diagram illustrating an example discovered network data model. In this example, the discovered network data model includes network resource 1 80, network resource 2 110, and device / node 1 90. Network resource 1 80 includes nodes D1 82, D2 84, and D3 86. Device / node 1 90 includes nodes D1 92, R1 94, R2 96, R3 98, and R4 100. Network resource 2 110 includes nodes D1 112, D2 114, and D3 116.
[0100] In this example, node D1 82 has reference edges to nodes D2 84, D3 86, R1 94, and R3 98, while node D2 has a reference edge to node D3 86. Similarly, node D1 112 has reference edges to nodes D2 114 and D3 116, while node D2 114 has a reference edge to node D3 116. Furthermore, the reference edge from node D1 82 to node D2 84 itself has a reference edge to node R3 98. Similarly, the reference edge from node D1 82 to node D3 86 has reference edges to the reference edges from node D1 112 to node D2 114 and from node D2 114 to node D3 116. Additionally, network resource 180 has a reference edge to network resource 2 110.
[0101] In this way, Figure 5 The illustrated model includes a network hierarchy demonstrating networks layered on top of each other (ie, a network stack).
[0102] Figure 6 is a flowchart illustrating an example method for constructing a resource dependency mathematical model and using the model to perform root cause analysis (RCA) according to the techniques of the present disclosure. In this example, for example, as discussed above, the controller device 10 ( Figure 1 and Figure 2 ) receives a script (130) including data representing a set of resource definitions, resource discovery rules, and resource merging policies.
[0103] Then, the controller device 10 may use the playbook to determine the resources of the network (132). For example, the controller device 10 may generate the database model as a graph and add nodes corresponding to the resources and edges between the nodes to the graph. In particular, the controller device 10 may discover the resources according to the resource definition, discovery rules, and merge policies in the playbook. The controller device 10 may also determine whether the resource is transient and process the resource differently when the resource is transient or non-transient.
[0104] The controller device 10 may also determine the relationships between resources (134). For example, the controller device 10 may determine whether the resources have a parent-child relationship or a containment relationship. The controller device 10 may update the resource dependency edges (136) in the graph accordingly, for example, by defining the edges as containment edges (i.e., parent-child edges) or dependency edges. The controller device 10 may use merge policy rules to add or delete dependency edges between nodes representing resources when determining dependent resources. Thus, the controller device 10 may generate a dependency model (138).
[0105] In addition, controller device 10 may monitor the network for modifications to resources, e.g., to detect resource additions, updates, and deletions. Controller device 10 may also determine whether such modifications correspond to transient resources. Controller device 10 may also update the dependency model, i.e., the nodes and edges in the graph, to synchronize the resources in the dependency model accordingly (140).
[0106] At some point, the controller device 10 may detect that a resource has failed (142). In response, the controller device 10 may use the dependency model to perform a root cause analysis (RCA) (144) to identify the root cause of the failed resource. Typically, the controller device 10 may trace the failure back through the resources that the failed resource depends on to ultimately identify the resource that caused the failure. For example, a failure of a network tunnel (e.g., an LSP) may be traced back to a failure of a network interface of a particular device.
[0107] In this way, Figure 6 The method shown represents an example of a method, comprising: determining, by a controller device that manages multiple network devices, resources provided by a network device; determining, by the controller device, relationships between the resources based on a programmed merging strategy; constructing, by the controller device, a dependency model using the determined relationships, the dependency model representing dependencies between the resources; determining, by the controller device, that at least one of the resources has experienced a failure; and performing a root cause analysis by the controller device using the dependency mathematical model to determine the root cause of the failure of the at least one of the resources.
[0108] The techniques described in the present disclosure may be implemented at least in part in hardware, software, firmware, or any combination thereof. For example, various aspects of the described techniques may be implemented in one or more processors, including: one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or any other equivalent integrated or discrete logic circuit devices and any combination of such components. The term "processor" or "processing circuit device" may generally refer to any of the aforementioned logic circuit devices (alone or in combination with other logic circuit devices) or any other equivalent circuit devices. A control unit including hardware may also perform one or more techniques of the present disclosure.
[0109] Such hardware, software and firmware can be implemented in the same device or in separate devices to support the various operations and functions described in this disclosure. Additionally, any of the described units, modules or components can be implemented together or individually as discrete but interoperable logical devices. Describing different features as modules or units is intended to highlight different functional aspects, and does not necessarily imply that such modules or units must be implemented by separate hardware or software components. On the contrary, the functionality associated with one or more modules or units can be performed by separate hardware or software components, or integrated in common or separate hardware or software components.
[0110] The techniques described in the present disclosure may also be implemented or encoded in a computer-readable medium (such as a computer-readable storage medium) containing instructions. Instructions embedded or encoded in a computer-readable medium may cause a programmable processor or other processor to perform the method, for example, when the instructions are executed. Computer-readable media may include non-transient computer-readable storage media and transient communication media. Computer-readable storage media (which are tangible and non-transient) may include: random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPRQM), flash memory, hard disk, CD-ROM, floppy disk, cassette, magnetic media, optical media, or other computer-readable storage media. The term "computer-readable storage medium" refers to a physical storage medium, not a signal, carrier wave, or other transient medium.
[0111] Various examples have been described. These and other examples are within the scope of the following claims.
Claims
1. A computer-implemented method comprising: Determining, by a controller device managing a plurality of network devices, resources provided by the network devices, the resources comprising one or more hardware units of the plurality of network devices, one or more software units of the plurality of network devices, or one or more network services provided by the plurality of network devices; Determining, by the controller device, the relationship between the resources according to a programmed merging strategy; The controller device uses the determined relationships to construct a dependency model, the dependency model representing dependencies between the resources, wherein constructing the dependency model includes performing a resource management phase, the resource management phase including, for each of the resources: determining whether the resource is transient; When the resource is not transient, retrieving an existing resource of the playbook, comparing the resource to the existing resource, creating or updating a node for the resource, and forming an updated resource ID list to include the created or updated node, the playbook representing a set of resource definitions, resource discovery rules, and the programmed merge strategy; as well as When the resource is transient, determining a parent / child ID for the resource; determining, by the controller device, that at least one of the resources has experienced a failure; as well as A root cause analysis is performed by the controller device using the dependency model to determine a root cause of the failure of the at least one of the resources.
2. The method according to claim 1, further comprising: Data defining the playbook is received, wherein determining the resource comprises determining the resource using the resource definition in the playbook, the resource discovery rules, and the programmed merge strategy.
3. The method according to claim 1, wherein constructing the dependency model comprises: constructing one or more nodes, each of the one or more nodes correspondingly representing one of the resources; as well as Edges between the nodes are constructed, wherein the edges represent the dependencies between the resources. The method of claim 3 , wherein each of the edges comprises one of a dependency edge or a parent-child edge.
5. The method according to claim 1, wherein constructing the dependency model comprises: using the updated list of resource IDs to determine the resources for which the dependencies have changed; executing the programmed merge strategy to add or delete dependency edges between the resources according to the dependencies; as well as An updated association list is formed.
6. The method according to claim 1, further comprising: determining that one or more of the resources have been added, updated, or deleted; Access existing resources; determining one or more differences between the existing resources and the one or more resources that have been added, updated, or deleted; updating associations for the one or more resources that have been added, updated, or deleted; as well as Reverse dependencies of the dependency model are updated using the updated associations.
7. The method of claim 1 , wherein determining the resource comprises: The resources are determined from an external controller device separate from the controller device.
8. A controller device for managing multiple network devices, the controller device comprising: a memory configured to store a dependency model and a programmed merge policy, the dependency model representing dependencies between resources provided by the network devices, the resources comprising one or more hardware units of the plurality of network devices, one or more software units of the plurality of network devices, or one or more network services provided by the plurality of network devices; as well as One or more processors implemented in a circuit arrangement and configured to: determining the resources provided by the network device; determining the relationship between the resources according to the programmed merge strategy; The dependency model is constructed using the determined relationships, wherein to construct the dependency model, the one or more processors are configured to perform a resource management phase to construct the dependency model, the resource management phase comprising: for each of the resources: determining whether the resource is transient; When the resource is not transient, retrieving an existing resource of the playbook, comparing the resource to the existing resource, creating or updating a node for the resource, and forming an updated resource ID list to include the created or updated node, the playbook representing a set of resource definitions, resource discovery rules, and the programmed merge strategy; and When the resource is transient, determining a parent / child ID for the resource; determining that at least one of the resources has experienced a failure; and A root cause analysis is performed using the dependency model to determine a root cause of the failure of the at least one of the resources.
9. A controller device according to claim 8, wherein the one or more processors are further configured to receive data defining the playbook, wherein the one or more processors are configured to determine the resources using the resource definitions in the playbook, the resource discovery rules, and the programmed merge strategy.
10. The controller device of claim 8, wherein to construct the dependency model, the one or more processors are configured to: constructing one or more nodes, each of the one or more nodes correspondingly representing one of the resources; and Edges between the nodes are constructed, wherein the edges represent the dependencies between the resources. 11 . The controller device of claim 10 , wherein each of the edges comprises one of a dependency edge or a parent-child edge.
12. The controller device of claim 8, wherein to construct the dependency model, the one or more processors are configured to: using the updated list of resource IDs to determine the resources for which the dependencies have changed; executing the programmed merge strategy to add or delete dependency edges between the resources according to the dependencies; as well as An updated association list is formed.
13. The controller device of claim 8, wherein the one or more processors are further configured to: determining that one or more of the resources have been added, updated, or deleted; Access existing resources; determining one or more differences between the existing resources and the one or more resources that have been added, updated, or deleted; updating associations for the one or more resources that have been added, updated, or deleted; and Reverse dependencies of the dependency model are updated using the updated associations.
14. The controller device of claim 8, wherein the one or more processors are configured to determine the resource from an external controller device separate from the controller device.
15. A non-transitory computer-readable storage medium having stored thereon instructions that, when executed, cause a processor of a controller device that manages a plurality of network devices to: determining resources provided by the network devices, the resources comprising one or more hardware units of the plurality of network devices, one or more software units of the plurality of network devices, or one or more network services provided by the plurality of network devices; determining relationships between the resources according to a programmed merge strategy; The determined relationships are used to construct a dependency model, the dependency model representing dependencies between the resources, wherein the instructions causing the processor to construct the dependency model include instructions causing the processor to execute the following: performing a resource management phase, including: for each of the resources: determining whether the resource is transient; When the resource is not transient, retrieving an existing resource of the playbook, comparing the resource to the existing resource, creating or updating a node for the resource, and forming an updated resource ID list to include the created or updated node, the playbook representing a set of resource definitions, resource discovery rules, and the programmed merge strategy; and When the resources are transient, determining parent / child IDs for the resources; determining that at least one of the resources has experienced a failure; and A root cause analysis is performed using the dependency model to determine a root cause of the failure of the at least one of the resources.
16. The non-transitory computer-readable storage medium of claim 15, further comprising instructions for causing the processor to execute the following: receiving data defining the playbook, wherein the instructions for causing the processor to determine the resources include instructions for causing the processor to execute the following: determining the resources using the resource definition in the playbook, the resource discovery rules, and the programmed merge strategy.
17. The non-transitory computer-readable storage medium of claim 15, wherein the instructions causing the processor to construct the dependency model include instructions causing the processor to: constructing one or more nodes, each of the one or more nodes correspondingly representing one of the resources; and Edges between the nodes are constructed, the edges representing the dependencies between the resources, wherein each of the edges comprises one of a dependency edge or a parent-child edge.
18. The non-transitory computer-readable storage medium of claim 15, further comprising instructions for causing the processor to: determining that one or more of the resources have been added, updated, or deleted; Access existing resources; determining one or more differences between the existing resources and the one or more resources that have been added, updated, or deleted; updating associations for the one or more resources that have been added, updated, or deleted; and Reverse dependencies of the dependency model are updated using the updated associations.
