Query agent for communicating dynamic system status

Abstract

The present disclosure relates to a query agent for transmitting dynamic system state. In general, a technique for providing dynamic system state for a distributed operating system is described. A network device includes a plurality of hardware computing nodes configured to execute a distributed operating system that provides an execution environment for executing a plurality of applications. A computing node of the plurality of computing nodes is configured to configure a dynamic application to operate as a proxy to receive one or more requests for state information stored in a data storage. The computing node retrieves the state information stored in the data storage; stores the retrieved state information in a local data storage of the computing node; receives one or more requests for state information; and sends a response including the requested state information from the local data storage.

Description

Technical Field

[0001] The present disclosure relates to network devices, and more particularly, to operating systems for network devices. Background Art

[0002] An operating system for a network device (e.g., a router) is used to provide an execution environment in which various applications (e.g., network protocols, maintenance interfaces, virtualization applications, etc.) can execute. One function of a router operating system is to establish a means by which state information can be propagated to the various components, or in other words, the computing nodes of the router to allow the router to properly route network traffic within the network.

[0003] A router may, for example, maintain state information representing the current state of resources of the router. For example, a router maintains state information representing the state of an interface between the router and a network. Such state information may include information representing the state of one or more interface cards (IFCs), such as the current configuration of the IFCs. As an additional example, a router may maintain state information representing the state of one or more packet forwarding engines (PFEs), one or more routing engines, or other resources within the router.

[0004] A control node called a "routing engine" running within a router may execute an instance of an operating system to propagate state information (and changes thereto) to various other processes or components within the router. These other processes or components are sometimes referred to as "consumers" because they receive and utilize (or in other words, "consume") state information maintained by an instance of the operating system. These consumers utilize the state information when performing their various functions.

[0005] In some examples, the operating system can be distributed on the computing nodes of the router, and the computing nodes can include routing engines, interface cards, packet forwarding engines, and non-network nodes, such as processors, central processing units (CPUs), application-specific integrated circuits (ASICs), and graphics processing units (GPUs). For example, the computing nodes can execute corresponding instances of the distributed operating system so that the operating system is distributed on all computing nodes. Each computing node can self-organize and merge to execute a single instance of the distributed operating system. The computing nodes executing the corresponding instances of the distributed operating system can be configured to store state information in corresponding data structures. Summary of the invention

[0006] Techniques for providing dynamic system state for distributed operating systems are described. For example, a distributed operating system for a network device can allow dynamic expansion or contraction (or in other words, "elasticity") of underlying hardware resources, while also potentially providing robust convergence of state information on production components (so-called "producers") and consumption components (so-called "consumers"). The operating system can be distributed on computing nodes (also referred to as "hardware computing nodes", "computing nodes" or "nodes"), which can include routing engines, interface cards, packet forwarding engines, and non-network nodes, such as processors, central processing units (CPUs), application-specific integrated circuits (ASICs), and graphics processing units (GPUs). The computing nodes can initially be merged from the kernel, detect each other via an object flooding protocol (OFP) based on topology discovery similar to a link state routing protocol, and organize via management processes to execute the distributed operating system.

[0007] Once started on a compute node, a distributed operating system can allow for real-time (or near real-time) construction of a synchronization tree for synchronizing a database of state information maintained by the distributed operating system. The operating system can synchronize the database using OFP. The operating system can handle consistency between the compute nodes executing instances of the distributed operating system so that consistent state information can be maintained by each instance in the event of various connections or failures of instances of the distributed operating system. In some examples, the synchronized database of state information of the distributed operating system is collectively referred to as a distributed data store, for example, an object daemon data store (DDS).

[0008] After being merged to execute a single instance of the distributed operating system, the computing nodes can execute the distributed application suite within the execution environment provided by the distributed operating system (e.g., network protocols, management interfaces, command line interface (CLI), etc.). The distributed operating system can distribute the application programs among the computing nodes supporting the execution of the distributed operating system based on the computing node capabilities and / or roles within the multi-rack router. The distributed operating system can manage the propagation of state information to support the execution of the distributed operating system and / or the application programs executed within the execution environment provided by the distributed operating system.

[0009] As an example of the technology described herein, a dynamic application is configured to operate as a proxy to receive one or more requests for state information in a DDS. For example, at least one computing node executing an instance of a distributed operating system may configure a dynamic application to operate as a proxy (e.g., a network server) to receive one or more requests from a client (e.g., a network client) and retrieve state information from the DDS. The dynamic application may retrieve state information from the DDS during initialization or each time the state information in the DDS is updated, and store the retrieved state information in a local data storage or memory. When the dynamic application receives one or more requests for state information in the DDS (e.g., a Hypertext Transfer Protocol (HTTP) request), the dynamic application may retrieve the state information stored in its local data storage or memory and configure a response (e.g., a Hypertext Markup Language (HTML) file) with the retrieved state information.

[0010] In this manner, a dynamic application can operate as a general proxy application to receive one or more requests for state information from a DDS that can utilize network-based queries, thereby providing a more flexible way to access state information from a DDS. That is, the dynamic application can decide what information to provide and / or change the presentation of the requested information. In addition, by configuring the dynamic application as a network server, the dynamic application is multi-threaded, which enables the dynamic application to handle multiple requests at a time.

[0011] In one example, a method includes executing a distributed operating system by a plurality of hardware computing nodes included in a network device, the distributed operating system providing an execution environment for executing a plurality of applications. The method also includes configuring a dynamic application by at least one of the plurality of hardware computing nodes to operate as an agent to receive one or more requests for state information stored in a data storage, wherein the state information includes information representing different states of the plurality of applications. The method also includes retrieving the state information stored in the data storage by the dynamic application. The method also includes storing the retrieved state information in a local data storage of at least one of the plurality of hardware computing nodes by the dynamic application. In addition, the method also includes receiving one or more requests for state information by the dynamic application. The method also includes sending a response including the requested state information from the local data storage in response to receiving the one or more requests by the dynamic application.

[0012] In another example, a network device includes: a plurality of hardware computing nodes configured to execute a distributed operating system, the distributed operating system providing an execution environment for executing a plurality of applications. At least one of the plurality of hardware computing nodes is configured to: configure a dynamic application to operate as an agent to receive one or more requests for state information stored in a data storage, wherein the state information includes information representing different states of the plurality of applications; retrieve the state information stored in the data storage; store the retrieved state information in a local data storage of at least one of the plurality of hardware computing nodes; receive one or more requests for state information; and in response to receiving the one or more requests, send a response including the requested state information from the local data storage.

[0013] In another example, a non-transitory computer-readable storage medium having instructions stored thereon, which, when executed, causes one or more processors of a network device to: execute a distributed operating system that provides an execution environment for executing multiple applications; configure a dynamic application to operate as an agent to receive one or more requests for state information stored in a data storage device, wherein the state information includes information representing different states of the multiple applications; retrieve the state information stored in the data storage device; store the retrieved state information in a local data storage device of at least one hardware computing node among a plurality of hardware computing nodes; receive one or more requests for state information; and in response to receiving the one or more requests, send a response including the requested state information from the local data storage device.

[0014] The details of one or more aspects of these techniques are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of these techniques will become apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a block diagram of an example network computing environment in which a service provider network includes multi-chassis routers configured to operate in accordance with the distributed operating system techniques described in this disclosure.

[0016] Figure 2 is a block diagram illustrating an example multi-chassis router executing a distributed operating system that implements dynamic applications according to the techniques described in this disclosure.

[0017] Figure 3 It is shown Figure 2 FIG. 1 is a block diagram of an example node of a multi-chassis router configured to operate according to respective aspects of the techniques described in this disclosure.

[0018] Figure 4 is a block diagram illustrating an example operation of a dynamic application according to the disclosed technology.

[0019] Figure 5 It is shown Figure 3 A flow diagram of example operations of a multi-chassis router in performing respective aspects of the techniques described in this disclosure is shown. DETAILED DESCRIPTION

[0020] Figure 1 is a block diagram illustrating an example computing environment 2, wherein a service provider network 6 includes a multi-chassis router 4. In this example, the multi-chassis router 4 communicates with edge routers 5A and 5B ("edge routers 5") to provide access to the network 6 to customer networks 8A-8C ("customer networks 8"). In one example, the multi-chassis router 4 includes a central switch card chassis (SCC) operating as a control node and one or more line card chassis (LCCs) operating as packet routing devices. The LCC may contain all physical interfaces for coupling to other devices within the network 6, while the SCC controls and routes traffic between the LCCs.

[0021] Although not shown, the service provider network 6 may be coupled to one or more networks managed by other providers and may therefore form part of a large-scale public network infrastructure (e.g., the Internet). Therefore, the customer network 8 may be considered an edge network of the Internet. The service provider network 6 may provide access to the Internet to computing devices within the customer network 8 and may allow computing devices within the customer network 8 to communicate with each other. The service provider network 6 may include various network devices other than the multi-chassis router 4 and the edge router 5, such as additional routers, switches, servers, or other devices.

[0022] In the illustrated example, edge router 5A is coupled to customer network 8A via access link 9A, and edge router 5B is coupled to customer networks 8B and 8C via access links 9B and 9C, respectively. Customer network 8 may be a network of geographically separate sites of an enterprise. Customer network 8 may include one or more computing devices (not shown), such as personal computers, laptop computers, handheld computers, cellular phones (including so-called "smart phones"), tablet computers, workstations, servers, switches, printers, or other devices. Figure 1 The configuration of the network 2 shown is merely an example. For example, the service provider network 6 may be coupled to any number of customer networks 8. However, for ease of description, Figure 1 Only customer networks 8A-8C are shown.

[0023] The multi-chassis router 4 may be configured to execute a single instance of an operating system on all computing nodes (which may be collectively referred to as all producers and consumers) of the multi-chassis router 4. The operating system described in the present disclosure may be referred to as a distributed operating system 10 ("DOS 10") because the execution of the operating system is distributed across all computing nodes. The computing nodes may include hardware computing nodes (e.g., routing engines, hardware forwarding units, which may include application specific integrated circuits and interface cards) and virtual computing nodes executed by hardware control units (e.g., one or more processors, one or more application specific integrated circuits, field programmable gate arrays, etc.).

[0024] Distributed operating system 10 can avoid redundant execution of separate instances of the same operating system while simplifying the propagation of state information through flooding in the form of multicast delivery of state information. In addition, distributed operating system 10 can be resilient to computing node failures, allowing a separate computing node supporting the distributed operating system to fail without requiring a restart of the remaining computing nodes supporting the execution of the single instance of the distributed operating system.

[0025] The computing nodes may be merged to execute the distributed operating system 10. For example, the computing nodes may discover each other (e.g., by using OFP) and be organized to execute a single instance of the distributed operating system 10. After merging to execute a single instance of the distributed operating system 10, the computing nodes may execute a distributed suite of applications in an execution environment provided by the distributed operating system. For example, the execution environment of the distributed operating system 10 may include applications 20A-20N (collectively referred to as "applications 20") in user space 18. Each of the applications 20 may represent a user-level process that may run network management software, execute routing protocols to communicate with peer routing devices, execute configuration commands received from an administrator, maintain and update one or more routing tables, manage subscriber flow processing, create one or more forwarding tables, and other functions. The application 20 may refer to any process other than low-level drivers and / or firmware not described above for the distributed operating system 10, including Daemon and PFE application (or in other words, software).

[0026] Once started on a computing node, the distributed operating system can allow for real-time (or near real-time) construction of a synchronization tree for synchronizing a state information database maintained by the distributed operating system. The state information can be, for example, state information of the distributed operating system 10 and / or one or more applications 20 executed in the execution space provided by the distributed operating system 10. The computing nodes of the multi-rack router 4 that execute the corresponding instance of the distributed operating system 10 can be configured to store the state information in a corresponding data structure, such as a tree data structure. For example, a computing node can instantiate a tree data structure, such as a radix trie. A radix trie is a tree composed of keys, for which each internal trie node has at least two child nodes. In order to locate a trie node with a specific key, start from the left and traverse the tree by checking the contents of the key. The radix trie is a minimum tree because there is no internal trie node with only one child node. Patricia Trie is a special form of radix trie.

[0027] The computing nodes may represent the state information as message fragments, wherein each message fragment is stored in a tree node of the tree data structure. The computing nodes may synchronize the tree data structure (e.g., using OFP) by flooding objects representing the message fragments to other computing nodes. For example, when a computing node of the multi-rack router 4 receives updated state information, the computing node of the multi-rack router 4 may update its local tree data structure and then flood the updated state information to other computing nodes. In this way, the computing nodes may maintain synchronization between their corresponding data structures for storing state information of the distributed operating system 10 and / or applications executed in the execution environment provided by the distributed operating system 10. Synchronous data structures may be collectively referred to herein as distributed data stores or object daemon data stores (DDS), such as DDS 24.

[0028] The distributed operating system 10 can manage the propagation of state information to support the execution of the distributed operating system 10 and / or the application 20 executed within the execution environment provided by the distributed operating system 10. For example, the distributed operating system 10 can utilize a hierarchical naming scheme to propagate state information from producers to consumers. For more information about hierarchical naming schemes and how hierarchical naming schemes use the Object Flooding Protocol (OFP) to more efficiently propagate state information, see U.S. application serial number 15 / 198,912, filed on June 30, 2016, entitled “HIERARCHICAL NAMING SCHEME FOR STATE PROPAGATION WITHIN NETWORK DEVICES,” the entire contents of which are incorporated herein by reference as if set forth in its entirety.

[0029] Additional examples of distributed operating systems are described in U.S. application serial number 15 / 637,775, filed on June 29, 2017, entitled “COMMUNICATING STATE INFORMATION INDISTRIBUTED OPERATING SYSTEMS,” U.S. application serial number 15 / 637,809, filed on June 29, 2017, entitled “ORGANIZING EXECUTION OF DISTRIBUTED OPERATING SYSTEMS FOR NETWORK DEVICES,” and U.S. application serial number 15 / 637,839, filed on June 29, 2017, entitled “MAINTAINING COHERENCY IN DISTRIBUTED OPERATING SYSTEMS FOR NETWORK DEVICES,” the entire contents of which are incorporated herein by reference.

[0030] In some cases, a client may request state information from DDS 24. In examples where different applications produce different state information stored in DDS 24, existing request processes need to send queries to specific applications to retrieve the corresponding state information from DDS 24. For example, application 20A is a producer of routing state information stored in DDS 24, while application 20N is a producer of interface state information stored in DDS 24. In this example, a request for routing state information is sent to application 20A, which in turn retrieves the routing state information from DDS 24. Similarly, a request for interface state information is sent to application 20N, which in turn retrieves the interface state information from DDS 24. Because applications are distributed throughout computing nodes in a distributed operating system, these existing request processes must discover the location of the applications in order to send requests to the appropriate applications.

[0031] The existing request process may also be multi-hop, which results in a slow response time. As an example of using the CLI to request status information, each request configured via the CLI (e.g., a request in an extensible markup language (XML) format) can be passed to a command line interface application, such as a command daemon (cmdd), which converts the request into a query that conforms to specific semantics used by an application (e.g., a database daemon) with an interface that supports the semantics. The application receiving the query can retrieve the status information from the DDS 24. The application can configure a response with the requested status information, which is converted by cmdd into a response compatible with the CLI (e.g., in XML format).

[0032] In some examples, an existing request process may use a proprietary syntax to query state information from DDS 24. Using the above example, cmdd may convert a request from a CLI (e.g., XML) into a proprietary query (e.g., a ddx request), and may also convert a proprietary response (e.g., a ddx response) into a response compatible with the CLI (e.g., XML). The use of a proprietary syntax limits the request process to only clients that are familiar with the proprietary syntax.

[0033] The existing request process is also a single-threaded process, in which the application performs a query on DDS each time it receives a request. That is, the CLI can only support the processing of a single request at a time. The CLI is a run-to-completion process, in which each instance of the CLI can only process a single request at a time until the corresponding response is available and the response is returned to the current response, which may cause delays in request processing. Thus, in some examples where the application receives multiple requests (e.g., a first request and a second request), the application cannot process the second request until the application completes the query on DDS based on the first request.

[0034] Furthermore, by requesting status information from the DDS 24 using the CLI, the entire XML file of status information is returned, rather than specific objects of the status information (e.g., discrete portions of the status information). As a result, the CLI may consume a significant amount of bandwidth (e.g., memory bus bandwidth) when transmitting the entire XML file, which may also result in further inefficiencies because the entire XML file must be processed to identify the relevant objects requested, thereby potentially consuming additional processing cycles.

[0035] According to the techniques described in this disclosure, the multi-chassis router 4 may include multiple computing nodes configured to execute a distributed operating system that provides an execution environment in which a dynamic application 22 is executed to operate as an agent to receive one or more requests for different state information in the DDS 24. For example, at least one computing node of the multi-chassis router 4 that executes an instance of the distributed operating system 10 may be configured with the dynamic application 22. In some examples, a remote node (e.g., an edge router 5) that may execute an instance of the distributed operating system 10 may execute the dynamic application 22 using the OFP.

[0036] In some examples, at least one computing node executing an instance of dynamic application 22 may receive configuration information (e.g., via a configuration file, command line options, or other input processing mechanism) that defines parameters under which dynamic application 22 may operate as a proxy to receive one or more requests for different state information. For example, the configuration information may include server parameters, service parameters (e.g., output parameters), and / or a structure of a data store generated on at least one computing node executing an instance of dynamic application 22.

[0037] The configuration information may instruct the computing node executing the corresponding instance of the dynamic application 22 to retrieve state information (e.g., a subset of the state information) from the DDS 24 and store the retrieved state information in a local data storage or memory of the computing device. For example, the configuration information may instruct the dynamic application 22 to retrieve state information from the DDS 24 during initialization and / or each time the state information in the DDS 24 is updated. The configuration information may also instruct the computing node executing the corresponding instance of the dynamic application 22 to construct a local data storage to store the retrieved state information. The local data storage may map the state information object retrieved from the DDS 24 to a response to a request from a client.

[0038] The server parameters of the configuration information may include parameters required for the computing node to operate as a server. The server parameters may include port numbers, authentication, etc. The computing node executing the corresponding instance of the dynamic application 22 may expose an interface (e.g., port) based on the configuration information so that the computing node can receive one or more requests from a service endpoint (e.g., a network client). In some examples, the computing node executing the instance of the dynamic application 22 may configure a port to receive a network request, such as a hypertext transfer protocol (HTTP) get request. In this way, the dynamic application 22 can operate as a network server at a predefined port to receive one or more network-based requests from a network client.

[0039] The service parameters of the configuration information may include parameters for responding to the state information request. For example, the service parameters may include the output format of the state information object. As an example, the configuration information may enable the computing node to be implemented according to a representational state transfer (REST) ​​software architecture to send and receive messages in a data interface format. The messages in the data interface format may conform to an open standard format that uses human-readable text to transmit data objects including attribute-value pairs. An example of a message format in the data interface format is the JavaScript Object Notation (JSON) described in the Internet Engineering Task Force (IETF), Request for Comments 7519, May 2015, the entire contents of which are incorporated herein by reference.

[0040] Dynamic application 22 may be multi-instanced in that multiple computing nodes of multi-chassis router 4 may implement respective instances of dynamic application 22. For example, configuration information may define a first computing node (e.g., a routing engine) to execute a first instance of dynamic application 22 to provide a view of a first subset of state information (e.g., routing state information), and define a second computing node to execute a second instance of dynamic application 22 to provide a view of a second subset of state information (e.g., interface state information).

[0041] In operation, the dynamic application 22 may retrieve state information from the DDS 24 during initialization or updating of the DDS 24 and store the retrieved state information in a local data store or memory. The dynamic application 22 may receive, for example, one or more requests (e.g., HTTP get requests) from a network client requesting state information in the DDS 24. The dynamic application 22 may retrieve the state information stored in its local data store or memory and may configure a response, such as a hypertext markup language file, with the retrieved state information. In some examples, the dynamic application 22 may include an HTML tag with the response.

[0042] In some examples, the dynamic application 22 can also register for notifications via a method suitable for obtaining the requested state information. For example, the dynamic application 22 can implement a network interface library (e.g., IFLIB) on an event handler for speed notifications. In some other examples, the dynamic application 22 can be configured to listen to requests from existing request processes and from request processes according to one or more aspects of the technology described herein.

[0043] The techniques described herein may provide one or more technical advantages. For example, by configuring a dynamic application to store state information, the dynamic application may process multiple requests at once by retrieving state information stored in its local data storage or memory without having to query the DDS for each request. In addition, the dynamic application running as a network server uses a non-proprietary syntax (e.g., HTTP requests) that is generic and does not require a proprietary syntax. In addition, the request may be a REST-style query that is easily integrated into an existing system. By implementing a generic query based on a network-based cache, external queries for state information are easier to configure, faster to process, and distributed to multiple systems.

[0044] Figure 2 is a block diagram illustrating an example multi-chassis router 120 configured to operate according to the techniques described in this disclosure. Figure 2 The multi-chassis router 120 may represent Figure 11. The multi-chassis router 120 is an example of a multi-chassis router 120. The multi-chassis router 120 routes packets between network devices through a network. In this example, the multi-chassis router 120 includes four substantially identical LCCs 128A-128D ("LCCs 128") and an SCC 122 that operates as a central control node. In other examples, the multi-chassis router may include more or fewer LCCs. The SCC 122 provides centralized switching and control for the multi-chassis router 120. The LCCs 128 provide an interface to the network using IFC groups 134A-134D ("IFCs 134").

[0045] SCC 122 includes a switch fabric 124 and a master routing engine 126. Figure 2 128, i.e., a connection that is separate from the network. The functions of the master routing engine 126 include maintaining routing information that describes the network topology and using this information to derive a forwarding information base (FIB). The routing engine 126 controls packet forwarding in the multi-chassis router 120 by installing the FIB in the LCC 128 via communication with the local routing engine 130 through the cable 137. The FIB of one LCC 128 may be the same or different from the FIB of other LCCs 128 and SCCs 122. Because the cable 137 provides a dedicated connection between the SCC 122 and the LCC 128, i.e., separate from the packet forwarding connection provided by the cable 136, the FIB in the LCC routing engine 130 can be updated without interrupting the packet forwarding performance of the multi-chassis router 120.

[0046] Each LCC 128 includes a local routing engine 130A-130D ("routing engine 130"), a switch fabric 125A-125D ("switch fabric 125"), at least one packet forwarding engine (PFE) (shown as PFE 132A-132D ("PFE 132")), and one or more IFCs 134. In some examples, when the multi-chassis router 120 is configured to provide high availability, in addition to the local routing engine 130, the LCC 128 may also include a backup local routing engine, which may be referred to as a master local routing engine 130 in a high availability configuration.

[0047] Multi-chassis router 120 performs routing functions in the following manner. An incoming packet is first received from the network by one IFC 134 (e.g., 134B), which directs the packet to one PFE 132 (e.g., PFE 132B). The PFE then uses the FIB provided by the primary local routing engine (e.g., routing engine 130B) to determine the appropriate route for the packet. If the packet is destined for an outbound link associated with one of the IFCs 134 that originally received the packet, the PFE forwards the packet to the outbound link. In this manner, the packet is sent by the same PFE that received the packet from the network bypass switch fabric 124 and switch fabric 125.

[0048] Otherwise, the PFE sends the packet to the switch fabric 125, where the packet is directed to the switch fabric 124 and routed to one of the other PFEs 132, e.g., PFE 132D. The PFE (e.g., PFE 132D) sends the packet through the network via one IFC 134 (e.g., IFC 134D). Thus, an input packet received by one LCC 128 may be sent to its destination by another LCC 128. Other multi-chassis routers operating in a manner consistent with the techniques described in the present disclosure may use different switching and routing mechanisms.

[0049] Local routing engine 130 controls and manages LCC 128, but is subordinate to master routing engine 126 of SCC 122. For example, after receiving a state information update from master routing engine 126, local routing engine 130 forwards the state information update to consumers on LCC 128 using a hierarchically ordered and temporarily linked data structure. For example, consumers that receive state information updates from local routing engine 130 include PFE 132 and IFC 134. Local routing engine 130 also distributes the FIB exported by master routing engine 126 to PFE 132.

[0050] Routing engines 126 and 130 may operate according to executable instructions obtained from one or more computer-readable media. Examples of such media include random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory, etc. The functions of multi-chassis router 120 may be implemented by executing instructions from computer-readable media using one or more processors, discrete hardware circuits, firmware, software executed on a programmable processor, or a combination thereof.

[0051] As described above, the nodes may include routing engine 126, routing engine 130, PFE 132, and IFC 134. The links may include switch fabric 124 and cables 136 and 137 as well as other cables shown but not listed for ease of illustration. The various nodes may be relative to Figure 3 A single node of the illustrated multi-chassis router 120 performs aspects of the techniques described in more detail below.

[0052] Figure 3 1 is a block diagram illustrating an example computing node 200 of a multi-chassis router 120 that is configured to operate according to respective aspects of the techniques described in the present disclosure. As examples, computing node 200 may represent a routing engine 126, one routing engine 130, a switch card chassis 122, or one line card chassis 128.

[0053] like Figure 3 As shown, computing node 200 executes kernel 202, which enables interaction with the hardware of computing node 200. For example, each computing node (including computing node 200) that executes an instance of distributed operating system 10 can execute kernel 202, for example, Each computing node may self-organize, coalesce, to execute a single instance of the distributed operating system 10. After self-organization (or, in other words, coalescing), the computing nodes 200 may execute a single distributed kernel 202 to the extent that the kernel 202 is aware of applications (e.g., application programs 230A-230N, dynamic applications 232) and / or other processes executed by other computing nodes.

[0054] Once the kernel 202 is operational, the computing node 200 can execute the OFP 204, which determines the topology 222 of the computing nodes executed within the multi-chassis router 120, which allows the computing nodes of a single instance of the distributed operating system 10 to be merged and executed. The OFP 204 provides a mechanism by which computing nodes (e.g., computing nodes other than the computing node 200) that can be used to participate in the execution of the distributed operating system 10 and the links interconnecting the computing nodes are discovered. The distributed operating system 10 initially executes the OFP 204 to determine the topology 222 of the computing nodes, which allows the computing nodes of a single instance of the distributed operating system 10 to be merged and executed. OFP physical topology discovery can occur in a manner similar to the link state protocol. The links to which a specific instance of the OFP 204 is coupled can be "discovered" by configuring the OFP 204. That is, a network administrator or other operator can configure the OFP 204 with the links to which each specific instance of the OFP 204 is coupled.

[0055] OFP 204 may discover or determine topology 222 by receiving notifications 224. OFP 204 may receive advertisements 224 from each other computing node supporting the execution of distributed operating system 10 as a producer or consumer of state information. Each advertisement 224 may specify a computing node and one or more links directly coupled to the computing node. OFP 204 may be configured (e.g., by a network administrator) with links directly coupled to computing nodes 200. Based on advertisements 224 and links 226, OFP 204 may construct topology 222. Topology 222 may represent a graph data structure including graph nodes representing computing nodes of multi-chassis router 120 and edges interconnecting graph nodes representing links interconnecting computing nodes of multi-chassis router 120. OFP 204 also includes state 228, representing a state data structure storing a plurality of objects, e.g., a tree data structure (e.g., a radix trie), each object defining a portion of state information of at least one of distributed operating system 10 or an application program executed in an application space provided by distributed operating system 10.

[0056] Based on the link 226, the OFP 204 can also generate an advertisement 224, and send the generated advertisement 224 via the link identified by the link 226, so that the OFP 204 executed by the adjacent computing node can also generate the topology 222. As with the link state routing protocol, the OFP 204 operates to form a local copy of the topology 222 at each computing node (or in some cases, only at the primary computing node). The OFP 204 can flood the changes in the topology 222 detected by the computing node 200 (for example, the interruption of the computing node or link) via the advertisement 224, thereby allowing the topology 222 to remain synchronized at each computing node 200 supporting the execution of the distributed operating system 10. The OFP can display the topology 222 (via the application programming interface API) to the system epoch management (SysEpochMan) process 206.

[0057] The SysEpochMan process 206 may organize the (hereto, disparate and separate) computing nodes to support the execution of a single distributed operating system 10. The SysEpochMan process 206 may also monitor the distributed operating system to ensure integrity in the event of a failure of one or more computing nodes. The SysEpochMan process 206 may provide for a transition from a previous system state to a new system state in the event of a change in the number of computing nodes, a loss of connectivity between nodes, a change in the organization of computing nodes, and / or a change in the rules for computing nodes, to name a few.

[0058] The SysEpochMan process 206 can create (and maintain) Plane (where Refers to Apache Project) and OFP domain (which may refer to the OFP domain, which is used by the distributed operating system 10 to propagate state information that is specific to the distributed operating system 10 and independent of the application). , but the techniques of the present disclosure may be implemented for any inter-process communication bus or mechanism. This is more generally referred to in this disclosure as the inter-process communication bus 208 ("IPCB 208").

[0059] IPCB 208 differs from OFP 204 in that OFP 204 is an asynchronous communication protocol (meaning that OFP can guarantee final object delivery, but not in-order delivery of objects), while IPCB 208 is a synchronous communication protocol (meaning that IPCB 208 can guarantee delivery in the proper order of changes, or in other words, all computing nodes receive changes in the order in which they occur). IPCB 208 can be executed within SysEpochMan process 206 to coordinate services such as leader election (within a computing node) and namespace allocation.

[0060] After forming IPCB 208 (and assuming OFP 204 is operational), the computing node can effectively communicate with other computing nodes of the multi-chassis router to merge and execute the distributed operating system 10. The computing node 200 can then execute a system manager ("SysMan") process 210, which coordinates the start and execution of applications (e.g., applications 230A-230N ("applications 230")) and dynamic applications 232 within the execution environment provided by the distributed operating system. In some examples, SysMan process 210 can use an initialization policy stored on a storage element (e.g., a hard disk) as an instruction with fixed state information to generate dynamic applications 232 at initialization. For example, dynamic applications 232 can be generated as part of a policy engine startup using configuration information provided by SysMan process 210. In other examples, SysMan process 210 can generate dynamic application 232 in response to configuration information pushed to a storage element (e.g., a dedicated hard disk area) of a network device (e.g., via a configuration file) and / or by executing a product via a CLI (e.g., a data flow architecture (DSA) type command). Each product of dynamic application 232 can be based on a corresponding configuration file and / or a global configuration file.

[0061] Each SysMan process 210 of a compute node may elect a SysMan master instance (eg, using ICPB 208) which may be responsible for the execution of applications on a particular compute node based on, for example, a policy engine.

[0062] The SysMan master process may communicate the application decision to the local SysMan process (e.g., via IPCB 208), which then executes the application according to the application decision. The local SysMan process monitors the execution of the application and provides the status of the application to the SysMan master process to allow the SysMan master process to monitor the status of the application. When the status indicates that the application has failed, the SysMan master process may restart execution of the application (via the same or a different compute node), or activate a standby instance of the application, to name a few.

[0063] Computing node 200 may also execute a distributor process 212 as part of distributed operating system 10. Distributor process 212 may form an object daemon data store (DDS), such as DDS 218, and coordinate with individual applications to transfer state information. DDS 218 may coordinate with state 228 of computing node 200 and state information data structures of other computing nodes executing instances of distributed operating system 10. For example, distributor process 212 identifies which objects stored in a state information data structure (e.g., state 228) are out of date and synchronizes with other computing nodes to synchronize objects of the data structure.

[0064] Distributor 212 may operate as a client of OFP 204 and transfer objects between distributors executed by different computing nodes.

[0065] Compute node 200 may be configured to synchronize state 228 with corresponding data structures of other nodes executing other instances of distributed operating system 10. Generally, state 228 may be considered synchronized with data structures of other nodes executing other instances of distributed operating system 10 when the tree data structures of each node executing an instance of distributed operating system 10 have a common arrangement and interconnection of tree nodes within each tree data structure, and when corresponding tree nodes of the tree data structures have the same summary value.

[0066] Computing node 200 may further determine whether state 228 is synchronized with state data structures of other nodes executing other instances of distributed operating system 10. Computing node 200 may recursively traverse tree nodes of the tree data structure representing state 228. If the local digest of the tree node of the tree data structure representing state 228 matches the digest of the corresponding tree node of the tree data structure of the node executing other instances of distributed operating system 10, computing node 200 may determine that the tree nodes are synchronized. Otherwise, computing node 200 may determine that an update of state 228 is necessary.

[0067] If an update is required, the computing node 200 may determine whether its version of the state 228 is up to date, or whether another version of the tree data structure storing state information of a different node executing another instance of the distributed operating system 10 is up to date. If the version of the tree node of the tree data structure representing the state 228 of the computing node 200 is up to date (e.g., has the latest logical clock value), the computing node 200 may send message fragment data of the tree node of the tree data structure representing the state 228 to one or more other nodes executing other instances of the distributed operating system 10. Otherwise, if the version of the tree node of the tree data structure representing the state 228 of the computing node 200 is not up to date, the computing node 200 may receive message fragment data of the tree node of the tree data structure representing the state 228 from different nodes among the other nodes executing another instance of the latest distributed operating system 10, and update the tree node of the tree data structure representing the state 228 with the received message fragment data.

[0068] Further examples of software stacks for distributed operating system 10 are described in above-incorporated U.S. Application Serial No. 15 / 637,775, U.S. Application Serial No. 15 / 637,809, and U.S. Application Serial No. 15 / 637,839.

[0069] According to the techniques described herein, compute node 200 , and in some cases other compute nodes, may configure dynamic application 232 to operate as an agent to receive one or more requests for state information stored in DDS 218 .

[0070] Computing node 200 may include a configuration interface 216 that receives configuration information for computing node 200. Configuration interface 216 may represent a command line interface; a graphical user interface; a simple network management protocol (SNMP), a network configuration protocol (Netconf), or other configuration protocol; or, in some examples, some combination of the above. Configuration interface 216 may receive configuration information, as described below with respect to Figure 4 The configuration information 404 is described in more detail.

[0071] The computing node 200 configures the dynamic application 232 as an application of the user space 214, for example, via the SysMan process 210. The user space 214 may represent the execution environment of the distributed operating system 10. The user space 214 may also include other applications, for example, applications 230A-230N, which may represent user-level processes that may run network management software, execute routing protocols to communicate with peer routing devices, execute configuration commands received from an administrator, maintain and update one or more routing tables, manage subscriber flow processing, create one or more forwarding tables, and other functions. The application 230 may refer to any process other than low-level drivers and / or firmware that is not described above with respect to the distributed operating system 10, including Daemon and PFE application (or in other words, software).

[0072] Based on the configuration information received by the configuration interface 216, the computing node 200 can configure the dynamic application 232 to operate as a proxy to receive one or more requests (e.g., HTTP requests) from a service endpoint and configure responses, such as Hypertext Markup Language (HTTP) documents, using state information stored in its local data storage or memory. To configure the dynamic application 232, the computing node 200 can receive configuration information defining one or more instances of the dynamic application 232 that the computing node 200 can implement and state information retrieved from the DDS 218 to be stored in the local data storage or memory of the computing device 200. The configuration information can further define instructions and the format of responses to the one or more requests, which instructions cause the computing node 200 to expose an interface (e.g., a port) to receive one or more requests from a service endpoint (e.g., a network client).

[0073] Figure 4 is a block diagram illustrating an example operation of a dynamic application according to the disclosed technology. Figure 4 In an example of , multiple computing nodes execute a distributed operating system that provides an execution environment for executing multiple applications. At least one computing node of the multiple computing nodes configures dynamic application 406 to operate as an agent to receive one or more requests for state information 410A-410N (collectively referred to as "state information 410") stored in DDS 408. The computing node executing the instance of dynamic application 406 can configure dynamic application 406 based on configuration information 404.

[0074] Configuration information 404 may define one or more instances in which computing nodes executing distributed operating system 10 may implement dynamic application 406 to serve different objects of state information 410. For example, a configuration file may define a first computing node (e.g., Figure 3 A computing node 200 of the embodiment of the present invention is defined to execute a first instance of a dynamic application 406 to provide a view of a first subset of state information, for example, state information 410A (for example, routing state information), and a second computing node is defined to execute a second instance of the dynamic application 406 to provide a view of a second subset of state information, for example, state information 410B (for example, interface state information).

[0075] The following is an example of configuration information 404 for a computing node that executes an instance of a dynamic application 406:

[0076]

[0077] A computing device executing an instance of the dynamic application 406 may use the configuration information 404 to configure the computing device to receive one or more requests from the client 402. In the example configuration information shown above, the computing device may expose an interface of the computing device (e.g., "port": "8888") to receive one or more requests from a service endpoint (e.g., the client 402). The configuration information may also define authentication (or no authentication) for accessing the dynamic application 22. In this manner, the dynamic application 406 may operate as a web server to receive one or more web-based requests (e.g., HTTP get requests) from a web client and retrieve state information 410 from the DDS 408.

[0078] The computing device executing the instance of the dynamic application 406 can use the configuration information 404 to define the format of the response. In the example configuration information shown above, the computing device is configured to generate a response to a request for an interface speed (e.g., "InterfaceSpeed") to include the interface name and the interface speed (e.g., (e.g., "{{interface}, {speed}}"), and / or a response to a request for a list of interfaces (e.g., "InterfaceList") to include the set of interfaces (e.g., "[interface").

[0079] Configuration information 404 may also instruct a computing node executing a corresponding instance of dynamic application 406 to construct a local data store 412 in which state information, such as state information 410, is stored. For example, configuration information may instruct dynamic application 406 to retrieve state information from DDS 408 during initialization and each time the state information in DDS 408 is updated. Local data store 412 may map objects of state information 410 retrieved from DDS 408 to responses to requests from client 402.

[0080] Assume, for example, that configuration information 404 defines a dynamic application 406 to serve state information 410A (e.g., interface state information) of a DDS 408. The dynamic application 406 may retrieve the state information 410A during initialization or updating of the DDS 408 (retrieval process 414) and store the retrieved state information 410A in a data store 412. The dynamic application 406 may then receive one or more requests 416A, 416B ("requests 416") from the client 402 for one or more objects of the state information 410A. For example, the client 402 may use a web client to configure and send a request 416A, e.g., an HTTP get request, to retrieve the state information 410A (e.g., interface speed) from the DDS 408. As an example, the client 402 may generate and send a request (e.g., an HTML request) 416A for the interface state information (e.g., interface speed), for example, as follows:

[0081] http: / / 192.168.1.70:8888 / DynamicState / InterfaceSpeed?InterfaceName=et-0 / 0 / 0.

[0082] Dynamic application 406 may receive request 416A and retrieve state information 410A from data store 412. Dynamic application 406 may use the state information in data store 412 to configure response 418A.

[0083] Continuing with the example above, the compute node may generate a response 418A to the request 416A for the interface speed as follows:

[0084]

[0085] As another example, client 402 may generate and send request 416B for a list of interfaces (e.g., interface speeds), e.g., as follows:

[0086] http: / / 192.168.1.70:8888 / DynamicState / InterfaceList. In some examples, client 402 may send requests 416A and 416B simultaneously or nearly simultaneously.

[0087] Dynamic application 406 may receive request 416B and retrieve state information 410A from data store 412. Dynamic application 406 may use the state information in data store 412 to configure response 418B.

[0088] Continuing with the example above, the compute node may generate a response 418B to the request 416B for the list of interfaces as follows:

[0089]

[0090] In some examples, the dynamic application 406 can configure the response 418A, 418B ("response 418") as an HTML file with one or more HTML tags. The HTML tag can indicate to the network client browser that the response is an HTML file and can also include other attributes associated with the HTML file.

[0091] Figure 5 It is shown Figure 3 Flowchart of example operations of a multi-chassis router in performing corresponding aspects of the techniques described in the present disclosure. In this example, multiple computing nodes included in a network device can execute a distributed operating system that provides an execution environment for executing multiple application programs (502). For example, multiple routing engines (e.g., Figure 2 The routing engine 130 of the distributed operating system can execute an instance of the distributed operating system. The distributed operating system provides an execution environment for executing user space applications. Each application can generate state information stored in the distributed data storage, for example, Figure 1 DDS 24.

[0092] At least one of the plurality of computing nodes configures the dynamic application to operate as an agent to receive one or more requests for state information (504). For example, the computing node configuring the dynamic application may receive a configuration file defining one or more instances of the dynamic application that the computing node may implement. For example, the configuration file may define a DDS field from which the dynamic application retrieves data. The configuration file may also instruct the computing node to expose an interface (e.g., a port) to receive one or more requests from a service endpoint (e.g., a network client). In this manner, the computing node executing the instance of the dynamic application may receive a network request, such as a hypertext transfer protocol (HTTP) get request. That is, the dynamic application may operate as a network server to receive one or more network-based requests from a network client. The configuration information may instruct the computing node executing the corresponding instance of the dynamic application to retrieve state information from the DDS 24 and store the retrieved state information in a local data storage or memory of the computing device.

[0093] The dynamic application may retrieve state information from the data store (506) and store the retrieved state information in a local data store (508). For example, the dynamic application may retrieve state information from the DDS 24 during initialization and store the retrieved state information in a local data store or memory. The dynamic application may also retrieve state information from the DDS 24 when updating state information.

[0094] The dynamic application may receive one or more requests for state information (510). For example, a computing node executing an instance of the dynamic application may receive one or more requests for state information in the DDS, such as an HTTP get request. The computing node may pass the request to the dynamic application, and the dynamic application may retrieve state information corresponding to the one or more requests from a local data store.

[0095] The dynamic application can configure and send a response with the state information retrieved from the local data storage (512). For example, the dynamic application can configure the response as an HTML document, including one or more HTML tags.

[0096] The technology of the present disclosure can be implemented in various devices or equipment, including network devices, integrated circuits (ICs), or a group of ICs (i.e., chipsets). Any components, modules, or units have been described to emphasize functional aspects and do not necessarily need to be implemented by different hardware units. The technology described herein can also be implemented in hardware or any combination of hardware and software and / or firmware. Any features described as modules, units, or components can be implemented together in an integrated logic device, or implemented separately as discrete but interoperable logic devices. In some cases, various features can be implemented as integrated circuit devices, for example, integrated circuit chips or chipsets.

[0097] If implemented in software, the techniques may be implemented at least in part by a computer-readable storage medium comprising instructions that, when executed in a processor, perform one or more of the above methods. The computer-readable storage medium may be a physical structure and may form part of a computer program product, which may include packaging materials. In this sense, the computer-readable medium may be non-transitory. The computer-readable storage medium may include a random access memory (RAM), such as a synchronous dynamic random access memory (SDRAM), a read-only memory (ROM), a non-volatile random access memory (NVRAM), an electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic or optical data storage media, and the like.

[0098] The code or instructions may be executed by one or more processors, for example, one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functionality described herein may be provided in a dedicated software module or hardware module configured for encoding and decoding, or included in a combined video codec. In addition, these techniques may be fully implemented in one or more circuits or logic elements.

Claims

1. A network method comprising: executing a distributed operating system by a plurality of hardware computing nodes included in the network device, the distributed operating system providing an execution environment for executing a plurality of application programs that provide network functions; configuring, by at least one of the plurality of hardware computing nodes, a dynamic application to operate as a proxy application to retrieve state information stored in a distributed data store that synchronizes the state information of the plurality of hardware computing nodes, and to receive one or more requests for state information from a client service endpoint, wherein the state information includes information representing different states of the plurality of applications distributed among the plurality of hardware computing nodes, wherein configuring the dynamic application comprises configuring, with an interface for receiving the one or more requests for the state information, the at least one hardware computing node of the plurality of hardware computing nodes configured with the dynamic application; retrieving, by the dynamic application, the state information stored in the distributed data storage; storing, by the dynamic application, the retrieved state information in a local data storage of at least one hardware computing node of the plurality of hardware computing nodes configured with the dynamic application; After storing the retrieved state information in the local data store, receiving, by the dynamic application via the interface, the one or more requests for the state information from the client service endpoint; and In response to receiving the one or more requests, a response including the requested state information from the local data store is sent by the dynamic application and to the client service endpoint without sending the one or more requests to the distributed data store.

2. The method according to claim 1, wherein: Configuring the dynamic application includes configuring the dynamic application to operate as a web server.

3. The method according to claim 1, wherein: Configuring the dynamic application includes exposing an interface of at least one of the plurality of hardware computing nodes to receive the one or more requests for the state information from a network client.

4. The method according to any one of claims 1 to 3, wherein: The one or more requests include a hypertext transfer protocol get request.

5. The method according to any one of claims 1 to 3, wherein: Sending the response includes sending a response including one or more hypertext markup language tags identifying the requested state information.

6. The method according to claim 1, wherein: Configuring the dynamic application includes configuring the dynamic application based on configuration information defining an instance of the dynamic application and constructing the local data store to store the state information of the distributed data store.

7. The method according to claim 6, wherein: Sending the response includes sending the response without performing a query to the distributed data store.

8. A network device, comprising: A plurality of hardware computing nodes are configured to execute a distributed operating system, wherein the distributed operating system provides an execution environment for executing a plurality of application programs that provide network functions, wherein at least one of the plurality of hardware computing nodes is configured to: configuring a dynamic application to operate as a proxy to retrieve state information stored in a distributed data store and receive one or more requests for state information stored in the distributed data store from a client service endpoint, the distributed data store synchronizing state information of the plurality of hardware computing nodes, wherein the state information includes information representing different states of the plurality of applications distributed among the plurality of hardware computing nodes, wherein configuring the dynamic application comprises configuring, with an interface for receiving the one or more requests for the state information, the at least one hardware computing node of the plurality of hardware computing nodes configured with the dynamic application; retrieving the state information stored in the distributed data storage; storing the retrieved state information in a local data storage of at least one hardware computing node of the plurality of hardware computing nodes configured with the dynamic application; After storing the retrieved state information in the local data store, receiving the one or more requests for the state information from the client service endpoint via the interface; and In response to receiving the one or more requests, sending a response including the requested state information from the local data store to the client service endpoint without sending the one or more requests to the distributed data store.

9. The network device according to claim 8, wherein: To configure the dynamic application, the at least one hardware computing node of the plurality of hardware computing nodes is further configured to configure the dynamic application to operate as a network server.

10. The network device according to claim 8, wherein: To configure the dynamic application, the at least one of the plurality of hardware computing nodes is further configured to expose an interface of the at least one of the plurality of hardware computing nodes to receive the one or more requests for state information from a network client.

11. The network device according to any one of claims 8 to 10, wherein: The one or more requests include a hypertext transfer protocol get request.

12. The network device according to any one of claims 8 to 10, wherein: To send the response, the at least one of the plurality of hardware computing nodes is further configured to send a response including one or more hypertext markup language tags identifying the requested state information.

13. The network device according to claim 8, wherein: In order to configure the dynamic application, at least one of the multiple hardware computing nodes is also configured to configure the dynamic application based on configuration information defining an instance of the dynamic application and constructing the local data storage to store the state information of the distributed data storage.

14. The network device according to claim 13, wherein: To send the response, the at least one hardware computing node of the plurality of hardware computing nodes is further configured to send the response without performing a query to the distributed data storage.

15. A non-transitory computer-readable storage medium encoded with instructions for causing one or more programmable processors to perform the method according to any one of claims 1 to 7.

Images