Edge devices using source identifiers for source identification

Abstract

The present application relates to an edge device for source identification using a source identifier. A device includes a processing circuit configured to identify a telemetry packet, the telemetry packet indicating telemetry data for a plurality of packets output by a network device in a plurality of network devices, and select a source identifier for the network device from a plurality of source identifiers. The processing circuit is further configured to modify the telemetry packet to further indicate the selected source identifier and output the modified telemetry packet.

Description

[0001] This application claims the benefit of U.S. Application No. 17 / 449,067, filed on September 27, 2021, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates to computer networks and, more particularly, to managing network devices. Background Art

[0003] Network devices typically include mechanisms for configuring the device locally or remotely, such as a management interface. By interacting with the management interface, a client can perform configuration tasks and execute operational commands to collect and view operational data of the managed device. For example, a client can configure the device's interface cards, adjust parameters for supported network protocols, specify physical components within the device, modify routing information maintained by a router, access software modules and other resources resident on the device, and perform other configuration tasks. In addition, a client can allow a user to view current operating parameters, system logs, information related to network connections, network activity or other status information from the device, and view and respond to event information received from the device.

[0004] Network configuration services can be performed by multiple different devices, such as routers with service cards and / or dedicated service devices. These services include connectivity services such as Layer 3 Virtual Private Network (L3VPN), Virtual Private LAN Service (VPLS), and Point-to-Point (P2P) services. Other services include network configuration services such as Dot1q VLAN services. Network management systems (NMS) and NMS devices, also known as controllers or controller devices, can support these services, making it easy for administrators to create and manage these advanced network configuration services.

[0005] Specifically, user configurations of devices can be referred to as "intents". Intent-based networking systems allow administrators to describe the expected network / compute / storage state. User intents can be classified as stateful intents (e.g., business policies) or stateless intents. Stateful intents can be resolved based on the current state of the network. Stateless intents can be a completely declarative way to describe the expected network / compute / storage state without considering the current network state.

[0006] Intents can be represented as an intent data model, which can be modeled using a unified graph model. The intent data model can be represented as a connection graph so that stateful intents can be implemented across a business computing architecture. For example, the data model can be represented using a data structure, such as a connection graph with vertices connected with edges and reference (ref) edges. The controller device can model the intent data model as a unified graph model. In this way, stateful intents can be implemented across intent data models. When intents are modeled using a unified graph model, extending new intent support may extend the graph model and compilation logic.

[0007] To configure a device to execute an intent, a user (e.g., a network administrator) can write a translation program that translates high-level configuration instructions (e.g., instructions according to an intent data model, which can be represented as a unified graphical model) into low-level configuration instructions (e.g., instructions according to a device configuration model). As part of the configuration service support, the user / administrator can provide an intent data model and a mapping between the intent data model and the device configuration model.

[0008] The sensor device may collect Internet Protocol (IP) network traffic as it enters and / or leaves an interface, and may output telemetry packets based on the IP network traffic. By analyzing the data provided by the telemetry packets, a user (such as a network administrator) may determine telemetry data such as the source and destination of the network traffic, the class of service of the network traffic, and the cause of network traffic congestion. A user and / or a controller device may use intents to modify the configuration of a network device using telemetry data. Summary of the invention

[0009] In general, the present disclosure describes a technique for identifying the source of telemetry data generated by a telemetry sensor. Some collector devices (e.g., devices configured to aggregate telemetry data for grouping) may use a source Internet Protocol (IP) address to identify a source device (e.g., a network device). However, such a collector may incorrectly identify a source device when, for example, a network address translator (NAT) translates the source address from a source IP address of the source device to another source IP address. In addition, some cloud-based collectors may include a NAT to translate the source address from a source IP address of the source device to a source address of a node of the cloud-based collector, which may confuse the source device.

[0010] Rather than relying solely on a source IP address to identify a source device, as described herein, a system may include an "edge device" that is arranged to identify a source device by adding a device identifier (e.g., a device ID). In some examples, the device identifier may be a unique identifier of a source device that is pre-configured by a network administrator. For example, an edge device may append the device identifier to each telemetry packet. In this example, a collector device may use the device identifier to identify the source device, which may improve the accuracy of identifying the source device compared to a system that uses only a source IP address. Improving the accuracy of identifying source devices may help improve the accuracy of telemetry information that may be used by a network administrator and / or controller to manage network devices. For example, a network administrator and / or controller may identify the largest resource consuming bandwidth in a network.

[0011] In one example, a method includes identifying, by an edge device implemented in a circuit, a telemetry packet indicating telemetry data for a plurality of packets (e.g., a packet flow or a packet stream) output by a network device of a plurality of network devices, and selecting, by the edge device, a source identifier for the network device from a plurality of source identifiers. The method also includes modifying, by the edge device, the telemetry packet to further indicate the selected source identifier and outputting, by the edge device, the modified telemetry packet.

[0012] In another example, a device includes processing circuitry configured to identify a telemetry packet indicating telemetry data for a plurality of packets output by a network device in a plurality of network devices, and select a source identifier for the network device from source identifiers of the plurality of network devices. The processing circuitry is further configured to modify the telemetry packet to further indicate the selected source identifier and output the modified telemetry packet.

[0013] In one example, a non-transitory computer-readable storage medium includes one or more instructions that cause a processing circuit to identify a telemetry packet indicating telemetry data for a plurality of packets output by a network device in a plurality of network devices and select a source identifier for the network device from a plurality of source identifiers. The one or more instructions further cause the processing circuit to modify the telemetry packet to further indicate the selected source identifier and output the modified telemetry packet.

[0014] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a block diagram illustrating an example of network elements including an enterprise network managed using a controller device in accordance with techniques of this disclosure.

[0016] Figure 2 It is used for Figure 1 A block diagram of an example component set of a controller device.

[0017] Figure 3 is a block diagram illustrating an example of a network device, an edge device, one or more network address translators (NATs), and a collector device for processing telemetry packets in accordance with the techniques of this disclosure.

[0018] Figure 4A is a conceptual diagram illustrating an example header of a telemetry packet generated by an example telemetry sensor in accordance with techniques of this disclosure.

[0019] Figure 4B This is to illustrate the technology according to the present disclosure Figure 4A A conceptual diagram of example details of a header.

[0020] Figure 5 is a block diagram illustrating an example of a network device, an edge device, one or more NATs, and a collector device for rate control of the network device in accordance with the techniques of this disclosure.

[0021] Figure 6 is a block diagram illustrating an example of a network device, an edge device, one or more NATs, and a collector device for processing telemetry packets from different tenants in accordance with the techniques of this disclosure.

[0022] Figure 7 is a flow diagram illustrating an example process for resources in accordance with the techniques of this disclosure.

[0023] Like reference numerals refer to like elements throughout the drawings and description. DETAILED DESCRIPTION

[0024] In general, the present disclosure describes techniques for identifying telemetry data sources for telemetry sensors. Some collector devices (e.g., devices configured to aggregate telemetry data for packets) may use a source Internet Protocol (IP) address to identify a source device (e.g., a network device or a telemetry sensor for a network device). However, a collector device that uses only a source IP address to identify a source device may have a "gap" in reliably identifying the source of telemetry data for a telemetry sensor. For example, a telemetry packet may not contain a host name or management IP in the payload. For example, the source IP for the telemetry packet may not be preserved across a network address translator (NAT). Additionally, or alternatively, a collector device that uses only a source IP address to identify a source device may be error-prone because the User Datagram Protocol (UDP) is unreliable, has no flow control, and is insecure compared to the Transmission Control Protocol (TCP). In addition, when the source IP address of a first tenant may conflict with that of a second tenant, a collector device that uses only a source IP address to identify a source device may not be suitable for a multi-tenant solution.

[0025] The techniques described herein may help address the problem of unreliable identification of telemetry sources at a collector device. For example, unreliable identification of sources may occur when using protocols such as NetFlow and other protocols, for example, where telemetry collection may be initiated from a device side (e.g., a telemetry sensor or network device) and the telemetry data (e.g., UDP messages) may flow to a collector device. A collector device may identify the source of the telemetry data (e.g., a telemetry sensor or network device), which may not be possible for some protocols like NetFlow due to, for example, NAT of the source IP address.

[0026] In some examples, there may be disadvantages to configuring a collector device to use only the source IP address of a UDP message to identify a source device. For example, UDP transport may be less reliable (from a reliability, packet loss, flow / rate control perspective) than the Transmission Control Protocol (TCP). In some examples, UDP is not secure. In some examples, a protocol like NetFlow may not contain anything to uniquely identify a source device on the collector side. There may be one or more deployment difficulties on the collector side where workarounds must be implemented for NetFlow to identify packets at the collector, which may prevent NAT of the source IP address using mechanisms like daemon sets, node ports, which may not scale.

[0027] Rather than relying solely on the source IP address to identify the source device or relying on the limitations of NAT that blocks the source IP address, a system may include an "edge device" that is arranged to modify the telemetry packet (e.g., device ID) by adding a source identifier. In some examples, the source identifier may be a unique identifier for a source device that is preconfigured by a network administrator. For example, the edge device may append the source identifier to each telemetry packet. In this example, the collector may use the source identifier to identify the source device, which may improve the accuracy of identifying the source device compared to a system that uses only the source IP address. Without the source identifier, there may be two devices (e.g., two different telemetry sensors for different network devices) using the same source address (private address), and the two devices may belong to two different customers. In addition, if the collector device is configured to use the source IP address of the packet, there may be a time when the source IP address may be changed by NAT on the way to the collector device and / or at the collector device before processing, thereby potentially losing the source identity.

[0028] The techniques described herein for modifying telemetry packets with source identifiers can be implemented without modifying the protocol itself or device-side behavior. For example, an edge device can provide an overlay mechanism to modify telemetry packets by adding source identifiers (e.g., device IDs and custom fields). The edge device can provide an overlay mechanism to use grpc remote procedure calls (gRPC) or HTTP 2.0 (both based on TCP), which may be a relatively reliable and secure transport compared to UDP. For example, the edge device can "fill" an entire UDP packet or multiplex multiple packets in the payload, and extend the payload by adding custom fields like source identifiers (e.g., device IDs) and each UDP packet. In this example, the collector device can receive a protocol data unit (PDU) for telemetry data from the edge device and can depacketize the PDU to reliably identify the telemetry sensor set for the telemetry data. In this way, the techniques described herein can configure the edge device and / or collector device to use telemetry packets with source identifiers so that the system can "bear" network address translation (NAT) operations.

[0029] The techniques described herein using source identifiers can allow network devices to perform flow control to allow edge devices to reduce or dial back (or optionally, increase) the sampling rate on the network device and ensure statistical sampling of telemetry packets rather than random discarding. In addition, the techniques described herein include modifying telemetry packets with source identifiers and tenant identifiers, which can support multi-tenancy. In some examples, the device ID can also be used to support multi-tenancy by configuring a tenant identifier (e.g., orgID) as part of a source identifier (e.g., device ID) so that a collector device can determine which tenant the source device belongs to. In this way, the collector device can be configured to explicitly determine which tenant sent a message indicating telemetry data. Techniques for configuring edge devices to modify telemetry packets using source identifiers can support multiple vendors (e.g., network devices configured with different operating software or network devices supporting different protocols).

[0030] While the above examples include NetFlow, other flow protocols may also be used, such as, for example, sampled flow (sflow), Juniper flow (Jflow), or other flow protocols. Additionally, while the techniques described herein may describe edge devices as being separate from network devices, some examples may implement techniques for decorating telemetry packets with a source identifier at a source device (locally or as an extension) and / or as part of an edge device.

[0031] Figure 11 is a block diagram illustrating an example of a network device including an enterprise network 2 managed using a controller device 10. Network elements 14A-14G (collectively referred to as "network elements 14" or "network devices 14") of the enterprise network 2 include network elements that are interconnected via communication links to form a communication topology for exchanging resources and information. Network elements 14 (also commonly referred to as network devices) may include, for example, routers, switches, gateways, bridges, hubs, servers, firewalls or other intrusion detection systems (IDS) or intrusion prevention systems (IDP), computing devices, computing terminals, printers, other network devices, or combinations of these devices. Network elements 14 may include one or more sensor devices configured to generate telemetry packets indicating telemetry data for multiple packets output by the corresponding network element. In some examples, each network element 14 may be associated with a corresponding set of sensor devices that are separate from the network element 14 (e.g., a separate circuit board or a separate processing device). Although described in the present disclosure as transmitting, conveying, or otherwise supporting packets, the enterprise network 2 may transmit data according to any other discrete data units defined by any other protocol, such as units defined by the Asynchronous Transfer Mode (ATM) protocol, or datagrams defined by the User Datagram Protocol (UDP). The communication links interconnecting the network elements 14 may be physical links (e.g., optical, copper wire, etc.), wireless, or any combination thereof.

[0032] The enterprise network 2 is shown coupled to a public network 18 (e.g., the Internet) via a communication link. The public network 18 may include, for example, one or more client computing devices. The public network 18 may provide access to network servers, application servers, public databases, media servers, end-user devices, and other types of network resource devices and content. As shown, the public network 18 may provide access to a collector device 20. The collector device 20 may represent one or more computing devices (e.g., servers, computers, or clouds).

[0033] The controller device 10 is communicatively coupled to the network element 14 via the enterprise network 2. In some examples, the controller device 10 forms part of a device management system, although in Figure 1Only one device of the device management system is shown for example purposes. The controller device 10 may be an example of a central controller. The controller device 10 may be coupled directly or indirectly to various network elements 14. Once the network elements 14 are deployed and activated, the administrator 12 may use the controller device 10 (or multiple such management devices) to manage the network devices using a device management protocol. One example device protocol is the Simple Network Management Protocol (SNMP), which allows the controller device 10 to traverse and modify a management information base (MIB) that stores configuration data within each managed network element 14. Another example device protocol may include NETCONF, however, the techniques described herein may be used with other device protocols. More detailed information about the SNMP protocol can be found in the Internet Engineering Task Force draft of the Network Working Group of Harrington et al., December 2002, RFC3411, "An Architecture for Describing Simple Network Management Protocol (SNMP) Management Frameworks" (available from http: / / tools.ietf.org / html / rfc3411), the entire contents of which are incorporated herein by reference. Another example device protocol may include NETCONF, however, the techniques described herein may be used with other device protocols. More detailed information about the NETCONF protocol can be found in the Internet Engineering Task Force draft of the Network Working Group of Enns et al., June 2011, RFC6241, "Network Configuration Protocol (NETCONF)" (available from https: / / tools.ietf.org / html / rfc6241), the entire contents of which are incorporated herein by reference.

[0034] In common practice, the controller device 10, also referred to as a network management system (NMS) or NMS device, and the network elements 14 are centrally maintained by the IT group of the enterprise. The administrator 12 interacts with the controller device 10 to remotely monitor and configure the network elements 14. For example, the administrator 12 can receive an alarm about any of the network elements 14 from the controller device 10, view the configuration data of the network elements 14, modify the configuration data of the network elements 14, add new network devices to the enterprise network 2, remove existing network devices from the enterprise network 2, or otherwise manipulate the enterprise network 2 and the network devices therein. Although described with respect to an enterprise network, the technology of the present disclosure is applicable to other network types, public and private networks, including LANs, VLANs, VPNs, etc.

[0035] In some examples, the administrator 12 uses the controller device 10 or a local workstation to directly interact with the network element 14, such as through telnet, secure shell (SSH), or other such communication sessions. That is, the network element 14 generally provides an interface for direct interaction, such as a command line interface (CLI), a network-based interface, a graphical user interface (GUI), etc., through which a user can interact with the device to directly issue text-based commands. For example, these interfaces generally allow a user to directly interact with the device, such as through telnet, secure shell (SSH), hypertext transfer protocol (HTTP), or other network sessions, to enter text according to a defined syntax to submit commands to the managed element. In some examples, the user uses the controller device 10 to initiate an SSH session 15 with one of the network elements 14, such as network element 14F, to directly configure the network element 14F. In this way, the user can directly provide commands in an execution format to the network element 14.

[0036] In addition, the administrator 12 may also create scripts that may be submitted by the controller device 10 to any or all of the network elements 14. For example, in addition to the CLI interface, the network element 14 may also provide an interface for receiving scripts that specify commands according to a scripting language. In a sense, the script may be output by the controller device 10 to automatically invoke a corresponding remote procedure call (RPC) on the managed network element 14. The script may conform to, for example, the Extensible Markup Language (XML) or another data description language.

[0037] The administrator 12 uses the controller device 10 to configure the network element 14 to specify certain operational characteristics that further achieve the goals of the administrator 12. For example, the administrator 12 can specify specific operational policies for the network element 14A regarding security, device accessibility, traffic engineering, quality of service (QoS), network address translation (NAT), packet filtering, packet forwarding, rate limiting, or other policies. The controller device 10 uses one or more network management protocols designed to manage configuration data within the network element 14, such as the SNMP protocol or the Network Configuration Protocol (NETCONF) protocol or its derivatives (such as the Juniper Network Device Management Interface) to perform configuration. In general, NETCONF provides a mechanism for configuring network devices and uses data encoding based on the Extensible Markup Language (XML) to configure data that may include policy data. NETCONF is described in RFC4741, "NETCONF Configuration Protocol" by the Network Working Group of Enns, December 2006 (available from tools.ietf.org / html / rfc4741). The controller device 10 can establish a NETCONF session with one or more network elements 14.

[0038] The controller device 10 may be configured to compare the new intent data model with the existing (or old) intent data model, determine the differences between the new intent data model and the existing intent data model, and apply a reaction mapper to the differences between the new intent data model and the old intent data model. Specifically, the controller device 10 determines whether the new configuration data set includes any additional configuration parameters relative to the old intent data model, and whether the new configuration data set modifies or omits any configuration parameters included in the old intent data model.

[0039] The intent data model can be a unified graphical model, while the low-level configuration data can be expressed in YANG, which is described in RFC 6020, "YANG—A Data Modeling Language for the Network Configuration Protocol (NETCONF)" by Bjorklund's Internet Engineering Task Force in October 2010 (available from tools.ietf.org / html / rfc6020). In some examples, the intent data model can be expressed in YAML Ain't Markup Language (YAML). The controller device 10 may include various reactive mappers for converting intent data model differences. These functions are configured to accept intent data models (which can be represented as structured input parameters, for example, according to YANG or YAML). The functions are also configured to output corresponding multiple groups of low-level device configuration data changes, such as device configuration additions and deletions. That is, y1=f1(x), y2=f2(x),...y N =f N (x).

[0040] The controller device 10 may use YANG modeling for intent data models and low-level device configuration models. The data may contain relationships across YANG entities, such as list items and containers. Traditionally, controller devices do not support real-time configuration management capabilities. As discussed in more detail below, the controller device 10 may convert a YANG data model into a database model and convert YANG validation into data validation. A technique for managing network devices using a graphical model for advanced configuration data is described in U.S. Patent Application No. 15 / 462,465, filed on March 17, 2017, “CONFIGURING AND MANAGING NETWORK DEVICES USING PROGRAM OVERLAY ON YANG-BASEDGRAPH DATABASE,” the entire contents of which are incorporated herein by reference.

[0041] The controller device 10 may receive data representing any or all create, update and / or delete actions with respect to the unified intent data model from one of the administrators 12. The controller device 10 may be configured to use the same compilation logic for each of create, update and delete as applied to the graphical model.

[0042] Typically, a controller like the controller device 10 uses a hierarchical data model for intent, a low-level data model, and resources. The hierarchical data model can be based on YANG or YAML. As described above, the hierarchical data model can be represented as a graph. Modern systems already support intent to simplify network management. Intent is declarative. To implement the intent, the controller device 10 tries to select the best resources.

[0043] According to the techniques of the present disclosure, network element 14A (e.g., a sensor device of network element 14A) may generate telemetry packets indicating a plurality of packets of telemetry data output by network element 14A of network element 14. The telemetry data may include any information that may be collected from network element 14 using a telemetry sensor (e.g., such as OpenConfig, Juniper Telemetry Interface (JTI) local, Netconf, SNMP, syslog, sFlow, NetFlow, or other telemetry sensors, for example). Example metrics in the telemetry data may include, but are not limited to, for example, interface status, routing status (including routing protocol information such as BGP, ISIS, MPLS, etc.), system status (CPU utilization, memory utilization, etc.), system alarm status (temperature alarm, critical alarm), security-related status, hardware chassis-related status (fans, power supplies, temperature), network service / flow details, or other telemetry data. For example, network element 14 may send the telemetry packets to edge device 16.

[0044] One or more edge devices 16 (referred to herein simply as “edge devices 16”) may identify the telemetry packets and select a source identifier for network element 14A from a plurality of source identifiers. In some examples, each of edge devices 16 may be associated with a respective tenant (see Figure 6 ).

[0045] For example, edge device 16 may be configured to store preconfigured and unique source identifiers for each of network elements 14 and associate each source identifier with a network element of network element 14. For example, network element 14A may be assigned a first source identifier and network element 14B may be assigned a second source identifier. For example, administrator 12 may assign a first source identifier to network element 14A and may assign a second source identifier to network element 14B.

[0046] In this instance, edge device 16 may be configured to select a first source identifier for telemetry packets output by network element 14A and a second source identifier for telemetry packets output by network element 14B. For example, in response to determining that the telemetry packet includes a source IP address that corresponds to (e.g., matches) network element 14A and that network element 14A is assigned the first source identifier, edge device 16 may select the first source identifier for the telemetry packet. However, in response to determining that the telemetry packet includes a source IP address that corresponds to (e.g., matches) network element 14B and that network element 14A is assigned the second source identifier, edge device 16 may select the second source identifier for the telemetry packet.

[0047] The edge device 16 may modify the telemetry packet to further indicate the selected source identifier. The network element 14 may output the modified telemetry packet to the collector device 20. The collector device 20 may associate the telemetry packet with the network element 14A based on the selected source identifier and may process the telemetry packet to generate telemetry information. The collector device 20 may output an indication of the telemetry information to the controller device 10. For example, the controller device 10 may output the telemetry information on a display. The telemetry information may help the administrator 12 understand the behavior of the network element 14, which may be further used to take action or feed to other controller components (e.g., the controller device 10). Examples of telemetry information may include visualizing the top N talkers in the network 2, the interface status of the network 2, the health of the server, routing information, the hardware status of one of the network elements 14 if a distributed denial of service (DDoS) attack occurs on the network 2, or other telemetry information.

[0048] Figure 2 It is used for Figure 1 1 is a block diagram of an example component set of a controller device 10. In this example, the controller device 10 includes a control unit 22, a network interface 34, and a user interface 36. The network interface 34 represents a means by which the controller device 10 can be communicatively coupled to an external device (e.g., Figure 1 The network interface 34 may represent a wireless and / or wired interface, such as an Ethernet interface or a wireless radio, configured to communicate according to a radio standard such as one or more IEEE 802.11 radio network protocols (e.g., 802.11a / b / g / n or other such radio protocols). In various examples, the controller device 10 may include multiple network interfaces, although only one network interface is shown for purposes of example.

[0049] The control unit 22 represents any combination of hardware, software and / or firmware for implementing the functions of the control unit 22 and its constituent modules and elements belonging to this article. When the control unit 22 includes software or firmware, the control unit 22 also includes any necessary hardware for storing and executing the software or firmware, such as one or more processors or processing units. Typically, the processing unit may include one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or any other equivalent integrated or discrete logic circuits, as well as any combination of these components. In addition, the processing unit is typically implemented using fixed and / or programmable logic circuits.

[0050] The user interface 36 represents one or more interfaces, such as the administrator 12 ( Figure 1 ) through which a user interacts with the controller device 10, e.g., to provide input and receive output. For example, the user interface 36 may represent one or more of a monitor, a keyboard, a mouse, a touch screen, a touch pad, a trackpad, a speaker, a camera, a microphone, etc. Furthermore, while the controller device 10 includes a user interface in this example, the administrator 12 need not interact directly with the controller device 10, but may instead remotely access the controller device 10, e.g., via the network interface 34.

[0051] The control unit 22 includes a user interface module 38, a network interface module 32, and a management module 24. The control unit 22 executes the user interface module 38 to receive input from the user interface 36 and / or provide output to the user interface 36. The control unit 22 also executes the network interface module 32 to send and receive data (e.g., packets) through the network interface 34. The user interface module 38, the network interface module 32, and the management module 24 may again be implemented as corresponding hardware units, or software or firmware, or a combination thereof.

[0052] The functions of the control unit 22 may be implemented as one or more processing units in fixed or programmable digital logic circuits. Such digital logic circuits may include one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or any other equivalent integrated or discrete logic circuits, as well as any combination of these components. When implemented as a programmable logic circuit, the control unit 22 may also include one or more computer-readable storage media that store hardware or firmware instructions to be executed by the processing units of the control unit 22.

[0053] The control unit 22 executes the management module 24 to manage various network devices, such as Figure 1 The management of network devices includes, for example, receiving requests from users (e.g., Figure 1The administrator 12) receives instructions to configure the network device and provides the user with the ability to submit instructions to configure the network device. The management of the network device also includes upgrading the network device with updated software (such as an updated software image).

[0054] The management module 24 is configured to receive intent unified graph modeling configuration data for a managed set of network devices from a user (e.g., an administrator 12). This intent unified graph modeling configuration data can be referred to as an "intent data model". Over time, the user can update the configuration data, for example to add new services, remove existing services, or modify existing services performed by managed devices. The unified intent data model can be constructed according to, for example, YANG or YAML. The graph model may include multiple vertices connected by edges in a hierarchical manner. In YANG, the edges of the graph model are represented by "leafref" elements. In the case of YAML, such edges can be represented by "ref" edges. Similarly, parent-child vertex relationships can be represented by "has" edges. For example, the vertex of element A using the vertex of element B with an edge reference can be understood as "element A has element B". In some examples, the management module 24 also provides the user with the ability to submit a reaction mapper, which the translation module 28 executes to convert the intent data model into device-specific low-level configuration instructions.

[0055] The controller device 10 also includes a configuration database 40. The configuration database 40 may include a data structure describing a managed network device, such as a network element 14. The configuration database 40 may be used as an intent data store, which may be used to persist and manage a collection of intent data models. For example, the configuration database 40 may include information indicating a device identifier (e.g., a MAC and / or IP address), a device type, a device vendor, a device category (e.g., a router, a switch, a bridge, a hub, etc.), etc. The configuration database 40 also stores current configuration information (e.g., an intent data model, or in some cases, both the intent data model and low-level configuration information) for a managed device (e.g., a network element 14). The configuration database 40 may include a database containing a unified intent data model. In some examples, the configuration database 40 may store a low-level configuration, which includes a pointer indicating the owner of the corresponding owned resource of the network element 14 for each corresponding owned resource in the owned resource set.

[0056] Management module 24 may maintain a data structure in configuration database 40. The data structure may include a plurality of vertices and a plurality of edges, each vertex in the plurality of vertices representing a corresponding network device (e.g., network element 14) in the plurality of network devices or a corresponding stateless intent in the plurality of stateless intents, and a plurality of edges defining relationships between the plurality of vertices. Management module 24 may receive an indication of a stateful intent. For example, management module 24 may receive intent unified graphical modeling configuration data for a set of managed network devices from a user, such as administrator 12.

[0057] Although for purposes of example, user interface 36 is described as allowing administrator 12 ( Figure 1 ) to interact with controller device 10, but other interfaces may be used in other examples. For example, controller device 10 may include a representational state transfer (REST) ​​client (not shown) that may be used as an interface to another device through which administrator 12 may configure controller device 10. Likewise, administrator 12 may configure network element 14 by interacting with controller device 10 through a REST client.

[0058] Management module 24 may model configuration database 40 as a graph database representing YANG configuration data elements. YANG specifies various types of data structures, including lists, leaf lists, containers, containers with existence, and features. Management module 24 may model each of the lists, containers, containers with existence, and features, and top-level containers as vertices in a graph database. Alternatively, configuration database 40 may represent YAML configuration data elements.

[0059] After building the graph database, the management module 24 can operate on the data of the graph database. For example, the management module 24 can map Netconf-based operations such as get-config, get-config with filters, and edit-config to graph query language queries such as Gremlin queries. Gremlin is described in GremlinDocs at gremlindocs.spmallette.documentup.com and in github.com / tinkerpop / gremlin / wiki. If the conditional attributes change, the management module 24 can execute the conditions mapped to the vertices and edges of the graph database. In response to these conditions, the management module 24 can process additional changes as a function discussed in more detail below. The management module 24 can further update all changes in transaction semantics.

[0060] Telemetry sensor 25 may be configured to generate telemetry packets indicating a plurality of packets of telemetry data output by network element 14A of network element 14. Telemetry data may include any information that may be collected from network element 14 using a telemetry sensor such as, for example, OpenConfig, Juniper Telemetry Interface (JTI) native, Netconf, SNMP, syslog, sFlow, NetFlow, or other telemetry sensors. Example metrics in the telemetry data may include, but are not limited to, for example, interface status, routing status (including routing protocol information such as BGP, ISIS, MPLS, etc.), system status (cpu utilization, memory utilization, etc.), system alarm status (temperature alarms, critical alarms), security-related status, hardware chassis-related status (fans, power supplies, temperature), network service / flow details, or other telemetry data.

[0061] Figure 3 is a block diagram illustrating an example of a network device 14, an edge device 16, one or more network address translators (NATs) 60, and a collector device 20 for processing telemetry packets in accordance with the techniques of this disclosure. Figure 3 In the example of FIG. 1 , two network elements 14 are shown, one ( 14A) being associated with a first provider (“Provider A”) and the other ( 14B) being associated with a second provider (“Provider B”). Figure 3 In the example of FIG. 1 , and for ease of illustration, only two elements 14 are shown, but any number of elements 14 may be used in other examples.

[0062] Network element 14A may generate telemetry packet 60A indicating telemetry data for a plurality of packets output by network element 14A. In some examples, network element 14A may generate telemetry packet 60A to indicate a source IP address of network element 14A. Network element 14B may generate telemetry packet 60B indicating telemetry data for a plurality of packets output by network element 14B. In some examples, network element 14B may generate telemetry packet 60B to indicate a source IP address of network element 14B.

[0063] Edge device 16 may identify telemetry packet 60A and select a first source identifier for network element 14A from a plurality of source identifiers. Similarly, edge device 16 may identify telemetry packet 60B and select a second source identifier for network element 14B from a plurality of source identifiers. Edge device 16 may modify telemetry packet 60A to further indicate the first selected source identifier to generate modified telemetry packet 62A. Similarly, edge device 16 may modify telemetry packet 60B to further indicate the second selected source identifier to generate modified telemetry packet 62B. NAT 66 may convert the source IP address for network element 14A to a modified IP address that is different from the source IP address for network element 14A. Thus, NAT 66 generates telemetry packet 64A from telemetry packet 62A. In some examples, NAT 66 may convert the source IP address for network element 14B to a modified IP address that is different from the source IP address for network element 14B. Thus, NAT 66 generates telemetry packet 64B from telemetry packet 62B.

[0064] Collector device 20 may associate telemetry packet 64A with network element 14A based on the first selected source identifier of telemetry packet 64A, and may process telemetry packet 64A to generate telemetry information for network element 14A. Similarly, collector device 20 may associate telemetry packet 64B with network element 14B based on the second selected source identifier of telemetry packet 64B, and may process telemetry packet 64B to generate telemetry information for network element 14B.

[0065] As shown, the technique for configuring edge device 16 to decorate telemetry packets with a source identifier can support multiple vendors (e.g., network devices configured with different operating software or network devices supporting different protocols). For example, network element 14A can be configured by a first vendor ("Vendor A") and network element 14B can be configured by a second vendor ("Vendor B") that is different from the first vendor.

[0066] Figure 4A is a conceptual diagram illustrating an example header 400 for a telemetry packet generated by an example telemetry sensor (eg, a NetFlow sensor), in accordance with techniques of this disclosure. Figure 4B This is to illustrate the technology according to the present disclosure Figure 4A A conceptual diagram of an example detail of a header 410. Figure 4A , Figure 4BIn the example of , the collector device 20 can be configured to use the source IP address plus the source ID 402 to associate the telemetry packet with a unique instance of a specific device. However, without the source identifier, there may be two devices (e.g., two different telemetry sensors for different network devices) using the same source address (private address), and the two devices may belong to two different customers. In addition, if the collector device 20 is configured to use the source IP address of the packet (and the source ID 402), there may be times when the source IP address may be changed by NAT on the way to the collector device and / or at the collector device before processing, thereby potentially losing the source identity.

[0067] Although the foregoing examples are described in the context of NetFlow, other flow protocols may be used, such as, for example, sampled flow (sflow), Juniper flow (Jflow), or other flow protocols. Additionally, although the techniques described herein may describe edge devices as being separate from network devices, some examples may implement techniques for decorating telemetry packets with a source identifier at a source device (locally or as an extension) and / or as part of an edge device.

[0068] Figure 5 1 is a block diagram illustrating an example of a network device 14, an edge device 16, one or more NATs 60, and a collector device 20 for rate control of a network device according to the techniques of the present disclosure. In this example, the edge device 16 can output feedback information 61 to the network element 14B based on the telemetry packets 60B. The network element 14B can be configured to modify the sampling rate for multiple packets based on the feedback information 61. Configuring the network element 14B to modify the sampling rate can be used to help alleviate dropped packets and / or socket congestion, which may occur under the aggressive sampling rate set on the network element 14B. Using the feedback information 61, the network element 14B can configure the sampling rate or packet export rate according to the needs and support of the telemetry protocol, which can help avoid random drops of packets and can help reduce congestion caused by telemetry. For example, in response to a change in the sFlow or NetFlow packet sampling rate from 1:1000 to 1:10000, the network element 14B can be configured to change the streaming local telemetry packet export rate from a frequency of 60 seconds to 120 seconds. As used herein, a packet export rate may refer to an interval at which telemetry information is exported from a device to a collector device 20 .

[0069] Figure 6 is a block diagram illustrating an example of a network device 14, an edge device 16, one or more NATs 60, and a collector device 20 for processing telemetry packets from different tenants according to the techniques of this disclosure. Figure 6In the example of , network element 14A may generate telemetry packet 60A that indicates multiple packets of telemetry data output by network element 14A. Network element 14B may generate telemetry packet 60B that indicates multiple packets of telemetry data output by network element 14B.

[0070] exist Figure 6 In an example of , one or more edge devices 16 may include an edge device 16A associated with tenant A (e.g., assigned to tenant A) and an edge device 16B associated with tenant B (e.g., assigned to tenant B). Edge device 16A may identify telemetry packet 60A and select a first source identifier for network element 14A from a plurality of source identifiers. Similarly, edge device 16B may identify telemetry packet 60B and select a second source identifier for network element 14B from a plurality of source identifiers. In addition, edge device 16A may select a first tenant identifier for network element 14A from a plurality of tenant identifiers. Edge device 16B may select a second tenant identifier for network element 14B from a plurality of tenant identifiers. Edge device 16A may modify telemetry packet 60A to further indicate the first selected source identifier and the first tenant identifier to generate a modified telemetry packet 72A. Similarly, edge device 16B may modify telemetry packet 60B to further indicate the second selected source identifier and the second tenant identifier to generate a modified telemetry packet 72B.

[0071] NAT 66 may translate the source IP address for network element 14A into a modified IP address that is different from the source IP address for network element 14A. Thus, NAT 66 generates telemetry packet 74A from telemetry packet 72A. In some examples, NAT 66 may translate the source IP address for network element 14B into a modified IP address that is different from the source IP address for network element 14B. Thus, NAT 66 generates telemetry packet 74B from telemetry packet 62B. In some examples, NAT 66 may be omitted.

[0072] Collector device 20 may associate telemetry packet 74A with network element 14A based on the first selected source identifier and the first tenant identifier of telemetry packet 74A, and may process telemetry packet 74A to generate telemetry information for network element 14A. Similarly, collector device 20 may associate telemetry packet 74B with network element 14B based on the second selected source identifier and the second tenant identifier of telemetry packet 74B, and may process telemetry packet 74B to generate telemetry information for network element 14B.

[0073] Figure 7 is a flow diagram illustrating an example process for resources in accordance with the techniques of this disclosure. Figure 7 It is a reference Figures 1 to 6 discussed for illustrative purposes only. Figure 7 In the example of , a sensor device for network element 14A may generate a telemetry packet indicating a plurality of packets of telemetry data output by network device 14A of network device 14 (702). For example, network element 14A (e.g., a telemetry sensor included in network element 14A) may assign a plurality of packets to a source IP address and a destination address, and network device 14A may generate each of the plurality of packets to include the source IP address and the destination address. The telemetry packet may conform to NetFlow. However, the telemetry packet may also conform to other flow protocols, such as sampled flow (sflow), Juniper flow (Jflow), or other flow protocols.

[0074] Edge device 16 may identify a telemetry packet that indicates a plurality of packets of telemetry data output by network device 14A (704). Edge device 16 may select a source identifier for network element 14A from a plurality of source identifiers (706). For example, edge device 16 may be configured to store preconfigured and unique source identifiers for each network element of network element 14. In some examples, edge device 16 may use a source IP address of a received telemetry packet as an input to a function (e.g., a lookup table) that generates a selected source identifier.

[0075] The edge device 16 may modify the telemetry packet to further indicate the selected source identifier (708). For example, the edge device 16 may modify the packet payload of the telemetry packet (e.g., Figure 4A 402) to include a selected source identifier. The edge device 16 (e.g., utilizing the network element 14) may output the modified telemetry packet to the collector device 20. In some examples, the edge device 16 may receive the telemetry packet using a connectionless protocol. Examples of connectionless protocols may include, for example, telemetry mechanisms (e.g., sFlow, Juniper Networks Telemetry Interface (JTI) Local, system logs, etc.) that use UDP transmission, which may be inherently less reliable than connection-oriented protocols (e.g., TCP-based protocols) and may drop packets, and may be less secure than connection-oriented protocols. In some examples, the edge device 16 may establish a connection between the edge device 16 and the collector device 20 and output the modified telemetry packet to the collector device 16 via the connection. For example, the edge device 16 may convert the telemetry packet from UDP to a connection-oriented protocol, such as gRPC or HTTP 2.0 (reliable and secure transport based on TCP), for example.

[0076] Collector device 20 may associate the telemetry packet with network element 14A based on the selected source identifier (710). Collector device 20 may process one or more flows and / or streams associated with a source device (e.g., network element 14A) to generate telemetry information. Collector device 20 may output an indication of the telemetry information to controller device 10. For example, controller device 10 may output the telemetry information on a display.

[0077] In some examples, the telemetry packet may further indicate a source IP address for network device 14A. In this example, NAT may be configured to change the source IP address for network device 14A to a translated IP address that is different from the source IP address for network device 14A. According to the techniques of this disclosure, edge device 16 may output a modified telemetry packet indicating a selected source identifier, and collector device 20 may associate the telemetry packet with network element 14A based on the selected source identifier. In this way, edge device 16 may help improve the accuracy of associating telemetry packets with network elements, particularly where NAT is used.

[0078] In some examples, edge device 16 may output feedback information to network device 14A based on the telemetry packets. In this example, network device 14A may be able to (e.g., be configured to) modify (e.g., increase or decrease) a set of parameters that control a telemetry export rate (e.g., a sampling rate or a sensing rate) based on the feedback information. For example, network device 14A may be configured to change a sampling rate of a streaming telemetry sensor (e.g., such as, for example, juniper network local telemetry (JTI), openconfig, or other streaming telemetry sensor) based on the feedback information. In some examples, network device 14A may be configured to change a sensor rate of a sensor configured for sFlow, NetFlow, or other telemetry sensor based on the feedback information.

[0079] The edge device 16 can select a tenant identifier for multiple groups from a plurality of tenant identifiers. In this example, the edge device 16 can modify the telemetry group to indicate the selected source identifier and further indicate the selected tenant identifier. The selected tenant identifier can be pre-configured by the administrator 12. For example, the administrator 12 can assign a first tenant identifier to the network device 14A associated with tenant "A" and can assign a second tenant identifier to the network device 14B associated with tenant "B". The first tenant identifier and the second tenant identifier can be different (e.g., unique). In addition to the source identifier and / or tenant identifier, the edge device 16 can be configured to use zero or more identifiers.

[0080] The techniques described in this disclosure may be implemented at least in part in hardware, software, firmware, or any combination thereof. For example, various aspects of the described techniques may be implemented within one or more processors including one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or any other equivalent integrated or discrete logic circuits, and any combination of these components. The term "processor" or "processing circuit" may generally refer to any of the aforementioned logic circuits alone, or in combination with other logic circuits, or any other equivalent circuit. A control unit including hardware may also perform one or more techniques of the present disclosure.

[0081] Such hardware, software, and firmware may be implemented within the same device or within separate devices to support the various operations and functions described in this disclosure. In addition, any of the described units, modules, or components may be implemented together or individually as discrete but interoperable logic devices. Describing different features as modules or units is intended to emphasize different functional aspects and does not necessarily imply that these modules or units must be implemented by separate hardware or software components. On the contrary, the functions associated with one or more modules or units may be performed by separate hardware or software components, or integrated in common or separate hardware or software components.

[0082] The techniques described in the present disclosure may also be embodied or encoded in a computer-readable medium (e.g., a computer-readable storage medium) containing instructions. Instructions embedded or encoded in a computer-readable medium may cause a programmable processor or other processor to perform the method, for example, when the instructions are executed. Computer-readable media may include non-transient computer-readable storage media and transient communication media. Tangible and non-transient computer-readable storage media may include random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, hard disk, CD-ROM, floppy disk, cassette, magnetic media, optical media, or other computer-readable storage media. The term "computer-readable storage medium" refers to a physical storage medium, not a signal, carrier wave, or other transient medium.

Claims

1. A method for source identification using a source identifier, the method comprising: identifying, by an edge device implemented in the circuit, a telemetry packet indicating telemetry data of a plurality of packets output by a network device among a plurality of network devices; In response to determining that the telemetry packet includes a source IP address that matches the network device and the network device is assigned a source identifier, selecting, by the edge device, the source identifier for the network device from a plurality of source identifiers; modifying, by the edge device, the telemetry packet to further indicate the selected source identifier; as well as A modified telemetry packet is outputted by the edge device, wherein the modified telemetry packet indicates the source IP address and the selected source identifier.

2. The method according to claim 1, wherein: The telemetry packets are output by telemetry sensors.

3. The method according to claim 2, wherein: The telemetry sensor is included within the network device.

4. The method according to claim 1, wherein: A network address translator (NAT) is configured to change the source IP address for the network device to a translated IP address that is different from the source IP address for the network device.

5. The method according to claim 1 also includes outputting feedback information from the edge device to the network device based on the telemetry packet, and enabling the network device to modify one or more of the sampling rate and sensor rate used for the multiple packets based on the feedback information.

6. The method according to claim 1, further comprising selecting, by the edge device, a tenant identifier for the plurality of groups from a plurality of tenant identifiers, wherein: Modifying the telemetry packet further includes modifying the telemetry packet to indicate the selected source identifier and to further indicate a selected tenant identifier.

7. The method according to claim 6, wherein: The selected tenant identifier is pre-configured by an administrator.

8. The method according to claim 1, wherein: The selected source identifier is different from other source identifiers of the plurality of source identifiers.

9. The method according to claim 1, wherein: The selected source identifier is pre-configured by an administrator.

10. The method according to claim 1, wherein: Outputting the modified telemetry packet includes establishing a connection between the edge device and a collector device and outputting the modified telemetry packet to the collector device via the connection.

11. The method according to claim 1, wherein: The telemetry packets may conform to NetFlow, OpenConfig, Juniper Telemetry Interface (JTI) native, netconf, SNMP, syslog, or sFlow.

12. The method according to any one of claims 1 to 11, wherein: The network device is configured to assign the plurality of packets to a source IP address and a destination address, and wherein the network device is configured to generate each packet of the plurality of packets to include the source IP address and the destination address.

13. A device comprising a processing circuit, the processing circuit being configured to: identifying a telemetry packet indicating a plurality of packets of telemetry data output by a network device among the plurality of network devices; responsive to determining that the telemetry packet includes a source IP address that matches the network device and the network device is assigned a source identifier, selecting the source identifier for the network device from a plurality of source identifiers; modifying the telemetry packet to further indicate the selected source identifier; as well as A modified telemetry packet is output, the modified telemetry packet indicating the source IP address and the selected source identifier.

14. The device according to claim 13, wherein: The telemetry packets are output by telemetry sensors.

15. The device according to claim 14, wherein: The telemetry sensor is included within the network device.

16. The apparatus according to claim 13, wherein: A network address translator (NAT) is configured to change the source IP address of the network device to a translated IP address that is different from the source IP address of the network device.

17. The apparatus according to claim 13, wherein: The processing circuit is configured to output feedback information to the network device based on the telemetry packets and enable the network device to modify a sampling rate of the plurality of packets based on the feedback information.

18. The apparatus of claim 13, wherein: The processing circuit is configured to select a tenant identifier for the plurality of packets from a plurality of tenant identifiers, wherein, to further modify the telemetry packet, the processing circuit is configured to modify the telemetry packet to indicate the selected source identifier and further indicate the selected tenant identifier.

19. The apparatus according to any one of claims 13 to 18, wherein: The network device is configured to assign the plurality of packets to a source IP address and a destination address, and wherein the network device is configured to generate each packet of the plurality of packets to include the source IP address and the destination address.

20. A computer readable storage medium encoded with instructions for causing one or more programmable processors to perform the method of any one of claims 1 to 12.

Images