Path visibility, packet loss and latency measurements for service chain data flows

Abstract

Disclosed herein are techniques for determining packet path visibility, packet loss, and latency measurements associated with a data flow of a networked computing environment. The techniques can include receiving flow data associated with a data flow of a networked computing environment and determining a packet path associated with the data flow. The packet path can indicate that a first leaf switch is configured to send a packet to a service chain device and a second leaf switch is configured to receive the packet from the service chain device. The techniques can also include receiving timestamp data indicating a first time at which the first leaf switch sends the packet to the service chain device and a second time at which the second leaf switch receives the packet from the service chain device. Based at least in part on the timestamp data, a latency associated with the service chain device can be calculated.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Patent Application No. 16 / 918,658, filed July 1, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to determining packet path visibility, packet loss, and latency measurements associated with data flows in a networked computing environment (e.g., a data center switching architecture) that utilizes service chains. Background Technology

[0004] Networked computing environments (e.g., data center switching infrastructures) typically include various hardware and software devices such as switches, routers, server computers, and network nodes. These devices are used to establish data flows within the networked computing environment to route data packets from one network to another. Furthermore, these networked computing environments often host mission-critical applications such as Internet of Things (IoT) applications, connected cars, gaming, and high-bandwidth internet. As these applications begin to utilize faster network speeds, such as 5G, determining whether specific nodes and / or devices in the data flow are causing packet loss and / or latency for the application is crucial. However, because these applications often have very complex service chaining requirements (including sending packets to service chaining devices located outside the networked computing environment), it is difficult to calculate the latency of specific nodes and / or devices, and difficult to determine whether specific nodes and / or devices are losing packets. Attached Figure Description

[0005] Detailed description is given below with reference to the accompanying drawings. In the drawings, the leftmost number(s) of the reference numerals indicate the drawing in which the reference numeral first appears. The same reference numerals are used in different drawings to denote similar or identical items. The systems shown in the drawings are not drawn to scale, and the components in the drawings may be drawn to different scales.

[0006] Figure 1 A system architecture diagram of an example networked computing environment is shown. As illustrated, the data flow in the networked computing environment utilizes a service chain.

[0007] Figure 2A-2C Together, flowcharts illustrate an example process for determining path visibility, packet loss, and / or latency of data flows in a networked computing environment utilizing service chains.

[0008] Figure 3 A flowchart of an example method for determining the latency associated with packets of data streams that traverse a networked computing environment utilizing a service chain is shown.

[0009] Figure 4 A flowchart is shown as an example method for determining the latency associated with a service chain device.

[0010] Figure 5 A computing system diagram of an example configuration of a data center that can be used to implement various aspects of the techniques disclosed herein is shown.

[0011] Figure 6 A computer architecture diagram is shown, illustrating an example computer hardware architecture for implementing a network device that can be used to implement various aspects of the various technologies presented herein. Detailed Implementation

[0012] Overview

[0013] Various aspects of the invention are set forth in the independent claims, and preferred features are set forth in the dependent claims. A feature of one aspect may be applied to any aspect alone or in combination with other aspects.

[0014] This disclosure describes techniques for determining packet path visibility, packet loss, and latency measurements associated with data flows in a networked computing environment utilizing service chains. Some techniques described in this disclosure may include receiving flow data (e.g., 5-tuple flow data, Virtual Routing and Forwarding Tables (VRFs), ingress and / or egress physical interface data, ingress and / or egress logical interface data, service chain configuration data, etc.) associated with the data flow from one or more leaf switches of the networked computing environment. Packet paths associated with the data flow can be determined, at least in part, based on the flow data. The packet paths may indicate that a first leaf switch among one or more leaf switches is configured to send packets to a service chain device, and a second leaf switch among one or more leaf switches is configured to receive packets from the service chain device. In the example, the service chain device is located outside the networked computing environment. In this way, by using flow data received from one or more leaf switches, packet path visibility can be determined for data flows in a networked computing environment utilizing service chains, resulting in improvements to computing techniques related to network visibility and troubleshooting.

[0015] Furthermore, the techniques described in this disclosure may include receiving first timestamp data instructing a first leaf switch to send a packet to a service chain device at a first time. The techniques may also include receiving second timestamp data instructing a second leaf switch to receive a packet from a service chain device at a second time. In this way, the latency associated with the service chain device can be calculated at least in part based on the first and second timestamp data. Therefore, by utilizing at least some of the techniques described herein, latency can be determined for devices not located within a networked computing environment (e.g., a data center switching infrastructure).

[0016] The techniques described herein may also include receiving first timestamp data from an ingress leaf switch of the networked computing environment, indicating the first time a packet enters the networked computing environment. Furthermore, second timestamp data from an egress leaf switch of the networked computing environment, indicating the second time a packet exits the networked computing environment, may be received. In this way, the latency associated with packets traversing the networked computing environment can be calculated, at least in part, based on the first and second timestamp data.

[0017] The techniques described in this disclosure can be performed as a method and / or executed by a system having a non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more processors, perform the techniques described above.

[0018] Example Implementation

[0019] As mentioned above, when networked computing environments host critical applications (such as 5G), it is crucial to determine whether specific nodes and / or devices in the data flow are causing packet loss and / or latency for the application. However, because these applications typically have very complex service chain requirements, including sending packets to service chain devices located outside the networked computing environment (e.g., firewalls, Transmission Control Protocol (TCP) optimizers, Carrier-Grade Network Address Translation (CGNAT), parental controls, Deep Packet Inspection (DPI), load balancers, etc.), it is difficult to calculate the latency of specific nodes and / or devices, and difficult to determine whether specific nodes and / or devices are losing packets.

[0020] For example, if any device in the data stream causes increased latency or packet loss, it can lead to downtime for 5G applications hosted in the networked computing environment, impacting service provider revenue. Traditionally, latency and / or packet loss can be determined, at least in part, based on the packet header of a particular packet as it traverses the data stream of the networked computing environment. The packet header can include timestamp data, 5-tuple stream data (including, for example, source Internet Protocol (IP) address, destination IP address, protocol, source Layer 4 (L4) port, and destination L4 port), TCP sequence number, identification fields, etc. However, this packet header is lost when the data stream includes external service chain devices, such as when packets are sent to service chain devices outside the networked computing environment (e.g., a data center switching infrastructure). Furthermore, using only the stream information stored in the packet header, it is impossible to determine whether a packet is entering the networked computing environment for the first time or returning from a service chain device, increasing the difficulty of determining end-to-end latency, service device latency, and packet loss.

[0021] Therefore, this disclosure describes techniques for determining packet path visibility, packet loss, and latency measurements associated with data flows in a networked computing environment utilizing service chains. As used herein, the term "networked computing environment" can include data center architectures, cloud computing networks, etc. Furthermore, a networked computing environment can include buildings, dedicated spaces within buildings, a group of buildings in a single physical location, or a group of buildings spread across multiple physical locations, which house computer systems and associated components, such as telecommunications and storage systems. A networked computing environment can include one or more leaf switches, spine switches, computing resources, data storage devices, routers, gateways, etc.

[0022] As used herein, the term "data flow" refers to a packet path that traverses a networked computing environment by routing packets through one or more specific devices (e.g., leaf switches, spine switches, etc.) within the networked computing environment. Furthermore, the packet path of a data flow can route packets outside the data center switching fabric to one or more service chain devices located outside the data center fabric. In various examples, the networked computing environment may include multiple data flows, each traversing through one or more different devices. For example, a first data flow may include a first ingress leaf switch that receives packets within the data center fabric, and a second data flow may include a second ingress leaf switch that also receives packets within the data center fabric. Additionally, the first data flow may include a third egress leaf switch that sends packets to service chain devices located outside the data center fabric, while the second data flow may include a fourth egress leaf switch that also sends packets to service chain devices located outside the data center fabric.

[0023] Furthermore, as used in this text, the term "stream data" can include telemetry data associated with a data stream. For example, stream data can include one or more of the following: 5-tuple stream data, Virtual Routing and Forwarding Tables (VRFs), switch ingress physical interface data, switch egress physical interface data, switch ingress logical interface data, switch egress logical interface data, etc. In various examples, and as further discussed below, stream data can be used to identify packet paths for one or more data streams in a networked computing environment (e.g., by determining the next hop of a data stream relative to a particular device upon receiving a packet).

[0024] By way of example and not limitation, a method according to the technology described herein may include receiving first-stream data associated with a data flow from a first leaf switch of a networked computing environment. The networked computing environment may include one or more leaf switches, one or more spine switches, one or more computing resources, etc. Furthermore, the data flow may traverse or otherwise pass through one or more physical and / or logical devices of the networked computing environment to route packets through the networked computing environment. In some examples, the first-stream data may include one or more of the following: 5-tuple flow data, Virtual Routing and Forwarding Table (VRF), ingress and / or egress physical interface data, ingress and / or egress logical interface data, etc. Additionally, the method may also include receiving second-stream data associated with the data flow from a second leaf switch of the networked computing environment. Similar to the first-stream data described above, the second-stream data may include one or more of the following: 5-tuple flow data, VRF, ingress and / or egress physical interface data, ingress and / or egress logical interface data, etc.

[0025] In some examples, based at least in part on first-stream and second-stream data, the method may include determining a packet path associated with the data stream. The packet path may indicate that a first leaf switch is configured to send packets to a service chain device, and a second leaf switch is configured to receive packets from the service chain device. In various examples, the service chain device may be located outside the networked computing environment and may include firewalls, TCP optimizers, CGNAT, parental controls, DPI, load balancers, etc. Furthermore, the packet path may indicate one or more other leaf switches, one or more spine switches, and / or one or more other devices associated with the data stream.

[0026] In some examples, service chain configuration data can be received for first-leaf nodes, second-leaf nodes, and other switches and / or devices in the networked computing environment. For example, if the networked computing environment is configured in Application Center Infrastructure (ACI) mode, the service chain configuration data may include contract, filter, and service graph configuration data received from the Application Policy Infrastructure Controller (APIC). Additionally or alternatively, if the networked computing environment is configured in a non-ACI mode, the service chain configuration data may include policy-based routing (PBR) and routing protocol configuration data that defines the service chain path for a given data flow. In at least one example, determining the packet path associated with the data flow may further be based at least in part on the service chain configuration data. The service chain configuration may instruct a first-leaf switch to be configured to export packets to a service chain device, and a second-leaf switch to be configured to receive packets from the service chain device into the networked computing environment.

[0027] The method may also include receiving first timestamp data from the first leaf switch indicating the first time the packet was sent to the service chain device. The first timestamp data may be included in the first packet header of the packet. The first packet header may additionally include packet identification data associated with the packet, such as 5-tuple data, TCP sequence number, etc. In some examples, the method may include receiving packet identification data associated with the packet. In this way, when the packet re-enters the networked computing environment from the service chain device, the packet can be tracked, as discussed further below.

[0028] In various examples, the method includes receiving second timestamp data from a second leaf switch indicating a second time at which a packet was received from a service chain device. The second timestamp data may be included in a second packet header of the packet, wherein the second packet header differs from the first packet header. For example, as described above, when a packet leaves the networked computing environment after being sent from the first leaf switch to the service chain device, the packet may lose its original (e.g., first) packet header, which included the first timestamp data. Furthermore, when the packet re-enters the networked computing environment from the service chain device, it may include a new (e.g., second) packet header that includes the second timestamp data. Therefore, in some examples, the method includes determining, at least in part, that the second leaf switch received a packet from the service chain device based on packet identification data, and associating the second timestamp data with the packet.

[0029] In some examples, the method includes calculating the latency associated with the service chain device based at least in part on first and second timestamp data. For example, the difference between a first time point associated with the first timestamp data and a second time point associated with the second timestamp data can be calculated, and this difference can indicate the latency associated with the service chain device. In various examples, the latency associated with the service chain device can include a first latency, and the method can also include calculating the end-to-end latency associated with packets of data streams traversing a networked computing environment. For example, calculating the end-to-end latency can be based at least in part on the first latency associated with the service chain device, and receiving (i) third timestamp data indicating a second latency associated with a first leaf switch, and (ii) fourth timestamp data indicating a third latency associated with a second leaf switch. In this way, the end-to-end latency can be the sum of the first latency, the second latency, and the third latency. Furthermore, it is contemplated that the end-to-end latency of the data stream and the individual latencies associated with a particular device of the data stream can be calculated for each of a data stream including multiple leaf switches, spine switches, service chain devices, etc.

[0030] The techniques described herein (including the example methods described above) can be implemented by software, hardware, or a combination thereof. For example, the techniques described herein can be implemented by software running on a central agent (e.g., computing resources, server computers, etc.) of a networked computing environment, or by switches in the networked computing environment (e.g., spine switches and / or leaf switches, etc.). By implementing some or all of the techniques described in this disclosure, packet path visibility, latency, and packet loss associated with data flows in a networked computing environment can be more easily determined. In this way, service providers can use these various techniques to enhance the reliability and efficiency of their systems by at least reducing latency and identifying where failures may occur, resulting in improvements to system monitoring techniques for networked computing environments utilizing service chains.

[0031] Certain implementations and embodiments of this disclosure will now be described more fully below with reference to the accompanying drawings, in which various aspects are illustrated. However, these aspects may be implemented in many different forms and should not be construed as limited to the implementations set forth herein. This disclosure includes variations of the embodiments as described herein. The same reference numerals consistently denote the same elements.

[0032] Figure 1A system architecture diagram of an example networked computing environment 102 is shown. As illustrated, data flow 106 in the networked computing environment utilizes service chaining. Typically, the networked computing environment 102 may include devices housed or located in one or more data centers 104, which may be located in different physical locations. For example, the networked computing environment 102 may be supported by a network of devices from a public cloud computing platform, a dedicated / enterprise computing platform, and / or any combination thereof. The one or more data centers 104 may be physical facilities or buildings located in a geographic area where networked devices are designated as part of the networked computing environment 102. Data centers 104 may include various networked devices, as well as redundant or backup components and infrastructure for power, data communication connectivity, environmental control, and various security devices. In some examples, data center 104 may include one or more virtual data centers, which are pools or collections of cloud infrastructure resources specifically designed for enterprise needs and / or cloud-based service provider needs. Typically, data centers 104 (physical and / or virtual) can provide basic resources such as processors (CPU), memory (RAM), storage devices (disks), and networking (bandwidth). However, in some examples, the devices in the networked computing environment 102 may not be located in the explicitly defined data center 104, but may be located in other locations or buildings.

[0033] The networked computing environment 102 includes various computing devices, such as a leaf switch layer 108, a spine switch layer 110, and a computing resource layer 112. The leaf switch layer 108 includes one or more leaf switches 108A-108N (where N represents any number greater than or equal to 1), the spine switch layer 110 includes one or more spine switches 110A-110N (where N represents any number greater than or equal to 1), and the computing resource layer 112 includes one or more computing resources 112A-112N (where N represents any number greater than or equal to 1). The leaf switches 108A-108N, spine switches 110A-110N, and computing resources 112A-112N can communicatively couple with each other via one or more wired and / or wireless connections. In this way, data can be transmitted back and forth between the various computing devices in the leaf switch layer 108, spine switch layer 110, and computing resource layer 112. In addition, one or more leaf switches 108A-108N, one or more spine switches 110A-110N, and / or one or more computing resources 112A-112N are capable of communicating with one or more service chain devices 114A-114N of service chain 114.

[0034] In some examples, software executing on compute resource layer 112 can determine the packet paths associated with data flow 106, such as Figure 1The data is shown in dashed lines. For example, the compute resource layer can receive streaming data 116 from various compute devices at leaf switch layer 108 and spine switch layer 110. This streaming data 116 may include one or more of the following: VRF, 5-tuple streaming data, ingress and / or egress physical interface data, ingress and / or egress logical interface data, etc. Based at least in part on the streaming data 116, software executing on compute resource layer 116 can determine the packet paths associated with the data stream 106. As an example, and as... Figure 1 As shown, the software executing on compute resource layer 112 can determine that leaf switch 108A is configured to receive packets of data stream 106, such as packet 118, from packet gateway 120 and then forward these packets to spine switch 110A. Next, spine switch 110A will receive packets of data stream 106 and then forward these packets to leaf switch 108B. Furthermore, based at least in part on stream data 116 and service chain configuration data, the software executing on compute resource layer 112 can determine that leaf switch 108B is configured to send packets of data stream 106 to service chain device 114A, and leaf switch 108C is configured to receive packets of data stream 106 from service chain device 114A. When leaf switch 108C receives packets of data stream 106, it is configured to send those packets to service chain device 114B. The software executing on compute resource layer 112 can also determine that leaf switch 108D is configured to receive packets of data stream 106 from service chain device 114B and then forward these packets to spine switch 110B. Finally, the software can determine that spine switch 110B is configured to receive packets of data stream 106 from leaf switch 108D, forward these packets to leaf switch 108N, and then leaf switch 108N is configured to forward these packets to one or more networks 122, such as the Internet. In this way, the software executing on compute resource layer 112 can determine the packet paths of data streams in a networked computing environment. Furthermore, although in Figure 1 The diagram shows the software executing on compute resource layer 112 to determine packet paths for data flows; however, it is conceivable that the software could execute on other hardware resources in a networked computing environment, such as leaf switch layer 108, spine switch layer 110, other devices not shown, and / or combinations thereof.

[0035] In some examples, software executing on compute resource layer 112 can determine the latency and / or packet loss associated with packets (e.g., packet 118) of data flow 106 traversing networked computing environment 102. For example, based at least in part on determining the packet path of data flow 106 as described above, the software executing on compute resource layer 112 can receive packet data 124 associated with packet 118 each time one of leaf switches 108A-108N and / or spine switches 110A-110N receives and / or transmits packet 118. This packet data 124 may include timestamp data and at least one of the following: TCP sequence number, 5-tuple stream (e.g., including source IP address, destination IP address, protocol, source L4 port, and destination L4 port), IP header identification field, etc. Using this packet data 124, the software can track packet 118 as it traverses the data stream 106 of the networked computing environment, and calculate the latency of each device sending and / or receiving packet 118, even if the IP header of packet 118 is changed when packet 118 is sent to service chain 114.

[0036] For example, when leaf switch 108B sends packet 118 to service link device 114A, packet 118 will lose its IP header, including timestamp data. In this way, when leaf switch 108C receives packet 118 back from service link device 114A, the timestamp data in the IP header of packet 118 will be different. However, at least one of the TCP sequence number and / or IP header identification fields will remain the same. Therefore, exporting this packet data 124 to the compute resource layer 112 allows software to identify when packet 118 re-enters the networked computing environment 102. For example, when leaf switch 108B sends packet 118 to service link device 114A, leaf switch 108B will also send packet data 124, including egress timestamp data, to compute resource layer 112. Furthermore, when leaf switch 108C receives packet 118 from service link device 114A, leaf switch 108C will send packet data 124, including ingress timestamp data, to compute resource layer 112. In this way, by using exit timestamp data and entry timestamp data, the latency associated with sending a packet to service chain device 114A can be calculated. A similar operation can be performed whenever packet 118 is sent to the device of service chain 114.

[0037] Furthermore, a latency can be determined for each device in the networked computing environment 102 as part of the data stream 106. In some examples, determining the latency associated with a device in the networked computing environment can be based at least in part on determining at least one of the packet paths, packet data 124, stream data 116, combinations thereof, etc., of the data stream 106. By calculating the latency of each device in the networked computing environment as part of the data stream 106 and the latency of each service chain device also as part of the data stream 106, the end-to-end latency of packets 118 traversing the data stream 106 can be calculated by summing all the latencies associated with the various devices in the data stream 106.

[0038] In some examples, the latency of service chain devices 114A-114N can be determined by sending a unique packet with a specific TCP port from a source leaf switch to a destination leaf switch (e.g., from leaf switch 108B to service chain device 114A, and then from service chain device 114A to leaf switch 108C) via the service chain device. This unique packet can be sent at regular time intervals (e.g., once per second, every 10 seconds, every 30 seconds, every minute, every 10 minutes, etc.) to ensure that the service chain device does not lose packets or cause longer latency than expected. In various examples, the unique packet may include a packet configuration that all switches can process to measure latency and determine the packet path of the data flow. Each egress and ingress switch connected to the service chain device can export the flow information and timestamp of this unique packet. Since this is a unique injected packet, software executing on a central agent (e.g., computing resources) can collect the timestamp of the packet from the egress switch before sending it to the service chain device. Similarly, the software can collect the timestamp of the packet from the ingress switch after it receives the packet from the service chain device. In the example, the time difference between these timestamps will be the latency caused by each service chain device.

[0039] Figure 2A-2C Together, flowcharts illustrate an example process for determining path visibility, packet loss, and / or latency of data flows in a networked computing environment utilizing service chains.

[0040] The process began at Figure 2ALeaf switches 108A-108E and spine switches 110A and 110B send flow data 116A-116G to computing resource 112A. In some examples, flow data 116A-116G may include one or more of the following: VRF table associated with each device, 5-tuple flow data, ingress and / or egress physical interface data associated with each device, ingress and / or egress logical interface data associated with each device, next-hop information, etc. Furthermore, in the case of an ACI-based networked computing environment 102 architecture, computing resource 112A may also receive service chain contracts, filters, and service graphs from the APIC controller that define service chain paths in ACI mode for a given flow. Alternatively, in the case of a non-ACI-based networked computing environment 102 architecture, computing resource 112A may receive PBR and routing protocol configuration data that define service chain paths for a given flow.

[0041] Therefore, using some or all of the above-mentioned stream data 116A-116G information, in Figure 2B In this example, the packet path of data flow 210 (shown as dashed lines) is determined by software and / or hardware executing on computing resource 112A. The packet path of data flow 210 begins at leaf switch 108A, where packets can be received from packet gateway 120. Next, leaf switch 108A forwards the packets associated with data flow 210 to spine switch 110A, which in turn forwards the packets of data flow 210 to leaf switch 108B. Once at leaf switch 108B, the packets of data flow 210 are sent outside the networked computing environment 102 to service chain device 114A, and leaf switch 108C receives the packets of data flow 210 returning to the networked computing environment from service chain device 114A. Next, once packets from data flow 210 are at leaf switch 108C, they are again sent outside of the networking computing environment 102 to service chain device 114B, and later received back into the networking computing environment 102 by leaf switch 108D. Leaf switch 108D then forwards packets from data flow 210 to spine switch 110B, which in turn forwards these packets to leaf switch 108E. Finally, leaf switch 108E forwards packets from data flow 210 outside of the networking computing environment 102 to one or more destination networks 122, such as the Internet.

[0042] In some examples, computing resource 112A may include one or more processors 202 and memory 204. The one or more processors 202 may include one or more cores. Furthermore, the one or more processors may include a single processor or one or more processors. By way of example and not limitation, memory 204 may include computer-readable storage media of volatile and non-volatile, removable and non-removable media implemented in any method or technology. Computer-readable storage media include, but are not limited to, RAM, ROM, erasable programmable ROM (“EPROM”), electrically erasable programmable ROM (“EEPROM”), flash memory or other solid-state memory technologies, compact disc ROM (“CD-ROM”), digital versatile disc (“DVD”), high-definition DVD (“HD-DVD”), BLU-RAY, or other optical storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information in a non-transitory manner. Memory 204 may store one or more data streams 206, including data stream 210 and service chain configuration 208.

[0043] Go to Figure 2C Based at least in part on determining the packet path of data stream 210, one or more delays can now be calculated for each segment and / or device (including multiple segments and / or devices) of data stream 210.

[0044] In step "1", leaf switch 108A receives packet 212 of data stream 210 from packet gateway 120 and forwards packet 212 to spine switch 110A. In various examples, leaf switch 108A may send packet data 214A associated with packet 212 to computing resource 112A. Computing resource 112A may store packet data 214A. Packet data 214A may include timestamp data (e.g., ingress and egress timestamp data of leaf switch 108A) and at least one of the following: TCP sequence number, 5-tuple stream, IP header identification field, etc. Using at least the timestamp data, the latency of leaf switch 108A can be calculated. Latency can indicate the amount of time it takes for leaf switch 108A to process packets and forward them to spine switch 110A.

[0045] In step “2”, spine switch 110A receives packet 212 of data stream 210 from leaf switch 108A and forwards packet 212 to leaf switch 108B. In various examples, spine switch 110A may send packet data 214B associated with packet 212 to computing resource 112A. Computing resource 112A may store packet data 214B. Packet data 214B may include timestamp data (e.g., ingress and egress timestamp data of spine switch 110A) and at least one of the following: TCP sequence number, 5-tuple stream, IP header identification field, etc. Using at least the timestamp data, the latency of spine switch 110A can be calculated. Latency may indicate the amount of time it takes for spine switch 110A to process the packet and forward it to leaf switch 108B, and / or may include the amount of time since the packet entered the networked computing environment on leaf switch 108A. The latency can be calculated by determining the difference between the local time when the spine switch 110A receives the packet and the timestamp value when the packet enters the networked computing environment on the leaf switch 108A.

[0046] In step “3”, leaf switch 108B receives packet 212 of data stream 210 from spine switch 110A and exports packet 212 outside the networked computing environment to service link device 114A. In various examples, leaf switch 108B may send packet data 214C associated with packet 212 to computing resource 112A. Computing resource 112A may store packet data 214C. Packet data 214C may include timestamp data (e.g., ingress and egress timestamp data of leaf switch 108B) and at least one of the following: TCP sequence number, 5-tuple stream, IP header identification field, etc. Using at least the timestamp data, the latency of leaf switch 108B can be calculated. Latency may indicate the amount of time spent by leaf switch 108B processing the packet and exporting it to service link device 114A, and / or may include the amount of time since the packet entered the networked computing environment on leaf switch 108A. The delay can be calculated by determining the difference between the local time when the leaf switch 108B receives the packet and the timestamp value when the packet enters the network computing environment on the leaf switch 108A.

[0047] In step 4, leaf switch 108C receives packet 212 of data stream 210 from service link device 114A and exports packet 212 outside the networked computing environment 102 to service link device 114B. In various examples, leaf switch 108C may send packet data 214D associated with packet 212 to computing resource 112A. Computing resource 112A may store packet data 214D. Packet data 214D may include timestamp data (e.g., ingress and egress timestamp data of leaf switch 108B) and at least one of the following: TCP sequence number, 5-tuple stream, IP header identification field, etc.

[0048] When packet 212 is received by leaf switch 108C, packet 212 may include a new packet header without the original timestamp information. However, based at least in part on stored packet data 214C and stored packet data 214D, computing resource 112A can determine that packet 212 received by leaf switch 108C is the same packet 212 exported by leaf switch 108B, even if the IP header has changed. Therefore, using at least the egress timestamp from stored packet data 214C (egress timestamp of leaf switch 108B) and the ingress timestamp from stored packet data 214D (ingress timestamp of leaf switch 108C), computing resource 112A can determine the latency of the path associated with service link device 114A for packet 212. Furthermore, using at least the timestamp data from packet data 214D, the latency of leaf switch 108C can be calculated. The latency can indicate the amount of time spent by leaf switch 108C processing the packet and exporting it to service link device 114B.

[0049] At "5", leaf switch 108D receives packet 212 of data stream 210 from service chain device 114B and forwards packet 212 to spine switch 110B. In various examples, leaf switch 108D may send packet data 214E associated with packet 212 to computing resource 112A. Computing resource 112A may store packet data 214E. Packet data 214E may include timestamp data (e.g., ingress and egress timestamp data of leaf switch 108B) and at least one of the following: TCP sequence number, 5-tuple stream, IP header identification field, etc.

[0050] When packet 212 is received by leaf switch 108D, packet 212 may include a new packet header and is not easily recognized by computing resource 112A as the same packet 212 derived from leaf switch 108C. However, based at least in part on stored packet data 214D and stored packet data 214E, computing resource 112A can determine that packet 212 received by leaf switch 108D is the same packet 212 derived from leaf switch 108C, even if the IP header has changed. Therefore, using at least the egress timestamp from stored packet data 214D (egress timestamp of leaf switch 108C) and the ingress timestamp from stored packet data 214E (ingress timestamp of leaf switch 108D), computing resource 112A can determine the latency associated with service chain device 114B that processes packet 212. Furthermore, using at least the timestamp data from packet data 214E, the latency of leaf switch 108D can be calculated. The delay can indicate the amount of time it takes for leaf switch 108D to process packet 212 and forward it to spine switch 110B.

[0051] At “6”, spine switch 110B receives packet 212 of data stream 210 from leaf switch 108D and forwards packet 212 to leaf switch 108E. In various examples, spine switch 110B may send packet data 214F associated with packet 212 to computing resource 112A. Computing resource 112A may store packet data 214F. Packet data 214F may include timestamp data (e.g., ingress and egress timestamp data of spine switch 110A) and at least one of the following: TCP sequence number, 5-tuple stream, IP header identification field, etc. Using at least the timestamp data, the latency of spine switch 110B can be calculated. Latency may indicate the amount of time it takes for spine switch 110B to process the packet and forward it to leaf switch 108E, and / or may include the amount of time since the packet entered the networked computing environment on leaf switch 108D. The latency can be calculated by determining the difference between the local time when the spine switch 110A receives the packet and the timestamp value when the packet enters the networked computing environment on the leaf switch 108D.

[0052] In step 7, leaf switch 108E receives packet 212 of data stream 210 from spine switch 110B and exports packet 212 outside the networked computing environment 102 to one or more networks 122 (e.g., the Internet). In various examples, leaf switch 108E may send packet data 214G associated with packet 212 to computing resource 112A. Computing resource 112A may store packet data 214G. Packet data 214G may include timestamp data (e.g., ingress and egress timestamp data of leaf switch 108A) and at least one of the following: TCP sequence number, 5-tuple stream, IP header identification field, etc. Using at least the timestamp data, the latency of leaf switch 108E can be calculated. Latency may indicate the amount of time spent by leaf switch 108E processing the packet and exporting it to one or more networks 122, and / or may include the amount of time since the packet entered the networked computing environment on leaf switch 108D. The latency can be calculated by determining the difference between the local time when the leaf switch 108E receives the packet and the timestamp value when the packet enters the networked computing environment on the leaf switch 108D.

[0053] Furthermore, computing resource 112A can calculate the end-to-end latency associated with packets 212 of the entire data flow 210 traversing the networked computing environment 102. The end-to-end latency can be calculated, at least in part, based on the summation of all latencies calculated for the individual devices and / or segments of the data flow 210 as described above. Additionally or alternatively, the end-to-end latency can be calculated by determining the difference between the ingress timestamp when leaf switch 108A receives packet 212 and the egress timestamp when leaf switch 108E exports packet 212 to one or more networks 122.

[0054] Figure 3 and Figure 4 A flowchart of an example method is shown, which illustrates at least in part the... Figure 1-2C This document describes various aspects of the functions performed by the various devices described herein. (References: [link to reference]) Figure 3 and Figure 4 The described logical operations can be implemented as (1) a sequence of actions or program modules implemented by a computer running on a computing system, and / or (2) interconnected machine logic circuits or circuit modules within the computing system.

[0055] The implementation of the various components described herein depends on the performance and other requirements of the computing system. Therefore, the logical operations described herein are referred to differently as operations, structural devices, actions, or modules. These operations, structural devices, actions, and modules can be implemented using software, firmware, special-purpose digital logic, and any combination thereof. It should also be understood that it is possible to perform operations that are more complex than... Figure 3 and Figure 4The operations shown and described herein may include more or fewer operations. These operations may also be performed in parallel or in a different order than those described herein. Some or all of these operations may also be performed by components other than those specifically identified. Although the techniques described in this disclosure are referenced to specific components, in other examples, these techniques may be implemented with fewer components, more components, different components, or any configuration of components.

[0056] Figure 3 A flowchart of an example method 300 for determining the latency associated with packets of data streams that traverse a networked computing environment utilizing a service chain is shown.

[0057] Method 300 begins at step 302, during which first timestamp data is received from the ingress leaf switch of the networked computing environment. The first timestamp data can indicate the first time a packet enters the data stream of the networked computing environment. In various examples, the ingress leaf switch can correspond to... Figure 1-2C Leaf switch 108A. The first timestamp data can be received by computing resources in a networked computing environment, such as computing resource 112A. Alternatively or additionally, the first timestamp data can be received by another device in the networked computing environment, such as a switch, router, computer, or other device capable of executing software to determine the packet path and latency of the data stream.

[0058] In 304, method 300 includes receiving first stream data associated with a data stream from a first intermediate leaf switch in a networked computing environment. In some examples, the first intermediate leaf switch may correspond to... Figure 1-2C The leaf switch is either 108B or 108C. The first-stream data may include one or more of the following: VRF table, 5-tuple stream data, ingress and / or egress physical interface data, ingress and / or egress logical interface data, service chain configuration data, next-hop information, TCP sequence number, IP header identification field, etc. The first-stream data may be received by computing resources in a networked computing environment, such as computing resource 112A.

[0059] In 306, method 300 includes determining, at least in part, that the first intermediate leaf switch is configured to export packets to a service chain device located outside the networked computing environment, based on first stream data. For example, the first stream data may indicate that packets associated with a data stream received by the first intermediate leaf switch will be exported to a service chain device. In some examples, the service chain device may correspond to... Figure 1-2C The service chain device is either 114A or 114B. Service chain devices can include firewalls, TCP optimizers, CGNAT, parental controls, DPI, load balancers, etc.

[0060] In 308, method 300 includes receiving second stream data associated with the data stream from a second intermediate leaf switch in a networked computing environment. In some examples, the second intermediate leaf switch may correspond to... Figure 1-2C The leaf switch is either 108C or 108D. The second-stream data may include one or more of the following: VRF table, 5-tuple stream data, ingress and / or egress physical interface data, ingress and / or egress logical interface data, service chain configuration data, next-hop information, TCP sequence number, IP header identification field, etc. The second-stream data may be received by computing resources in a networked computing environment such as computing resource 112A.

[0061] In method 300, at least in part, the second intermediate leaf switch determines that it received the packet when it returned from the service link device to the networked computing environment, based on second stream data. For example, the second stream data may indicate that the packet associated with the data stream received by the second intermediate leaf switch was sent by the service link device. Additionally or optionally, the second stream data may include data corresponding to the first stream data, such as TCP sequence numbers, 5-tuple stream data, IP header identification fields, etc. In this way, determining that the second intermediate leaf switch received the packet may be based at least in part on first data of the first stream data corresponding to the second data of the second stream data. For example, the first data and the second data may include the same IP header identification fields, the same TCP sequence numbers, etc.

[0062] In 312, method 300 includes receiving second timestamp data from an egress leaf switch of the networked computing environment, indicating a second time in which a data stream exits the networked computing environment. In various examples, the egress leaf switch may correspond to... Figure 1-2C The leaf switches 108N and / or 108E are used. The second timestamp data can be received by computing resources in a networked computing environment, such as computing resource 112A. Additionally or alternatively, the second timestamp data can be received by another device in the networked computing environment, such as a switch, router, computer, or other device capable of executing software to determine the packet path and latency of the data stream.

[0063] In 314, method 300 includes calculating a delay associated with packets of a data stream traversing a networked computing environment, based at least in part on first and second timestamp data. For example, the delay can be determined by calculating the difference between the first and second timestamp data. Additionally or alternatively, individual delays associated with various devices and / or segments of the data stream can be determined and added together to calculate the delay.

[0064] In some examples, if the latency exceeds a threshold latency, one or more devices in the data stream causing the extended latency can be identified. In this way, the packetization path of the data stream can be adjusted to exclude one or more devices causing the extended latency, or one or more devices can be repaired or replaced.

[0065] Figure 4 A flowchart of an example method 400 for determining latency associated with a service chain device is shown. Method 400 can be derived from the above reference. Figure 1-2C The one or more devices and Figure 1-2C Other devices not shown in the diagram are used to achieve this.

[0066] In 402, method 400 includes receiving first-flow data associated with a data flow from a first-leaf switch of a networked computing environment. The first-flow data may include one or more of the following: a VRF table, 5-tuple flow data, ingress and / or egress physical interface data, ingress and / or egress logical interface data, service chain configuration data, next-hop information, TCP sequence number, IP header identification field, etc. The first-flow data may indicate one or more next hops for packets routed through the first-leaf switch.

[0067] In method 404, method 400 includes receiving second-stream data associated with a data stream from a second-leaf switch in a networked computing environment. Similar to the first-stream data, the second-stream data may include one or more of the following: a VRF table, 5-tuple stream data, ingress and / or egress physical interface data, ingress and / or egress logical interface data, service chain configuration data, next-hop information, TCP sequence number, IP header identification field, etc. The second-stream data may indicate one or more next hops for packets routed through the second-leaf switch.

[0068] In 406, method 400 includes determining a packet path associated with the data stream, based at least in part on the first and second stream data, the packet path indicating that a first leaf switch is configured to send packets to the service chain device and a second leaf switch is configured to receive packets from the service chain device. In some examples, the service chain device is located outside the networked computing environment. The service chain device may include a firewall, TCP optimizer, CGNAT, parental controls, DPI, load balancer, etc.

[0069] At 408, method 400 includes receiving from a first leaf switch first timestamp data indicating a first time when a packet was sent to a serving chain device. For example, the first timestamp data may correspond to an egress time associated with the first leaf switch that sent the packet. At 410, method 400 includes receiving from a second leaf switch second timestamp data indicating a second time when a packet was received from the serving chain device. For example, the second timestamp data may correspond to an ingress time associated with the second leaf switch that received the packet.

[0070] In method 400, at least in part, a delay associated with the service chain device is calculated based on first timestamp data and second timestamp data. For example, the difference between the first and second timestamp data can be calculated to determine the delay. Furthermore, the delay calculation can also be based at least in part on the packet path associated with the data flow. For example, when a packet is received by a second leaf switch, the packet may include a new packet header, and IP header identification fields, TCP sequence numbers, etc., can be used to determine that the received packet is the same packet sent from the first leaf switch to the service chain device. Additionally, the delay calculation can also be based at least in part on packet identification data associated with the packet received from both the first and second leaf switches. In some examples, the packet identification data may include 5-tuple stream data, TCP sequence numbers, IP header identification fields, etc.

[0071] Figure 5 The computing system diagram illustrates the configuration of a data center 500 that can be used to implement various aspects of the techniques disclosed herein. Figure 5 The example data center 500 shown includes several server computers 502A-502F (which may be referred to herein as "server computer 502" in either the singular or plural form) for providing computing resources. In some examples, the resources and / or server computers 502 may include or correspond to any type of computing device described herein. Although described as a server, server computer 502 may include any type of networking device, such as a server, switch, router, hub, bridge, gateway, modem, repeater, access point, etc.

[0072] Server computer 502 may be a standard tower, rack-mount, or blade server computer appropriately configured to provide computing resources. In some examples, server computer 502 may provide computing resources 504, including data processing resources (e.g., VM instances or hardware computing systems, database clusters, compute clusters, storage clusters, data storage resources, database resources, network resources, VPNs, etc.). Some servers 502 may also be configured to execute a resource manager 506 capable of instantiating and / or managing computing resources. For example, in the case of VM instances, resource manager 506 may be a hypervisor or other types of programs configured to enable the execution of multiple VM instances on a single server computer 502. Server computer 502 in data center 500 may also be configured to provide network services and other types of services.

[0073] exist Figure 5 In the example data center 500 shown, server computers 502A-502F are also interconnected using appropriate LAN 508. It should be understood that the configuration and network topology described herein have been greatly simplified, and more computing systems, software components, networks, and networking devices can be used to interconnect the various computing systems disclosed herein and provide the aforementioned functionality. Appropriate load balancing devices or other types of network infrastructure components can also be used to balance the load between data centers 500, between each server computer 502A-502F in each data center 500, and potentially among the computing resources in each server computer 502. It should be understood that references... Figure 5 The configuration of the data center 500 described is merely illustrative and may be implemented using other methods.

[0074] In some cases, data center 500 can provide computing resources permanently or on demand, such as packet path visibility, packet loss, latency measurement results, VM instances, VPN instances, and storage. In other types of functions, computing resources provided by the cloud computing network can be used to implement the various services and technologies mentioned above. Computing resources 504 provided by the cloud computing network can include various types of computing resources, such as data processing resources like tenant containers and VM instances, data storage resources, networking resources, data communication resources, network services, VPN instances, etc.

[0075] Each type of computing resource 504 provided by the cloud computing network can be general-purpose or available in multiple specific configurations. For example, data processing resources can be used as physical computers or VM instances in a variety of different configurations. VM instances can be configured to run applications, including web servers, application servers, media servers, database servers, some or all of the aforementioned network services, and / or other types of programs. Data storage resources can include file storage devices, block storage devices, etc. The cloud computing network can also be configured to provide other types of computing resources 504 not specifically mentioned herein.

[0076] The computing resources 504 provided by the cloud computing network, in one embodiment, may be enabled by one or more data centers 500 (which may be referred to herein as "data center 500" in the singular or plural form). A data center 500 is a facility for housing and operating computer systems and related components. A data center 500 typically includes redundant and backup power, communication, cooling, and security systems. Data centers 500 may also be located in geographically distinct locations. References will follow. Figure 6 This describes an illustrative embodiment of a data center 500 that can be used to implement the techniques disclosed herein.

[0077] Figure 6 The computer architecture diagram illustrates an example computer hardware architecture 600 for implementing a computing device that can be used to implement various aspects of the various technologies presented herein. Figure 6 The computer hardware architecture 600 shown illustrates a conventional server computer 502, computing resources 112A-112N, network devices (e.g., leaf switches 108A-108N, spine switches 110A-110N, service chain devices 114A-114N, etc.), workstations, desktop computers, laptop computers, tablet computers, network equipment, e-readers, smartphones, or other computing devices, and can be used to execute any of the software components presented herein. In some examples, computer 600 may correspond to network device 102 described herein and may include networking devices such as servers, switches, routers, hubs, bridges, gateways, modems, repeaters, access points, etc.

[0078] Computer 600 includes a substrate 602 or “motherboard”, which is a printed circuit board, to which multiple components or devices can be connected via a system bus or other electrical communication paths. In one illustrative configuration, one or more central processing units (“CPU”) 604 operate together with chipset 606. CPU 604 may be a standard programmable processor that performs the arithmetic and logic operations required to perform the operation of computer 600.

[0079] The CPU 604 performs operations by manipulating switching elements that distinguish and change these states, transitioning from one discrete physical state to the next. Switching elements typically include electronic circuitry, such as flip-flops, that maintains one of two binary states, and electronic circuitry that provides an output state based on a logical combination of the states of one or more other switching elements, such as logic gates. These basic switching elements can be combined to create more complex logic circuits, including registers, adder-subtractor units, arithmetic logic units, floating-point units, and so on.

[0080] Chipset 606 provides an interface between CPU 604 and the remaining components and devices on substrate 602. Chipset 606 can provide an interface to RAM 608, which serves as the main memory in computer 600. Chipset 606 can also provide an interface to computer-readable storage media such as read-only memory (“ROM”) 610 or non-volatile RAM (“NVRAM”), which stores basic routines that help boot computer 600 and transfer information between various components and devices. Depending on the configuration described herein, ROM 610 or NVRAM can also store other software components required for the operation of computer 600.

[0081] Computer 600 can operate in a networked environment using logical connections to remote computing devices and computer systems via a network (e.g., network 106). Chipset 606 may include functionality for providing network connectivity via a network interface controller (NIC) 612 (e.g., a Gigabit Ethernet adapter). NIC 612 enables computer 600 to connect to other computing devices via network 106. It should be understood that multiple NICs 612 may exist in computer 600, connecting the computer to other types of networks and remote computer systems. In some examples, NIC 612 may be configured to perform at least some of the techniques described herein, such as packet redirection and / or other techniques described herein.

[0082] Computer 600 can be connected to storage device 618, which provides non-volatile storage for the computer. Storage device 618 can store operating system 620, programs 622, and data, which have been described in more detail herein. Storage device 618 can be connected to computer 600 via storage controller 614 connected to chipset 606. Storage device 618 may include one or more physical storage units. Storage controller 614 can interface with physical storage units via interfaces such as Serial Attached SCSI (“SAS”), Serial Advanced Technology Connect (“SATA”), Fibre Channel (“FC”), or other types of interfaces used for physical connections and data transfer between the computer and physical storage units.

[0083] Computer 600 can store data on storage device 618 by transforming the physical state of physical storage units to reflect the stored information. In different embodiments of this specification, the specific transformation of the physical state may depend on various factors. Examples of these factors may include, but are not limited to, the technology used to implement the physical storage units, whether storage device 618 is characterized as a primary storage device or a secondary storage device, etc.

[0084] For example, computer 600 can store information in storage device 618 by issuing instructions via storage controller 614 to change the magnetic properties of a specific location within a disk drive unit, the reflection or refraction properties of a specific location in an optical storage unit, or the electrical properties of a specific capacitor, transistor, or other discrete component in a solid-state storage unit. Other transformations of the physical medium are also possible without departing from the scope and spirit of this specification; the foregoing examples are provided for illustrative purposes only. Computer 600 can also read information from storage device 618 by detecting the physical state or characteristics of one or more specific locations within the physical storage unit.

[0085] In addition to the aforementioned high-capacity storage device 618, computer 600 may also access other computer-readable storage media to store and retrieve information, such as program modules, data structures, or other data. Those skilled in the art will understand that a computer-readable storage medium is any available medium that provides non-transitory storage of data and can be accessed by computer 600. In some examples, operations performed by network 106 and / or any components included therein may be supported by one or more devices similar to computer 600. In other words, some or all of the operations performed by network 106 and / or any components included therein may be performed by one or more computer devices 600 operating in a cloud-based configuration.

[0086] By way of example and not limitation, computer-readable storage media can include volatile and non-volatile, removable and non-removable media implemented in any method or technology. Computer-readable storage media include, but are not limited to, RAM, ROM, erasable programmable ROM (“EPROM”), electrically erasable programmable ROM (“EEPROM”), flash memory or other solid-state memory technologies, compact disc ROM (“CD-ROM”), digital versatile disc (“DVD”), high-definition DVD (“HD-DVD”), BLU-RAY or other optical storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information in a non-transitory manner.

[0087] As briefly described above, storage device 618 can store operating system 620 for controlling the operation of computer 600. According to one embodiment, the operating system includes a LINUX operating system. According to another embodiment, the operating system includes one from Microsoft Corporation of Redmond, Washington. SERVER operating system. According to another embodiment, the operating system may include one of the UNIX operating systems or variants thereof. It should be understood that other operating systems may also be used. Storage device 618 may store other systems or applications and data used by computer 600.

[0088] In one embodiment, storage device 618 or other computer-readable storage medium is encoded with computer-executable instructions that, when loaded into computer 600, transform the computer from a general-purpose computing system into a special-purpose computer capable of implementing the embodiments described herein. As described above, these computer-executable instructions transform computer 600 by specifying how CPU 604 transitions between states. According to one embodiment, computer 600 has access to a computer-readable storage medium storing computer-executable instructions that, when executed by computer 600, perform the above-described... Figure 1-4 The various processes described herein. Computer 600 may also include a computer-readable storage medium having instructions stored thereon for performing any other computer-implemented operations described herein.

[0089] Computer 600 may also include one or more input / output controllers 616 for receiving and processing input from multiple input devices such as a keyboard, mouse, touchpad, touchscreen, electronic pen, or other types of input devices. Similarly, input / output controllers 616 may provide output to a display such as a computer monitor, flat panel display, digital projector, printer, or other types of output devices. It should be understood that computer 600 may not include... Figure 6 All components shown may include Figure 6 Other components not explicitly shown in the document, or those that can be utilized with Figure 6 The architecture shown is completely different.

[0090] As described herein, computer 600 may include one or more of client device 106 or network devices (e.g., server computer 502, computing resource 114, router 110, etc.). Computer 600 may include one or more hardware processors 604 (processors) configured to execute one or more stored instructions. The processors 604 may include one or more cores. Furthermore, computer 600 may include one or more network interfaces configured to provide communication between computer 600 and other devices, such as the communication performed by client device 106 and computing resource 114 as described herein. Network interfaces may include devices configured to couple to personal local area networks (PANs), wired and wireless local area networks (LANs), wired and wireless wide area networks (WANs), etc. For example, network interfaces may include interfaces with Ethernet, Wi-Fi, etc. TM Compatible devices.

[0091] Program 622 may include any type of program or process that performs the techniques described in this disclosure to determine packet path visibility, packet loss, and latency measurements associated with data flows in a networked computing environment utilizing service chains. Program 622 may enable leaf switch 108, spine switch 110, and / or computing resource 112 to perform various operations.

[0092] In summary, this paper discloses techniques for determining packet path visibility, packet loss, and latency measurements associated with data flows in a networked computing environment. These techniques may include receiving streaming data associated with a data flow in the networked computing environment and determining packet paths associated with that data flow. The packet path may indicate that a first leaf switch is configured to send packets to a service chain device, and a second leaf switch is configured to receive packets from the service chain device. These techniques may also include receiving timestamp data indicating a first time when the first leaf switch sends packets to the service chain device and a second time when the second leaf switch receives packets from the service chain device. Latency associated with the service chain device can be calculated, at least in part, based on the timestamp data.

[0093] While the invention has been described with reference to specific examples, it should be understood that the scope of the invention is not limited to these specific examples. Since other modifications and variations to suit specific operational requirements and environments will be readily apparent to those skilled in the art, the invention is not to be considered limited to the examples chosen for purposes of disclosure, but rather covers all changes and modifications that do not depart from the true spirit and scope of the invention.

[0094] Although this application describes embodiments with specific structural features and / or methodological actions, it should be understood that the claims are not necessarily limited to the specific features or actions described. Rather, the specific features and actions are merely illustrative embodiments falling within the scope of the claims of this application.

Claims

1. A method for a networked computing environment, comprising: Receive first timestamp data from the ingress leaf switch of the networked computing environment, the first timestamp data indicating the first time the packet enters the data stream of the networked computing environment; Receive first stream data associated with the data stream from a first intermediate leaf switch in the networked computing environment; Receive first service chain configuration data associated with the data stream from the first intermediate leaf switch; Based at least in part on the first stream data and the first service chain configuration data, it is determined that the first intermediate leaf switch is configured to export the packets to a service chain device located outside the networked computing environment; Receive second stream data associated with the data stream from the second intermediate leaf switch in the networked computing environment; Receive second service chain configuration data associated with the data stream from the second intermediate leaf switch; Based at least in part on the second stream data and the second service chain configuration data, it is determined that the second intermediate leaf switch received the packet when the packet returned from the service chain device to the networked computing environment; Receive second timestamp data from the egress leaf switch of the networked computing environment, the second timestamp data indicating the second time at which the packet exits the data stream of the networked computing environment; as well as The latency associated with the data stream traversing the networked computing environment by the group is calculated, based at least in part on the first timestamp data and the second timestamp data.

2. The method according to claim 1, further comprising: Receive third-stream data associated with the data stream from the inlet leaf switch; Receive fourth stream data associated with the data stream from the exit leaf switch; Based at least in part on the first stream data, the second stream data, the third stream data, and the fourth stream data, a packet path associated with the data stream traversing the networked computing environment by the packet is determined; and The calculation of the delay is also based at least in part on the packet path.

3. The method according to claim 1, further comprising: Receive third timestamp data from the first intermediate leaf switch, the third timestamp data indicating that the first intermediate leaf switch will export the packet to the third time of the service chain device; Receive fourth timestamp data from the second intermediate leaf switch, the fourth timestamp data indicating the fourth time at which the second intermediate leaf switch received the packet from the service chain device; as well as Calculate the second delay associated with exporting the packet to the service chain device.

4. The method according to any one of claims 1 to 3, wherein, The first stream data and the second stream data include at least one of the following: TCP sequence number, Internet Protocol (IP) header identification field, or 5-tuple stream.

5. The method according to any one of claims 1 to 3, wherein, The first stream data and the second stream data include at least one of the following: a virtual routing and forwarding (VRF) table, an ingress interface, or an egress interface. The method further includes determining, at least in part, a packet path associated with a data stream traversing the networked computing environment based on the first stream data and the second stream data.

6. The method according to any one of claims 1 to 3, wherein, The first service chain configuration data indicates that the first intermediate leaf switch is configured to export the packet to the service chain device, and the second service chain configuration data indicates that the second intermediate leaf switch is configured to receive the packet from the service chain device.

7. A system for a networked computing environment, comprising: One or more processors; as well as One or more non-transitory computer-readable media storing instructions, which, when executed by the one or more processors, cause the one or more processors to perform operations, the operations including: Receive first stream data associated with the data stream from the first leaf switch of the networked computing environment; Receive second stream data associated with the data stream from the second leaf switch of the networked computing environment; Receive service chain configuration data associated with the data stream from at least one of the first leaf switch or the second leaf switch; Based at least in part on the first stream data, the second stream data, and the service chain configuration data, a packet path associated with the data stream is determined, the packet path indicating that the first leaf switch is configured to send packets to the service chain device, and the second leaf switch is configured to receive the packets from the service chain device, wherein the service chain device is located outside the networked computing environment; Receive first header data associated with a first packet of the data stream from the first leaf switch, the first packet being sent by the first leaf switch to the service chain device; Receive second header data associated with a second packet of the data stream from the second leaf switch, the second packet being received by the second leaf switch from the service chain device; Based at least in part on the packet path, the first header data, and the second header data, it is determined that the first packet and the second packet are the same packet; The latency associated with the service chain device is calculated based at least in part on the first timestamp data included in the first header data and the second timestamp data included in the second header data.

8. The system according to claim 7, wherein, The first header data and the second header data include at least one of the following: TCP sequence number, Internet Protocol (IP) header identification field, or 5-tuple stream.

9. The system according to claim 7, wherein, At least a portion of the first header data is different from the second header data.

10. The system according to any one of claims 7 to 9, wherein, The delay includes a first delay, and the operation further includes: Receive third timestamp data from the first leaf switch, the third timestamp data indicating a second delay associated with the first leaf switch; Receive fourth timestamp data from the second leaf switch, the fourth timestamp data indicating a third delay associated with the second leaf switch; and The end-to-end delay associated with traversing the data stream with the same group is calculated, based at least in part on the first delay, the second delay, and the third delay.

11. The system according to any one of claims 7 to 9, wherein, The first stream data or the second stream data includes at least one of the following: a Virtual Routing and Forwarding (VRF) table, an ingress interface, an egress interface, or a 5-tuple stream.

12. The system according to any one of claims 7 to 9, wherein, The service chain configuration data indicates that the first leaf switch is configured to send the packet to the service chain device, and the second leaf switch is configured to receive the packet from the service chain device.

13. A method for a networked computing environment, comprising: Receive streaming data associated with the data stream from multiple leaf switches in the networked computing environment; Receive service chain configuration data associated with the data stream from the plurality of leaf switches; The packet path associated with the data stream is determined at least in part based on the streaming data and the service chain configuration data. The packet path indicates that a first leaf switch among the plurality of leaf switches is configured to send packets to the service chain device, and a second leaf switch among the plurality of leaf switches is configured to receive the packets from the service chain device, wherein the service chain device is located outside the networked computing environment. Receive first header data associated with a first packet of the data stream from the first leaf switch, the first packet being sent by the first leaf switch to the service chain device; The second header data associated with a second packet of the data stream is received from the service chain device by the second leaf switch, and the second header data is different from the first header data. Based at least in part on the packet path, the first header data, and the second header data, it is determined that the first packet and the second packet are the same packets including different header data; and The latency associated with the service chain device is calculated based at least in part on the first timestamp data included in the first header data and the second timestamp data included in the second header data.

14. The method according to claim 13, wherein, The first header data and the second header data further include at least one of the following: TCP sequence number, Internet Protocol (IP) header identification field, or 5-tuple stream.

15. The method according to claim 13, wherein, The delay includes a first delay, and the method further includes: Determine the second delay associated with the first leaf switch; Determine the third delay associated with the second leaf switch; and The end-to-end delay associated with traversing the data stream with the same group is calculated, based at least in part on the first delay, the second delay, and the third delay.

16. The method according to any one of claims 13 to 15, wherein, The stream data includes at least one of the following: a Virtual Routing and Forwarding (VRF) table, an ingress interface, an egress interface, or a 5-tuple stream.

17. The method according to any one of claims 13 to 15, wherein, The service chain configuration data indicates that the first leaf switch is configured to send packets of the data stream to the service chain device and the second leaf switch is configured to receive packets of the data stream from the service chain device.

18. An apparatus for a networked computing environment, comprising: A module for receiving first timestamp data from the ingress leaf switch of the networked computing environment, the first timestamp data indicating the first time a packet enters the data stream of the networked computing environment; A module for receiving first stream data associated with the data stream from a first intermediate leaf switch in the networked computing environment; A module for receiving first service chain configuration data associated with the data stream from the first intermediate leaf switch; A module for determining, at least in part, based on the first stream data and the first service chain configuration data, that the first intermediate leaf switch is configured to export the packets to a service chain device located outside the networked computing environment; A module for receiving second stream data associated with the data stream from a second intermediate leaf switch in the networked computing environment; A module for receiving second service chain configuration data associated with the data stream from the second intermediate leaf switch; A module for determining, at least in part, based on the second stream data and the second service chain configuration data, that the second intermediate leaf switch receives the packet when the packet returns from the service chain device to the networked computing environment; A module for receiving second timestamp data from the egress leaf switch of the networked computing environment, the second timestamp data indicating a second time when the packet exits the data stream of the networked computing environment; as well as A module for calculating the latency associated with the data stream traversing the networked computing environment by the group, based at least in part on the first timestamp data and the second timestamp data.

19. The apparatus of claim 18, further comprising a module for implementing the method of any one of claims 2 to 6.

20. An apparatus for a networked computing environment, comprising: A module for receiving streaming data associated with a data stream from multiple leaf switches in the networked computing environment; A module for receiving service chain configuration data associated with the data stream from the plurality of leaf switches; A module for determining a packet path associated with the data stream based at least in part on the stream data and the service chain configuration data, the packet path indicating that a first leaf switch among a plurality of leaf switches is configured to send packets to a service chain device, and a second leaf switch among a plurality of leaf switches is configured to receive packets from the service chain device, wherein the service chain device is located outside the networked computing environment; A module for receiving first header data associated with a first packet of the data stream from the first leaf switch, the first packet being sent by the first leaf switch to the service chain device; A module for receiving second header data associated with a second packet of the data stream from the second leaf switch, the second packet being received by the second leaf switch from the service chain device, the second header data being different from the first header data; A module for determining, at least in part, that the first packet and the second packet are the same packet including different header data, based on the packet path, the first header data, and the second header data; and A module for calculating the latency associated with the service chain device based at least in part on the first timestamp data included in the first header data and the second timestamp data included in the second header data.

21. The apparatus of claim 20, further comprising a module for implementing the method of any one of claims 14 to 17.

22. A computer program product or computer-readable medium comprising instructions that, when executed by a computer, cause the computer to perform the steps of the method according to any one of claims 1 to 6 or 13 to 17.

Images