Proactive and targeted data plane traffic monitoring for wired networks

By directly analyzing the encapsulated packet header of the actual service flow in a wired network, the load and complexity problems caused by the synthetic service flow method are solved, and more accurate and efficient network service monitoring and damage detection are achieved.

CN112956159BInactive Publication Date: 2024-11-12CISCO TECHNOLOGY INC
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
CN201980071409.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-04-22
Filing Date
2019-11-06
Publication Date
2024-11-12
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In wired networks, methods of monitoring network traffic flows and detecting network damage often rely on synthetic traffic flows, resulting in increased loading of control planes and data planes, increasing complexity, and potentially providing inaccurate results.

Method used

By receiving and analyzing the encapsulated packet header of the actual service flow at the network node, determining the source address, timestamp and serial number to monitor the delay, jitter and loss of data packets, truncating the header copy from the network node and sending it to the analysis platform , avoid generating synthetic business flows.

Benefits of technology

Reduces control plane and data plane load, reduces the complexity of network monitoring, performance measurement and damage detection, provides more accurate network service analysis, and improves monitoring flexibility and fineness.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112956159B_ABST
    Figure CN112956159B_ABST
Patent Text Reader

Abstract

Systems, methods, and computer-readable media for monitoring traffic in a network include: receiving, at an analysis platform connected to the network, one or more encapsulated packet headers from one or more network nodes of the network. Based on the one or more encapsulated packet headers, at least one or more source addresses of the one or more network nodes that sent the encapsulated packet headers and one or more timestamps when the one or more encapsulated packet headers were sent from the one or more network nodes can be determined. Based at least on the one or more source addresses and timestamps, network traffic information, such as one or more of a delay or jitter of a data packet passing through the one or more network nodes, can be determined, wherein the one or more encapsulated packet headers can correspond to the data packets passing through the one or more network nodes.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of and priority to U.S. Non-Provisional Patent Application No. 16 / 390,695, filed on April 22, 2019, entitled “ACTIVE TARGETED DATA PLANETRAFFIC MONITORING FOR WIRED NETWORKS,” and claims the benefit of U.S. Provisional Patent Application No. 62 / 769,290, filed on November 19, 2018, entitled “ACTIVE TARGETED DATA PLANE TRAFFIC MONITORING FOR WIRED NETWORKS,” which are incorporated herein by reference in their entireties. Technical Field

[0003] The disclosed subject matter relates generally to the field of computer networking and, more particularly, to monitoring and analysis of network performance and detection of network impairments that adversely affect communication sessions. Background Art

[0004] Currently, it is very difficult, time-consuming, laborious and error-prone to monitor network traffic flows at multiple locations in a wired network and then correlate the results to determine the meaningful impact of any network impairments on actual end-user network traffic flows. Examples of currently utilized methods include generating synthetic traffic flows in a network to simulate user traffic flows. Metrics measured for these synthetic traffic flows are analyzed to measure network performance, determine packet loss or transmission rate, and detect network impairments. However, reliance on synthetic traffic flows introduces increased control plane and data plane loads, increased overhead, and additional analysis complexity. In addition, reliance on synthetic traffic flows to monitor traffic flows may be deceptive because actual or real user traffic may exhibit characteristics that are different from those of the synthetic traffic flows. Therefore, monitoring synthetic traffic flows may provide inaccurate results. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] In order to describe the manner in which the above and other advantages and features of the present disclosure can be obtained, a more particular description of the principles briefly described above will be presented by reference to specific embodiments thereof illustrated in the accompanying drawings. Understanding that these drawings depict only exemplary embodiments of the present disclosure and are therefore not to be considered limiting of its scope, the principles herein will be described and illustrated with additional specificity and detail through the use of the accompanying drawings, in which:

[0006] Figure 1 illustrates an example of a physical topology of an enterprise network according to some examples;

[0007] Figure 2illustrates an example of a logical architecture of an enterprise network according to some examples;

[0008] Figure 3 An example network structure and analysis platform for monitoring traffic in a network according to some examples is shown;

[0009] Figure 4 shows an example encapsulated packet header format for providing analysis information according to some examples;

[0010] Figure 5 An example method for monitoring network traffic according to some examples is shown;

[0011] Figure 6 An example network device according to various examples is shown; and

[0012] Figure 7 An example computing device architecture according to some examples is shown. DETAILED DESCRIPTION

[0013] Various embodiments of the present disclosure are discussed in detail below. Although specific implementations are discussed, it should be understood that this is done for illustrative purposes only. Those skilled in the relevant art will recognize that other components and configurations may be used without departing from the spirit and scope of the present disclosure.

[0014] Overview

[0015] Various aspects of the invention are set out in the independent claims and preferred features are set out in the dependent claims. Features of one aspect may apply to each aspect alone or in combination with the other aspects.

[0016] Additional features and advantages of the present disclosure will be set forth in the following description, and in part will be apparent from the description, or may be learned by practicing the principles disclosed herein. The features and advantages of the present disclosure may be realized and obtained by the instruments and combinations particularly pointed out in the appended claims. These and other features of the present disclosure may become more fully apparent from the following description and the appended claims, or may be learned by practicing the principles set forth herein.

[0017] Disclosed herein are systems, methods, and computer-readable media for the following schemes, according to which network services can be monitored based on truncated copies of encapsulated packet headers received from selected packets passing through network nodes. In some examples, a method for monitoring services in a network may include: receiving one or more encapsulated packet headers from one or more network nodes of the network at an analysis platform connected to the network. Based on the one or more encapsulated packet headers, at least one or more source addresses of one or more network nodes that send the encapsulated packet headers, and one or more timestamps when the one or more encapsulated packet headers are sent from one or more network nodes can be determined. In addition, based at least on one or more source addresses and one or more timestamps, network service information (e.g., at least one or more of the following: delay or jitter of data packets passing through one or more network nodes) can be determined, wherein the one or more encapsulated packet headers can correspond to the data packets passing through the one or more network nodes.

[0018] The method may also include determining one or more sequence numbers based on each of the one or more encapsulated packet headers, and determining one or more of the following based on the one or more sequence numbers: loss in one or more encapsulated packet headers or data packets. In some examples, the one or more sequence numbers include at least a first sequence number introduced by a first network node in one or more network nodes, wherein the first network node is an ingress node of the network, and the data packet enters the network from an endpoint located outside the network through the ingress node. The first sequence number is incremented at each sequential network node in the one or more network nodes that the data packet passes through. In some examples, the one or more sequence numbers may also optionally include a second sequence number, wherein the second sequence number is constant between the first network node and the sequential network nodes in the one or more network nodes that the data packet passes through.

[0019] In some cases, each of the one or more encapsulated packet headers may include: a truncated copy of a header of a data packet traversing a corresponding network node in the one or more network nodes. The one or more encapsulated packet headers may be received from the one or more network nodes based on one or more selection criteria provided to the one or more network nodes, wherein the encapsulated packet headers are generated by the one or more network nodes for data packets traversing the one or more network nodes and matching the selection criteria. In some cases, the one or more selection criteria correspond to a granularity of network traffic to be monitored.

[0020] In some cases, one or more encapsulated packet headers are received from a first network node of the one or more network nodes based on one or more selection criteria provided to the first network node and a flag inserted by the first network node. Flag insertion by the first network node is optional, and in cases where such flag insertion is used, the encapsulated packet headers are generated by the first network node for data packets that traversed the first network node and matched the selection criteria. In these cases, the one or more encapsulated packet headers may be received from one or more intermediate network nodes other than the first network node based on data packets that traversed the one or more intermediate network nodes and matched the flag.

[0021] In some examples, a system is provided. The system includes one or more processors and a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more processors, cause the one or more processors to perform operations. The operations may include receiving one or more encapsulated packet headers from one or more network nodes of a network. The operations may also include: based on the one or more encapsulated packet headers, at least determining one or more source addresses of one or more network nodes that send the encapsulated packet headers, and one or more timestamps when the one or more encapsulated packet headers are sent from the one or more network nodes. The operations may also include: based on at least one or more source addresses and one or more timestamps, at least determining one or more of the delay or jitter of data packets passing through the one or more network nodes, wherein the one or more encapsulated packet headers correspond to data packets passing through the one or more network nodes.

[0022] In some cases, the operation may also include: determining one or more sequence numbers based on each of the one or more encapsulated packet headers, and determining one or more of the following based on the one or more sequence numbers: loss in one or more encapsulated packet headers or data packets. In some examples, the one or more sequence numbers include at least a first sequence number introduced by a first network node in one or more network nodes, wherein the first network node is an ingress node of the network, wherein the data packet enters the network from an endpoint located outside the network through the ingress node. The first sequence number is incremented at each sequential network node in the one or more network nodes that the data packet passes through. In some examples, the one or more sequence numbers may also optionally include a second sequence number, wherein the second sequence number is constant between the first network node and the sequential network nodes in the one or more network nodes that the data packet passes through.

[0023] In some cases of the example system, each of the one or more encapsulated packet headers may include: a truncated copy of a header of a data packet that traversed a corresponding network node in the one or more network nodes. In these cases, the one or more encapsulated packet headers may be received from the one or more network nodes based on one or more selection criteria provided to the one or more network nodes, wherein the encapsulated packet headers are generated by the one or more network nodes for the data packets that traversed the one or more network nodes and matched the selection criteria.

[0024] In some cases, the one or more encapsulated packet headers are received from a first network node of the one or more network nodes based on one or more selection criteria provided to the first network node and a flag inserted by the first network node. The flag is optional, and in the case where the flag is inserted by the first network node, the encapsulated packet headers are generated by the first network node for data packets that traversed the first network node and matched the selection criteria. In these cases, the one or more encapsulated packet headers may be received from one or more intermediate network nodes other than the first network node based on data packets that traversed the one or more intermediate network nodes and matched the flag.

[0025] In some examples, a non-transitory machine-readable storage medium is provided. The non-transitory machine-readable storage medium may include instructions configured to cause a data processing device to perform operations. The operations may include: receiving one or more encapsulated packet headers from one or more network nodes of a network; and, based on the one or more encapsulated packet headers, at least determining one or more source addresses of one or more network nodes that send the encapsulated packet headers, and one or more timestamps when the one or more encapsulated packet headers are sent from the one or more network nodes. The operations may also include: determining at least one or more of the delay or jitter of data packets passing through the one or more network nodes based on at least one or more source addresses and one or more timestamps, wherein the one or more encapsulated packet headers correspond to data packets passing through the one or more network nodes.

[0026] Description of Example Embodiments

[0027] Disclosed herein are systems, methods, and computer-readable media for the following schemes, according to which real-time and original traffic flows transmitted between a source node and a destination node can be selectively selected, their headers truncated, and sent to a network controller for network analysis without interfering with the real-time and original traffic flows being forwarded to their destinations. Some example advantages of using real-time and original traffic flows are reduced load on both the control plane and the data plane, and significantly reduced complexity in performing network monitoring, performance measurement, and detecting network impairments.

[0028] Conventional methods for network traffic monitoring include Internet Protocol Service Level Agreement (IP SLA) and other similar methods, which rely on the traffic flow or the traffic flow of simulation generated for the network to be monitored.Synthetic traffic flow is used to simulate user / equipment traffic flow, and these synthetic traffic flows are driven across the network from IP SLA source to receiver (usually hosted on network devices such as switches and routers).By analyzing the various metrics measured for these synthetic traffic flows, determine the quality (for example, delay, jitter, packet loss etc.) of the intermediate network transmission.However, these conventional methods suffer from various shortcomings, which cause inefficiency, inaccuracy and expense increase.Some of these shortcomings are described below.

[0029] Synthesized traffic flows introduce a large amount of control plane load on the source network device that generates the synthesized traffic flow and the sink / destination network device that receives the synthesized traffic flow. Since IP SLA and similar synthesized traffic flows are created and absorbed by the central processing unit or "CPU" of the network device (because they are too complex and / or variable to be directly generated in the dedicated integrated circuit or "ASIC" hardware), the synthesized traffic flows constitute an additional (and usually heavy, large-scale) load on the control plane of the network device, which may have been delegated many other active tasks. As the complexity of the synthesized traffic flow increases (for example, for simulating IP-based video services, compared to simpler IP voice or "VoIP") and / or as the amount of synthesized traffic flows increases, the control plane load may further increase. For this type of large-scale synthetic traffic flow sources and sinks, the size of the typical network device CPU is not suitable.

[0030] Note that the generation of synthetic traffic probes between pairs of nodes in a network for obtaining traffic information may increase exponentially with respect to the number of nodes in the network being monitored. For example, the number of probes that may be generated may be on the order of n(n-1) / 2, where n represents the number of nodes in the network. This exponential increase in synthetic traffic probes may increase the control plane load to the point where synthetic traffic generation may be unsustainable in a large network environment.

[0031] In addition, the data plane load caused by the generated synthetic traffic flow introduces additional complexity and expense. Although bandwidth availability may not be an important limitation in some network situations, bandwidth limitations may be important in some networks such as wide area networks (WANs), remote locations, satellite links, etc. Synthetic traffic flows generate additional data plane traffic loads, which are transmitted end-to-end between source and destination, and are usually bidirectional. Some synthetic traffic flow loads may be extremely large (e.g., simulated video traffic or file transfers), and may directly compete for buffer space and other limited resources with real-time or actual user / device traffic on transit network devices. Synthetic traffic flows may require the same packet headers as actual traffic (e.g., differential service field code point (DSCP) values) to enable sharing of the same queue structure on the network, so synthetic traffic flows may directly compete for queues and buffer structures that may be scarce resources in some network deployments, so synthetic traffic flows also cause additional network load and congestion in the data plane.

[0032] Another disadvantage of composite traffic flows is their inherent complexity. For both normal operation mode and failover mode in the network, composite traffic flows may need to be carefully planned and configured before traffic transmission of the composite traffic flows. This results in an additional burden on network administrators to plan and execute custom composite traffic flows for specific networks and modes, resulting in increased complexity.

[0033] While there is significant value in being able to monitor the network and detect out-of-profile conditions, at least the above and other disadvantages of IP SLA and similar approaches combine to impose constraints on their deployment.

[0034] Example aspects of the present disclosure include monitoring techniques that are designed to overcome the above-mentioned obstacles of conventional techniques while enhancing the flexibility and ease of designing and implementing finely-targeted business tracking and analysis. In particular, the example techniques disclosed herein analyze actual business flows through the network, and do not require the generation and use of synthetic business flows. Therefore, the example monitoring techniques have minimal impact on the control plane and data plane loads of the network. For example, compared with conventional implementations utilizing synthetic business flows such as IPSLA, the example techniques also have lower complexity and are easier to deploy. In addition, the example techniques are flexible because a subset of business flows can be selected for analysis, and the type and granularity of the selected business flows can be customized for specific use cases.

[0035] In one or more examples, techniques for selecting at least a subset of actual traffic flows in a network are disclosed. The selected traffic flows may be annotated in one or more of the ways described herein so that they can be identified at various network nodes. In order to analyze the selected traffic flows, data therein may be truncated to reduce bandwidth and control plane overhead. The truncated information is provided to at least one analysis platform, which may be configured to perform network analysis on traffic flows from various network nodes of the network.

[0036] In one example, a traffic flow is analyzed as it progresses through one or more locations or network nodes on a network infrastructure such as an enterprise network infrastructure. For example, at an ingress node, one or more triggers are used to select at least a subset of the traffic flowing through the network infrastructure. The selected traffic flow or portion thereof is tracked and captured from one or more network nodes to be analyzed (e.g., truncated headers, using encapsulated packet headers such as a remote switching port analyzer or "ERSPAN" (such as ERSPANv3) to preserve bandwidth). In one example, a timestamp and one or more sequence numbers are added to captured packets from the selected traffic flow being tracked, for example, at each network node through which the selected traffic flow is forwarded. Captured information from selected traffic flows is correlated and analyzed at a central analysis platform to enable an integrated presentation of the analysis, for example in an aggregated manner that clearly allows network managers to determine one or more of: actual network traffic trajectories (which network nodes the traffic provably traverses (or is proven to traverse); analysis of possible network impairments (such as latency between nodes or across the entire path, as well as jitter and packet loss) in a holistic end-to-end manner that can reveal what impairments may have occurred and where in the network path of the traffic flow in question the impairments occurred; and in-depth analysis of packet headers of the traffic flow in question to identify protocol-level responses to traffic impairments on the network path (which can be used to determine the application experience of end users and devices).

[0037] Figure 1An example of a physical topology of an enterprise network 100 that can be configured according to various aspects of the present disclosure is shown. For example, the enterprise network 100 may include a wired network whose services can be monitored according to the example techniques herein. In one example, the enterprise network 100 may provide intent-based networking. It should be understood that for the enterprise network 100 and any network discussed herein, additional or fewer nodes, devices, links, networks, or components may exist in similar or alternative configurations. Example embodiments with different numbers and / or types of endpoints, nodes, cloud components, servers, software components, devices, virtual or physical resources, configurations, topologies, services, devices, or deployments are also contemplated herein. In addition, the enterprise network 100 may include any number or type of resources that can be accessed and utilized by an endpoint or network device. The illustrations and examples provided herein are for clarity and simplicity.

[0038] In this example, the enterprise network 100 includes a management cloud 102 and a network fabric 120. Although shown in this example as a network or cloud external to the network fabric 120, the management cloud 102 may alternatively or additionally reside on the organization's premises or in a hosting center (also, hosted by a cloud provider or similar environment). The management cloud 102 may provide a central management plane for building and operating the network fabric 120. The management cloud 102 may be responsible for forwarding configuration and policy distribution as well as device management and analysis. The management cloud 102 may include one or more network controller devices 104, one or more authentication authorization and accounting (AAA) devices 106, one or more wireless LAN controllers (WLCs) 108, and one or more fabric control plane nodes 110. In other embodiments, one or more elements of the management cloud 102 may be co-located with the network fabric 120.

[0039] The network controller device(s) 104 may serve as a command and control system for one or more network structures and may accommodate automated workflows for deploying and managing the network structures. Figure 2 As further discussed, the network controller device(s) 104 may include automation, design, policy, provisioning, and assurance capabilities, among others. In some embodiments, one or more Cisco Digital Network Architecture (Cisco DNA TM ) device can be used as (one or more) network controller devices 104.

[0040] AAA device(s) 106 may control access to computing resources, facilitate enforcement of network policies, audit usage, and provide information necessary for billing services. AAA devices may interact with network controller device(s) 104 and with databases and directories containing information for users, devices, transactions, policies, billing, and the like to provide authentication, authorization, and billing services. In some embodiments, AAA device(s) 106 may utilize Remote Authentication Dial-In User Service (RADIUS) or Diameter to communicate with devices and applications. In some embodiments, one or more AAA devices 106 may communicate with devices and applications using a remote authentication dial-in user service (RADIUS). An Identity Services Engine (ISE) device may be used as the AAA device(s) 106 .

[0041] The WLC(s) 108 can support fabric-enabled access points attached to the network fabric 120, handling traditional tasks associated with WLCs as well as interactions with the fabric control plane for wireless endpoint registration and roaming. In some embodiments, the network fabric 120 can implement a wireless deployment that moves data plane terminations (e.g., virtual extensible LANs or "VXLANs") from a centralized location (e.g., utilizing a previous overlay Control and Provisioning of Wireless Access Points (CAPWAP) deployment) to access points / fabric edge nodes. This can enable distributed forwarding and distributed policy application for wireless traffic while retaining the benefits of centralized provisioning and management. In some embodiments, one or more Wireless controller, Wireless LAN, and / or other Cisco DNA TM -ready wireless controller can be used as (one or more) WLC108.

[0042] The network fabric 120 may include fabric border nodes 122A and 122B (collectively 122), fabric intermediate nodes 124A-D (collectively 124), and fabric edge nodes 126A-F (collectively 126). Although the fabric control plane node(s) 110 are shown as being external to the network fabric 120 in this example, in other embodiments, the fabric control plane node(s) 110 may be co-located with the network fabric 120. In embodiments where the fabric control plane node(s) 110 are co-located with the network fabric 120, the fabric control plane node(s) 110 may include a dedicated node or set of nodes, or the functionality of the fabric control plane node(s) 110 may be implemented by the fabric border node(s) 122.

[0043] The fabric control plane node(s) 110 can be used as a central database for tracking all users, devices, and things as they attach to the network fabric 120 and as they roam. The fabric control plane node(s) 110 can allow the network infrastructure (e.g., switches, routers, WLCs, etc.) to query the database to determine the location of users, devices, and things attached to the fabric, rather than using a flood and learn mechanism. In this way, the fabric control plane node(s) 110 can be used as a single source of truth about where each endpoint attached to the network fabric 120 is located at any point in time. In addition to tracking specific endpoints (e.g., / 32 addresses for IPv4, / 128 addresses for IPv6, etc.), the fabric control plane node(s) 110 can also track larger aggregation routers (e.g., IP / mask). This flexibility can help aggregate across fabric sites and improve overall scalability.

[0044] The fabric border nodes 122 can connect the network fabric 120 to a traditional layer 3 network (e.g., a non-fabric network) or to different fabric sites. The fabric border nodes 122 can also translate context (e.g., user, device, or thing mapping and identity) from one fabric site to another fabric site or to a traditional network. When the encapsulation is the same across different fabric sites, the translation of the fabric context is typically mapped 1:1. The fabric border nodes 122 can also exchange reachability and policy information with fabric control plane nodes at different fabric sites. The fabric border nodes 122 also provide border functions for internal and external networks. The internal border can advertise a defined set of known subnets, such as those leading to a set of branch sites or to a data center. On the other hand, the external border can announce unknown destinations (e.g., to the Internet, which operates similarly to the function of a default route).

[0045] The fabric intermediate nodes 124 may function as pure layer 3 repeaters that connect the fabric border nodes 122 to the fabric edge nodes 126 and provide a layer 3 underlay for fabric overlay traffic.

[0046] The fabric edge nodes 126 can connect endpoints to the network fabric 120 and can encapsulate / decapsulate and forward traffic from these endpoints to and from the network fabric. The fabric edge nodes 126 can operate at the periphery of the network fabric 120 and can be the first point for attaching users, devices, and things and implementing policies. In some embodiments, the network fabric 120 may also include fabric extension nodes (not shown) that are used to attach downstream non-fabricated layer 2 network devices to the network fabric 120 and thereby extend the network fabric. For example, the extension node can be a small switch (e.g., a compact switch, an industrial Ethernet switch, a building automation switch, etc.) that is connected to the fabric edge node via layer 2. Devices or things connected to the fabric extension nodes can use the fabric edge node 126 to communicate to external subnets.

[0047] In this example, the network fabric may represent a single fabric site deployment, which may be distinguished from a multi-site fabric deployment.

[0048] In some embodiments, all subnets hosted in a fabric site can be provisioned on each fabric edge node 126 in the fabric site. For example, if subnet 10.10.10.0 / 24 is provisioned in a given fabric site, that subnet can be defined on all fabric edge nodes 126 in that fabric site, and endpoints located in that subnet can be placed on any fabric edge node 126 in that fabric. This can simplify IP address management and allow for the deployment of fewer but larger subnets. In some embodiments, one or more Catalyst switches, Cisco Switch, Cisco MS switches, Integrated Services Router (ISR), Aggregation Services Router (ASR), Enterprise Network Computing System (ENCS), Cloud Services Virtual Router (CSRv), Cisco Integrated Services Virtual Router (ISRv), Cisco MX devices, and / or other Cisco DNA-ready TM Devices may be used as fabric nodes 122 , 124 , and 126 .

[0049] The enterprise network 100 may also include wired endpoints 130A, 130C, 130D, and 130F and wireless endpoints 130B and 130E (collectively 130). The wired endpoints 130A, 130C, 130D, and 130F may be connected to fabric edge nodes 126A, 126C, 126D, and 126F, respectively, by wires, and the wireless endpoints 130B and 130E may be wirelessly connected to wireless access points 128B and 128E, respectively (collectively 128), which in turn may be connected to fabric edge nodes 126B and 126E, respectively, by wires. In some embodiments, Cisco Access point, Cisco MR access points, and / or other Cisco DNA TM -ready access point can be used as wireless access point 128.

[0050] Endpoints 130 may include general computing devices (e.g., servers, workstations, desktop computers, etc.), mobile computing devices (e.g., laptop computers, tablet computers, mobile phones, etc.), wearable devices (e.g., watches, glasses or other head-mounted displays (HMDs), ear devices, etc.), and the like. Endpoints 130 may also include: Internet of Things (IoT) devices or equipment, such as agricultural equipment (e.g., livestock tracking and management systems, watering equipment, unmanned aerial vehicles (UAVs), etc.); connected cars and other vehicles; smart home sensors and devices (e.g., alarm systems, security cameras, lighting, appliances, media players, HVAC equipment, electric meters, windows, automatic doors, doorbells, locks, etc.); office equipment (e.g., desk phones, copiers, fax machines, etc.); medical equipment (e.g., pacemakers, biometric sensors, medical equipment, etc.); industrial equipment (e.g., robots, factory machinery, construction equipment, industrial sensors, etc.); retail equipment (e.g., vending machines, point of sale (POS) devices, radio frequency identification (RFID) tags, etc.); smart city equipment (e.g., street lights, parking meters, waste management sensors, etc.); transportation and logistics equipment (e.g., turnstiles, rental car trackers, navigation equipment, inventory monitors, etc.); and the like.

[0051] In some embodiments, the network fabric 120 can support wired and wireless access as part of a single integrated infrastructure, so that connectivity, mobility, and policy enforcement behavior are similar or identical for both wired and wireless endpoints. This can bring a unified experience to users, devices, and things independent of the access medium.

[0052] In an integrated wired and wireless deployment, control plane integration can be achieved by notifying the joining, roaming, and disconnection of wireless endpoints 130 by (one or more) WLCs 108 to (one or more) fabric control plane nodes 110 so that (one or more) fabric control plane nodes can have connectivity information about both wired and wireless endpoints in the network fabric 120 and can be used as a single source of truth for endpoints connected to the network fabric. For data plane integration, (one or more) WLCs 108 can instruct the fabric wireless access point 128 to form a VXLAN overlay tunnel to its neighboring fabric edge node 126. The AP VXLAN tunnel can carry segmentation and policy information to and from the fabric edge node 126, thereby allowing the same or similar connectivity and functionality as wired endpoints. When wireless endpoints 130 join the network fabric 120 via the fabric wireless access point 128, (one or more) WLCs 108 can join these endpoints to the network fabric 120 and notify (one or more) fabric control plane nodes 110 of the media access control (MAC) addresses of these endpoints. The (one or more) WLCs 108 can then instruct the fabric wireless access point 128 to form a VXLAN overlay tunnel to the adjacent fabric edge node 126. Next, the wireless endpoint 130 can obtain its own IP address via the Dynamic Host Configuration Protocol (DHCP). Once the above operations are completed, the fabric edge node 126 can register the IP address of the wireless endpoint 130 with the (one or more) fabric control plane node 110 to form a mapping between the MAC address and the IP address of the endpoint, and traffic to and from the wireless endpoint 130 can begin to flow.

[0053] Figure 2 An example of a logical architecture 200 of an enterprise network (e.g., enterprise network 100) is shown. One of ordinary skill in the art will appreciate that additional or fewer components may be present in similar or alternative configurations for the logical architecture 200 and any systems discussed in this disclosure. The illustrations and examples provided in this disclosure are for simplicity and clarity. Other embodiments may include different numbers and / or types of elements, but one of ordinary skill in the art will understand that such variations do not depart from the scope of this disclosure. In this example, the logical architecture 200 includes a management layer 202, a controller layer 220, a network layer 230 (such as embodied by the network structure 120), a physical layer 240 (such as embodied by the network structure 120), and a plurality of network layers 240 (such as embodied by the network structure 120). Figure 1 embodied in various elements), and a shared service layer 250.

[0054] The management layer 202 can abstract the complexity and dependencies of other layers and provide users with tools and workflows for managing an enterprise network (e.g., enterprise network 100). The management layer 202 can include a user interface 204, a design function 206, a policy function 208, a provisioning function 210, an assurance function 212, a platform function 214, and a basic automation function 216. The user interface 204 can provide users with a single point for managing and automating the network. The user interface 204 can be implemented in a web application / web server accessible through a web browser, and / or in an application / application server accessible through a desktop application, a mobile app, a shell or other command line interface (CLI), an application program interface (e.g., static state transfer (REST), simple object access protocol (SOAP), service-oriented architecture (SOA), etc.), and / or other suitable interfaces, in which users can configure cloud-managed network infrastructure, devices, and things; provide user preferences; specify policies, enter data; view statistics; configure interactions or operations; and the like. The user interface 204 can also provide visibility information, such as views of networks, network infrastructure, computing devices, and things. For example, user interface 204 may provide views of: the state or status of the network, ongoing operations, services, performance, topology or layout, implemented protocols, running processes, errors, notifications, alerts, network structure, ongoing communications, data analysis, and the like.

[0055] The design function 206 may include tools and workflows for managing site profiles, maps and floor plans, network settings, and IP address management, etc. The policy function 208 may include tools and workflows for defining and managing network policies. The configuration function 210 may include tools and workflows for deploying networks. The assurance function 212 may use machine learning and analysis to provide end-to-end visibility of the network by learning from network infrastructure, endpoints, and other contextual information sources. The platform function 214 may include tools and workflows for integrating network management systems with other technologies. The basic automation function 216 may include tools and workflows to support the policy function 208, the configuration function 210, the assurance function 212, and the platform function 214.

[0056] In some embodiments, design functions 206, policy functions 208, provisioning functions 210, assurance functions 212, platform functions 214, and foundation automation functions 216 may be implemented as microservices (where the respective software functions are implemented in multiple containers that communicate with each other), rather than merging all tools and workflows into a single software binary. Each of design functions 206, policy functions 208, provisioning functions 210, assurance functions 212, and platform functions 214 may be viewed as a set of related automated microservices to cover the design, policy authoring, provisioning, assurance, and cross-platform integration phases of the network lifecycle. Foundation automation functions 214 may support top-level functions by allowing users to perform certain network-wide tasks.

[0057] Figure 3 An example physical topology of a network 300 configured to implement aspects of the present disclosure is shown. The network 300 may be an enterprise network that can monitor its business in an efficient, cost-effective, and scalable manner. In some aspects, the network 300 may include Figure 1 An example implementation of the enterprise network 100, wherein the logical architecture 200 of the enterprise network 100 is described above with reference to Figure 2 is described.

[0058] For example, network 300 shows a user device or endpoint 302, which may be Figure 1 In the structure of network 300, several network nodes are shown, among which the network node 304a connected to the endpoint 302 can be Figure 1 An example of an ingress node or fabric edge node 126, and the network nodes 304b-c may be Figure 1 The network nodes 304b-c may be connected to one or more network devices 306x-y, which may be examples of network devices such as Figure 1 The one or more network controller devices 104, one or more AAA devices 106, one or more WLCs 108, or one or more fabric control plane nodes 110 of the network 300. The analysis platform 308 configured according to the example aspects for monitoring the traffic of the network 300 can be implemented in one of the above network devices (e.g., in one or more AAA devices 106), or can be implemented in a separate network device.

[0059] In an exemplary aspect, normal traffic flow 312 represents example traffic that originates from a source such as endpoint 302, traverses network nodes 304a-c, and reaches a destination such as network device 306y. Although shown in a direction from source (endpoint 302) to destination (network device 306y) for illustrative purposes, normal traffic flow 312 may be bidirectional.

[0060] Normal traffic flow 302 can be monitored by analysis platform 308. In some aspects, a subset of normal traffic flow 312 can be selected by analysis platform 308 for analysis to reduce the complexity and expense of the analysis. Several ways to select a subset of normal traffic flow 312 will be discussed in the following sections. In addition, data packets from normal traffic flow 312 that are forwarded to analysis platform 308 can be truncated to reduce bandwidth overhead. For example, only truncated headers using ERSPANv3 can be forwarded from multiple locations in network 300 (e.g., network nodes 304a-c).

[0061] In some aspects, timestamps and one or more sequence numbers are associated with captured packets from all traffic flows being tracked, which are performed at each network node through which these traffic flows are forwarded. For example, analysis platform 308 uses these timestamps and one or more sequence numbers to associate packets received from various network nodes and analyze the track of normal traffic flow 312. This analysis can reveal which network nodes normal traffic flow 312 has passed through, for example, providing proof of passing through these nodes. This analysis can also reveal possible network impairments in an overall end-to-end manner, such as, delays and jitter and packet loss between nodes or on the entire path. A more in-depth analysis of the packet header can also reveal the protocol-level response to possible traffic impairments on the network path.

[0062] refer to Figure 3 , when normal traffic flow 312 enters network 300 from endpoint 302, for example, at network node 304a, one or more triggers may be used to select a specific traffic flow or packet or portion thereof for monitoring. One or more such triggers according to the present disclosure may be customized (e.g., defined by a network administrator) to signal a specific traffic flow that may be of interest to be tracked and analyzed. The trigger may include selection criteria that enable flexibility in terms of the type and granularity of the traffic flow of interest. For example, a trigger may be used to select a specific portion of a traffic flow of a VoIP or video stream of interest (e.g., a specific 5-tuple source + 5-tuple destination). In another example, the trigger may be more extensive to select a match of one or more subnets, a security group tag (SGT), a match of a set of SGTs, a match on a L4 port number range, a match on an input interface or subinterface, a virtual network identifier (VNID), or many other possible selectors. Regardless of the specific granularity or selection criteria, an appropriate trigger may be implemented to determine whether a portion of a traffic flow (e.g., one or more packets) at node 304a in this example matches the selection criteria, and if so, the process of monitoring the traffic flow is initiated.

[0063] Once the selected traffic is specified based on the trigger, any traffic matching the criteria may be forwarded as normal upon entering the configured network node in addition to the specific functions performed for monitoring. For example, in some implementations, if a data packet has been selected for monitoring at network node 304a based on a trigger being satisfied, in addition to forwarding the packet along the normal traffic flow 312, a truncated copy of the header of the packet is captured in each node (e.g., nodes 304b-c) through which the traffic flows based on the same trigger.

[0064] In some aspects, the network node 304a-c that generates the truncated copy of the packet header for the packet selected to be monitored may also add a timestamp to the truncated copy of the packet header that indicates the time when the packet passed through the corresponding network node 304a-c during the normal traffic flow 312 of the packet.

[0065] Additionally, in some aspects, the network node may also add one or more sequence numbers to the truncated copy of the packet header before forwarding to the analysis platform 308. In some examples, the first sequence number may also be inserted by the originating network node. The first sequence number may be locally meaningful for each network node and thus be changed at each sequential network node that the packet passes through. For example, for a packet selected for monitoring, the first sequence number may be initialized to an initial value at the network node 304a, and the first sequence number may be incremented at each sequential network node 304b-c that the packet passes through. The first sequence number may be used to determine whether there is a loss of sampled traffic. For example, if a truncated copy of a packet header from a particular network node (e.g., network node 304b) is not received at analysis platform 308, but the packet has traversed normal traffic flow 312 from network node 304a to network node 304c, analysis platform 308 may identify this situation based on the following: truncated copies of packet headers for the packet have been received from network nodes 304a and 304c, but the first sequence number has been incremented twice between network nodes 304a and 304c. This may indicate that the value of the first sequence number that was incremented once was not received, and therefore there may have been a loss of sampled traffic (i.e., the truncated copy of the packet header from network node 304b).

[0066] In addition and optionally, a second sequence number may also be used. The second sequence number may be inserted, for example, by an originating network node such as network node 304a, which selects a packet to be monitored based on a trigger. For example, the second sequence number remains constant for a particular packet during its passage through network 300, and does not change when the packet passes through subsequent network nodes 304b-c. Intermediate or subsequent network nodes 304b-c do not modify or change the second sequence number. In the case where an operation management and maintenance (OAM) bit or a similar header is used for a service in a data plane, a second sequence number may be used. If a specific expected value of the second sequence number is not received at the analysis platform 308, this may indicate that there is a packet loss of a packet with an expected second sequence number. This is distinguished from the first sequence number above, which is applicable to a situation where a sampled service loss can be detected instead of an actual service loss.

[0067] At each of the network nodes 304a-c in the above example, the first sequence number and (optionally) the second sequence number may be added to a truncated copy of the packet header and encapsulated into an ERSPANv3 packet header forwarded to the analysis platform 308. The encapsulated ERSPANv3 packet header from the network nodes 304a-c is already in Figure 3 The first sequence number and the second sequence number may be used to determine whether a packet has traversed the network 300 as expected, or in the event of packet loss, to determine whether the failure to receive a truncated copy of a packet header from one of the network nodes 304a-c indicates that the failure to receive a truncated copy of a packet header is related to a loss rather than an actual packet loss, or indicates that an actual packet loss has occurred in the normal traffic flow 312.

[0068] In an exemplary aspect, a truncated copy of the packet header 310a-c is encapsulated at each of the corresponding network nodes 304a-c with a timestamp and a first sequence number and an optional second sequence number and forwarded to the analysis platform 308 for analysis. For example, a truncated copy of the packet header 310a-c (typically the first 64-96 bytes, but this will be a configurable value in an implementation) can be captured in all nodes 304a-c (e.g., in ASIC hardware) through which the packet flows, and the truncated copy of the packet header 310a-c is encapsulated in ERSPANv3 with the associated timestamp and sequence number and then sent to a designated ERSPAN collector (such as the analysis platform 308) for analysis.

[0069] Leave for a short time Figure 3 And refer to Figure 4, shows an example encapsulated packet header format for one implementation, where an ERSPANv3 format may be used for encapsulated packet headers 310a-c. Figure 4 The illustrated ERSPANv3 packet header is indicated by reference numeral 400. As shown, the ERSPANv3 packet header 400 may include fields illustrated as an ERSPAN Type III (or ERSPANv3) header 402, which may be 12 octets, a platform-specific sub-header 404, which may be an optional 8-octet field, and wherein the above two fields of the composite header may be followed by the original mirrored frame indicated by reference numeral 406 and a standard 4-octet Ethernet cyclic redundancy check (CRC).

[0070] In one implementation, header 402 of ERSPANv3 packet header 400 may include a timestamp in a timestamp 402a field, where the timestamp field utilizes, for example, ASIC-based IEEE 1588 timing capabilities to achieve microsecond accuracy. Platform-specific subheader 404 fields may include a first sequence number and an optional second sequence number in field 404a, which is shown as having been reserved for platform-specific information.

[0071] Now combine the reference Figure 3 to Figure 4 , the truncated copy of the packet header (e.g., encapsulated as an ERSPANv3 packet header 400) may include information useful for traffic analysis, such as source and destination addresses or IPs, ports, flags, Transmission Control Protocol (TCP) sequence numbers, etc. It is recognized that transmitting the complete actual packet data rather than a truncated copy of the packet header to the analysis platform may not only consume more bandwidth, but may also raise privacy issues, which may not be necessary for the purpose of monitoring traffic. This is because the truncated copy of the packet header may omit information that is sensitive to privacy, security, etc. when such information may not be useful in obtaining the desired traffic analysis. For example, the truncated copy of the packet header for the traffic flow being analyzed may include information such as a timestamp and packet sequence number inserted by the network nodes 304a-c, which may not be specific to the data contained in the packet, and thus avoid the above-mentioned privacy / security issues.

[0072] Since the actual user equipment traffic header from the normal traffic flow 312 is used for analysis, the generation and absorption of synthetic traffic flows, such as previously discussed for conventional IP SLAs, is avoided in the exemplary aspects. Accordingly, the operations for implementing the exemplary traffic monitoring techniques can be performed entirely in hardware by the network nodes 304a-c, thereby avoiding the control plane load exhibited by synthetic traffic generation. In this way, the exemplary techniques are scalable on any number of network nodes through which traffic flows pass, without the constraints imposed by control plane load issues.

[0073] When a central point such as analysis platform 308 receives a truncated copy of a packet header (eg, an ERSPAN encapsulated truncated packet header such as ERSPANv3 packet header 400 discussed above), the following analysis of traffic related to the selected packet may be performed.

[0074] In one aspect of the analysis, the IP address of the network node that sent the truncated copy of the packet header to the analysis platform 308 can be analyzed. The IP address can correspond to a loopback address in the network nodes 304a-c, for example, which can effectively determine the network nodes that the actual user / device packets passed through as part of the normal traffic flow 312. This information can be associated with a network map to show the actual (unexpected) path taken by the user traffic, thereby enabling traffic trajectory inspection and analysis in a complex environment. In some aspects, the information about the network nodes that the packets passed through provides a proof of the network path used by the traffic flow involved.

[0075] Another aspect of the analysis can utilize timestamps from the sending network nodes. As previously mentioned, each of the network nodes 304a-c that the selected packet (identified by the trigger) traverses can send an encapsulated, truncated copy of the packet header (e.g., 310a-c) to the analysis platform 308, and include in the encapsulation a timestamp related to the time when the packet traversed the network node. By examining the timestamp and comparing it to the timestamp for the same packet, the analysis platform 308 can obtain useful information related to the transition of the packet through the network 300. For example, the analysis platform 308 can determine that the encapsulated copies of the truncated header from different network nodes 304a-c correspond to the same packet, for example, based on the temporal adjacency of the different encapsulated copies when they are received in conjunction with the TCP sequence numbers. For example, by observing differently encapsulated copies of the same packet at preceding / subsequent network nodes with synchronized clocks, actual inter-packet delay and jitter can be calculated (note that such calculations of actual inter-packet delay and jitter correspond to actual transmitted user traffic in normal traffic flow 312, rather than for simulated or synthetic traffic flows), thereby providing an inherently more accurate and representative analysis of the actual user / device network experience.

[0076] Another aspect of the analysis involves determining whether a truncated copy of a packet header has been captured at a particular network node (e.g., network node 304b in one illustrative example). As previously mentioned, the lack of a truncated copy of a packet header received by the analysis platform 308 from network node 304b may not necessarily itself lead to a conclusion that the actual packets of the user / device traffic being tracked are discarded before reaching network node 304b. This is because the truncated copy of the packet header itself may have been discarded on the way from network node 304b to the analysis platform 308, while the actual packets in the normal traffic flow 312 may have passed through network node 304b to network node 304c and possibly further, as expected. However, if a truncated packet header is received from a subsequent network node 304c in the direction of normal traffic flow 312 (the truncated packet header having the same second sequence number and first sequence number incremented twice as that present in the truncated copy of the packet header from network node 304a), this can be used as a strong indicator that the loss is limited to the loss of the truncated copy of the packet header from network node 304b (having the same second sequence number and first sequence number incremented once), rather than the loss of the original packet itself. Conversely, the failure to capture any subsequent copies of the packet header at any downstream network node can be used as a strong indicator that actual user / device packets are lost at the network node 304b from which the analysis platform 308 observes the loss.

[0077] Thus, by comparing one or more sequence numbers in the encapsulated, truncated copies 310a-c of the packet headers at the analysis platform 308 in the network 300, it can be determined whether the truncated copy or the actual packet has been dropped in the event of a loss of service. Machine learning tools can be used for the above analysis and determination of possible service disruption and loss of service.

[0078] As mentioned previously, the triggers for selecting the packets to be monitored can be customized for different granularities. By using directional triggers for smaller granularities (e.g., specific packets of a traffic flow), targeted troubleshooting tasks can be performed. In other examples, a wider granularity can capture / analyze all traffic in a subnet, for example to enable batch traffic analysis. Non-limiting example selection criteria are provided below, which can be used as triggers for selecting traffic to be monitored. A packet can be triggered by performing a match on the following non-exhaustive list of one or more of the following:

[0079] - the source receiving interface or subinterface (physical or virtual),

[0080] - one or more source / destination IP addresses,

[0081] - Source / destination L4 port number (exact match or range),

[0082] - source / destination subnets,

[0083] - source and / or destination SGT,

[0084] - source virtual routing domain (e.g., virtual routing and forwarding (VRF) domain / instance, virtual private network (e.g., virtual private cloud (VPC), virtual network (VN), etc.), VNID, etc.),

[0085] - Source / destination MAC address (for L2 traffic),

[0086] - matched DSCP value, and / or

[0087] - The matching protocol (TCP, UDP, GRE, etc.).

[0088] Once a match is obtained, for example at network node 304a using selection criteria such as one or more of the above non-exhaustive list of selection criteria, the traffic can be forwarded as usual on the normal traffic flow 312 as discussed above, and an encapsulated, truncated copy of the packet header (e.g., typically the first 64 / 96 bytes) is forwarded to the analysis platform 308.

[0089] In addition to the above triggering options, in some aspects, an application program interface (API) driven dynamic triggering capability can also be configured. For example, a call control system (such as Cisco Unified Communications Manager CUCM, or a similar call control element) can be configured by a system manager or voice system manager of the network 300 so that, for example, traffic from a specific user can be directed for capture (for example, based on the user having reported a problem and customer service personnel wanting to analyze the user's traffic flow through the network 300 to help determine the root cause of the problem).

[0090] Accordingly, in some aspects, with respect to matching the traffic using the above example selection criteria at the first network node (e.g., network node 304a) when the traffic enters the network (which is typically an ingress switch, but alternative aspects may use other nodes designated as the first network node in the desired tracking path), the first network node may optionally set or insert a flag in the transmitted traffic flow. The flag may indicate to other network nodes in the path (e.g., network nodes 304b-c in the normal traffic flow 312 of the traffic) that the marked traffic or packets are to be subject to capture and forwarding (e.g., an encapsulated and truncated copy of the packet header is forwarded to the analysis platform 308) when the marked packets pass through these subsequent network nodes 304b-c.

[0091] This use of the flag in the transmitted traffic flow avoids the need to distribute and implement complex traffic matching on each subsequent "mid-span" or intermediate capture-enabled network node (such as network nodes 304b-c). Therefore, detailed matching strategies or selection criteria do not need to be distributed to all nodes of network 300, but only the first network node such as network node 304a can be configured to implement matching logic for the selection criteria. Subsequent network nodes 304b-c can identify packets as selected for monitoring based on the presence of the flag on the transmitted traffic arriving at these subsequent network nodes 304b-c, so as to trigger the desired functionality for monitoring the packets. In this way, the configuration of these subsequent or intermediate network nodes 304b-c can be simplified to include, for example: flag matching functionality, which can trigger packet truncation, ERSPAN encapsulation with timestamps, etc.; and functionality for forwarding to analysis platform 308.

[0092] In addition, the use of flags in exemplary aspects also improves the scalability and stability of, for example, intermediate network nodes 304b-c. Because it is not necessary to configure the intermediate network nodes 304b-c to perform checks for various rules, for example, using a ternary content addressable memory (TCAM) load associated with carrying multiple different business matches created by downstream network sources for marked business flows. When network managers change relevant policies, intermediate network nodes 304b-c can be similarly excluded from implementing TCAM disturbances (churn). Instead, these intermediate network nodes 304b-c can be configured with flag matching criteria for triggering their business capture capabilities, while the actual flags on the business of interest are being generated and / or changed at the edge of the network 300 (e.g., at the first network node 304a). In this way, intermediate network nodes 304b-c (typically used for aggregation, core switching, and routing platforms) are more stable and less subject to CPU load and hardware programming disturbances associated with the ever-changing designation of the business of interest.

[0093] Some example implementations of the flag will now be discussed. In the structure of network 300, for example, in a network environment using a network overlay such as VXLAN (Virtual Extensible LAN), a bit within an overlay (e.g., VXLAN) packet header can be used as a flag for specifying a service of interest. A reserved bit in an overlay (e.g., VXLAN) header can also be used for the flag, or in another format such as a VXLAN-GPE format, an operations management and maintenance (OAM) bit can be used for the flag. In such an implementation, any subsequent network node (such as intermediate network nodes 304b-c) that receives a packet with a flag bit set in an overlay (e.g., VXLAN) header can trigger a desired function for service monitoring.

[0094] In an IPv6 network, a bit in the flow label (which is usually underutilized or unused in IPv6) can be used as a flag indicator. Alternatively, another bit in the IPv6 header can be used as a flag.

[0095] In the non-encapsulated business of the traditional IPv4 network, when there is no structure based on overlay (e.g., VXLAN) or similar structure and IPv4 is in use, the most significant bit (MSB) of the two-byte fragment field in the IPv4 header can be used as a flag. The MSB is currently an unused bit in the IPv4 header (while the other bits in the 16-bit fragment field in IPv4 are used as follows: 13 bits are used for fragment offset, 1 bit is used for "no fragmentation" flag, and 1 bit is used for "more fragments" flag, and the MSB is not used). If the MSB of the fragment field is used for the flag, in order to avoid the counterfeiting of the MSB by a malicious host, the MSB can be cleared for all packets except those packets on the first hop entry node such as network node 304a: those packets have been designated as interesting services that can be marked with MSB. In addition, in order to maintain compatibility and compliance with Internet protocols such as RFC 791, the MSB can be cleared when the service leaves the network area where the service capture function is required or when it is transferred out to the Internet or other partner networks.

[0096] If in any implementation of the above aspects (wherein, at any network node in the transit service path, an undesirable clearing of the MSB of the fragmentation field in the IPv4 header is encountered), or in the case where setting the MSB would result in an undesirable network impact at the network node, the MSB may be cleared, and the original matching criteria (as previously specified) may be implemented instead at the subsequent network node encountering the undesirable impact. Thus, in these cases, the matching criteria may be selectively implemented at specific intermediate network nodes without having to implement the complete service matching criteria at each transit service network node.

[0097] Accordingly, aspects of the present disclosure may be implemented at the platform level in ASIC-based hardware (such as a Unified Access Data Plane (UADP) ASIC) at various network nodes of the network structure being monitored, which may provide relevant assurance capabilities. In addition, since the functionality for detecting packets to be monitored and providing encapsulated portions thereof for analysis is implemented entirely in hardware, no additional processing load is introduced on the network node device platform. By providing a hardware-based solution using enhancements to ERSPANv3 functionality in switches and routers that may constitute network nodes, improved scalability may be achieved, and the impact on network devices (e.g., on local device CPU resources) used to implement the exemplary functionality may be reduced or eliminated.

[0098] For example, ingesting ERSPAN-based data from network nodes and analyzing the data to provide assurance results in the analysis platform 308 can be implemented by various elements of the network. For example, the analysis platform 308 can be implemented by a virtual network analysis module (vNAM) and Cisco DNA Center. The vNAM function can be hosted on network devices (such as Catalyst 9000 series switches or IOS-XE-based routers) as a containerized application, allowing the potential computing resources in these platforms to be utilized in a distributed manner for assurance, which allows the solution to scale.

[0099] Accordingly, for example, aspects of the present disclosure enable network managers to have network monitoring capabilities for selected data flows as they move across an enterprise network. Accordingly, hop-by-hop data on latency, jitter, and packet loss can be observed with high accuracy, and accurate attestations on the network paths involved can be determined. Furthermore, offline analysis of truncated copies of packet headers captured from network nodes can enable full L4-L7 business analysis, providing network managers with information for determining, for example, application performance, TCP retransmissions, and other network pathologies.

[0100] Having described example systems and concepts, the present disclosure now turns to Figure 5 The method 500 is shown. The steps outlined herein are examples and may be implemented in any combination thereof (including combinations that exclude, add, or modify certain steps).

[0101] At step 502, method 500 may include receiving, at an analysis platform (e.g., 308) connected to a network (e.g., 300), one or more encapsulated packet headers (e.g., 310a-c) from one or more network nodes (304a-c) of the network. The one or more encapsulated packet headers received from the one or more network nodes may be based on a trigger or one or more selection criteria provided to the one or more network nodes, wherein the encapsulated packet headers are generated by the one or more network nodes for data packets passing through the one or more network nodes and matching the selection criteria. The one or more selection criteria may correspond to a granularity of network traffic to be monitored.

[0102] At step 504, method 500 may include determining, based on the one or more encapsulated packet headers, at least: one or more source addresses (e.g., IP addresses) of one or more network nodes that sent the encapsulated packet headers, and one or more timestamps (e.g., 402a) when the one or more encapsulated packet headers were sent from the one or more network nodes. Each of the one or more encapsulated packet headers may include a truncated copy of a header of a data packet that traversed a corresponding network node in the one or more network nodes (e.g., including it in a truncated copy of a header of a data packet that traversed the one or more network nodes). Figure 4 ERSPANv3 packet header 400 shown).

[0103] At step 506, the method 500 may include: determining at least one or more of the delay or jitter of the data packet passing through one or more network nodes based on at least one or more source addresses and one or more timestamps, wherein the one or more encapsulated packet headers correspond to the data packet passing through one or more network nodes (e.g., as part of the normal business flow 312). The analysis platform 308 may also determine one or more sequence numbers based on each of the one or more encapsulated packet headers to determine one or more of the following based on the one or more sequence numbers: loss in one or more encapsulated packet headers, or loss in the data packet. For example, the one or more sequence numbers may include at least a first sequence number introduced by a first network node, wherein the first network node is an ingress node of the network, wherein the data packet enters the network from an endpoint located outside the network through the ingress node. The first sequence number may be incremented at each sequential network node (e.g., 304b-c) in the one or more network nodes passed by the data packet. The one or more sequence numbers may additionally and optionally include a second sequence number, which is constant at all network nodes, or in other words, is unchanged between the first network node and the one or more network nodes passed by the data packet.

[0104] In some aspects, in addition to the selection criteria, a flag can optionally be used for the data packets or services of interest, wherein one or more encapsulated packet headers received from a first network node of the one or more network nodes can be based on the one or more selection criteria provided to the first network node and the flag inserted by the first network node. The encapsulated packet headers are generated by the first network node for data packets that pass through the first network node and match the selection criteria and the flag. In these aspects, one or more encapsulated packet headers received from one or more intermediate network nodes other than the first network node can be based on data packets that pass through the one or more intermediate network nodes and match the flag.

[0105] Figure 6 An example network device 600 according to the present disclosure (e.g., an implementation of the network nodes 304a-c discussed above) is shown, which is suitable for implementing a policy agent and performing switching, routing and other networking operations related to selecting data packets based on trigger conditions and forming encapsulated and truncated copies of their packet headers for monitoring network services. The network device 600 includes a central processing unit (CPU) 604, an interface 602, and a connection 610 (e.g., a PCI bus). When acting under the control of appropriate software or firmware, the CPU 604 is responsible for performing packet management, error detection and / or routing functions. The CPU 604 preferably performs all of these functions under the control of software including an operating system and any appropriate application software. The CPU 604 may include one or more processors 608, such as processors from the INTEL X86 series of microprocessors. In some cases, the processor 608 may be specially designed hardware for controlling the operation of the network device 600. In some cases, a memory 606 (e.g., non-volatile RAM, ROM, etc.) also forms part of the CPU 604. However, there are many different ways to couple the memory to the system.

[0106] Interface 602 is usually provided as a modular interface card (sometimes referred to as a "line card"). Generally speaking, they control the sending and receiving of data packets on the network, and sometimes support other peripheral devices used with network device 600. Among the interfaces that can be provided are Ethernet interfaces, frame relay interfaces, cable interfaces, DSL interfaces, token ring interfaces, and the like. In addition, various ultra-high-speed interfaces can be provided, such as fast token ring interfaces, wireless interfaces, Ethernet interfaces, Gigabit Ethernet interfaces, ATM interfaces, HSSI interfaces, POS interfaces, FDDI interfaces, WIFI interfaces, 3G / 4G / 5G cellular interfaces, CAN BUS, LoRA, and the like. Generally speaking, these interfaces may include ports suitable for communicating with appropriate media. In some cases, they may also include independent processors, and in some cases include volatile RAM. Independent processors can control communication-intensive tasks such as packet switching, media control, signal processing, cryptographic processing, and management. By providing separate processors for communication-intensive tasks, these interfaces allow the main microprocessor 604 to effectively perform routing calculations, network diagnostics, security functions, and the like.

[0107] although Figure 6 The system shown is a specific network device of the present technology, but it is by no means the only network device architecture on which the present technology can be implemented. For example, an architecture with a single processor that handles communications as well as routing calculations, etc. is often used. In addition, other types of interfaces and media can also be used with network device 600.

[0108] Regardless of the configuration of the network device, it can use one or more memories or memory modules (including memory 606), which are configured to store program instructions for general network operations and for the mechanisms of roaming, route optimization and routing functions described herein. For example, program instructions can control the operation of an operating system and / or one or more applications. One or more memories can also be configured to store tables such as mobility binding, registration and association tables, etc. Memory 606 can also accommodate various software containers and virtualized execution environments and data.

[0109] Network device 600 may also include an application specific integrated circuit (ASIC), which may be configured to perform routing and / or switching operations. For example, the ASIC may communicate with other components in network device 600 via connection 610 to exchange data and signals and coordinate various types of operations performed by network device 600, such as routing, switching, and / or data storage operations.

[0110] Figure 7An example computing device architecture 700 of an example computing device that may implement the various techniques described herein is shown. Components of the computing device architecture 700 are shown in electrical communication with each other using a connection 705, such as a bus. The example computing device architecture 700 includes a processing unit (CPU or processor) 710 and a computing device connection 705 that couples various computing device components, such as a read-only memory (ROM) 720 and a random access memory (RAM) 725, including a computing device memory 715 to the processor 710.

[0111] The computing device architecture 700 may include a cache of high-speed memory directly connected to the processor 710, closely adjacent to the processor 710, or integrated as part of the processor 710. The computing device architecture 700 may copy data from the memory 715 and / or the storage device 730 to the cache 712 for quick access by the processor 710. In this way, the cache can provide a performance boost, which avoids the processor 710 from delaying while waiting for data. These and other modules can control the processor 710 to perform various actions, or can be configured to control the processor 710 to perform various actions. Other computing device memories 715 may also be available. The memory 715 may include a variety of different types of memory with different performance characteristics. The processor 710 may include any general-purpose processor and hardware or software services configured to control the processor 710 (such as service 1 732, service 2 734, and service 3 736 stored in the storage device 730), as well as a dedicated processor that incorporates software instructions into the processor design. The processor 710 may be an independent system containing multiple cores or processors, buses, memory controllers, caches, etc. Multi-core processors can be symmetric or asymmetric.

[0112] To enable user interaction with the computing device architecture 700, the input device 745 may represent any number of input mechanisms, such as a microphone for voice, a touch-sensitive screen for gesture or graphic input, a keyboard, a mouse, motion input, voice, etc. The output device 735 may also be one or more of a variety of output mechanisms known to those skilled in the art, such as a display, a projector, a television, a speaker device, etc. In some cases, a multi-modal computing device may enable a user to provide multiple types of input to communicate with the computing device architecture 700. The communication interface 740 may generally govern and manage user input and computing device output. There is no limitation to operating on any particular hardware arrangement, so the basic features herein may be easily replaced with improved hardware or firmware arrangements as they are developed.

[0113] The storage device 730 is a non-volatile memory and can be a hard disk or other type of computer-readable medium that can store data accessible by a computer, such as a magnetic tape cartridge, a flash memory card, a solid-state memory device, a digital versatile disk, a cassette tape, a random access memory (RAM) 725, a read-only memory (ROM) 720, and a mixture thereof. The storage device 730 may include services 732, 734, 736 for controlling the processor 710. Other hardware or software modules are contemplated. The storage device 730 may be connected to the computing device connection 705. In one aspect, a hardware module that performs a particular function may include a software component stored in a computer-readable medium in conjunction with necessary hardware components such as the processor 710, the connection 705, the output device 735, etc. to perform the function.

[0114] In summary, a system, method, and computer-readable medium for monitoring traffic in a network include: receiving, at an analysis platform connected to the network, one or more encapsulated packet headers from one or more network nodes of the network. Based on the one or more encapsulated packet headers, at least one or more source addresses of the one or more network nodes that sent the encapsulated packet headers, and one or more timestamps when the one or more encapsulated packet headers were sent from the one or more network nodes can be determined. Based at least on the one or more source addresses and timestamps, network traffic information, such as one or more of a delay or jitter of a data packet passing through the one or more network nodes, can be determined, wherein the one or more encapsulated packet headers can correspond to the data packets passing through the one or more network nodes.

[0115] For clarity of explanation, in some cases, the present technology may be presented as including various functional blocks including functional blocks consisting of devices, device components, steps or routines in methods embodied in software, or a combination of hardware and software.

[0116] In some embodiments, computer-readable storage devices, media, and memories may include wired or wireless signals containing bit streams, etc. However, when referred to, non-transitory computer-readable storage media explicitly excludes media such as energy, carrier signals, electromagnetic waves, and signals themselves.

[0117] The method according to the above examples can be implemented using computer executable instructions stored or otherwise available from a computer readable medium. Such instructions may, for example, include instructions and data that cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a certain function or group of functions or otherwise configure a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a certain function or group of functions. Portions of the computer resources used may be accessible via a network. Computer executable instructions may, for example, be binary files, intermediate format instructions such as assembly language, firmware, or source code. Examples of computer readable media that can be used to store instructions, information used, and / or information created during the method according to the described examples include disks or optical disks, flash memory, USB devices provided with non-volatile memory, networked storage devices, and the like.

[0118] Devices implementing methods according to these disclosures may include hardware, firmware, and / or software, and may be implemented in any of a variety of form factors. Some examples of such form factors include general-purpose computing devices such as servers, rack-mounted devices, desktop computers, laptop computers, etc., or general-purpose mobile computing devices such as tablet computers, smart phones, personal digital assistants, wearable devices, etc. The functions described herein may also be embodied in peripheral devices or expansion cards. As a further example, such functions may also be implemented on a circuit board between different chips or different processes executed in a single device.

[0119] Instructions, media for transmitting such instructions, computing resources for executing them, and other structures for supporting such computing resources are the means for providing the functions described in these disclosures.

[0120] Although various examples and other information are used to illustrate various aspects within the scope of the appended claims, limitations on the claims should not be implied based on specific features or arrangements in such examples, because ordinary technicians will be able to use these examples to derive a variety of implementations. In addition, although a subject matter may have been described in language specific to examples of structural features and / or method steps, it should be understood that the subject matter defined in the appended claims is not necessarily limited to these described features or actions. For example, such functions may be distributed in different ways or performed in components other than those identified herein. Instead, the described features and steps are disclosed as examples of components of methods and systems within the scope of the appended claims.

[0121] Claim language reciting "at least one of" a set indicates that one member of the set or multiple members of the set satisfy the claim. For example, claim language reciting "at least one of A and B" means A, B, or A and B.

Claims

1. A method for monitoring services in a network, comprising: At an analysis platform connected to the network, receiving one or more encapsulated packet headers from one or more network nodes of the network, wherein each of the one or more encapsulated packet headers comprises: a truncated copy of a header of a data packet in a data packet that traverses a network node in the one or more network nodes; and a timestamp indicating a time when the data packet traverses the network node; determining at least the following based on the one or more encapsulated packet headers: one or more source addresses of the one or more network nodes that sent the encapsulated packet header; determining the timestamp based on each of the one or more encapsulated packet headers; and determining at least one or more of a delay or a jitter of a data packet passing through the one or more network nodes based at least on the one or more source addresses and the timestamp, wherein the one or more encapsulated packet headers are received from a first network node of the one or more network nodes based on one or more selection criteria provided to the first network node and a flag inserted by the first network node, wherein the encapsulated packet header is generated by the first network node for a data packet passing through the first network node and matching the selection criteria, The one or more encapsulated packet headers are received from one or more intermediate network nodes other than the first network node based on a data packet that passes through the one or more intermediate network nodes and matches the marker.

2. The method of claim 1, further comprising: determining one or more sequence numbers based on each of the one or more encapsulated packet headers; and Based on the one or more sequence numbers, one or more of: a loss in the data packet, or a loss in the one or more encapsulated packet headers is determined.

3. The method of claim 2, wherein: The one or more sequence numbers include at least a first sequence number introduced by a first network node among the one or more network nodes, the first sequence number being incremented at each sequential network node among the one or more network nodes traversed by the data packet.

4. The method of claim 3, wherein: The first network node is an ingress node of the network, wherein the data packet enters the network from an endpoint located outside the network through the ingress node.

5. The method according to claim 3 or 4, wherein: The one or more sequence numbers further include a second sequence number, wherein the second sequence number is constant between the first network node and sequential network nodes among the one or more network nodes through which the data packet passes.

6. The method of claim 1, wherein: The one or more selection criteria correspond to a granularity of traffic of the network to be monitored.

7. A system for monitoring services in a network, comprising: one or more processors; and A non-transitory computer-readable storage medium comprising instructions that, when executed on the one or more processors, cause the one or more processors to perform operations including: receiving one or more encapsulated packet headers from one or more network nodes of the network, wherein each of the one or more encapsulated packet headers comprises: a truncated copy of a header of a data packet of a data packet traversing a network node of the one or more network nodes; and a timestamp indicating a time when the data packet traversed the network node; determining at least the following based on the one or more encapsulated packet headers: one or more source addresses of the one or more network nodes that sent the encapsulated packet header; determining the timestamp based on each of the one or more encapsulated packet headers; and determining at least one or more of a delay or a jitter of a data packet passing through the one or more network nodes based at least on the one or more source addresses and the timestamp, wherein the one or more encapsulated packet headers are received from a first network node of the one or more network nodes based on one or more selection criteria provided to the first network node and a flag inserted by the first network node, wherein the encapsulated packet header is generated by the first network node for a data packet passing through the first network node and matching the selection criteria, The one or more encapsulated packet headers are received from one or more intermediate network nodes other than the first network node based on a data packet that passes through the one or more intermediate network nodes and matches the marker.

8. The system of claim 7, wherein: The operations also include: determining one or more sequence numbers based on each of the one or more encapsulated packet headers; and Based on the one or more sequence numbers, one or more of: a loss in the data packet, or a loss in the one or more encapsulated packet headers is determined.

9. The system of claim 8, wherein: The one or more sequence numbers include at least a first sequence number introduced by a first network node among the one or more network nodes, the first sequence number being incremented at each sequential network node among the one or more network nodes traversed by the data packet.

10. The system of claim 9, wherein: The first network node is an ingress node of the network, wherein the data packet enters the network from an endpoint located outside the network through the ingress node.

11. The system of claim 9 or 10, wherein: The one or more sequence numbers further include a second sequence number, wherein the second sequence number is constant between the first network node and sequential network nodes among the one or more network nodes through which the data packet passes.

12. A non-transitory machine-readable storage medium comprising instructions configured to cause a data processing apparatus to perform operations comprising: receiving one or more encapsulated packet headers from one or more network nodes of a network, wherein: Each of the one or more encapsulated packet headers comprises: a truncated copy of a header of a data packet of a data packet traversing a network node of the one or more network nodes; and a timestamp indicating a time when the data packet traversed the network node; determining at least the following based on the one or more encapsulated packet headers: one or more source addresses of the one or more network nodes that sent the encapsulated packet header; determining the timestamp based on each of the one or more encapsulated packet headers; and determining at least one or more of a delay or a jitter of a data packet passing through the one or more network nodes based at least on the one or more source addresses and the timestamp, wherein the one or more encapsulated packet headers are received from a first network node of the one or more network nodes based on one or more selection criteria provided to the first network node and a flag inserted by the first network node, wherein the encapsulated packet header is generated by the first network node for a data packet passing through the first network node and matching the selection criteria, The one or more encapsulated packet headers are received from one or more intermediate network nodes other than the first network node based on a data packet that passes through the one or more intermediate network nodes and matches the marker.

13. An apparatus for monitoring services in a network, comprising: means for receiving, at an analysis platform connected to the network, one or more encapsulated packet headers from one or more network nodes of the network, wherein each of the one or more encapsulated packet headers comprises: a truncated copy of a header of a data packet of a data packet traversing a network node of the one or more network nodes; and a timestamp indicating a time when the data packet traversed the network node; means for determining, based on the one or more encapsulated packet headers, at least: one or more source addresses of the one or more network nodes that sent the encapsulated packet headers; determining the timestamp based on each of the one or more encapsulated packet headers; and means for determining at least one or more of a delay or a jitter of a data packet passing through the one or more network nodes based at least on the one or more source addresses and the timestamp, wherein the one or more encapsulated packet headers are received from a first network node of the one or more network nodes based on one or more selection criteria provided to the first network node and a flag inserted by the first network node, wherein the encapsulated packet header is generated by the first network node for a data packet passing through the first network node and matching the selection criteria, The one or more encapsulated packet headers are received from one or more intermediate network nodes other than the first network node based on a data packet that passes through the one or more intermediate network nodes and matches the marker.

14. The apparatus according to claim 13, further comprising means for implementing the method according to any one of claims 2 to 6.

15. A computer readable medium comprising instructions which, when executed by a computer, cause the computer to perform the steps of the method of any one of claims 1 to 6.

Citation Information

Patent Citations

  • Active targeted data plane traffic monitoring for wired networks

    US20200162357A1

  • Data stream mirroring method and device

    CN104243211A

  • Traceroute for multi-path routing

    CN108667681A

  • Message mirroring method and network equipment with message mirroring function

    CN1744534A

  • Arrangement for monitoring performance of network connection

    US20100125661A1