A data center network telemetry information collection method based on backtracking mechanism

Abstract

The present invention discloses a data center network telemetry information collection method based on a backtracking mechanism, including real-time flow information collection and storage by each switch. If a switch receives a telemetry information collection task, it generates a fast packet according to the task and sends it directly to a designated source edge switch, and generates a slow packet according to the task and sends it to the previous hop switch in the network. The flow information and task-related information of the intermediate switches are collected by tracing back the slow packet hop by hop until the slow packet reaches the designated source edge switch. The source edge switch receives the fast packet and the slow packet respectively and generates an alarm data packet or a data report and forwards it to the controller. The present invention can achieve efficient and low-overhead data collection, realizes the ability of upstream devices to obtain information beyond a multi-hop distance without the participation of the controller, can effectively provide more detailed and rich decision-making basis for adaptive algorithms such as load balancing and fast rerouting, and has good scalability and ease of use.

Description

Technical Field

[0001] The present invention relates to the field of computer network communications, and in particular to a data center network telemetry information collection method based on a backtracking mechanism. Background Art

[0002] In recent years, with the gradual maturity of data center models, an increasing number of critical services have been deployed within data center networks, and network topologies have become increasingly complex and expansive. As a crucial bridge connecting data center network measurement and management, the performance of telemetry data collection and the ability to support further analysis and management directly impact the efficiency of data center network operations and maintenance. In particular, in reality, telemetry information is normally exchanged only within neighbor relationships. During the collection and aggregation process, upstream switches are either collected along with the flow to downstream switches or interact directly with the controller. Their ability to access network telemetry information is relatively limited. However, as upstream switches, their ability to make decisions based on data directly impacts critical data center optimization functions such as load balancing and fast rerouting. Therefore, an efficient telemetry collection method that enables upstream switches to obtain detailed data can directly improve the effectiveness of data center network management and network operations and maintenance.

[0003] Data center network telemetry data collection refers to the collection of measured network information at specific times to obtain accurate network status. Generally speaking, existing telemetry data collection methods can be categorized into two types: PULL mode and PUSH mode. The PULL mode primarily relies on controller polling, requiring the collector to poll each network element and extract information. To support intelligent decision-making by upstream switches, the controller must aggregate and collect data before distributing it to upstream switches for global optimization. However, as network scale expands, the number of collectors and the amount of information required to be collected increases, forcing network elements to handle an increasing number of queries, which can quickly lead to controller bottlenecks. Furthermore, data must first be collected by the controller before being distributed, resulting in low timeliness and a slow response to unexpected events.

[0004] Another approach is the PUSH method, which relies on the switch proactively uploading telemetry data. With the advancement of hardware, network devices can proactively aggregate and upload fine-grained information to the controller. Representative PUSH methods can be further categorized into the POSTCARD and PASSPORT modes. The POSTCARD method allows each switch to interact with the controller in real time as traffic passes through it, gaining fine-grained awareness and convenient control capabilities. However, for upstream switches, this still follows a collection-first-then-distribution process. While this method can support data collection under high concurrency, its support for upstream switches is still limited. The PASSPORT method continuously appends payload information to specific packets to record hop-by-hop data and associations. However, this type of in-stream detection typically streams data to downstream switches, preventing upstream switches from effectively and timely obtaining data from downstream switches to make intelligent decisions.

[0005] At the same time, thanks to the continuous development of programmable devices, data packet processing has become increasingly flexible and efficient, making it very convenient to implement flow-level telemetry information collection in data centers. As data packets pass through the ingress and egress processing pipelines of programmable switches, the programmable switches can extract the current flow's five-tuple information (source IP address, destination IP address, source port number, destination port number, and protocol number) from the packet header and payload. This five-tuple information can be used to uniquely identify a flow, and data can be stored and collected for specific flows, thus achieving flow-level telemetry information collection. Summary of the Invention

[0006] Technical problem to be solved by the present invention: In response to the above-mentioned problems of the prior art, a data center network telemetry information collection method based on a backtracking mechanism is provided. The present invention can achieve efficient and low-overhead data collection, and realize the ability of upstream devices to obtain information beyond multi-hop distances without the participation of a controller. It can effectively provide more detailed and rich decision-making basis for adaptive algorithms such as load balancing and fast rerouting, and has good scalability and ease of use.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] A data center network telemetry information collection method based on a backtracking mechanism includes each switch collecting and storing flow information in real time. If a switch receives a telemetry information collection task, it generates a fast packet based on the task and sends it directly to a designated source edge switch, and also generates a slow packet based on the task and sends it to the previous hop switch in the network. The slow packet is traced back hop by hop to collect flow information and task-related information from intermediate switches until the slow packet reaches the designated source edge switch. The source edge switch receives the fast packet and slow packet respectively and generates an alarm data packet or data report and forwards it to the controller.

[0009] Optionally, when collecting and accessing flow information, the flow information collection includes: reading the five-tuple of the data packet pkt header from the inbound queue of the switch interface InGress of the current switch as a binary string Q-tuple, querying the hardware forwarding table to obtain the inbound port number in_num and the outbound port number out_num of the data packet, and querying the internal clock to obtain the flow occurrence timestamp timestamps_in; querying the field data flow_data of the current switch, and arranging the binary string Q-tuple, the inbound port number in_num, the outbound port number out_num, the flow occurrence timestamp timestamps_in, and the field data flow_data into a binary string flow_raw_data separated by separators as the collected flow information.

[0010] Optionally, when the flow information is collected and accessed, the flow information storage includes: using two specified hash functions to respectively calculate the hash values ​​of the header five-tuple information of the data packet in the current flow information to obtain hash value hash1 and hash value hash2; judging the corresponding positions of hash value hash1 and hash value hash2 in the cuckoo filter, if the position corresponding to hash value hash1 or hash value hash2 is empty, selecting an empty position to insert the current flow information, and the storage ends; if the position corresponding to hash value hash1 is empty and the value is equal to hash value hash1, or the position corresponding to hash value hash2 is empty and the value is equal to hash value hash2, it is determined that the current flow information has been stored, and the storage ends; if the positions corresponding to hash value hash1 and hash value hash2 are both not empty, recursively moving data according to the insertion algorithm in the cuckoo filter so that the position corresponding to hash value hash1 or hash value hash2 is empty, and finally inserting the data into the position vacated by the moved data.

[0011] Optionally, when the flow information is collected and accessed, the flow information storage also includes: regularly calculating the time difference between the flow occurrence timestamps_in and the current timestamp Time.time in all stored flow information; if the time difference of a certain flow information exceeds a preset threshold, the flow information is located and deleted according to the process of the deletion algorithm in the cuckoo filter.

[0012] Optionally, the generating of a fast packet according to the task and sending it directly to the designated source edge switch includes: obtaining the five-tuple hash value hash_flow of the data packet pkt header in the current flow and the IP address srcIP of the source edge switch specified by the task; generating a new backtrace packet according to the specified message format, and setting the destination IP to the IP address srcIP of the source edge switch, filling the five-tuple hash value hash_flow, the switching device number Switch ID of the current device, the task triggering time task_ini_timestamps, and the event number event ID specified by the task into the backtrace packet, marking the new backtrace packet with the fast packet mark fast_pkt and sending it out to directly reach the designated source edge switch through the network.

[0013] Optionally, generating a slow packet according to the task and sending it to the previous-hop switch in the network includes: obtaining a five-tuple hash value hash_flow, an ingress port number in_num, an egress port number out_num, and field data flow_data of a data packet pkt header in the current flow, obtaining a source MAC address srcMAC and a destination MAC address dstMAC in a normal direction of the current flow according to the ingress port number in_num and the egress port number out_num, generating a new backtrace packet according to a specified message format, using the source MAC address srcMAC as the MAC address of the previous-hop switch, filling the five-tuple hash value hash_flow and the event number event ID specified by the task into the backtrace packet, marking the new backtrace packet with a slow packet tag slow_pkt, and sending it to the previous-hop switch.

[0014] Optionally, the process of collecting flow information and task-related information of intermediate switches by tracing back the slow packet hop by hop until the slow packet reaches the designated source edge switch, and the processing of the slow packet by the previous hop switch that receives the slow packet includes: first parsing the slow packet to obtain the five-tuple hash value hash_flow of the current flow and the event number event ID specified by the task; according to the five-tuple hash value hash_flow, querying the hardware forwarding table to obtain the ingress port number in_num and egress port number out_num of the data packet, as well as the flow data and field data flow_data of the current switch, generating a flow data list of the current switch; and generating a flow data list of the current switch according to the event number event specified by the task. ID, extract the flow data list of the downstream switch in the payload of the slow packet, the flow data list including flow data and field data flow_data, and form multiple two-tuples (switchID, relavant_flow_data) of the flow data list of the current switch and the flow data list of the downstream switch in stack order as the telemetry data in the slow packet, where switchID is the switch number and relavant_flow_data is the flow data list; at the same time, obtain the source MAC address srcMAC and destination MAC address dstMAC in the normal direction of the current flow based on the ingress port number in_num and egress port number out_num, use the source MAC address srcMAC as the MAC address of the new previous-hop switch in the slow packet, and send the slow packet to the previous-hop switch or the designated source edge switch.

[0015] Optionally, the data area of ​​the slow packet includes an Ethernet data frame header, a traceback data header, and a telemetry data stack. The Ethernet data frame header includes the MAC address of the previous-hop switch and the MAC address of the local device. When the source MAC address srcMAC is used as the MAC address of the new previous-hop switch in the slow packet, it also includes filling the MAC address of the local switch into the MAC address of the local device in the Ethernet data frame header; the traceback data header includes the five-tuple hash value hash_flow of the current flow, the event number event ID specified by the task, the data stack depth and the overall length. When forming multiple two-tuples (switch ID, relavant_flow_data) as the telemetry data stack in the slow packet, the data stack depth and overall length in the updated traceback data header are also reported, so that the data stack depth is the size of the two-tuple (switch ID, relavant_flow_data), and the overall length is the overall length of the slow packet.

[0016] Optionally, when the source edge switch receives fast packets and slow packets respectively, the processing after receiving the fast packet includes: parsing the packet format of the fast packet, obtaining the five-tuple hash value hash_flow, the switching device number SwitchID of the current device and the task trigger time task_ini_timestamps and the event number event ID specified by the task in the fast packet; according to pre-set rules, identifying the event number event ID specified by the task, if the event corresponding to the event number event ID is a fault event, forwarding the fast packet directly to the controller as an alarm data packet; obtaining the five-tuple of the data packet pkt header corresponding to the current data flow in this switch through the five-tuple hash value hash_flow as a binary string Q-tuple, forming an initial data report and waiting for the corresponding slow packet to arrive.

[0017] Optionally, when the source edge switch receives fast packets and slow packets respectively, the processing after receiving the slow packet includes: parsing the packet format of the slow packet, obtaining the five-tuple hash value hash_flow, the switching device number SwitchID of the current device, the telemetry data stack, the data stack depth and the overall length in the slow packet, searching for the corresponding fast packet according to the five-tuple hash value hash_flow, and obtaining the generated initial data report; then extracting the two-tuple (switch ID, relavant_flow_data) from the telemetry data stack and storing them in the data report in sequence, where switchID is the switch number and relavant_flow_data is the flow data list; after all the data in the telemetry data stack are extracted and stored in the data report, the overall length is verified, and if the verification passes, the data report is forwarded to the controller, and the telemetry information collection task of the current flow is completed.

[0018] 1. The present invention addresses the problem that the telemetry information collection method in the data center network scenario has weak support capabilities for the intelligent decision-making of upstream switches. The present invention collects flow information and task-related information of intermediate switches by tracing back slow packets hop by hop until the slow packets reach the designated source edge switch to achieve efficient and low-overhead data collection.

[0019] 2. This method enables upstream devices to obtain information beyond multi-hop distances without controller involvement, effectively providing more detailed and comprehensive decision-making information for adaptive algorithms such as load balancing and fast reroute. Furthermore, this method offers excellent scalability and ease of use, enabling rapid and convenient on-demand collection of specific information and supporting a variety of root cause location and fault recovery algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of the principle of telemetry information collection in an embodiment of the present invention.

[0021] Figure 2 Schematic diagram of the data structure of the slow packet in an embodiment of the present invention.

[0022] Figure 3 This is a workflow diagram for recursively collecting slow packets when performing backtracking to collect data in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] To more clearly illustrate the steps of the data center network telemetry information collection method based on a backtracking mechanism, the present invention will be further described in detail using the data plane of a programmable switch as an example. In this description, the switching devices within the network are divided into two roles: edge switches and other switches. Edge switches refer to access switches directly connected to terminals and servers. Other switches refer to the set of switches within the network, excluding edge switches.

[0024] like Figure 1 As shown, this embodiment provides a data center network telemetry information collection method based on a backtracking mechanism, including each switch collecting and storing flow information in real time. If a switch receives a telemetry information collection task ( Figure 1 Indicated as ① backtracking task trigger), the fast packet generated according to the task is sent directly to the specified source edge switch ( Figure 1 In the figure, it is represented as ② fast packet direct transmission), and generates slow packets according to the task and sends them to the previous hop switch in the network. The flow information and task-related information of the intermediate switches are collected by tracing back the slow packets hop by hop until the slow packets reach the designated source edge switch ( Figure 1 The source edge switch receives the fast packet and the slow packet respectively and generates an alarm data packet ( Figure 1 ④ in the figure) or data report ( Figure 1 (as shown in ⑤) and forwarded to the controller. Figure 1 In the figure, H1 and H2 are two hosts, S1 to S6 are switches, S1 and S6 are edge switches, and S2 to S5 are other switches. Figure 1 It can be seen that switch S5 generates a fast packet according to the task and sends it directly to the designated source edge switch S1, and generates a slow packet according to the task and sends it to the previous hop switch S4 in the network. Starting from the previous hop switch S4, the slow packet is traced back hop by hop to collect the flow information and task related information of the intermediate switches until the slow packet reaches the designated source edge switch S1. The backtracking process passes through S4, S3, S2 in sequence to reach S1. During this process, the flow information and task related information in S4, S3, and S2 are collected, so that S1 can obtain the flow information and task related information in S4, S3, and S2 without the intervention of the controller; the source edge switch S1 receives the fast packet and slow packet respectively and generates an alarm data packet ( Figure 1④ in the figure) or data report ( Figure 1 (as shown in ⑤) and forwarded to the controller.

[0025] As an optional implementation method, this embodiment includes adding a flow information collection module to the switch data plane, and adding a customized cuckoo filter, data access module, backtracking data generation module, backtracking packet generation module, and data packet integration module to the control plane, wherein: the flow information collection module is used to collect flow information; the data access module is used to perform flow information storage; the cuckoo filter is used to store flow information and filter it; the backtracking data generation module is used to generate backtracking data; the backtracking packet generation module is used to generate backtracking packets, including fast packets and slow packets; and the data packet integration module is used to integrate backtracking data into slow packets. The flow information collection module is connected to the data access module. The flow information collection module is mainly responsible for obtaining the five-tuple information of the flow, the inlet and outlet port numbers of the current flow, the time of first arrival, the ID number of the current device, and other data that can support subsequent algorithms, such as the current maximum queue depth and flow priority, flow queue delay, flow packet loss rate, etc., through specific rules. After collecting the above information, the flow information collection module sends it to the data access module for storage. The data access module is connected to the cuckoo filter, flow information collection module, and retrospective data generation module. The data access module can read, write, and delete elements in the cuckoo filter. The data access module's input is a data structure consisting of a five-tuple hash value, input and output port numbers, device interface, collection time, and other variable-length data. The data access module's output is a subset of data relevant to the collection purpose, filtered by specific identifiers. The retrospective packet generation module is connected to the retrospective data generation module and is responsible for generating fast and slow data packets for retrospective collection according to a custom message structure. The retrospective data generation module is also connected to the data access module. When a telemetry information collection task is initiated, the generation module first generates a fast packet. It obtains the current collection purpose, site information, and collection time from the retrospective data generation module. It then generates a retrospective packet using the source edge switch's IP address and returns it directly to the edge switch. It then generates a slow packet, obtains the source and destination MAC addresses required for retrospective collection from the retrospective data generation module, enters site information, and forwards it to the switch, collecting telemetry information hop by hop. The packet consolidation module is connected to the data access module. Upon receiving slow packet traceback information, it collects relevant information from the payload according to specific requirements and records any anomalies in the downstream switch for that specific flow, facilitating optimization decisions for other links. The edge switch is directly connected to the controller, and its packet consolidation module receives input from the fast and slow packets of a specific flow. Through consolidation, it collects and organizes the collected information on fast and slow packets, generates a data report, and submits it to the controller.

[0026] In this embodiment, when collecting and accessing flow information, the flow information collection includes: reading the quintuple in the packet pkt header from the ingress queue of the switch interface InGress of the current switch as a binary string Q-tuple (composed of the quintuple source IP address srcIP, destination IP address dstIP, source port number srcPort, destination port number dstPort, and protocol protocol), querying the hardware forwarding table to obtain the packet's ingress port number in_num and egress port number out_num, and querying the internal clock to obtain the flow occurrence timestamp timestamps_in; querying the current switch's field data flow_data, and organizing the binary string Q-tuple, ingress port number in_num, egress port number out_num, flow occurrence timestamp timestamps_in, and field data flow_data into a delimited binary string flow_raw_data as the collected flow information. After receiving the binary string flow_raw_data uploaded by the flow information collection module, the data access module parses the (Q-tuple, in_num, out_num, flow_data, timestamps_in) data based on the delimiter.

[0027] After receiving the binary string flow_raw_data, the data access module first calculates the hash value of the binary string Q-tuple to uniquely mark a flow. By executing the workflow of the search algorithm in the cuckoo filter, it determines whether the flow has been recorded. If it has been recorded, it will not be recorded again. If it has not been recorded, the data will be further sorted. In addition, other methods can be used to store the collected flow data as needed, or other methods other than the cuckoo filter can be used to filter the collected flow data. In this embodiment, for the edge switch, what needs to be inserted is (Hash, Q-tuple, in_num, out_num, flow_data, timestamps_in), and for other switches, only (Hash, srcIP, in_num, out_num, flow_data, timestamps_in) needs to be stored. The reason is that other switches only act as intermediate forwarding and do not need to store the binary string Q-tuple. In this embodiment, when collecting and accessing flow information, the flow information storage includes: using two specified hash functions to respectively calculate the hash value of the header five-tuple information of the data packet in the current flow information to obtain hash value hash1 and hash value hash2; judging the corresponding positions of hash value hash1 and hash value hash2 in the cuckoo filter, if the position corresponding to hash value hash1 or hash value hash2 is empty, then selecting an empty position to insert the current flow information, and the storage ends; if the position corresponding to hash value hash1 is empty and the value is equal to hash value hash1, or the position corresponding to hash value hash2 is empty and the value is equal to hash value hash2, then it is determined that the current flow information has been stored, and the storage ends; if the positions corresponding to hash value hash1 and hash value hash2 are both non-empty, recursively moving data according to the insertion algorithm in the cuckoo filter so that the position corresponding to hash value hash1 or hash value hash2 is empty, and finally inserting the data into the position vacated by the moved data. It should be noted that the cuckoo filter and its insertion algorithm are both existing technologies. The cuckoo filter structure consists of two fixed-size bucket arrays, uniquely identified by the hash value of the flow's quintuple. In network switches (excluding edge switches), each element in the array contains three fields: a hash field for the quintuple information, a field for the collected data, and a field for the time of entry. Edge switches also require the addition of quintuple information fields: a hash field, a quintuple information field, a data field, and a field for the time of entry. The flow quintuple information field is derived from the flow's source IP address, destination IP address, source port number, destination port number, and protocol number. When the switch starts running, the cuckoo filter is initialized to an empty field.

[0028] In this embodiment, when collecting and accessing flow information, the flow information storage also includes: regularly calculating the time difference between the flow occurrence timestamps_in and the current timestamp Time.time in all stored flow information; if the time difference of a certain flow information exceeds a preset threshold (such as 300 milliseconds), the flow information is located and deleted according to the process of the deletion algorithm in the cuckoo filter to reduce the required storage space.

[0029] In this embodiment, the flow information collection module and the data access module work in parallel to complete the collection and access of telemetry information. The flow information collection module obtains the five-tuple information and entry time from the flow packet header and sends it to the data access module. The data access module processes and organizes the data, generates a five-tuple hash value, and writes the data into the cuckoo filter. At the same time, the data access module regularly scans the entry time of each data point and deletes any data that exceeds the time threshold to reduce the required storage space.

[0030] In this embodiment, after the telemetry information collection task is initiated, the data access module, the data integration module, the backtracking data and packet generation module work in parallel to cooperate to complete the hop-by-hop backtracking collection of the required telemetry data. The method is that after the telemetry data collection task is initiated, the backtracking data generation module collaborates with the backtracking packet generation module to obtain simple information of the flow to be collected through the data access module, and generates a fast packet in combination with the collection purpose (such as only collecting data related to abnormal events of queuing delay indicators), and sends it directly to the source edge switch for rapid alarm or indication of the type of event occurring in the network and simple information of the telemetry collection task; then, the two collaborate to obtain the address of the previous hop switching device, generate a slow packet in combination with the collection purpose, and send it to the previous hop switching device. After receiving the slow packet, other network devices store the event-related information in the slow packet payload according to the collection purpose identified by the specific message field. This information is used to support other intelligent tuning algorithms to obtain information about downstream switches beyond multiple hops. Simultaneously, the data access module collects the corresponding telemetry data and inserts it into the slow packet payload. The backtracking data generation module obtains the address of the previous hop switch and modifies the slow packet address. The packet is then sent to the previous hop switch according to the address. This backtracking process is repeated until the slow packet reaches the source edge switch.

[0031] In this embodiment, generating a fast packet based on a task and sending it directly to the specified source edge switch includes: obtaining the five-tuple hash value hash_flow of the data packet pkt header in the current flow and the IP address srcIP of the source edge switch specified by the task; generating a new backtrace packet according to the specified message format, and setting the destination IP to the IP address srcIP of the source edge switch, filling the backtrace packet with the five-tuple hash value hash_flow, the switching device number Switch ID of the current device, the task trigger time task_ini_timestamps, and the event number event ID specified by the task, and marking the new backtrace packet with the fast packet mark fast_pkt and sending it out to directly reach the specified source edge switch through the network (without collecting data hop by hop, and then directly sending it to the data plane for forwarding). In this embodiment, the fast packet mainly consists of three parts: an Ethernet data frame header, a standard IP header filled with the source edge switch IP, and a payload formed using the UDP protocol that contains simple information about the collection task.

[0032] In this embodiment, generating a slow packet according to a task and sending it to the previous hop switch in the network includes: obtaining the five-tuple hash value hash_flow, the ingress port number in_num, the egress port number out_num, and the field data flow_data of the data packet pkt header in the current flow, obtaining the source MAC address srcMAC and the destination MAC address dstMAC in the normal direction of the current flow according to the ingress port number in_num and the egress port number out_num, generating a new backtrace packet according to the specified message format, using the source MAC address srcMAC as the MAC address of the previous hop switch, filling the five-tuple hash value hash_flow and the event number event ID specified by the task into the backtrace packet, marking the new backtrace packet with the slow packet tag slow_pkt and sending it to the previous hop switch. When other switching devices receive a slow packet with the slow packet tag slow_pkt from a downstream switch, they collect the time information and field information of the downstream switch, parse the data packet to obtain the collection purpose, obtain data and the MAC address of the previous hop from the backtrace data generation module according to the corresponding collection purpose, and forward it. Figure 3As shown in the figure, the flow information and task-related information of the intermediate switches are collected by tracing back the slow packet hop by hop until the slow packet reaches the designated source edge switch. The previous hop switch that receives the slow packet processes the slow packet by: first parsing the slow packet to obtain the five-tuple hash value hash_flow of the current flow and the event number event ID specified by the task; according to the five-tuple hash value hash_flow, querying the hardware forwarding table to obtain the ingress port number in_num and egress port number out_num of the data packet, as well as the flow data and field data flow_data of the local switch to generate the flow data list of the local switch; according to the event number event specified by the task The switch uses the ID to extract the flow data list of the downstream switch in the slow packet's payload. The flow data list includes flow data and field data flow_data. The flow data list of the current switch and the flow data list of the downstream switch are stacked in order to form multiple two-tuples (switchID, relavant_flow_data) as the telemetry data in the slow packet, where switchID is the switch ID and relavant_flow_data is the flow data list. Simultaneously, the source MAC address srcMAC and destination MAC address dstMAC of the current flow in the normal direction are obtained based on the ingress port number in_num and egress port number out_num. The source MAC address srcMAC is used as the MAC address of the new previous-hop switch in the slow packet, and the slow packet is sent to the previous-hop switch or the designated source edge switch. At this point, the upstream switch has obtained field information about the downstream switch beyond multiple hops. This information can be used for intelligent decision-making in methods such as load balancing and fast rerouting. When the previous-hop device receives a packet marked as slow, it parses the backtrace header to obtain the event ID. Based on the event ID field and predefined rules, it extracts the corresponding data to form a flow data list called relavant_flow_data, which is then appended to the payload to form a new payload. Simultaneously, the data stack depth (data left) and total length (total length) in the backtrace header are modified according to the new payload length. The backtrace packet generation module uses the hash_flow value to query the corresponding in / out_num, obtain the srcMAC and dstMAC in the normal direction, and modify the corresponding values ​​in the data packet. The destination address of the current packet is now srcMAC, and the source address is now dstMAC. The new slow_pkt, with the payload and modified header attached, is then sent to the data plane for forwarding. This process is repeated, allowing the slow packet marked as slow_pkt to be advertised hop by hop along the real path to downstream switches, and iterative backtrace collection reaches the edge switch.At the same time, in order to alleviate storage pressure and prevent redundant data from occupying storage space, each previous-hop switch also regularly calculates the time difference between the flow occurrence timestamp timestamps_in and the current timestamp Time.time in all stored flow information. If the time difference of a flow information exceeds the preset threshold (such as 300 milliseconds), the flow information is located and deleted according to the deletion algorithm process in the cuckoo filter to reduce the required storage space.

[0033] like Figure 2 As shown, the data area of ​​the slow packet includes an Ethernet data frame header, a traceback data header, and a telemetry data stack. The Ethernet data frame header includes the MAC address of the previous-hop switch and the MAC address of the local device. When the source MAC address srcMAC is used as the MAC address of the new previous-hop switch in the slow packet, the MAC address of the local switch is also filled in with the MAC address of the local device in the Ethernet data frame header. The traceback data header includes the five-tuple hash value hash_flow of the current flow, the event number event ID specified by the task, the data stack depth data left, and the total length total length. When forming multiple two-tuples (switch ID, relavant_flow_data) as the telemetry data stack in the slow packet, the data stack depth data left and the total length total length in the updated traceback data header are also reported, so that the data stack depth data left is the size of the two-tuple (switch ID, relavant_flow_data), and the total length total length is the total length of the slow packet.

[0034] The source edge switch begins its operation upon receiving a fast packet labeled fast_pkt. Upon receiving the fast packet, it first extracts the event information contained in the fast packet. If the event information is an exception alarm or fault alarm, it is directly forwarded to the controller for timely alerting. It then waits for the arrival of a slow packet from the same flow. When the slow packet from the same flow reaches the edge switch, it aggregates the data to generate a data report and uploads it to the controller.

[0035] In this embodiment, when the source edge switch receives fast packets and slow packets respectively, the processing after receiving the fast packet includes: parsing the packet format of the fast packet, obtaining the five-tuple hash value hash_flow, the switching device number SwitchID of the current device and the task trigger time task_ini_timestamps and the event number event ID specified by the task in the fast packet; according to the pre-set rules, identifying the event number event ID specified by the task, if the event corresponding to the event number event ID is a fault event (for example, the flow queuing delay is too long, the packet loss rate is too high, and other indicators are abnormal or the link is not connected), the fast packet is directly forwarded to the controller as an alarm data packet; obtaining the five-tuple of the data packet pkt header corresponding to the current data flow in this switch through the five-tuple hash value hash_flow as a binary string Q-tuple (this type of information is only stored on the edge switch), forming an initial data report and waiting for the corresponding slow packet to arrive. The processing after receiving the slow packet includes: parsing the packet format of the slow packet, obtaining the five-tuple hash value hash_flow, the switch device number Switch ID of the current device, the telemetry data stack, the data stack depth and the overall length in the slow packet, searching for the corresponding fast packet according to the five-tuple hash value hash_flow, and obtaining the generated initial data report; then extracting the two-tuple (switch ID, relavant_flow_data) from the telemetry data stack and storing them in the data report in sequence, where switchID is the switch number and relavant_flow_data is the flow data list; after all the data in the telemetry data stack is extracted and stored in the data report, the overall length is checked to prevent data missing or overflow. If the check passes, the data report is forwarded to the controller, and the telemetry information collection task of the current flow is completed.

[0036] In summary, in order to address the problem of weak support capabilities for intelligent decision-making of upstream switches in telemetry information collection methods in data center network scenarios, the method of this embodiment collects a small amount of data related to the collection purpose, and exchanges remote (outside the multi-hop range) data with the upstream switch hop by hop along the real path of the flow until the data is backtracked and collected to the edge switch to achieve efficient and low-overhead data collection. At the same time, the ability of upstream devices to obtain information beyond the multi-hop distance without the participation of the controller is realized, which can effectively provide more detailed and rich decision-making basis for adaptive algorithms such as load balancing and fast rerouting. In addition, the method also has good scalability and ease of use, can quickly and conveniently collect specific information according to needs, and supports different types of root cause location and fault recovery algorithms.

[0037] This embodiment utilizes slow packet backtracking to collect telemetry information while simultaneously notifying upstream switches of downstream switch event and fault information, providing remote information acquisition capabilities. Since fault information carried by slow packets passes through upstream switches hop by hop along the link during the backtracking process, the upstream switch can simultaneously obtain status information and possible fault information from downstream switches multiple hops away while collecting data. This ability to transmit information beyond neighbor relationships allows upstream switches multiple hops away to independently perform remote load balancing and traffic scheduling when encountering network failures, such as link congestion, without the involvement of a global controller. Furthermore, the ability to obtain on-site information from devices multiple hops away provides more detailed data for other adaptive algorithms deployed on upstream switches. When multiple devices possess the capabilities of remote information acquisition and adaptive decision-making, local intelligent adaptive capabilities can be implemented. The fast and slow packet mechanisms simultaneously achieve rapid event alerts and reliable data collection. This embodiment utilizes customized cuckoo filters to enable data search, insertion, and deletion operations in highly concurrent environments, ensuring high data access availability under high traffic conditions. This embodiment combines a cuckoo filter structure to implement a method for searching, storing, and deleting data required by flow data and backtracking mechanisms. By maintaining a cuckoo filter structure for each device on the data plane, each time the flow information collection module reports flow data, a cuckoo filter search algorithm is used to determine whether the flow has been stored. If it has been stored, the insertion is ignored; if it has not been stored, the cuckoo filter is inserted using an insertion algorithm. The data structure in the network device is (five-tuple hash value, outgoing port number, incoming port number, source edge switch IP, filled timestamp, network data). The network data is a data structure composed of field information, and its specific content can be quickly customized according to the requirements of algorithms such as network measurement, anomaly detection, and fault location. The data structure in the edge switch also needs to add specific information of the five-tuple, namely (five-tuple hash value, five-tuple specific information, outgoing port number, incoming port number, source edge switch IP, filled timestamp, network data). Since the duration of a flow is generally short, a method of periodically deleting entries is adopted to effectively alleviate storage pressure and improve access efficiency. Specifically, through periodic scanning, data entries whose filling time exceeds a certain threshold are deleted. The method of this embodiment uses a self-designed traceback data packet format. The fast packet mainly consists of three parts: an Ethernet data frame header, a standard IP header filled with the source edge switch IP, and a payload formed using the UDP protocol containing simple information about the collection task.The slow packet (backtrace data packet) mainly consists of three parts: an Ethernet data frame header with the source and destination addresses filled in reverse, a backtrace data packet header containing four fields: the hash value of the current flow five-tuple, event ID, payload data depth, and total packet size, and a backtrace data packet payload consisting of valid data collected in each device. Among them, in order to have better scalability and ease of use, the payload in the backtrace data packet adopts a stack structure, allowing the collecting device to be placed in the payload in the form of (device ID, variable-length data structure). The information in the variable-length data structure is specifically determined by the supported network analysis, positioning, recovery algorithm and event ID, achieving better scalability and ease of use. The method of this embodiment realizes the mechanism of backtrace collection and remote information notification. Specifically, by obtaining the inlet and outlet port numbers of a specific flow when flowing through the device, marking the specific source and destination MAC addresses and reversely filling in the backtrace data frame header when backtrace is required, the function of returning to the previous hop along the real path can be realized; by continuously collecting data and filling in the backtrace address, the corresponding data collected hop by hop along the real path and the notification function of the downstream switch information to the upstream switch can be realized, and finally returned to the edge switch. It can effectively improve the control and grasp of the status of the downstream switch network of the upstream switch fleet, and has the possibility of significantly improving the adaptive decision-making ability of the upstream switch. This process does not require the participation of the controller in planning and control, which can effectively alleviate the storage overhead and computing overhead of the centralized controller. The method of this embodiment generates two types of fast and slow backtracking packets, and realizes the functions of fast event alarm and reliable data collection at the same time. Mainly when the telemetry data collection task is initiated, the fast packet is generated first, and the simple information of the telemetry data collection task is filled in. It is sent directly through the source edge switch IP, without the need for hop-by-hop backtracking and data collection, and plays the role of fast notification and fast alarm; after the fast packet is generated, the slow packet continues to be generated, and the data is collected by step-by-step backtracking, which plays the role of reliable data collection.

[0038] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A data center network telemetry information collection method based on a backtracking mechanism, characterized in that: This involves each switch collecting and storing flow information in real time. If a switch receives a telemetry information collection task, it generates a fast packet based on the task and sends it directly to the designated source edge switch. It also generates a slow packet based on the task and sends it to the previous hop switch in the network. The slow packet is then traced back hop by hop to collect flow information and task-related information from the intermediate switches until the slow packet reaches the designated source edge switch. The source edge switch receives the fast and slow packets respectively, generates an alarm data packet or data report, and forwards it to the controller. Generating a fast packet according to the task and sending it directly to the designated source edge switch includes: obtaining a five-tuple hash value hash_flow of a data packet pkt header in the current flow and an IP address srcIP of the source edge switch designated by the task; generating a new backtrace packet according to a designated message format, setting the destination IP address to the IP address srcIP of the source edge switch, filling the backtrace packet with the five-tuple hash value hash_flow, the switch device number of the current device Switch ID, the task triggering time task_ini_timestamps, and the event number event ID designated by the task; marking the new backtrace packet with a fast packet tag fast_pkt and sending it out to directly reach the designated source edge switch through the network; The generating of a slow packet according to the task and sending it to the previous-hop switch in the network includes: obtaining a five-tuple hash value hash_flow, an ingress port number in_num, an egress port number out_num, and field data flow_data of a data packet pkt header in the current flow; obtaining a source MAC address srcMAC and a destination MAC address dstMAC in a normal direction of the current flow according to the ingress port number in_num and the egress port number out_num; generating a new backtrace packet according to a specified message format; using the source MAC address srcMAC as the MAC address of the previous-hop switch; filling the five-tuple hash value hash_flow and an event number event ID specified by the task into the backtrace packet; marking the new backtrace packet with a slow packet tag slow_pkt and sending the packet to the previous-hop switch.

2. The data center network telemetry information collection method based on the backtracking mechanism according to claim 1 is characterized in that: When collecting and storing flow information, the flow information collection includes: reading the five-tuple of the data packet pkt header from the inbound queue of the switch interface InGress of the current switch as a binary string Q-tuple, querying the hardware forwarding table to obtain the inbound port number in_num and the outbound port number out_num of the data packet, and querying the internal clock to obtain the flow occurrence timestamp timestamps_in; querying the field data flow_data of the current switch, and arranging the binary string Q-tuple, the inbound port number in_num, the outbound port number out_num, the flow occurrence timestamp timestamps_in, and the field data flow_data into a binary string flow_raw_data separated by delimiters as the collected flow information.

3. The data center network telemetry information collection method based on the backtracking mechanism according to claim 1 is characterized in that: When the flow information is collected and stored, the flow information storage includes: using two specified hash functions to respectively calculate the hash values ​​of the header five-tuple information of the data packet in the current flow information to obtain hash value hash1 and hash value hash2; judging the corresponding positions of hash value hash1 and hash value hash2 in the cuckoo filter, if the position corresponding to hash value hash1 or hash value hash2 is empty, selecting an empty position to insert the current flow information, and the storage ends; if the position corresponding to hash value hash1 is empty and the value is equal to hash value hash1, or the position corresponding to hash value hash2 is empty and the value is equal to hash value hash2, it is determined that the current flow information has been stored, and the storage ends; if the positions corresponding to hash value hash1 and hash value hash2 are both not empty, recursively moving data according to the insertion algorithm in the cuckoo filter so that the position corresponding to hash value hash1 or hash value hash2 is empty, and finally inserting the data into the position vacated by the moved data.

4. The data center network telemetry information collection method based on the backtracking mechanism according to claim 2 is characterized in that: When the flow information is collected and stored, the flow information storage also includes: regularly calculating the time difference between the flow occurrence timestamps_in and the current timestamp Time.time in all stored flow information; if the time difference of a certain flow information exceeds a preset threshold, the flow information is located and deleted according to the process of the deletion algorithm in the cuckoo filter.

5. The data center network telemetry information collection method based on the backtracking mechanism according to claim 1, characterized in that: The slow packet is traced back hop by hop to collect flow information and task-related information from the intermediate switches until the slow packet reaches the designated source edge switch. The previous hop switch that receives the slow packet processes the slow packet by first parsing the slow packet to obtain the five-tuple hash value hash_flow of the current flow and the event number event specified by the task. The system queries the hardware forwarding table based on the five-tuple hash value hash_flow to obtain the ingress port number in_num and egress port number out_num of the data packet, as well as the flow data and field data flow_data of the local switch, to generate the flow data list of the local switch. According to the event number eventID specified by the task, the system extracts the flow data list of the downstream switch from the payload of the slow packet, which includes the flow data and field data flow_data. The flow data list of the local switch and the flow data list of the downstream switch are stacked in order to form multiple two-tuples (switchID, relavant_flow_data) as the telemetry data in the slow packet, where switchID is the switch number and relavant_flow_data is the flow data list. Simultaneously, the system obtains the source MAC address srcMAC and destination MAC address dstMAC in the normal direction of the current flow based on the ingress port number in_num and egress port number out_num. The source MAC address srcMAC is used as the MAC address of the new previous-hop switch in the slow packet, and the slow packet is sent to the previous-hop switch or the designated source edge switch.

6. The data center network telemetry information collection method based on the backtracking mechanism according to claim 5, characterized in that: The data area of ​​the slow packet includes an Ethernet data frame header, a traceback data header, and a telemetry data stack. The Ethernet data frame header includes the MAC address of the previous-hop switch and the MAC address of the local device. When the source MAC address srcMAC is used as the MAC address of the new previous-hop switch in the slow packet, the MAC address of the local switch is also filled in with the MAC address of the local device in the Ethernet data frame header. The traceback data header includes the five-tuple hash value hash_flow of the current flow, the event number event ID specified by the task, the data stack depth and the overall length. When forming multiple two-tuples (switch ID, relavant_flow_data) as the telemetry data stack in the slow packet, the data stack depth and overall length in the updated traceback data header are also reported, so that the data stack depth is the size of the two-tuple (switch ID, relavant_flow_data) and the overall length is the overall length of the slow packet.

7. The data center network telemetry information collection method based on the backtracking mechanism according to claim 1, characterized in that: When the source edge switch receives fast packets and slow packets respectively, the processing after receiving the fast packet includes: parsing the packet format of the fast packet, obtaining the five-tuple hash value hash_flow, the switching device number Switch ID of the current device, the task trigger time task_ini_timestamps, and the event number event ID specified by the task in the fast packet; according to a pre-set rule, identifying the event number event ID specified by the task, if the event corresponding to the event number event ID is a fault event, forwarding the fast packet directly to the controller as an alarm data packet; obtaining the five-tuple of the data packet pkt header corresponding to the current data flow in this switch through the five-tuple hash value hash_flow as a binary string Q-tuple, forming an initial data report and waiting for the corresponding slow packet to arrive.

8. The data center network telemetry information collection method based on the backtracking mechanism according to claim 6, characterized in that: When the source edge switch receives a fast packet and a slow packet respectively, the processing after receiving the slow packet includes: parsing the packet format of the slow packet, obtaining the five-tuple hash value hash_flow, the switch device number Switch ID of the current device, the telemetry data stack, the data stack depth and the overall length in the slow packet, searching for the corresponding fast packet according to the five-tuple hash value hash_flow, and obtaining the generated initial data report; then extracting the two-tuple (switch ID, relavant_flow_data) from the telemetry data stack and storing them in sequence in the data report, where switchID is the switch number and relavant_flow_data is the flow data list; after all data in the telemetry data stack is extracted and stored in the data report, the overall length is verified. If the verification passes, the data report is forwarded to the controller, and the telemetry information collection task of the current flow is completed.

Images