Method and device for collecting telemetry flow data messages

Abstract

The present application provides a method and device for collecting telemetry stream data packets. The method for collecting telemetry stream data packets includes: setting a data stream to be collected; setting a committed access rate for the data stream to be collected; adding a color mark to each duplicate data packet of the data stream to be collected according to the committed access rate; discarding each duplicate data packet of the data stream to be collected with a color mark that prohibits forwarding; adding collection information to each duplicate data packet of the data stream to be collected with a color mark that allows forwarding and encapsulating them into a telemetry stream data packet; and sending each encapsulated telemetry stream data packet to a collector.

Description

Technical Field

[0001] The present application relates to communication technology, and in particular to a method and device for collecting telemetry stream data messages. Background Art

[0002] Telemetry Stream is a network monitoring technology that collects data from devices. After configuring the Telemetry Stream feature, the device will actively and continuously send collected data to the collector.

[0003] Each monitored device configures access control list (ACL) entries and a sampler on its inbound interface. The sampler extracts and copies data packets matching the inbound ACL entries at the sampler's configured sampling rate. The copied data packets are encapsulated with a timestamp, an outbound timestamp, a UDP header containing the collector's port number, an Ethernet header with the collector's MAC address as the destination MAC address, and a Layer 3 header with the collector's IP address as the destination IP address. The encapsulated Telemetry Stream message is then sent to the collector, which calculates the path and latency of the received Telemetry Stream messages and the traffic rate of the original data messages based on the sampling rate.

[0004] However, the sampler resources configured on the inbound interface of the monitored device and the sampling rate of the sampler are preconfigured by the chip manufacturer. When the sampling rate is high, the volume of Telemetry Stream packets sent from the monitored device to the collector increases, occupying bandwidth and causing controller congestion. If the sampling rate is low, the sampler granularity is too large, hindering refined control. Summary of the Invention

[0005] The purpose of this application is to provide a method and device for collecting telemetry flow data messages, which flexibly control the flow of telemetry flow data messages sent to a collector through a token bucket.

[0006] To achieve the above-mentioned objectives, the present application provides a method for collecting telemetry stream data packets; wherein, the method comprises setting a data stream to be collected; setting a committed access rate for the data stream to be collected; adding a color mark to each duplicate data packet of the data stream to be collected according to the committed access rate; discarding each duplicate data packet of the data stream to be collected with a color mark prohibiting forwarding; adding collection information to each duplicate data packet of the data stream to be collected with a color mark allowing forwarding and encapsulating them into a telemetry stream Telemetry Stream data packet; and sending each encapsulated telemetry stream Telemetry Stream data packet to a collector.

[0007] To achieve the above-mentioned purpose of the invention, the present application also provides a device for collecting telemetry stream data packets, which includes: a setting module for setting the data stream to be collected; setting the committed access rate of the data stream to be collected; a control module for adding a color mark to each copy data packet of the data stream to be collected according to the committed access rate; discarding each copy data packet of the data stream to be collected with a color mark prohibiting forwarding; a telemetry module for adding collection information to each copy data packet of the data stream to be collected with a color mark allowing forwarding and encapsulating it into a telemetry stream Telemetry Stream data packet; and sending each encapsulated telemetry stream Telemetry Stream data packet to a collector.

[0008] The beneficial effects of the present application are that, by means of a set committed access rate, traffic regulation is performed on the data stream to be collected, and duplicate messages that exceed the traffic specification are discarded through a token bucket instead of sampling the data stream with collection through a sampler; and the sampling rate is controlled by flexibly controlling the number of tokens in the token bucket to flexibly control the duplicate messages allowed to be sent to the collector. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 The flowchart of the embodiment of the method for collecting telemetry stream data packets provided by the present application is shown;

[0010] Figure 2 Shown is a schematic diagram of collecting telemetry stream data packets in this application;

[0011] Figure 3 Shown is a flow chart of an embodiment of a device for collecting telemetry stream data packets provided by the present application. DETAILED DESCRIPTION

[0012] The present invention will be described in detail with reference to a plurality of examples shown in the accompanying drawings. In the following detailed description, a number of specific details are provided to provide a comprehensive understanding of the present invention. Known methods, steps, components, and circuits are not described in detail in the examples to avoid obscuring the understanding of the examples.

[0013] Among the terms used, the term "including" means including but not limited to; the term "containing" means including but not limited to; the terms "above," "within," and "below" are inclusive; the terms "greater than" and "less than" are exclusive. The term "based on" means based on at least a portion.

[0014] Figure 1 The method embodiment of collecting telemetry stream data packets of the present application shown includes the following steps:

[0015] Step 101, setting the data stream to be collected and the committed access rate of the data stream to be collected;

[0016] Step 102: adding a color mark to each replicated data message of the data stream to be collected according to the committed access rate;

[0017] Step 103: discard the duplicate data packets with the color mark of prohibiting forwarding of the data stream to be collected;

[0018] Step 104: Add collection information to each replicated data message with a forwarding-allowed color mark in the data stream to be collected and encapsulate it into a telemetry stream data message;

[0019] Step 105: Send each encapsulated telemetry stream data message to the collector.

[0020] Figure 1 The beneficial effect of the illustrated embodiment is that, by setting a committed access rate to perform traffic regulation on the data stream to be collected, duplicate messages that exceed the traffic specification are discarded through a token bucket instead of sampling the data stream to be collected through a sampler; and by flexibly setting the number of tokens in the token bucket, the sampling rate is controlled by flexibly controlling the duplicate messages allowed to be sent to the collector.

[0021] Figure 2 The figure shows a schematic diagram of collecting telemetry stream data packets in this application. ACL entries are configured on the interface of the device to be monitored for data streams 1, 2, and 3 to be collected. These entries are used to match data packets received by the interface that belong to data streams 1, 2, and 3. A CAR (Committed Access Rate) list is configured on the interface to limit the traffic of data streams 1, 2, and 3, respectively, in the data packets received by the interface.

[0022] When the interface of the supervised device receives a data message, the switching chip first searches for a match with the configured ACL entry based on the flow feature information (eg, triplet, quintuple, or septuple) in the data message.

[0023] When the switching chip finds the ACL entry that matches the data packet of data stream 1 received by the interface, it copies the data packet and adds the QoS local ID1 (quality of service local action identifier) ​​corresponding to data stream 1 to the copied data packet, and then continues to forward the original data packet of data stream 1.

[0024] If the switching chip does not find an ACL entry that matches the data packets of data flow 2, it continues to forward the data packets of data flow 2.

[0025] When the switching chip finds an ACL entry matching the data packet of data flow 3 received by the interface, it copies the data packet and adds QoS local ID 3 corresponding to data flow 3 to the copied data packet, and then continues to forward the original data packet of data flow 3.

[0026] Switch chip 3 searches the CAR list based on the duplicated data packets with added QoS Local IDs 1 and 3. It determines the current number of tokens in bucket C (i.e., committed burst size (CBS)) and the current number of tokens in bucket E (i.e., peak information rate (PIR)) for duplicated data streams 1 and 3. Bucket C and bucket P are used to mark duplicated data stream 1. Bucket P has a larger capacity than bucket C. If the input packet rate of duplicated data stream 1 exceeds the PIR, meaning the tokens in bucket P cannot meet the incoming packets, the duplicated data stream 1 is marked red. If the input packet rate of duplicated data stream 1 is less than the PIR but greater than the CIR, the duplicated data stream 1 is marked yellow. If the input packet rate of duplicated data stream 1 is less than the CIR, meaning the tokens in bucket C can meet the incoming packets, the duplicated data stream 1 is marked green. The red, yellow, and green marking methods are similar for duplicated data stream 3.

[0027] However, in this application, the traffic rate limits set for the duplicated messages of data stream 1 (i.e., duplicated flow 1) and data stream 3 (i.e., duplicated flow 3) in the CAR list are different; the rate limit can be set according to the collection requirements, and the data messages of the two duplicated flows can be marked. At the same time, the switching chip counts the total number of red, yellow, and green marked data messages of data stream 1 and the total number of red, yellow, and green marked data messages of data stream 3 in each reporting cycle.

[0028] Figure 2 In the example, the switching chip discards the duplicate data packets marked in red of data flow 1 and data flow 3 based on the CAR list entry of data flow 1 and the CAR list entry of data flow 3; or discards the duplicate data packets marked in red and yellow of data flow 1 and data flow 3.

[0029] The switching chip sends the duplicate data packets of data stream 1 and data stream 3 with color marks that allow forwarding (for example, duplicate data packets with green marks or duplicate data packets with green and yellow marks) to the TelemetryStream unit, encapsulates them into TelemetryStream data packets, and then sends the TelemetryStream data packets of data stream 1 and data stream 3 to the collector.

[0030] This application uses the CAR list to limit the speed and discard some color messages, instead of sampling the data messages of the data stream received by the interface through the interface sampler. It can not only flexibly set the CAR speed limit rate according to business needs, but also flexibly control the copy data messages of each data stream that needs to be discarded through the token bucket to replace the control of the sampling rate. In addition, the number of table items in the CAR list is much larger than the number of samplers of the interface, so the sampling rate can be set more precisely according to the business characteristics of the collected data stream.

[0031] Since the sampler is replaced by CAR rate limiting, the collector cannot perform path and delay calculations based on the sampling rate to ensure the collector can calculate the traffic rate of the data packets of the original data stream 1 and data stream 3. The monitored device will send the total number of red, yellow, and green marked data packets of data stream 1 and the total number of red, yellow, and green marked data packets of data stream 3 in each reporting cycle to the collector via a remote procedure call protocol message, such as gRPC (Google Remote Procedure Call), so that the collector can obtain the total number of copied data packets of data stream 1 and data stream 3 with the color mark of allowing forwarding and the color mark of prohibiting forwarding.

[0032] Figure 3 The present application provides a device for collecting telemetry stream data packets. The device 30 includes at least a network interface, a switching unit, a telemetry stream unit, a CPU, and memory. The switching unit includes at least a control module, and the telemetry stream unit includes at least a telemetry module. The processor executes the setup module and the control module by running processor-executable instructions in the memory.

[0033] The setting module is used to set the data stream to be collected; set the committed access rate of the data stream to be collected; the control module is used to add a color mark to each replicated data message of the data stream to be collected according to the committed access rate; discard each replicated data message with a prohibited forwarding color mark of the data stream to be collected; the telemetry module is used to add collection information to each replicated data message with a permitted forwarding color mark of the data stream to be collected and encapsulate it into a telemetry stream data message; and send each encapsulated telemetry stream data message to the collector; wherein the prohibited forwarding color mark is a red mark; or a red mark or a yellow mark.

[0034] The control module is further configured to count the number of duplicate data messages of each color mark in the data stream to be collected; and send the counted number of duplicate data messages of each color mark to the collector according to a statistical cycle.

[0035] The control module adds a color mark to each copied data message of the data stream to be collected according to the committed access rate, including: copying each data message of the data stream to be collected; adding a unique local service quality action identifier of the data stream to be collected to each copied data message of the data stream to be collected; and adding a color mark to each copied data message according to the committed access rate corresponding to the local service quality action identifier.

[0036] The control module adds a color mark to each copied data message of the data stream to be collected according to the committed access rate, including: copying each data message of the data stream to be collected; adding a universal local service quality action identifier to each copied data message of the data stream to be collected; adding a color mark to each copied data message according to the flow characteristic parameters of the data stream to be collected and the committed access rate corresponding to the universal local service quality action identifier.

[0037] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for collecting telemetry stream data packets, characterized in that: The method comprises: Set the data stream to be collected; Setting a committed access rate for the data stream to be collected; Adding color marks to each replicated data message of the data flow to be collected according to the C bucket and P pass corresponding to the committed access rate; discarding each duplicate data message with a color mark prohibiting forwarding of the data stream to be collected; Adding collection information to each replicated data message with a forwarding-allowed color mark of the data stream to be collected and encapsulating it into a telemetry stream data message; Send each encapsulated telemetry stream data packet to the collector.

2. The method according to claim 1, characterized in that Before discarding each duplicate data message with a forwarding prohibited color mark of the data stream to be collected, the method further includes: Counting the number of duplicate data packets marked with each color of the data stream to be collected; The number of duplicate data messages of each type of color mark counted is sent to the collector according to the statistical period.

3. The method according to claim 1, characterized in that adding a color mark to each replicated data message of the data stream to be collected according to the committed access rate; Copying each data message of the data stream to be collected; Adding a unique local service quality action identifier of the data stream to be collected to each duplicate data message of the data stream to be collected; A color mark is added to each replicated data message according to the committed access rate corresponding to the local quality of service action identifier.

4. The method according to claim 1, wherein adding a color mark to each replicated data message of the data stream to be collected according to the committed access rate; Copying each data message of the data stream to be collected; Adding a common local service quality action identifier to each duplicate data message of the data stream to be collected; A color mark is added to each of the replicated data packets according to the flow characteristic parameters of the data flow to be collected and the committed access rate corresponding to the universal local service quality action identifier.

5. The method according to claim 3 or 4, characterized in that The prohibited forwarding color mark is red mark; or, red mark or yellow mark.

6. A device for collecting telemetry stream data packets, characterized in that: The device comprises: A setting module is used to set the data stream to be collected; set the committed access rate of the data stream to be collected; A control module, configured to add a color mark to each duplicate data message of the data flow to be collected according to the C bucket and P pass corresponding to the committed access rate; and discard each duplicate data message of the data flow to be collected with a prohibited forwarding color mark; The telemetry module is used to add collection information to each replicated data message with a forwarding-allowed color mark of the data stream to be collected and encapsulate it into a telemetry stream Telemetry Stream data message; and send each encapsulated telemetry stream Telemetry Stream data message to the collector.

7. The device according to claim 6, characterized in that The control module is further configured to count the number of duplicate data messages of each color mark in the data stream to be collected; and send the counted number of duplicate data messages of each color mark to the collector according to a statistical cycle.

8. The device according to claim 6, characterized in that The control module adds a color mark to each copied data message of the data stream to be collected according to the committed access rate, including: copying each data message of the data stream to be collected; adding a unique local service quality action identifier of the data stream to be collected to each copied data message of the data stream to be collected; and adding a color mark to each copied data message according to the committed access rate corresponding to the local service quality action identifier.

9. The device according to claim 6, characterized in that The control module adds color marks to each copied data message of the data stream to be collected according to the committed access rate, including: copying each data message of the data stream to be collected; adding a universal local service quality action identifier to each copied data message of the data stream to be collected; and adding color marks to each copied data message according to the flow characteristic parameters of the data stream to be collected and the committed access rate corresponding to the universal local service quality action identifier.

10. The device according to claim 8 or 9, characterized in that The prohibited forwarding color mark is red mark; or, red mark or yellow mark.

Images