A programmable network service one-hop transmission packet loss rate measurement method

Abstract

The application discloses a programmable network service single-hop transmission packet loss rate measuring method, which comprises the following steps: deploying specific data structures on upstream and downstream switches; utilizing the cooperation mechanism of the upstream and downstream switches to discover service streams that are likely to cause packet loss in real time on the downstream switch, and recording the stream ID; and performing fine-grained statistics on the service streams on the upstream and downstream switches, so as to obtain the packet loss rate of the service streams that cause packet loss on the single hop.

Description

Technical Field

[0001] This invention belongs to the field of network measurement technology, and more specifically, relates to a method for measuring packet loss rate in single-hop transmission of programmable network services. Background Technology

[0002] In recent years, with the widespread application of virtualization technology and the development of emerging application models such as cloud computing, coupled with the rapid growth in the number of users of online services, the amount of data transmitted in the network has shown an explosive growth trend. This massive data traffic easily leads to sudden network congestion, which often results in packet loss, directly causing a decline in overall network performance, including reduced flow completion time and link utilization. In severe cases, it can even lead to network malfunction, seriously affecting user experience and operator revenue. Along with the increase in data volume, the scale of the network and the various hardware and software resources and critical business systems within it are also increasing rapidly. However, network failures are unavoidable, including but not limited to hardware failures, which typically manifest as packet loss in network services. Locating the fault location and diagnosing the cause within the vast physical structure is challenging. Operators often need tens of hours to determine where the fault occurred, which traffic was affected, and ultimately, the root cause of the fault. Therefore, timely and accurate measurement of packet loss rate per hop can help operators identify the root cause of the fault and repair it, minimizing the adverse impact of packet loss on application and network performance.

[0003] Common packet loss rate measurement methods often rely on specific protocols or dedicated hardware, such as Ping and Traceroute solutions based on the ICMP protocol. These solutions often cannot achieve business-level packet loss rate measurement. Some advanced packet loss rate measurement solutions require dedicated hardware, which can increase implementation costs. Due to these limitations, it is necessary to seek solutions that are applicable to various protocols, do not require dedicated hardware, and can provide business-level measurement accuracy in a cost-effective manner.

[0004] In recent years, the development of programmable networks, especially programmable network hardware, has provided new possibilities for packet loss rate measurement. Programmable network hardware allows network administrators to customize the processing of network traffic in the data plane according to specific needs and collect relevant information from network flows. This technology brings more flexible and accurate packet loss rate measurement methods, making it easier to implement service-level measurements in various network environments. However, service-level measurements often consume the hardware resources of network devices, and the hardware resources of programmable network hardware are very limited. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for measuring packet loss rate in single-hop transmission of programmable network services. This method utilizes programmable network hardware to promptly and accurately detect service flows that generate packet loss and perform fine-grained statistics, thereby minimizing the consumption of hardware resources.

[0006] To achieve the above-mentioned objectives, the present invention provides a method for measuring packet loss rate in single-hop transmission of programmable network services, characterized by comprising the following steps:

[0007] (1) Data structure deployment;

[0008] (1.1) Deploy a coarse-grained counting data structure CM Sketch with d rows on the upstream and downstream switches. Deploy w counters on each row, and label the counters from left to right as 0, 1, ..., w-1. At the same time, deploy a hash function on each row. The structure of each hash function is different. The hash functions for each row are denoted as h1, h2, h3, ..., h d Each hash function calculates a hash value that is an integer in the range [0, w-1], and each hash value corresponds to a counter number.

[0009] (1.2) Deploy a fine-grained counting data structure hash table with 2 rows on the upstream switch. The first row is used to store the flow ID and the second row is used to store the number of packets. In addition, deploy m counters on each row, and the counters are labeled 0, 1, ..., m-1 from left to right. Deploy the same hash function h on each row. The hash value calculated by the hash function is an integer in the interval [0, m-1], which corresponds to the counter labeled with the hash value.

[0010] (1.3) Deploy a fine-grained counting data structure hash table with 3 rows on the downstream switch. The first row is used to store the flow ID, the second row is used to store the number of data packets arriving at the downstream switch, and the third row is used to store the number of detected link packet loss. Deploy m counters on each row, and the counters are marked as 0, 1, ..., m-1 from left to right. Deploy the same hash function h on each row. The hash value calculated by the hash function is an integer in the interval [0, m-1], which corresponds to the counter labeled with the hash value.

[0011] (2) Perform fine-grained statistics on business flows based on data structures;

[0012] (2.1) Initialize the data structure;

[0013] (2.2) When the upstream switch receives a data packet, it first parses the data packet and extracts the frame header and flow IDF of the data packet. If the frame header type of the data packet is different from the frame header type of the custom data packet, then step (2.3) is executed; otherwise, it is determined that the received data packet is a custom data packet, and then operation (2.8) is executed.

[0014] (2.3) Call the hash function h in the upstream switch, take the flow ID as the input of the hash function h, and perform hash calculation to obtain the hash value H;

[0015] Use the hash value H as an index to query the hash table of the upstream switch. Compare the value at the query position with the flow ID. If they do not match, the query fails and proceed to step (2.4); if they match, the query succeeds and proceed to step (2.6).

[0016] (2.4) Call the hash functions h1, h2, h3, ..., h in the upstream switch. d The stream IDs are used as hash functions h1, h2, h3, ..., h1 respectively. d The input is processed and a hash calculation is performed to obtain hash values ​​H1, H2, H3, ..., H. d ;

[0017] H1, H2, H3, ..., H d As an index operation, increment the count value of the corresponding counter in each row of CM-Sketch by 1, and then add a label SketchFlag to mark the count of this business flow in CM-Sketch;

[0018] Read the count values ​​UpstreamSketchData-1, UpstreamSketchData-2, ..., UpstreamSketchData-d of each line in CM-Sketch, as well as the tag SketchFlag, write them all into the header of the data packet, and then send them to the downstream switch to proceed to step (2.5).

[0019] (2.5) After receiving the data packet, the downstream switch parses the data packet and extracts the flow ID, tag SketchFlag, and count values ​​DownstreamSketchData-1, DownstreamSketchData-2, ..., DownstreamSketchData-d;

[0020] Then call the hash functions h1, h2, ..., h in the downstream switches. d The stream IDs are used as hash functions h1, h2, ..., h3 respectively. d The input is processed and a hash calculation is performed to obtain hash values ​​H1, H2, H3, ..., H. d ;

[0021] H1, H2, H3, ..., H dAs an indexing operation, the counter value corresponding to each row in CM-Sketch is incremented by 1;

[0022] Compare the count values ​​of each row in CM-Sketch before and after the update. If the count values ​​at the corresponding positions are inconsistent, a custom data packet is generated, the flow ID of the current flow is written into the packet header and sent to the upstream switch, and the data packet of the current service flow is forwarded normally according to the flow table, and then jump to step (3); otherwise, no custom data packet is generated, the current data packet is forwarded directly according to the flow table, and then jump to step (3).

[0023] (2.6) Increment the counter value of the corresponding counter according to the hash value H, and add a tag HashTableFlag to mark the count of this service flow in the hash table. Then read the count value HashTableData and tag HashTableFlag in the hash table, write them all into the packet header and send them to the downstream switch, and execute step (2.7).

[0024] (2.7) After receiving the data packet, the downstream switch parses the data packet and extracts the flow ID, HashTableFlag and HashTableData of the data packet;

[0025] Then, the hash function h in the downstream switch is called, with the flow ID as the input to the hash function h, and the hash value H is obtained.

[0026] Use the hash value H as an index to operate the counter at the corresponding position in each row of the hash table. Set the counter of the first row to equal the stream ID, set the counter of the second row to +1, and set the counter of the third row to equal the difference between the counters of the upstream and downstream second rows. After the update is completed, proceed to step (3).

[0027] (2.8) When the upstream switch receives a custom data packet, it parses the data packet and extracts the flow ID recorded in the data packet;

[0028] Then, the hash function h in the upstream switch is called, and the flow ID is used as the input of the hash function h to perform hash calculation and obtain the hash value H;

[0029] The hash value H is used as an index to operate the counter at the corresponding position in the first row of the hash table. First, if the count value in the counter is 0 or equal to the stream ID, the count value is set to equal the stream ID, the second row counter is not operated, and then the custom data packet is discarded; if the value is not equal to the stream ID and is not 0, then no counter in the hash table is operated, and the current custom data packet is directly discarded.

[0030] (3) Detect packet loss rate;

[0031] With a fixed time interval as the cycle, after each cycle of measurement, data is read from the hash table of the fine-grained counting data structure of the downstream switch, and then the packet loss rate is calculated based on the service flow ID, the number of data packets arriving at the downstream switch, and the number of packet losses on the link.

[0032] The objective of this invention is achieved as follows:

[0033] This invention is a method for measuring packet loss rate in single-hop transmission of programmable network services. First, specific data structures are deployed on upstream and downstream switches. Then, the coordination mechanism of upstream and downstream switches is used to detect service flows that may cause packet loss in real time on the downstream switch and record their flow IDs. Then, fine-grained statistics are performed on this part of the service flow on the upstream and downstream switches to obtain the packet loss rate of the service flow that causes packet loss on a single hop.

[0034] Meanwhile, the programmable network service single-hop transmission packet loss rate measurement method of the present invention also has the following beneficial effects:

[0035] (1) Compared with traditional measurement methods based on specific protocols and specific hardware devices, this solution does not rely on any protocol or specific hardware device and can realize service-level packet loss rate measurement in programmable networks, thus improving measurement accuracy and convenience.

[0036] (2) In the upstream and downstream interaction mechanism, the Cm-Sketch data structure, which is commonly used in network measurement, was used. However, its counting function was only used to detect packet loss in real time without paying attention to its counting accuracy. The data structure was used to reduce the error caused by hash collisions, which reduced the false alarm rate of packet loss service flow and also reduced hardware resource consumption. Attached Figure Description

[0037] Figure 1 This is a flowchart of a method for measuring packet loss rate in single-hop transmission of programmable network services according to the present invention.

[0038] Figure 2 This is a schematic diagram of the coarse-grained counting data structure Cm-Sketch;

[0039] Figure 3 This is a schematic diagram of the structure of a hash table, a fine-grained counting data structure of the upstream switch;

[0040] Figure 4 This is a schematic diagram of the structure of a hash table, a fine-grained counting data structure for downstream switches.

[0041] Figure 5 This is a schematic diagram of a data packet structure carrying coarse-grained counting data structure information;

[0042] Figure 6 This is a diagram illustrating a custom data packet structure;

[0043] Figure 7 This is a schematic diagram of a data packet structure carrying fine-grained counting data structure information; Detailed Implementation

[0044] The specific embodiments of the present invention will now be described with reference to the accompanying drawings to enable those skilled in the art to better understand the invention. It should be particularly noted that in the following description, detailed descriptions of known functions and designs that might obscure the main content of the invention will be omitted here.

[0045] Example

[0046] Figure 1 This is a flowchart of a method for measuring packet loss rate in single-hop transmission of programmable network services according to the present invention.

[0047] In this embodiment, as Figure 1 As shown, the present invention provides a method for measuring the single-hop packet loss rate of programmable network services, which mainly includes three steps: S1, the cooperation mechanism between upstream and downstream switches; S2, fine-grained statistics of service flows based on data structures; S3, single-hop packet loss rate measurement; the three steps are described in detail below.

[0048] S1, the collaboration mechanism between upstream and downstream switches;

[0049] Traditional packet loss rate measurement schemes have certain limitations and are not suitable for service-level packet loss rate measurement. While packet loss rate measurement schemes based on programmable networks offer the possibility of service-level measurement, the technical challenge of this scheme lies in the limited hardware resources of programmable network devices. Therefore, this invention designs a scheme for collaborative measurement between upstream and downstream switches, so as to enable downstream switches to detect service flows that may cause packet loss in real time.

[0050] To implement the collaboration mechanism between upstream and downstream switches, we need to deploy specific data structures on both switches, specifically:

[0051] S1.1 Deploy a coarse-grained counting data structure (CM Sketch) on the upstream and downstream switches;

[0052] Deploy a CM Sketch with row d on the upstream and downstream switches. The structure of the CM Sketch is as follows: Figure 2 As shown, w counters are deployed in each row, labeled 0, 1, ..., w-1 from left to right; simultaneously, a hash function is deployed in each row, with each hash function having a different structure. The hash functions for each row are denoted as h1, h2, h3, ..., h... dEach hash function calculates a hash value that is an integer in the range [0, w-1], and each hash value corresponds to a counter number.

[0053] S1.2 Deploy fine-grained counting data structure hash tables on upstream and downstream switches;

[0054] Deploy a 2-row fine-grained counting data structure hash table on the upstream switch. The structure of the fine-grained counting data structure hash table is as follows: Figure 3 As shown, the first row is used to store the stream IDs (F1, F2, ..., Fm), and the second row is used to store the number of data packets (data). In addition, m counters are deployed on each row, and the counters are labeled 0, 1, ..., m-1 from left to right. The same hash function h is deployed on each row. The hash value calculated by the hash function is an integer in the interval [0, m-1], which corresponds to the counter labeled with the hash value.

[0055] Deploy a 3-row fine-grained counting data structure hash table in the downstream switch. The structure of the fine-grained counting data structure hash table is as follows: Figure 4 As shown, the first row stores the flow IDs (F1, F2, ..., Fm), the second row stores the number of data packets arriving at the downstream switch, and the third row stores the number of detected link packet losses. Each row contains m counters, labeled 0, 1, ..., m-1 from left to right. Each row also contains the same hash function h, where the hash value calculated by the hash function is an integer within the interval [0, m-1], corresponding to a counter labeled with that hash value.

[0056] S2. Perform fine-grained statistics on business flows based on data structures;

[0057] S2.1 Initialize the data structure;

[0058] S2.2 When the upstream switch receives a data packet, it first parses the data packet and extracts the frame header and flow IDF of the data packet. If the frame header type of the data packet is different from the frame header type of the custom data packet, then proceed to step S2.3; otherwise, it determines that the received data packet is a custom data packet and then proceeds to operation S2.8.

[0059] S2.3 Call the hash function h in the upstream switch, take the flow ID as the input of the hash function h, and perform hash calculation to obtain the hash value H;

[0060] Use the hash value H as an index to query the hash table of the upstream switch, compare the value at the query position with the flow ID, if they do not match, proceed to step S2.4; if they match, proceed to step S2.6.

[0061] S2.4, Call the hash functions h1, h2, h3, ..., h in the upstream switch. d The stream IDs are used as hash functions h1, h2, h3, ..., h1 respectively. d The input is processed and a hash calculation is performed to obtain hash values ​​H1, H2, H3, ..., H. d ;

[0062] H1, H2, H3, ..., H d As an index operation, increment the count value of the corresponding counter in each row of CM-Sketch by 1, and then add a label SketchFlag to mark the count of this business flow in CM-Sketch;

[0063] Read the count values ​​UpstreamSketchData-1, UpstreamSketchData-2, ..., UpstreamSketchData-d of each line in CM-Sketch, as well as the tag SketchFlag, and write them all into the header of the data packet. The structure of the written data packet is as follows. Figure 5 As shown, it is then sent to the downstream switch and proceeds to step (2.5);

[0064] S2.5 After receiving the data packet, the downstream switch parses the data packet and extracts the flow ID, tag SketchFlag, and count values ​​DownstreamSketchData-1, DownstreamSketchData-2, ..., DownstreamSketchData-d.

[0065] Then call the hash functions h1, h2, ..., h in the downstream switches. d The stream IDs are used as hash functions h1, h2, ..., h3 respectively. d The input is processed and a hash calculation is performed to obtain hash values ​​H1, H2, H3, ..., H. d ;

[0066] H1, H2, H3, ..., H d As an indexing operation, the counter value corresponding to each row in CM-Sketch is incremented by 1;

[0067] The count values ​​of each row in CM-Sketch before and after the update are compared sequentially. If the count values ​​at corresponding positions are inconsistent, a custom data packet is generated. Then, the flow ID of the current flow is written into the packet header and sent to the upstream switch. The custom data packet format is as follows: Figure 6As shown, the data packets of the current service flow are forwarded normally according to the flow table, and then the process jumps to step S3; otherwise, no custom data packets are generated, the current data packets are forwarded directly according to the flow table, and then the process jumps to step S3.

[0068] S2.6. Increment the counter value corresponding to the hash value H by 1, and add a tag HashTableFlag to mark the count of this service flow in the hash table. Then, read the count value HashTableData and the tag HashTableFlag from the hash table, write them all into the packet header, and send them to the downstream switch. The packet structure is as follows. Figure 7 Proceed to step S2.7;

[0069] S2.7 After receiving the data packet, the downstream switch parses the data packet and extracts the flow ID, HashTableFlag, and HashTableData of the data packet;

[0070] Then, the hash function h in the downstream switch is called, with the flow ID as the input to the hash function h, and the hash value H is obtained.

[0071] Use the hash value H as an index to operate the counter at the corresponding position in each row of the hash table. Set the counter of the first row to equal the stream ID, increment the counter of the second row by 1, and set the counter of the third row to equal the difference between the counters of the upstream and downstream second rows. After the update is completed, proceed to step S3.

[0072] S2.8 When the upstream switch receives a custom data packet, it parses the data packet and extracts the flow ID recorded in the data packet;

[0073] Then, the hash function h in the upstream switch is called, and the flow ID is used as the input of the hash function h to perform hash calculation and obtain the hash value H;

[0074] The hash value H is used as an index to operate the counter at the corresponding position in the first row of the hash table. First, if the count value in the counter is 0 or equal to the stream ID, the count value is set to equal the stream ID, the second row counter is not operated, and then the custom data packet is discarded; if the value is not equal to the stream ID and is not 0, then no counter in the hash table is operated, and the current custom data packet is directly discarded.

[0075] S3, Single-hop transmission packet loss rate measurement;

[0076] With a fixed time interval as the cycle, after each cycle of measurement, data is read from the hash table of the fine-grained counting data structure of the downstream switch, and then the packet loss rate is calculated based on the service flow ID, the number of data packets arriving at the downstream switch, and the number of packet losses on the link.

[0077] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. A method for measuring packet loss rate in single-hop transmission of programmable network services, characterized in that, Includes the following steps: (1) Data structure deployment (1.1) Deploy a coarse-grained counting data structure CM Sketch with d rows on the upstream and downstream switches. Deploy w counters on each row, and label the counters from left to right as 0, 1, ..., w-1. At the same time, deploy a hash function on each row. The structure of each hash function is different. The hash functions for each row are denoted as h1, h2, h3, ..., h d Each hash function calculates a hash value that is an integer in the range [0, w-1], and each hash value corresponds to a counter number. (1.2) Deploy a fine-grained counting data structure hash table with 2 rows on the upstream switch. The first row is used to store the flow ID and the second row is used to store the number of packets. In addition, deploy m counters on each row, and the counters are labeled 0, 1, ..., m-1 from left to right. Deploy the same hash function h on each row. The hash value calculated by the hash function is an integer in the interval [0, m-1], which corresponds to the counter labeled with the hash value. (1.3) Deploy a fine-grained counting data structure hash table with 3 rows on the downstream switch. The first row is used to store the flow ID, the second row is used to store the number of data packets arriving at the downstream switch, and the third row is used to store the number of detected link packet loss. Deploy m counters on each row, and the counters are marked as 0, 1, ..., m-1 from left to right. Deploy the same hash function h on each row. The hash value calculated by the hash function is an integer in the interval [0, m-1], which corresponds to the counter labeled with the hash value. (2) Packet loss rate measurement (2.1) Initialize the data structure; (2.2) When the upstream switch receives a data packet, it first parses the data packet and extracts the frame header and flow ID F of the data packet. If the frame header type of the data packet is different from the frame header type of the custom data packet, then step (2.3) is executed; otherwise, it is determined that the received data packet is a custom data packet, and then operation (2.8) is executed. (2.3) Call the hash function h in the upstream switch, take the flow ID as the input of the hash function h, and perform hash calculation to obtain the hash value H; Use the hash value H as an index to query the hash table of the upstream switch. Compare the value at the query position with the flow ID. If they do not match, the query fails and proceed to step (2.4); if they match, the query succeeds and proceed to step (2.6). (2.4) Call the hash functions h1, h2, h3, ..., h in the upstream switch. d The stream IDs are used as hash functions h1, h2, h3, ..., h1 respectively. d The input is processed and a hash calculation is performed to obtain hash values ​​H1, H2, H3, ..., H. d ; H1, H2, H3, ..., H d As an index operation, increment the count value of the corresponding counter in each row of CM-Sketch by 1, and then add a label SketchFlag to mark the count of this business flow in CM-Sketch; Read the count values ​​UpstreamSketchData-1, UpstreamSketchData-2, ..., UpstreamSketchData-d of each line in CM-Sketch, as well as the tag SketchFlag, write them all into the header of the data packet, and then send them to the downstream switch to proceed to step (2.5). (2.5) After receiving the data packet, the downstream switch parses the data packet and extracts the flow ID, tag SketchFlag, and count values ​​DownstreamSketchData-1, DownstreamSketchData-2, ..., DownstreamSketchData-d; Then call the hash functions h1, h2, ..., h in the downstream switches. d The stream IDs are used as hash functions h1, h2, ..., h3 respectively. d The input is processed and a hash calculation is performed to obtain hash values ​​H1, H2, H3, ..., H. d ; H1, H2, H3, ..., H d As an indexing operation, the counter value corresponding to each row in CM-Sketch is incremented by 1; Compare the count values ​​of each row in CM-Sketch before and after the update. If the count values ​​at the corresponding positions are inconsistent, a custom data packet is generated, the flow ID of the current flow is written into the packet header and sent to the upstream switch, and the data packet of the current service flow is forwarded normally according to the flow table, and then jump to step (3); otherwise, no custom data packet is generated, the current data packet is forwarded directly according to the flow table, and then jump to step (3). (2.6) Increment the counter value of the corresponding counter according to the hash value H, and add a tag HashTableFlag to mark the count of this service flow in the hash table. Then read the count value HashTableData and tag HashTableFlag in the hash table, write them all into the packet header and send them to the downstream switch, and execute step (2.7). (2.7) After receiving the data packet, the downstream switch parses the data packet and extracts the flow ID, HashTableFlag and HashTableData of the data packet; Then, the hash function h in the downstream switch is called, with the flow ID as the input to the hash function h, and the hash value H is obtained. Use the hash value H as an index to operate the counter at the corresponding position in each row of the hash table. Set the counter of the first row to equal the stream ID, set the counter of the second row to +1, and set the counter of the third row to equal the difference between the counters of the upstream and downstream second rows. After the update is completed, proceed to step (3). (2.8) When the upstream switch receives a custom data packet, it parses the data packet and extracts the flow ID recorded in the data packet; Then, the hash function h in the upstream switch is called, and the flow ID is used as the input of the hash function h to perform hash calculation and obtain the hash value H; The hash value H is used as an index to operate the counter at the corresponding position in the first row of the hash table. First, if the count value in the counter is 0 or equal to the stream ID, the count value is set to equal the stream ID, the second row counter is not operated, and then the custom data packet is discarded. If the value is not equal to the stream ID and is not 0, then no counter in the hash table will be operated, and the current custom data packet will be discarded directly. (3) Detect packet loss rate; With a fixed time interval as the cycle, after each cycle of measurement, data is read from the hash table of the fine-grained counting data structure of the downstream switch, and then the packet loss rate is calculated based on the service flow ID, the number of data packets arriving at the downstream switch, and the number of packet losses on the link.

Images