Communication delay estimation method and related device

Abstract

An embodiment of the present application provides a communication delay estimation method and related devices, the method comprising: a network device obtains flow information from a received first data packet, the flow information being identification information of a first flow to which the first data packet belongs; then, determining a first recording unit for mapping the first flow based on the flow information; the first recording unit includes the maximum delay of a single data packet of the target flow passing through the target device, and the number of data packets of the target flow received by the network device; the target flow includes a flow mapped to the first recording unit; then, updating the number of packets, and when the delay of the first data packet passing through the target device is greater than the first delay, updating the first delay to the delay of the first data packet passing through the target device; when the number of packets is less than a threshold, the first delay is used to estimate the tail delay of the first flow passing through the target device. The present application can save storage and processing resources for each flow, so that the tail delay of the full flow can be estimated on existing resources.

Description

Technical Field

[0001] The present invention relates to the technical field of communication network measurement, and in particular to a communication delay estimation method and related devices. Background Art

[0002] Network performance monitoring has always been a key topic within the highly valuable field of network measurement. Latency measurement has been a hot topic in academia and an area of continuous innovation within the industry. Traditional tools that use sampling or probes to measure end-to-end latency (such as ping and traceroute) have been widely deployed and used, playing a significant role for a long time. Currently, with the rapid expansion of network scale and the continuous improvement of network device capabilities, users and products are placing more sophisticated demands on network operations and maintenance. Network managers hope to improve the ability to quickly detect and locate network anomalies, such as quickly discovering distributed denial of service (DDOS) attacks or detecting potential microbursts in data centers. These more challenging issues rely on more fine-grained measurements of flow performance metrics, such as tail latency.

[0003] Currently, commonly used flow tail latency measurement technologies can be divided into two categories: one is in-network measurement technology, represented by NetFlow. This technology supports filtering, sampling, and copying relevant packet data within network devices, and then uploading the packet data to the controller for tail latency measurement. The other is in-band operation administration and maintenance (iOAM) technology, represented by in-band network telemetry (INT). This technology directly encapsulates the required operation administration and maintenance (OAM) information (such as latency measurement data) in data packets. At the last hop in the monitoring domain, the edge node, the OAM data is removed and forwarded to the controller for tail latency measurement. However, NetFlow technology supports network devices obtaining relevant information about data packets and sending it to the controller, but this brings a large additional load and resource consumption, making sampling or flow selection often required in actual use, making it difficult to cover the tail delay measurement of all flows passing through the network device. iOAM technology, represented by INT, carries the required information in the packet header and uploads the collected information to the controller at the last hop, reducing the upload cost. However, this also occupies link bandwidth, resulting in increased flow completion time and reduced throughput. Therefore, it is often not deployed packet by packet, which also makes it difficult to achieve tail delay measurement for all flows.

[0004] In summary, how to reduce the resource overhead of network devices during the tail delay measurement process of each flow is a technical problem that those skilled in the art urgently need to solve. Summary of the Invention

[0005] An embodiment of the present application provides a communication delay estimation method and related apparatus, which can reduce the resource overhead of a network device during the tail delay measurement process of each flow, thereby enabling the network device to measure and estimate the tail delay of all flows passing through the network device based on existing storage and processing resources.

[0006] In a first aspect, the present application provides a communication delay estimation method, the method comprising: a network device obtains flow information from a received first data packet, the flow information being identification information of a first flow to which the first data packet belongs; the network device determines a first recording unit for mapping the first flow based on the flow information; the first recording unit includes a first delay and a number of packets, the first delay being the maximum delay for a single data packet of the target flow to pass through the target device, and the number of packets being the number of data packets of the target flow received by the network device; the target flow includes a flow mapped to the first recording unit; the network device updates the number of packets, and when the delay for the first data packet to pass through the target device is greater than the first delay, updates the first delay to the delay for the first data packet to pass through the target device; when the number of packets is less than a threshold, the first delay is used to estimate the tail delay of the first flow passing through the above-mentioned target device.

[0007] Optionally, the target device is the network device; or, the network device is a device on a data plane in a network, the network device is the last hop device of the target flow in the data plane, and the target device includes multiple devices that the single data packet passes through in the data plane.

[0008] In an embodiment of the present application, the above-mentioned network device is any network device used to forward data in the data plane. By configuring the above-mentioned first recording unit for each flow received by the network device in the network device to record the maximum delay of the data packets in each flow passing through the above-mentioned target device, the tail delay of each flow passing through the target device is estimated when the number of packets in each flow is less than a threshold. Since in the embodiment of the present application, each flow is mapped to the corresponding first recording unit, the first recording unit only records the maximum delay of the data packets in the flow passing through the target device, rather than recording the delay of all data packets, thereby saving storage resources. Compared with the existing NetFlow and INT technical solutions, the embodiment of the present application does not need to send a large number of data packets or a large amount of measurement data to the controller, thereby saving additional link bandwidth resources. In addition, since in the embodiment of the present application, the resources required for measuring and estimating the tail delay of a single flow are saved, there is no need to filter and sample the flow when measuring the tail delay of the flow, that is, compared with the existing solution, the present application can measure and estimate the tail delay of all flows passing through the network device based on the existing storage and processing resources of the network device.

[0009] In one possible implementation, the above-mentioned network device is a device on a data plane in a network, and the network device is the last-hop device of the data packet of the target flow in the data plane. The target device includes multiple devices through which the single data packet passes in the data plane; the delay of the first data packet passing through the target device is the delay of the first data packet passing through the multiple devices; the first data packet includes the delay of the first data packet passing through other devices, and the other devices are other network devices among the multiple devices except the network device.

[0010] In an embodiment of the present application, a delay acquisition module can be deployed on all network devices that the first flow passes through in the data plane to obtain the delay of each data packet of the first flow passing through each network device, and then carry these delays in the data packet and send it to the above-mentioned last-hop device. The last-hop device also obtains the delay of each data packet passing through itself, and then calculates the total delay of each data packet passing through all network devices, and takes the maximum delay and records it in the above-mentioned first recording unit. This implementation method only requires the last-hop device to record and process information such as delay, and does not require every network device that the first flow passes through to record it, thereby saving storage and processing resources of the network device.

[0011] In one possible embodiment, the above-mentioned network device is a device on the data plane in the network, and the network device is the last-hop device of the data packet of the target flow in the data plane. The target device includes multiple devices through which the single data packet passes in the data plane; the delay of the first data packet passing through the target device is the delay of the first data packet passing through the multiple devices; the first data packet includes the timestamp of the entry of the first-hop device through which the first data packet passes in the data plane, and the timestamp is used to calculate the delay of the first data packet passing through the multiple devices.

[0012] In an embodiment of the present application, the ingress timestamp of the data packet of the flow can be obtained on the first device that the first flow passes through in the data plane, and the timestamp is carried in the data packet and sent to the above-mentioned last-hop device. After the last-hop device receives the data packet, it can obtain the timestamp of the data packet sent from itself. The latter timestamp minus the former timestamp is the delay of the data packet passing through the above-mentioned target device. Then, the maximum delay of the flow passing through the above-mentioned target device is taken and recorded in the above-mentioned first recording unit. Similarly, this implementation method can make it only necessary to record and process information such as delay at the last-hop device, without the need for recording every network device that the first flow passes through, thereby saving storage and processing resources of the network device.

[0013] In one possible implementation, the data packets of the target flow received by the above-mentioned network device are data packets received by the network device within an estimation window, and the estimation window is a time window used to estimate the tail delay of the first flow passing through the above-mentioned target device; the first delay is the maximum delay of a single data packet of the target flow passing through the target device, including: the first delay is the maximum delay of a single data packet of the target flow passing through the target device within the time window; the tail delay of the first flow passing through the above-mentioned target device is the tail delay of the first flow passing through the target device within the time window.

[0014] In an embodiment of the present application, by dividing the transmission process of a stream into one or more time windows and estimating the tail delay of the stream in each time window, the feasibility and operability of the network device can be improved.

[0015] In one possible implementation, there are multiple first recording units mapped to the first flow; the method also includes: after the estimation window ends, when the number of packets recorded by at least one of the multiple first recording units is less than the threshold, the network device determines that the first delay recorded in the first unit is the tail delay estimate of the first flow passing through the target device within the estimation window, and sends the first delay recorded in the first unit to the controller; the first unit is the unit with the smallest first delay recorded among the multiple first recording units.

[0016] In an embodiment of the present application, when the number of packets of the target flow received by the network device within the estimation window is less than a threshold, the smallest first delay recorded in the plurality of first recording units can be used to estimate the tail delay of the first flow passing through the target device. This is simple to implement and highly operable. This is because, when the number of packets is small, the tail delay is slightly different from the maximum delay of the packets recorded in the first recording unit passing through the target device, so the maximum delay can be used to estimate the tail delay of the flow.

[0017] In a possible implementation, the first recording unit of the first stream mapping is multiple; the method further includes: when the estimation window ends, the network device sends the information recorded in the multiple first recording units to the controller.

[0018] In an embodiment of the present application, the network device may send the information recorded by the first recording unit to the controller, and the controller may perform an estimated calculation of the tail delay, thereby reducing the processing burden of the network device.

[0019] In one possible implementation, when the number of packets recorded in the first recording unit is greater than or equal to the threshold, the method further includes: the network device allocates a second recording unit to the first recording unit; the second recording unit is used to record the delay of each of n data packets passing through the target device, the n data packets being data packets in the target flow received by the network device after the first data packet, and n being an integer greater than 1; the n delays are used to estimate the tail delay of the first flow passing through the above-mentioned target device.

[0020] In an embodiment of the present application, when the number of packets is greater than the above-mentioned threshold, the tail delay is significantly different from the maximum delay of the data packets recorded in the above-mentioned first recording unit passing through the target device, so it is necessary to record the delay additionally to estimate the tail delay of the flow. Therefore, the above-mentioned second recording unit is configured to record the delay of part of the data packets of the subsequent received flow passing through the above-mentioned target device, and then estimate the tail delay of the flow based on the delay information recorded by the second recording unit to improve the accuracy of the estimation.

[0021] In one possible embodiment, the method also includes: the network device receives a second data packet, which is a data packet of the target flow; the network device obtains the second delay among the n delays of the second recording unit according to a polling manner; when the delay of the second data packet passing through the target device is greater than the second delay, the network device updates the second delay in the second recording unit to the delay of the second data packet passing through the target device.

[0022] In an embodiment of the present application, the delay in the second recording unit may be updated by polling to ensure that the n delays are the maximum n delays.

[0023] In one possible embodiment, the method also includes: the network device receives a third data packet, which is a data packet of the target flow, and before the third data packet and after the second data packet, the network device receives m data packets of the target flow, where m is an integer greater than 1; the network device randomly obtains a third delay from the n delays in the second recording unit; when the delay of the third data packet passing through the target device is less than the third delay, the network device updates the third delay in the second recording unit to the delay of the third data packet passing through the target device.

[0024] When a large number of packets are received within the estimation window, using the n largest delays to estimate the tail delay of the flow can also result in significant errors. For example, if the number of packets reaches 10,000 and n is 10, and if these 100 delays represent the 10 largest delays, when estimating the 99th percentile tail delay, the 9900th delay, sorted from smallest to largest, of the 10,000 packets passing through the target device, is the accurate 99th percentile tail delay. However, each of the 10 largest delays differs significantly from the 9900th delay, making it impossible to use these 10 delays to estimate the 99th percentile tail delay. Therefore, in an embodiment of the present application, in the process of updating the above-mentioned second recording unit, based on the above-mentioned polling update method, every certain number of data packets, one of the n delays in the second recording unit is randomly selected and updated to a smaller delay, so that the n delays recorded in the second recording unit are no longer the maximum delay, but as many other percentile delay information as possible are recorded in the second recording unit according to a certain probability, thereby reducing the estimation error of the tail delay.

[0025] In a possible implementation, the method further includes: when the estimation window ends, the network device estimates the tail delay of the first flow passing through the target device within the estimation window based on the n delays in the second recording unit.

[0026] Optionally, the tail delay of the first flow passing through the above-mentioned target device is the p-percentile tail delay, and p is greater than 80 and less than 100; the network device estimates the tail delay of the first flow passing through the above-mentioned target device based on the n delays in the second recording unit, including: the network device determines the w-th delay among the n delays as the estimated tail delay value of the first flow passing through the target device within the estimation window, and the w-th delay is the delay ranked w among the n delays sorted from small to large, and w is an integer greater than 0 and less than or equal to n.

[0027] In an embodiment of the present application, based on the above-mentioned method of updating the above-mentioned second recording unit, the above-mentioned p-percentile tail delay can be estimated by using the delay among the n delays in the second recording unit, and the above-mentioned w can be determined by calculating the mathematical expectation of the number of delays greater than the p-percentile tail delay among the above-mentioned n delays.

[0028] In a possible implementation, the method further includes: when the estimation window ends, the network device sends the information recorded in the first recording unit and the second recording unit to a controller.

[0029] In an embodiment of the present application, the network device may send the information recorded by the first recording unit and the second recording unit to a controller, and the controller may perform an estimated calculation of the tail delay, thereby reducing the processing burden of the network device.

[0030] In one possible implementation, the network device determines the first record unit of the first flow mapping based on the flow information, including: the network device maps the flow information to multiple bits in a bitmap through multiple hash functions; a bit in the bitmap is associated with one of the first record units; and the network device determines the first record unit of the first flow mapping based on the multiple bits.

[0031] In the embodiment of the present application, the stream is mapped to the corresponding first recording unit through hash calculation, which is simple to calculate, convenient to process, and can also save storage resources.

[0032] In a possible implementation, the first recording unit further includes an XOR result of the number of streams and the stream information, where the number of streams is the number of target streams, and the XOR result of the stream information is a result obtained by performing an XOR operation on the stream information of the target stream.

[0033] In an embodiment of the present application, the XOR result of the stream number and the stream information is recorded in the first recording unit, which can be used to mark the recording unit where the hash conflict occurs, and can be used to subsequently decode the stream information of the hash conflict.

[0034] In a second aspect, the present application provides a communication delay estimation method, the method comprising: a network device obtaining flow information from the received first data packet, the flow information being used to indicate that the first data packet is a data packet in a first flow; the network device determining a first recording unit for mapping the first flow based on the flow information; the first recording unit being used to record a first delay and a number of packets, the first delay being the maximum delay for a single data packet of the target flow to pass through the target device, the number of packets being the number of data packets of the target flow received by the network device; the target flow including the flow mapped to the first recording unit; the network device updating the number of packets, and When the delay of the first data packet passing through the target device is greater than the first delay, the first delay is updated to the delay of the first data packet passing through the target device; when the number of packets is greater than or equal to a threshold, the network device allocates a second recording unit to the first recording unit; the second recording unit is used to record the delay of each of n data packets passing through the target device, where the n data packets are part of the data packets of the target flow received by the network device after the first data packet, and n is an integer greater than 1; the n delays are used to estimate the tail delay of the first flow passing through the above-mentioned target device.

[0035] In an embodiment of the present application, based on the above-mentioned method of updating the above-mentioned second recording unit, the above-mentioned p-percentile tail delay can be estimated by using the delay among the n delays in the second recording unit, and the above-mentioned w can be determined by calculating the mathematical expectation of the number of delays greater than the p-percentile tail delay among the above-mentioned n delays.

[0036] In a third aspect, the present application provides a communication delay estimation method, which includes: a controller receives information recorded in multiple first recording units, the information recorded in each of the first recording units includes a first delay and the number of packets, the first delay is the maximum delay of a single data packet of the target flow passing through the target device, and the number of packets is the number of data packets of the target flow received by the network device; the target flow includes a flow mapped to the first recording unit, and the target flow includes a first flow; when the number of packets is less than a threshold, the controller estimates the tail delay of the first flow passing through the above-mentioned target device based on the first delay recorded in the multiple first recording units.

[0037] Optionally, the target device is the network device; or, the network device is a device on a data plane in a network, the network device is the last hop device of the target flow in the data plane, and the target device includes multiple devices that the single data packet passes through in the data plane.

[0038] In an embodiment of the present application, the controller can perform an estimation calculation of the tail delay. When the number of packets of the target flow received by the network device within the estimation window is less than a threshold, the first delays recorded in the plurality of first recording units can be used to estimate the tail delay of the first flow passing through the target device. This is simple to implement and highly operable. This is because when the number of packets is small, the difference between the tail delay and the maximum delay of the packets recorded in the first recording unit passing through the target device is small, so the maximum delay can be used to estimate the tail delay of the flow.

[0039] In one possible embodiment, the method also includes: the controller also receives an identifier of an estimation window, which indicates that the information recorded in the multiple first recording units is recorded within the estimation window, and the estimation window is a time window for estimating the tail delay of the first flow passing through the above-mentioned target device; the controller estimates the tail delay of the first flow passing through the above-mentioned target device based on the first delay recorded in the multiple first recording units, including: the controller determines that the first delay recorded in the first unit is the estimated value of the tail delay of the first flow passing through the target device within the estimation window; the first unit is the unit with the smallest first delay recorded in the multiple first recording units.

[0040] In an embodiment of the present application, by dividing the transmission process of the stream into one or more time windows and estimating the tail delay of the stream in each time window, the feasibility and operability of the network device can be improved. On this basis, the tail delay of the first stream passing through the target device can be estimated by using the minimum first delay recorded in the above-mentioned multiple first recording units. This is because if the first delays in the multiple first recording units are different, it means that a hash conflict has occurred in a recording unit, and only a delay greater than the recorded first delay can be updated to the first recording unit. Therefore, the minimum first delay in the multiple first recording units mapped by the first stream is a relatively accurate estimate of the tail delay of the first stream. Therefore, using the minimum first delay to estimate the tail delay of the first stream can improve the accuracy of the estimation.

[0041] In a fourth aspect, the present application provides a communication delay estimation method, which includes: a controller receives information recorded in a first recording unit and a second recording unit, the information recorded in the first recording unit includes a first delay and the number of packets, the first delay is the maximum delay of a single data packet of the first flow passing through the target device, and the number of packets is the number of data packets of the first flow received by the network device; the second recording unit is associated with the first recording unit, and the information recorded in the second recording unit includes the delay of each of the n data packets in the first flow passing through the target device, where n is an integer greater than 1; the controller estimates the tail delay of the first flow passing through the above-mentioned target device based on the n delays.

[0042] In an embodiment of the present application, when the number of packets of the target flow received by the network device in the above estimation window is greater than a threshold, the recorded information may also be sent to the controller, and the controller may perform an estimation calculation of the tail delay.

[0043] In one possible implementation, the method further includes: the controller further receiving an identifier of an estimation window, where the identifier indicates that the information recorded in the first recording unit and the second recording unit is recorded within the estimation window, where the estimation window is a time window for estimating the tail delay of the first flow passing through the target device;

[0044] The tail delay of the first flow passing through the target device is a p-percentile tail delay, where p is greater than 80% and less than 100%. The controller estimates the tail delay of the first flow passing through the target device based on the n delays, including:

[0045] The controller determines the wth delay among the n delays as the estimated tail delay of the first flow passing through the target device within the estimation window, the wth delay is the delay ranked w among the n delays sorted from small to large, and w is an integer greater than 0 and less than or equal to n.

[0046] In an embodiment of the present application, the p-percentile tail delay can be estimated using the delay among the n delays in the second recording unit, and the w can be determined by calculating the mathematical expectation of the number of delays greater than the p-percentile tail delay among the n delays.

[0047] In a fifth aspect, the present application provides a network device, the network device including a processing unit, configured to:

[0048] Acquire flow information from the received first data packet, where the flow information is identification information of a first flow to which the first data packet belongs;

[0049] Determining a first record unit of the first flow mapping based on the flow information; the first record unit includes a first delay and a number of packets, the first delay being a maximum delay for a single data packet of the target flow to pass through the target device, and the number of packets being the number of data packets of the target flow received by the network device; the target flow includes the flow mapped to the first record unit;

[0050] Update the number of packets, and when the delay of the first data packet passing through the target device is greater than the first delay, update the first delay to the delay of the first data packet passing through the target device; when the number of packets is less than a threshold, the first delay is used to estimate the tail delay of the first flow passing through the above-mentioned target device.

[0051] Optionally, the target device is the network device; or, the network device is a device on a data plane in a network, the network device is the last hop device of the target flow in the data plane, and the target device includes multiple devices that the single data packet passes through in the data plane.

[0052] In one possible implementation, the above-mentioned network device is a device on a data plane in a network, and the network device is the last-hop device of the data packet of the target flow in the data plane. The target device includes multiple devices through which the single data packet passes in the data plane; the delay of the first data packet passing through the target device is the delay of the first data packet passing through the multiple devices; the first data packet includes the delay of the first data packet passing through other devices, and the other devices are other network devices among the multiple devices except the network device.

[0053] In one possible embodiment, the above-mentioned network device is a device on the data plane in the network, and the network device is the last-hop device of the data packet of the target flow in the data plane. The target device includes multiple devices through which the single data packet passes in the data plane; the delay of the first data packet passing through the target device is the delay of the first data packet passing through the multiple devices; the first data packet includes the timestamp of the entry of the first-hop device through which the first data packet passes in the data plane, and the timestamp is used to calculate the delay of the first data packet passing through the multiple devices.

[0054] In one possible implementation, the data packets of the target flow received by the above-mentioned network device are data packets received by the network device within an estimation window, and the estimation window is a time window used to estimate the tail delay of the first flow passing through the above-mentioned target device; the first delay is the maximum delay of a single data packet of the target flow passing through the target device, including: the first delay is the maximum delay of a single data packet of the target flow passing through the target device within the time window; the tail delay of the first flow passing through the above-mentioned target device is the tail delay of the first flow passing through the target device within the time window.

[0055] In one possible implementation, there are multiple first recording units mapped to the first flow; the above-mentioned processing unit is also used to: after the end of the estimation window, when the number of packets recorded by at least one unit among the multiple first recording units is less than the threshold, determine that the first delay recorded in the first unit is the estimated value of the tail delay of the first flow passing through the target device within the estimation window, and send the first delay recorded in the first unit to the controller; the first unit is the unit with the smallest first delay recorded among the multiple first recording units.

[0056] In a possible implementation, the first recording unit of the first stream mapping is multiple; the network device further includes a sending unit, configured to send information recorded in the multiple first recording units to the controller when the estimation window ends.

[0057] In a possible implementation, when the number of packets recorded in the first recording unit is greater than or equal to the threshold, the processing unit is further configured to:

[0058] A second recording unit is allocated to the first recording unit; the second recording unit is used to record the delay of each of n data packets passing through the target device, where the n data packets are data packets of the target flow received by the network device after the first data packet, and n is an integer greater than 1; the n delays are used to estimate the tail delay of the first flow passing through the above-mentioned target device.

[0059] In a possible implementation, the network device further includes a receiving unit, configured to receive a second data packet, where the second data packet is a data packet of the target flow;

[0060] The above-mentioned processing unit is also used to obtain the second delay among the n delays of the second recording unit according to a polling manner; when the delay of the second data packet passing through the target device is greater than the second delay, the second delay in the second recording unit is updated to the delay of the second data packet passing through the target device.

[0061] In one possible implementation, the receiving unit is further configured to receive a third data packet, where the third data packet is a data packet of the target flow, and before the third data packet and after the second data packet, the receiving unit receives m data packets of the target flow, where m is an integer greater than 1.

[0062] The above-mentioned processing unit is also used to randomly obtain a third delay from the n delays of the second recording unit; when the delay of the third data packet passing through the target device is less than the third delay, the third delay in the second recording unit is updated to the delay of the third data packet passing through the target device.

[0063] In a possible implementation, the processing unit is further configured to: when the estimation window ends, estimate the tail delay of the first flow passing through the target device within the estimation window based on the n delays in the second recording unit.

[0064] Optionally, the tail delay of the first stream passing through the above-mentioned target device is the p-percentile tail delay, and p is greater than 80 and less than 100; the above-mentioned processing unit estimates the tail delay of the first stream passing through the above-mentioned target device based on the n delays in the second recording unit, including: determining the w-th delay among the n delays as the estimated tail delay value of the first stream passing through the target device within the estimation window, the w-th delay is the delay ranked w among the n delays sorted from small to large, and w is an integer greater than 0 and less than or equal to n.

[0065] In a possible implementation, the network device further includes a sending unit, configured to send the information recorded in the first recording unit and the second recording unit to a controller when the estimation window ends.

[0066] In one possible implementation, the processing unit determines the first record unit of the first flow mapping based on the flow information, including: mapping the flow information to multiple bits in a bitmap through multiple hash functions; a bit in the bitmap is associated with one of the first record units; and the network device determines the first record unit of the first flow mapping based on the multiple bits.

[0067] In a possible implementation, the first recording unit further includes an XOR result of the number of streams and the stream information, where the number of streams is the number of target streams, and the XOR result of the stream information is a result obtained by performing an XOR operation on the stream information of the target stream.

[0068] In a sixth aspect, the present application provides a network device, the network device including a processing unit, configured to:

[0069] Acquire flow information from the received first data packet, where the flow information is used to indicate that the first data packet is a data packet in a first flow;

[0070] Determining a first recording unit of the first flow mapping based on the flow information; the first recording unit is used to record a first delay and a number of packets, the first delay being a maximum delay for a single data packet of the target flow to pass through the target device, and the number of packets being the number of data packets of the target flow received by the network device; the target flow includes the flow mapped to the first recording unit;

[0071] updating the number of packets, and, if a delay of the first data packet passing through the target device is greater than the first delay, updating the first delay to a delay of the first data packet passing through the target device;

[0072] When the number of packets is greater than or equal to a threshold, a second recording unit is allocated to the first recording unit; the second recording unit is used to record the delay of each of the n data packets passing through the target device, where the n data packets are part of the data packets of the target flow received by the network device after the first data packet, and n is an integer greater than 1; the n delays are used to estimate the tail delay of the first flow passing through the above-mentioned target device.

[0073] In a seventh aspect, the present application provides a controller, comprising:

[0074] a receiving unit, configured to receive information recorded in a plurality of first recording units, wherein the information recorded in each of the first recording units includes a first delay and a number of packets, wherein the first delay is a maximum delay for a single data packet of a target flow to pass through a target device, and the number of packets is a number of data packets of the target flow received by the network device; the target flow includes a flow mapped to the first recording unit, and the target flow includes a first flow;

[0075] The processing unit is configured to estimate a tail delay of the first flow passing through the target device based on the first delays recorded in the plurality of first recording units when the number of packets is less than a threshold.

[0076] Optionally, the target device is the network device; or, the network device is a device on a data plane in a network, the network device is the last hop device of the target flow in the data plane, and the target device includes multiple devices that the single data packet passes through in the data plane.

[0077] In one possible implementation, the receiving unit is further configured to receive an identifier of an estimation window, where the identifier indicates that the information recorded in the plurality of first recording units is recorded within the estimation window, where the estimation window is a time window for estimating a tail delay of the first flow passing through the target device.

[0078] The above-mentioned processing unit estimates the tail delay of the first flow passing through the above-mentioned target device based on the first delay recorded in the multiple first recording units, including: determining that the first delay recorded in the first unit is the estimated value of the tail delay of the first flow passing through the target device within the estimation window; the first unit is the unit with the smallest first delay recorded in the multiple first recording units.

[0079] In an eighth aspect, the present application provides a controller, comprising:

[0080] a receiving unit, configured to receive information recorded in a first recording unit and a second recording unit, the information recorded in the first recording unit including a first delay and a number of packets, the first delay being a maximum delay for a single data packet of the first flow to pass through a target device, and the number of packets being a number of data packets of the first flow received by the network device; the second recording unit being associated with the first recording unit, the information recorded in the second recording unit including a delay for each of n data packets in the first flow to pass through the target device, where n is an integer greater than 1;

[0081] The processing unit is configured to estimate a tail delay of the first flow passing through the target device based on the n delays.

[0082] In one possible implementation, the receiving unit is further configured to receive an identifier of an estimation window, where the identifier indicates that the information recorded in the first recording unit and the second recording unit is recorded within the estimation window, where the estimation window is a time window used to estimate the tail delay of the first flow passing through the target device.

[0083] The tail delay of the first flow passing through the target device is a p-percentile tail delay, where p is greater than 80% and less than 100%. The processing unit estimates the tail delay of the first flow passing through the target device based on the n delays, including:

[0084] The wth delay among the n delays is determined as the estimated tail delay value of the first flow passing through the target device within the estimation window, the wth delay is the delay ranked w among the n delays sorted from small to large, and w is an integer greater than 0 and less than or equal to n.

[0085] In a ninth aspect, an embodiment of the present application provides a network device, which may include: a memory and a processor coupled to the memory, a sending interface, and a receiving interface. The sending interface is used to support the network device in executing the step of sending messages and / or data packets in the communication delay estimation method provided in the first aspect. The receiving interface is used to support the network device in executing the step of receiving messages and / or data packets in the communication delay estimation method provided in the first aspect. The sending interface and the receiving interface may be integrated into a transceiver. The processor is used to support the network device in executing other processing operations of the network device in addition to sending and receiving in the communication method provided in the first aspect.

[0086] It should be noted that the transmitting interface and receiving interface in the embodiments of the present application can be integrated or coupled via a coupler. The memory is used to store the computer program for the communication delay estimation method described in the first aspect, and the processor is used to execute the computer program stored in the memory. The memory and processor can be integrated or coupled via a coupler.

[0087] In addition, the computer program in the memory of the present application can be pre-stored or downloaded from the Internet and stored when the device is used. The present application does not specifically limit the source of the computer program in the memory. The coupling in the embodiments of the present application is an indirect coupling or connection between devices, units or modules, which can be electrical, mechanical or other forms, for information exchange between devices, units or modules.

[0088] In one possible implementation, the processor is configured to execute program instructions stored in the memory, causing the network device to perform the following operations:

[0089] Acquire flow information from a received first data packet, where the flow information is identification information of a first flow to which the first data packet belongs; determine a first record unit for mapping the first flow based on the flow information; the first record unit includes a first delay and a number of packets, where the first delay is the maximum delay for a single data packet of the target flow to pass through a target device, and the number of packets is the number of data packets of the target flow received by the network device; the target flow includes a flow mapped to the first record unit; update the number of packets, and when the delay for the first data packet to pass through the target device is greater than the first delay, update the first delay to the delay for the first data packet to pass through the target device; when the number of packets is less than a threshold, the first delay is used to estimate the tail delay of the first flow to pass through the above-mentioned target device.

[0090] In a tenth aspect, an embodiment of the present application provides a network device, which may include: a memory and a processor coupled to the memory, a sending interface, and a receiving interface. The sending interface is used to support the network device in executing the step of sending messages and / or data packets in the communication delay estimation method provided in the second aspect. The receiving interface is used to support the network device in executing the step of receiving messages and / or data packets in the communication delay estimation method provided in the second aspect. The sending interface and the receiving interface can be integrated into a transceiver. The processor is used to support the network device in executing other processing operations of the network device in addition to sending and receiving in the communication method provided in the second aspect.

[0091] It should be noted that the transmitting interface and receiving interface in the embodiments of the present application can be integrated together or coupled via a coupler. The memory is used to store the computer program for the communication delay estimation method described in the second aspect, and the processor is used to execute the computer program stored in the memory. The memory and processor can be integrated together or coupled via a coupler.

[0092] In addition, the computer program in the memory of the present application can be pre-stored or downloaded from the Internet and stored when the device is used. The present application does not specifically limit the source of the computer program in the memory. The coupling in the embodiments of the present application is an indirect coupling or connection between devices, units or modules, which can be electrical, mechanical or other forms, for information exchange between devices, units or modules.

[0093] In one possible implementation, the processor is configured to execute program instructions stored in the memory, causing the network device to perform the following operations:

[0094] Obtaining flow information from the received first data packet, the flow information is used to indicate that the first data packet is a data packet in the first flow; determining a first recording unit for mapping the first flow based on the flow information; the first recording unit is used to record a first delay and a number of packets, the first delay being the maximum delay for a single data packet of the target flow to pass through a target device, and the number of packets being the number of data packets of the target flow received by the network device; the target flow includes the flow mapped to the first recording unit; updating the number of packets, and if the delay for the first data packet to pass through the target device is greater than the first delay, updating the first delay to the delay for the first data packet to pass through the target device; if the number of packets is greater than or equal to a threshold, allocating a second recording unit to the first recording unit; the second recording unit is used to record the delay for each of n data packets to pass through the target device, the n data packets being part of the data packets of the target flow received by the network device after the first data packet, where n is an integer greater than 1; the n delays are used to estimate the tail delay of the first flow passing through the above-mentioned target device.

[0095] In an eleventh aspect, an embodiment of the present application provides a controller, which may include: a memory and a processor coupled to the memory, a sending interface, and a receiving interface. The sending interface is used to support the controller in executing the step of sending messages and / or data packets in the communication delay estimation method provided in the third aspect. The receiving interface is used to support the controller in executing the step of receiving messages and / or data packets in the communication delay estimation method provided in the third aspect. The sending interface and the receiving interface can be integrated into a transceiver. The processor is used to support the controller in executing other processing operations of the controller in the communication method provided in the third aspect except for sending and receiving.

[0096] It should be noted that the transmitting interface and receiving interface in the embodiments of the present application can be integrated together or coupled via a coupler. The memory is used to store the computer program for the communication delay estimation method described in the third aspect, and the processor is used to execute the computer program stored in the memory. The memory and processor can be integrated together or coupled via a coupler.

[0097] In addition, the computer program in the memory of the present application can be pre-stored or downloaded from the Internet and stored when the device is used. The present application does not specifically limit the source of the computer program in the memory. The coupling in the embodiments of the present application is an indirect coupling or connection between devices, units or modules, which can be electrical, mechanical or other forms, for information exchange between devices, units or modules.

[0098] In one possible implementation, the processor is configured to execute program instructions stored in the memory, causing the controller to perform the following operations:

[0099] Receive information recorded in multiple first recording units, where the information recorded in each of the first recording units includes a first delay and a number of packets, the first delay being the maximum delay for a single data packet of the target flow to pass through a target device, and the number of packets being the number of data packets of the target flow received by the network device; the target flow includes a flow mapped to the first recording unit, and the target flow includes a first flow; when the number of packets is less than a threshold, estimate the tail delay of the first flow passing through the above-mentioned target device based on the first delay recorded in the multiple first recording units.

[0100] In a twelfth aspect, an embodiment of the present application provides a controller, which may include: a memory and a processor coupled to the memory, a sending interface, and a receiving interface. The sending interface is used to support the controller in executing the step of sending messages and / or data packets in the communication delay estimation method provided in the fourth aspect. The receiving interface is used to support the controller in executing the step of receiving messages and / or data packets in the communication delay estimation method provided in the fourth aspect. The sending interface and the receiving interface can be integrated into a transceiver. The processor is used to support the controller in executing other processing operations of the controller in the communication method provided in the fourth aspect except for sending and receiving.

[0101] It should be noted that the transmitting interface and receiving interface in the embodiments of the present application can be integrated together or coupled via a coupler. The memory is used to store the computer program of the communication delay estimation method described in the fourth aspect, and the processor is used to execute the computer program stored in the memory. The memory and processor can be integrated together or coupled via a coupler.

[0102] In addition, the computer program in the memory of the present application can be pre-stored or downloaded from the Internet and stored when the device is used. The present application does not specifically limit the source of the computer program in the memory. The coupling in the embodiments of the present application is an indirect coupling or connection between devices, units or modules, which can be electrical, mechanical or other forms, for information exchange between devices, units or modules.

[0103] In one possible implementation, the processor is configured to execute program instructions stored in the memory, causing the controller to perform the following operations:

[0104] Receive information recorded in a first recording unit and a second recording unit, the information recorded in the first recording unit including a first delay and a number of packets, the first delay being the maximum delay for a single data packet of the first flow to pass through a target device, and the number of packets being the number of data packets of the first flow received by the network device; the second recording unit is associated with the first recording unit, the information recorded in the second recording unit including the delay for each of n data packets in the first flow to pass through the target device, where n is an integer greater than 1; estimate the tail delay of the first flow passing through the above-mentioned target device based on the n delays.

[0105] In the thirteenth aspect, an embodiment of the present application provides a system, which includes a network device and a controller, wherein the network device is the network device described in the fifth aspect above, and the controller is the controller described in the seventh or eighth aspect above; or, the network device is the network device described in the sixth aspect above, and the controller is the controller described in the eighth aspect above; or, the network device is the network device described in the ninth aspect above, and the controller is the controller described in the eleventh or twelfth aspect above; or, the network device is the network device described in the tenth aspect above, and the controller is the controller described in the twelfth aspect above.

[0106] In the fourteenth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method described in any one of the first aspects above; or, the computer program is executed by a processor to implement the method described in any one of the second aspects above.

[0107] In the fifteenth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method described in any one of the third aspects above; or, the computer program is executed by a processor to implement the method described in any one of the fourth aspects above.

[0108] In the sixteenth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the method described in any one of the first aspects above; or, it enables the computer to execute the method described in any one of the second aspects above.

[0109] In the seventeenth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the method described in any one of the third aspects above; or, it enables the computer to execute the method described in any one of the fourth aspects above.

[0110] It is understandable that the network device, controller, system, computer storage medium, and computer program product provided above are all used to perform the method provided in any one of aspects 1 to 4. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding method and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] The following is an introduction to the drawings required for use in the embodiments of this application.

[0112] Figure 1 and Figure 2 The figure shows a schematic diagram of a system architecture in an embodiment of the present application;

[0113] Figure 3 The figure shows a flow chart of a method provided by an embodiment of the present application;

[0114] Figure 4 and Figure 5 Shown is a schematic diagram of information carried by a data packet during transmission;

[0115] Figure 6 The figure shows a schematic diagram of a network device processing flow architecture provided by the implementation of this application;

[0116] Figure 7 The figure shows a schematic diagram of information flow in a network provided by an embodiment of the present application;

[0117] Figure 8 The figure shows a schematic diagram of the contents included in a data packet provided in an embodiment of the present application;

[0118] Figure 9 The figure shows a schematic diagram of simulation results provided by an embodiment of the present application;

[0119] Figures 10 to 13 The figure shows a schematic diagram of the logical structure of a device provided in an embodiment of the present application;

[0120] Figures 14 to 17 The figure shows a schematic diagram of the hardware structure of a device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0121] The embodiments of the present application are described below with reference to the accompanying drawings.

[0122] First, let’s introduce the terms involved in the embodiments of this application:

[0123] 1. Data plane: The data plane refers to the forwarding surface in the network, which describes the system and capabilities of the forwarding chips and forwarding storage of devices within the network. It corresponds to the control plane with ubiquitous computing capabilities.

[0124] 2. Latency: The latency used in the embodiments of this application is the time it takes for a packet to travel within the network, calculated based on the hardware clock timestamp provided by the network device. In actual deployment scenarios, this can be a single-hop latency (e.g., the time it takes for a packet to be processed within the network device) or an end-to-end latency (e.g., the time from the first hop network device ingress to the last hop network device egress).

[0125] 3. Full flow: Full flow refers to all flows in the network, and full flow latency refers to the latency information of each flow in the network.

[0126] 4. Tail Delay: If p% of packets in a flow have a delay less than a certain delay, then that delay is called the p-percentile delay. In this application, when the value of p% is between 80% and 100%, that delay is called the tail delay, or the p-percentile tail delay. For example, if 95% of packets in a flow have a delay less than a certain delay, then that delay is called the 95th percentile tail delay.

[0127] An application scenario of an embodiment of the present application is introduced below.

[0128] See also Figure 1 , Figure 1 Shown is a schematic diagram of the system architecture of an application scenario of an embodiment of the present application. Figure 1 As can be seen in Figure 1, the system architecture includes a control plane 110 and a data plane 120. The control plane 110 includes a controller 111, which can be a software-defined network (SDN) controller or an operations and maintenance center (OMC). Controller 111 can be deployed in one or more servers, and multiple servers can form a server cluster.

[0129] The data plane 120 includes g network devices 121 , where g is an integer greater than 1. Figure 1 Multiple network devices may be included between network device 1 and network device g shown in FIG. These multiple network devices may be connected in any topology. Network device 121 is configured to forward data in the data plane. Network device 121 may be, for example, a forwarding device such as a switch, a router, or an access device. The access device may be a base station or a wireless access point.

[0130] In one possible implementation, see Figure 2 The network devices in the data plane have strong resources and processing capabilities. In addition to implementing data forwarding logic, they can also implement their own control logic. Therefore, the ability to implement their own control logic can also be called the device control plane, which can also be a capability of the control plane 110. In the embodiment of the present application, the device control plane can be used to process the latency information of the collected data packets to estimate the tail latency of the corresponding flow.

[0131] It should be noted that the above Figure 1 and Figure 2 The system architecture shown is only an example, and the system architecture applicable to the embodiments of the present application is not limited to the system architecture described above.

[0132] The current commonly used flow tail delay measurement technologies in the industry can be divided into two categories: one is the in-network measurement technology represented by NetFlow, which supports filtering, sampling, and copying relevant message data in network devices, and then uploading the message data to the controller for tail delay measurement; the other is the in-band operation administration and maintenance (iOAM) technology represented by in-band network telemetry (INT) technology. Through the analysis of these existing solutions, it can be concluded that the main challenge of the full-flow delay measurement task is the insufficient storage resources and logical capabilities of network devices in the data plane, or in other words, the existing solutions require a large amount of storage resources and processing resources to perform full-flow delay measurement, and the resource overhead is very large. Specifically:

[0133] On the one hand, the current network throughput is extremely high. If the full-flow delay measurement task is implemented on the control plane, it will often bring a huge additional bandwidth load (copying) or cause the coverage to decrease and fail to achieve full measurement (sampling / filtering). In addition, the number of concurrent flows in the current network is extremely large, and data centers often have 100K-1M concurrent flows at the same time. As a result, if you want to record flow-by-flow information, the storage space available for each flow is extremely limited due to the current limited data plane storage capacity.

[0134] On the other hand, the logical resources and supported operations of network devices are more limited than those of general-purpose computers. Therefore, although current algorithmic theoretical research has reached good conclusions on flow model quantile estimation, how to apply and deploy them has become a problem. As a result, existing methods either make strong assumptions about device capabilities, making them difficult to deploy in practice; or they abandon accuracy and adopt coarser-grained estimation; or they use more resources to implement compromised versions of existing algorithms, giving up scalability and making it difficult to support flow-by-flow delay measurement.

[0135] Therefore, this application focuses on achieving the goal of low-overhead, high-precision measurement of tail delay for all flows on the data plane within network devices with limited resources and capabilities. Specifically, it focuses on reducing the resource overhead of network devices during tail delay measurement. The following are specific implementation examples.

[0136] See also Figure 3 , shows a communication delay estimation method provided by an embodiment of the present application, which is used to solve the above technical problems. This method can be applied to the above Figure 1 or Figure 2 The system architecture shown. The method may include but is not limited to the following steps:

[0137] 301. A first network device obtains flow information from a received first data packet. The flow information is a flow identifier of a first flow to which the first data packet belongs.

[0138] In a specific embodiment, the first network device may be the above-mentioned Figure 1 or Figure 2 Any network device in the data plane 120. The first network device can be used to forward data packets of one or more flows.

[0139] Specifically, after the first network device receives a data packet (hereinafter referred to as the first data packet for the sake of convenience of description), it can parse the first data packet and obtain the flow information in the first data packet. Exemplarily, the flow information can be, for example, a four-tuple, a five-tuple, or a seven-tuple, etc. The four-tuple includes the source Internet protocol (IP) address, destination IP address, source port, and destination port of the flow; the five-tuple includes the source IP address, destination IP address, protocol number, source port, and destination port; the seven-tuple includes the source IP address, destination IP address, protocol number, source port, destination port, service type, and interface index. After the first network device obtains the flow information in the above-mentioned first data packet, it can know the flow to which the first data packet belongs (hereinafter referred to as the first flow for the sake of convenience of description).

[0140] In a possible implementation, the first network device may encode the obtained flow information. For example, the flow information may be represented by a symbol or a numerical value, which may uniquely identify the first flow within the first network device.

[0141] 302. The first network device determines a first record unit of the first flow mapping based on the flow information; the first record unit includes a first delay and a number of packets, the first delay being the maximum delay of a single data packet of the flow mapped to the first record unit passing through the target device, and the number of packets being the number of data packets of the flow mapped to the first record unit received by the first network device.

[0142] In a specific embodiment, the first network device includes a data structure bitmap, which includes multiple bits. In the embodiment of the present application, the bitmap can be used to determine whether a flow is a new flow. Specifically, after the first network device obtains the flow information of a flow, it can use multiple hash functions to perform hash calculations on the flow information to obtain multiple hash values. The multiple hash values correspond to the position numbers of the multiple bits in the bitmap. Therefore, the multiple hash values can map the flow corresponding to the flow information to the multiple bits. The multiple hash functions can be two or more hash functions.

[0143] The initial value of each bit in the bitmap may be a first value, and after mapping the stream to the multiple bits based on the calculated multiple hash values, the values in the multiple bits may be set to a second value. For example, optionally, assuming that the initial value of each bit in the bitmap is 0, after mapping the stream to the multiple bits based on the calculated multiple hash values, the values in the multiple bits may be set to 1. Alternatively, optionally, assuming that the initial value of each bit in the bitmap is 1, after mapping the stream to the multiple bits based on the calculated multiple hash values, the values in the multiple bits may be set to 0.

[0144] Based on the above logic, the first network device performs hash calculations on the flow information of the above-mentioned first flow through the above-mentioned multiple hash functions to obtain multiple hash values, and maps the first flow to multiple bits in the bitmap based on the multiple hash values. If the value in at least one bit in the multiple bits is the above-mentioned first value, it can be determined that the first flow is a new flow, and the above-mentioned first data packet is the first data packet in the first flow received by the first network device. The new flow refers to a flow that has not yet been mapped to the bitmap. If the value in each bit in the multiple bits is the above-mentioned second value, it indicates that the first flow is not a new flow, and the above-mentioned first data packet is a data packet in the first flow received again by the first network device.

[0145] In the embodiment of the present application, each bit in the above bitmap is associated with a first recording unit, and the first recording unit associated with a bit is also a unit of the stream mapping mapped to the bit. The first recording unit can be used to record the number of packets and the first delay.

[0146] The above number of packets is the number of data packets of the stream mapped to the first recording unit received by the first network device. Optionally, for a certain first recording unit, if there is only one stream mapped to the first recording unit, then the number of packets in the first recording unit is the number of data packets of the stream received by the first network device. Optionally, for the certain first recording unit, if there are multiple streams mapped to the first recording unit, that is, a hash conflict occurs, then the number of packets in the first recording unit is the sum of the number of data packets of the multiple streams received by the first network device.

[0147] The first delay is the maximum delay of a single data packet of the flow mapped to the first recording unit passing through the target device.

[0148] In one possible implementation, the target device is the first network device. Alternatively, in another possible implementation, the first network device is the last hop device in the data plane of the flow mapped to the first recording unit, and the target device includes multiple devices, each of which is a device that a single data packet in the flow mapped to the first recording unit passes through in the data plane.

[0149] Optionally, whether the target device is the first network device or the target device includes multiple devices, compared with the basically fixed link delay, the processing delay on the network device, especially the queuing delay waiting for processing in the network device, is the main factor causing the delay fluctuation. Therefore, in the embodiment of the present application, the processing time of the network device can be used as the delay. The processing time is the time between a data packet being received from the network device and being sent from the network device, that is, the processing time is the delay of the data packet passing through the network device. For the case where the target device is the first network device, the first delay is the maximum delay of the single data packet passing through the first network device. For the case where the target device includes multiple devices, the first delay is the maximum value of the sum of the delays of the single data packet passing through each of the multiple devices. For example, assuming the multiple devices are device 1 and device 2, and device 2 receives the first and second data packets in the first flow, the delays for the first data packet through device 1 and device 2 are 1 millisecond and 0.9 milliseconds, respectively, and the delays for the second data packet through device 1 and device 2 are 1.1 milliseconds and 1 millisecond, respectively. Therefore, the first delay in the first record unit of the first flow mapping in device 2 is 2.1 milliseconds. Based on the above description, the sum of the processing time of the data packet by all network devices in the data plane is the delay of the entire transmission process of the data packet in the data plane. How to obtain the delay will be described later and will not be detailed here.

[0150] Alternatively, if the target device includes multiple devices, the transmission delay of a data packet along the transmission path in the data plane can be used as the delay of the entire transmission process of the data packet in the data plane. This transmission delay includes the link delay in the transmission path and the delay caused by the processing time of the multiple devices in the transmission path. In this application, this transmission delay is also referred to as the delay passing through the multiple devices. Since the multiple devices are the target devices, the transmission delay is the delay passing through the target devices. In this case, the first delay can be the maximum value of the delay of a single data packet in the stream mapped to the first recording unit during the entire transmission process. For example, assuming that the first network device receives the first and second data packets in the first stream, the delay of the first data packet during the entire transmission process is 2.5 milliseconds, and the delay of the second data packet during the entire transmission process is 2.8 milliseconds. Then, the first delay in the first recording unit mapped to the first stream in the first network device is 2.8 milliseconds. The transmission delay is the delay between the ingress network device receiving the data packet and the egress network device sending the data packet. The ingress network device is the first hop device that forwards the data packet in the data plane, and the egress network device is the last hop device that forwards the data packet in the data plane. How to obtain the latency will be described later and will not be detailed here.

[0151] In a possible implementation, the first recording unit can also be used to record the XOR result of the stream information and the number of streams. The number of streams is the number of streams mapped to the first recording unit, and the XOR result of the stream information is the XOR result of the stream information mapped to the first recording unit. Specifically, if the number of streams is 1, the XOR result of the stream information is the stream information itself, or in other words, when the number of streams mapped to the first recording unit is one, the location of the XOR result of the stream information in the first recording unit records the stream information itself. If the number of streams is greater than 1, that is, there are multiple streams mapped to the same first recording unit, then the XOR result of the stream information is the result obtained after the XOR operation of the multiple stream information. In a computer, all information is represented by binary data composed of 0 and 1, so the XOR result of the stream information is also binary data composed of 0 and 1.

[0152] Based on the above description, the first network device maps the first stream to a plurality of bits in a bitmap, and a plurality of first recording units mapped to the first stream can be determined through the plurality of bits.

[0153] By mapping the stream into the bitmap, storage resources can be saved for the first network device and the computation difficulty can be reduced to save computation resources.

[0154] 303. The first network device updates the number of packets, and if the delay of the first data packet passing through the target device is greater than the first delay, updates the first delay to the delay of the first data packet passing through the target device.

[0155] In a specific embodiment, after the first network device determines the multiple first recording units of the first flow mapping, it can adaptively update the information in the multiple first recording units, which is described below in different situations.

[0156] In the first case, the first stream is a new stream, and no hash collision occurs in the multiple bits mapped by the first stream. In this case, the multiple first recording units mapped by the first stream do not yet contain any information or the recorded information is the original default value. In this case, the first network device may set the value of the position recording the number of packets in the multiple first recording units to 1. In addition, for the first delay in the multiple first recording units, the first network device may obtain the delay of the first data packet passing through the target device, and then record the delay in the position recording the first delay in the multiple first recording units.

[0157] Specifically, a delay acquisition module may be deployed in the first network device, and the delay acquisition module may acquire the delay of the first data packet passing through the target device.

[0158] Optionally, if the target device is the first network device, the latency acquisition module can obtain the processing time of the first data packet in the first network device through measurement and calculation. This processing time is the latency of the first data packet passing through the first network device. Specifically, the first network device can measure the timestamp of the first data packet's arrival at the first network device and estimate the timestamp of the first data packet's transmission from the first network device. Subtracting the previous timestamp from the latter timestamp to obtain a time quantity, which is the processing time of the first data packet in the first network device, i.e., the latency. The first network device then records this calculated latency in the location where the first latency is recorded in the multiple first recording units.

[0159] It should be noted that the timestamp of the first data packet being sent from the network device is calculated based on the state of the data packet transmission queue. In this embodiment of the present application, the delay calculated by the delay acquisition module allows for an acceptable error. In other words, in this embodiment of the present application, the delay calculated by the delay acquisition module can be used as the correct delay of a single data packet passing through the network device. Optionally, the delay acquisition module can estimate the time with an accuracy of nanoseconds, thereby improving the accuracy of the delay estimation.

[0160] Optionally, if the target device includes multiple devices, one possible implementation may use the processing time of the network device as the latency. Then, for the first data packet, each network device forwarding the first data packet in the data plane needs to be equipped with processing logic for calculating the processing time of the first data packet in the network device.

[0161] Specifically, the ingress network device calculates the delay of the first data packet passing through the ingress network device and adds the delay to the first data packet's header, which is then sent along with the first data packet to the next-hop network device. Optionally, the header can be a custom header specifically added to carry the delay. After receiving the first data packet carrying the delay, the next-hop network device similarly calculates the processing time, i.e., the delay, of the first data packet within itself according to the aforementioned logic. It then adds the calculated delay to the delay in the received first data packet's header to obtain a new delay, which it then updates to the header of the first data packet and continues sending it to the next-hop device. This continues until the first data packet reaches the last hop device in the data plane, i.e., the first network device. The first network device then obtains the delay in the first data packet's header, calculates the delay in its own header, and then adds the two delays to obtain the total delay. This total delay is the delay of the first data packet passing through the multiple devices. After obtaining the total delay, the first network device can remove the delay-carrying header from the first data packet. For example, the operation of adding the delay to the existing delay in the data packet in each hop network device and sending the data packet can be implemented by using the INT technology introduced above. Then, the first network device records the total delay in the position of recording the first delay in the plurality of first recording units. For easier understanding of this embodiment, please refer to the example of Figure 4 .

[0162] Figure 4 A schematic diagram of the network device topology in a simple data plane is shown, along with the information carried by data packets during transmission. Assume that the transmission path of the first data packet in the data plane is: network device 1 - network device 2 - network device 5 - network device 6. The delay carried in the first data packet sent by network device 1 is the delay a, which is the time it takes for the first data packet to pass through network device 1. The first data packet is then sent to network device 2, which obtains the delay b of the first data packet passing through itself and then encapsulates the delay a+b into the first data packet. The first data packet is then sent to network device 5, which obtains the delay c of the first data packet passing through itself and then encapsulates the delay a+b+c into the first data packet. The first data packet carrying the delay a+b+c is then sent to network device 6, which obtains the delay d of the first data packet passing through itself and adds d to the delay a+b+c to obtain the delay a+b+c+d. The delay a+b+c+d is then recorded in the location where the first delay is recorded in the plurality of first recording units.

[0163] It should be noted that in the embodiment of the present application, although the delay carried in the first data packet forwarded in different network devices is different, as long as the load (payload) in the first data packet remains unchanged, the data packet carrying different delay information is still called the first data packet.

[0164] Optionally, if the target device includes multiple devices, one possible implementation method can use the transmission delay of the data packet along the transmission path in the data plane as the latency of the entire transmission process of the data packet in the data plane. For the first data packet, on the ingress network device forwarding the first data packet in the data plane, the ingress network device can measure the timestamp of the first data packet's arrival at the ingress network device. This timestamp is then added to the header of the first data packet, which is then forwarded along with the first data packet to the egress network device in the data plane, i.e., the first network device. Optionally, this header can be a custom header specifically added to carry timestamps. The first network device can obtain the timestamp in the first data packet, then estimate the timestamp of the first data packet's transmission from the first network device. Subtracting the previous timestamp from the latter timestamp to obtain a time quantity, which is the latency of the first data packet passing through the target device. After obtaining this time quantity, the first network device can remove the header specifically used to carry the timestamp from the first data packet. For example, the operation of adding a timestamp to a data packet in the ingress network device and transmitting it to the egress network device along with the data packet can be implemented using the INT technology described above. Then, the first network device records the delay in the location where the first delay is recorded in the plurality of first recording units. In this implementation, the clocks of the network devices in the data plane need to be synchronized so that the calculated delay is accurate. For an easier understanding of this implementation, please refer to the example of Figure 5 .

[0165] Figure 5 The topology diagram of a simple network device in a data plane is shown, and the information carried by the data packet during transmission is shown. Assume that the transmission path of the first data packet in the data plane is: network device 1 - network device 2 - network device 5 - network device 6. If network device 1 is the ingress network device, the first data packet sent from network device 1 carries the timestamp of when the first data packet arrives at network device 1, that is, Figure 5The first timestamp shown in is then sent to network device 6 via network device 2 and network device 5. Network device 6 estimates the timestamp of the first data packet sent from network device 6 and subtracts the previous timestamp from the latter timestamp to obtain a time quantity, which is the delay of the first data packet passing through the target device. Network device 6 records this time quantity in the location where the first delay is recorded in the plurality of first recording units.

[0166] In one possible implementation, the first recording unit may also be used to record the XOR result of the flow information and the number of flows. If the first recording unit includes the XOR result of the flow information, since no hash collision occurs in the multiple bits of the first flow mapping, the first network device may save the flow information of the first flow to the location of the XOR result of the flow information recorded in the first recording unit of the first flow mapping. If the first recording unit includes the number of flows, the first network device may set the number of flows recorded in the first recording unit of the first flow mapping to 1.

[0167] In the second case, the first stream is a new stream, and a hash conflict occurs in at least one bit among the multiple bits mapped by the first stream.

[0168] In this case, for the first record unit of the bit mapping without hash conflict in the multiple bits, the update of the number of packets and the first delay can refer to the corresponding description in the first case above, and will not be repeated here. For the at least one first record unit of the at least one bit mapping with hash conflict, the first network device can increase the number of packets in the at least one first record unit by 1 count, that is, replace the original number of packets with the number of packets obtained after increasing the count by 1. For the first delay in the at least one first record unit, the first network device can obtain the delay of the above-mentioned first data packet passing through the above-mentioned target device according to the method described in the first case above, and then compare the obtained delay with the first delay recorded in each unit in the at least one first record unit. If the obtained delay is greater than the minimum first delay in the at least one first record unit, the minimum first delay in the at least one first record unit is replaced with the obtained delay.

[0169] In a possible implementation, the first recording unit may also be used to record an XOR result of the stream information and the number of streams.

[0170] If the first recording unit includes the XOR result of the flow information, for the first recording unit in which there is no hash conflict in the bit mapping of the above-mentioned multiple bits, the first network device can save the flow information of the first flow to the location of the XOR result of the flow information recorded in the first recording unit. For at least one first recording unit in which at least one bit mapping has a hash conflict, the first network device can perform an XOR operation on the flow information of the first flow and the information in the location of the XOR result of the flow information recorded in the first recording unit (which can be the information of one flow, or the XOR result of the flow information of multiple flows) to obtain an XOR result, and then save the calculated XOR result to the location of the XOR result of the flow information recorded in the first recording unit.

[0171] If the first recording unit includes the number of flows, then, for the first recording unit of the bit mapping in which there is no hash conflict among the plurality of bits, the first network device may set the number of flows recorded in the first recording unit of the first flow mapping to 1. For at least one first recording unit of at least one bit mapping in which a hash conflict occurs, the first network device may increase the number of flows in the at least one first recording unit by 1, i.e., replace the original number of flows with the number of flows obtained by increasing the number by 1.

[0172] In the third case, the first stream is not a new stream, and no hash conflict occurs in the multiple bits mapped by the first stream.

[0173] In this case, it indicates that the above-mentioned first data packet is a data packet in the first flow that is received again by the first network device, and the number of packets and the first delay of the data packets of the first flow that were previously received are recorded in the multiple first record units of the first flow mapping. The first network device can increase the number of packets in the first record unit of the first flow mapping by 1 count, that is, replace the original number of packets with the number of packets obtained after increasing the count by 1. In addition, for the first delay in the multiple first record units, the first network device can obtain the delay of the above-mentioned first data packet passing through the above-mentioned target device, and then compare the obtained delay with the first delay recorded in the first record unit. If the obtained delay is greater than the first delay recorded in the first record unit, the first network device updates the first delay in the multiple first record units to the obtained delay. If the obtained delay is not greater than the first delay recorded in the first record unit, the first delay in the multiple first record units remains unchanged.

[0174] In a possible implementation, the first recording unit can also be used to record the XOR result of the flow information and the number of flows. If the first recording unit includes the XOR result of the flow information, since there is no hash conflict in the multiple bits of the first flow mapping, the flow information of the first flow is still recorded in the position where the XOR result of the flow information is recorded in the multiple first recording units of the first flow mapping, and remains unchanged. If the first recording unit includes the number of flows, since there is no hash conflict in the multiple bits of the first flow mapping, the flow mapped in the corresponding multiple first recording units is only the above-mentioned first flow, and therefore, the number of flows recorded in the multiple first recording units of the first flow mapping remains 1.

[0175] In a fourth case, the first stream is not a new stream, and a hash conflict occurs in at least one bit among the multiple bits mapped by the first stream.

[0176] In this case, for the first recording unit of the bit mapping without a hash conflict among the multiple bits, the update of the number of packets and the first delay therein can refer to the corresponding description in the third case above, and will not be repeated here. For the at least one first recording unit of the at least one bit mapping with a hash conflict, the update of the number of packets and the first delay therein can refer to the corresponding description in the second case above, and will not be repeated here.

[0177] In one possible implementation, the first recording unit may also be used to record an XOR result of the stream information and the number of streams. Since the first stream is not a new stream, the XOR result of the stream information and the number of streams in the multiple first recording units mapped to the first stream remain unchanged. For details, see the corresponding description in the third scenario above.

[0178] In one possible implementation, the first data packet is a data packet received by the first network device within a preset time window, and the size of the preset time window can be any size between 10 milliseconds and 100 milliseconds; in this case, the first network device estimates the delay of the first flow in segments based on the time window as the estimated duration, and then the first delay is the maximum delay of a single data packet of the flow mapped to the first recording unit passing through the target device within the time window.

[0179] Alternatively, in another possible implementation, the first data packet is any data packet in the first stream received by the first network device. In this case, the first network device estimates the delay of the first stream based on the overall transmission time of the first stream. Then, the first delay is the maximum delay of a single data packet of the stream passing through the target device within the overall transmission time of the stream mapped to the first recording unit.

[0180] In this embodiment of the present application, the aforementioned preset time window or the overall transmission time of the aforementioned stream can be collectively referred to as an estimation window, i.e., the estimation window is the time window used to estimate the stream's latency. In this embodiment of the present application, the purpose is to estimate the stream's tail latency, so the estimation window can also be considered the time window used to estimate the stream's tail latency. Furthermore, the information recorded in the aforementioned first recording unit is also estimated within this estimation window.

[0181] In an embodiment of the present application, the estimation of the tail delay of the above-mentioned first flow passing through the above-mentioned target device includes two different processing methods, and the specific processing method can be determined by judging whether the number of data packets of the first flow received by the first network device within the above-mentioned estimation window is greater than a threshold. Specifically, after the above-mentioned first network device updates the multiple first recording units mapped by the first flow, it can be judged whether the number of packets recorded in the multiple first recording units is greater than a threshold. Of course, the data packets corresponding to the number of packets are received by the first network device within the estimation window. The threshold can be, for example, 100, 1000 or 10000, etc., and can be set according to the desired measurement percentile and the desired accuracy, etc., and this application does not impose any restrictions on this.

[0182] In one possible implementation, if the number of packets recorded in the plurality of first recording units is less than the threshold, then, if the estimation window has not yet ended, the first network device continues to receive data packets of the first flow, and then updates the information in the plurality of first recording units mapped to the first flow. For specific updates, see the corresponding description in step 303. After the estimation window ends, the first network device may find the plurality of first recording units mapped to the first flow based on the hash value of the flow information of the first flow, obtain the plurality of first delays recorded in the plurality of first recording units mapped to the first flow, and then take the minimum value of the plurality of first delays as the estimated value of the tail delay of the first flow passing through the target device within the estimation window.

[0183] In the above processing method, since the number of data packets of the first flow in the above estimation window is less than the above threshold, it indicates that the number of data packets is relatively small. Then, the difference in tail delay between the 80th percentile tail delay and the 100th percentile tail delay of the first flow in the estimation window is small, and the first delay recorded in the first record unit of the first flow mapping is the maximum delay of a single data packet passing through the target device in the estimation window. Therefore, the first delay recorded in the first record unit of the first flow mapping can be used to estimate any tail delay between the 80th percentile tail delay and the 100th percentile tail delay of the first flow in the estimation window. This estimation error is small and within an acceptable range. Because hash conflicts may exist, if the first delay of the unit record with hash conflicts in the multiple first record units mapped by the first stream is different from the first delay of the unit record without hash conflicts, then, based on the above-mentioned principle of replacing the smaller delay with the larger delay, the first delay of the unit record with hash conflicts shall not be less than the first delay of the unit record without hash conflicts. Therefore, it is more accurate to select the smallest first delay among the multiple first record units mapped by the first stream as the estimated value of the tail delay of the first stream passing through the above-mentioned target device within the estimation window.

[0184] In one possible implementation, for at least one first recording unit of a bit mapping in which a hash conflict exists in the bit positions of the above-mentioned first flow mapping, after the above-mentioned estimation window ends, the first network device, in the process of decoding the information in the first recording unit, may perform an XOR operation on the XOR result of the flow information in the at least one first recording unit and the flow information of the first flow again, and record the result of the XOR operation in the at least one first recording unit, so that the flow information of the first flow can be deleted from the at least one first recording unit. At the same time, the first network device may reduce the number of flows in the at least one first recording unit by 1, and the number of packets by the number of data packets of the first flow received within the estimation window, but the first delay is not changed. Through these processes, the relevant information of the first flow can be deleted from the at least one first recording unit. In the case of multiple estimation windows, the at least one first recording unit can be prepared to record the relevant information of the first flow in the next estimation window.

[0185] As for the first recording units of the bit mapping in which there is no hash conflict in the bits of the above-mentioned first flow mapping, after the above-mentioned estimation window ends, the first network device can clear all the information recorded in these first recording units. In the case of multiple estimation windows, these first recording units can be prepared to record the relevant information of the first flow in the next estimation window.

[0186] In one possible implementation, if the number of packets recorded in the above-mentioned multiple first recording units is greater than or equal to the above-mentioned threshold, then when the number of packets is equal to the threshold, if the estimation window has not yet ended, the first network device configures an associated second recording unit for each of the multiple first recording units, and the first recording unit and the second recording unit are associated by an index. The first network device can configure multiple index recording units to specifically record the index information between the first recording unit and the second recording unit, and each index recording unit records the index information between a first recording unit and a second recording unit. The second recording unit can record n time delays, where n is an integer greater than 1. The n time delays are the delays for n data packets in the target flow received by the first network device after the number of packets is equal to the above-mentioned threshold to pass through the above-mentioned target device. If the number of packets recorded in the first recording unit just reaches the threshold after the first network device receives the above-mentioned first data packet, then the n data packets are the n data packets of the target flow received by the first network device after the first data packet. The target stream is the stream mapped to the first recording unit. If the stream mapped to the first recording unit is only the first stream, then the target stream is the first stream; if the stream mapped to the first recording unit includes multiple streams including the first stream, then the target stream is the multiple streams.

[0187] Specifically, when configuring the above-mentioned second recording unit, the initial values of n delays can be configured by default in the second recording unit, or the second recording unit may be empty at the beginning. In an embodiment of the present application, the n delays in the second recording unit can be updated by polling. After the number of packets recorded in the above-mentioned first recording unit is equal to the above-mentioned threshold, the delays corresponding to the first n data packets of the target flow received by the first network device passing through the above-mentioned target device can be recorded in sequence in the position where the n delays are recorded in the second recording unit. Then, after the n data packets, each time the first network device receives a data packet of the target flow, it can determine a delay in the second recording unit by polling, and then, when the delay is less than the delay of the received data packet passing through the above-mentioned target device, replace the one delay in the second recording unit with the delay of the received data packet passing through the above-mentioned target device.

[0188] The above-mentioned polling method refers to starting from one of the n delays and querying each delay one by one in a preset order until the query of the n delays is completed. Then, starting from the same delay again and querying each delay one by one in the preset order until the query of the n delays is completed again, the query operation is executed in a loop. For ease of understanding, the following example is used. See Table 1.

[0189] Table 1

[0190] Delay location 1 2 3 4 5 Delay value 0.3 milliseconds 0.35 milliseconds 0.28 milliseconds 0.32 milliseconds 0.26 milliseconds

[0191] Table 1 exemplifies the delay recorded in a second recording unit. Assuming that the second recording unit can record the delays of the data packets of 5 target streams passing through the above-mentioned target device, the delays at the corresponding positions can be found in the above-mentioned Table 1. Assuming that the 5 delays recorded in the above-mentioned Table 1 are the delays of the first 5 data packets of the first stream received by the first network device passing through the above-mentioned target device after the number of packets recorded in the above-mentioned first recording unit is equal to the above-mentioned threshold. In addition, assuming that the above-mentioned polling method is to start from the delay at position 1 and query the delay at position 5 in sequence, then, query the delay at position 1 in turn, and then query the delay at position 5 in sequence, and so on.

[0192] Based on Table 1, the first network device receives the first data packet of the target flow after the above-mentioned number of packets equals the above-mentioned threshold value, and obtains the delay of the first data packet passing through the above-mentioned target device, which is 0.32 milliseconds. Then, based on the polling method, the first network device first queries the delay in position 1 in the second recording unit, and the delay is 0.3 milliseconds. Since 0.32 milliseconds is greater than 0.3 milliseconds, the first network device replaces the delay of 0.3 milliseconds in position 1 with 0.32 milliseconds. After one round of operation, that is, the first network device receives 5 data packets of the first flow after the above-mentioned number of packets equals the above-mentioned threshold value, and polls and updates the second recording unit, the 5 delays in the second recording unit are shown in Table 2.

[0193] Table 2

[0194] Delay location 1 2 3 4 5 Delay value 0.32 milliseconds 0.35 milliseconds 0.3 milliseconds 0.32 milliseconds 0.28 milliseconds

[0195] Based on Table 2, the first network device receives the sixth packet of the target flow after the number of packets reaches the threshold, and obtains the delay of this sixth packet through the target device, which is 0.25 milliseconds. Then, based on the polling method, the first network device queries the delay in position 1 of the second record unit, which is 0.32 milliseconds. Since 0.25 milliseconds is less than 0.32 milliseconds, the first network device leaves the delay in position 2 unchanged, i.e., does not update the delay in position 1.

[0196] Optional, because the subsequent estimation of the tail delay of the target flow passing through the target device within the estimation window is based on the n delays recorded in the second recording unit, if the number of packets of the target flow is very large, then the n delays in the second recording unit are updated in the above-mentioned polling manner, then the n tail delays finally obtained are the largest n tail delays, and at this time, the n tail delays can be used to more accurately estimate the tail delay close to the 100th percentile, but for estimating the 80th percentile tail delay, 85th percentile tail delay, 90th percentile tail delay or 95th percentile tail delay, or even the 99th percentile tail delay, the error is large and the estimated tail delay is inaccurate. In order to solve this problem, the embodiment of the present application provides the following operations:

[0197] Based on the above-mentioned polling method for updating the above-mentioned second recording unit, after the above-mentioned packet number reaches the above-mentioned threshold, each time the first network device receives a data packet of the target flow after receiving a preset number of data packets of the target flow, it can randomly query a delay in the second recording unit, and then compare the delay of the data packet passing through the above-mentioned target flow with the query delay. If the delay of the data packet passing through the above-mentioned target flow is less than the query delay, the query delay is replaced with the delay of the data packet passing through the above-mentioned target flow and recorded in the second recording unit; otherwise, the query delay is not updated.

[0198] The above-mentioned preset number can be 10, 50 or 100, etc., and the present application does not impose any restriction on the specific value of the preset number.

[0199] Still based on the above Table 1 as an example for further description, assuming that the preset number is 8, then after the above number of packets is equal to the above threshold, the first network device receives 8 data packets of the target flow and polls and updates the second recording unit. The 5 delays in the second recording unit are shown in Table 3.

[0200] Table 3

[0201] Delay location 1 2 3 4 5 Delay value 0.32 milliseconds 0.35 milliseconds 0.31 milliseconds 0.32 milliseconds 0.28 milliseconds

[0202] Based on Table 3, the first network device receives the ninth data packet of the target flow after the number of packets reaches the threshold, and obtains the delay of the ninth data packet passing through the target device, which is 0.27 milliseconds. The first network device then randomly searches for a delay among the five delays in the second recording unit. Assume that the delay at position 5 is 0.28 milliseconds. Since 0.27 milliseconds is less than 0.28 milliseconds, the first network device replaces the 0.28 millisecond delay at position 5 with 0.27 milliseconds. For details, see Table 4.

[0203] Table 4

[0204] Delay location 1 2 3 4 5 Delay value 0.32 milliseconds 0.35 milliseconds 0.31 milliseconds 0.32 milliseconds 0.27 milliseconds

[0205] It should be noted that, due to the existence of hash conflicts, the number of packets in the unit with bit mapping with hash conflicts among the above-mentioned multiple first recording units is greater than the number of packets in the unit with bit mapping without hash conflicts. For the unit with bit mapping with hash conflicts, as long as its number of packets reaches a threshold, a second recording unit can also be configured for the unit, and then, the second recording unit is used to record the delay of some data packets received by the first network device through the above-mentioned target device after the number of packets in the multiple streams mapped to the unit is greater than the above-mentioned threshold. The operation of updating the second recording unit can be referred to the above description and will not be repeated here.

[0206] Based on the above-mentioned method of updating the second recording unit, after the above-mentioned estimation window ends, the first network device finds the associated second recording unit based on the multiple first recording units mapped by the first flow. Specifically, the second recording unit associated with the first recording unit can be found through the index information recorded in the above-mentioned index recording unit. Then, the first network device obtains the n delays recorded in the second recording unit, and estimates the tail delay of the first flow passing through the target device within the estimation window based on the n delays. Specifically, assuming that the n delays are sorted from small to large, the delay ranked as w among the n delays can be used to estimate the p-percentile tail delay of the first flow within the estimation window, and the delay ranked before w is less than or equal to the p-percentile tail delay, and w is an integer greater than 0 and less than or equal to n.

[0207] In one possible implementation, in order to avoid data confusion caused by hash conflicts, n delays in the second recording unit associated with the first recording unit with 1 or the smallest number of flows can be selected to estimate the tail delay of the first flow passing through the target device within the estimation window.

[0208] In one possible implementation, it is assumed that the number of first-stream data packets received by the first network device within the estimation window is N, where N is an integer greater than or equal to n. Then, the above w can be n(1-1 / n) N(1-p) , that is, the order of the above n delays can be n(1-1 / n) N(1-p) The largest delay is used to estimate the p-th tail delay of the first stream in the estimation window, and the nth (1-1 / n) N(1-p) The maximum delay is the estimated value of the N*pth maximum delay among the N delays corresponding to the N data packets in the first flow passing through the above-mentioned target device. The specific estimation process can be exemplified by the following calculation formula:

[0209]

[0210] Among them, aK represents the Kth largest delay among the N delays passing through the target device, a j represents the jth largest delay among the n delays above, C represents the set of n delays in the second recording unit, c i represents the delay of the i-th position record in the second recording unit, P(a K ∈C) represents a K The probability of being recorded at any position of the second recording unit, a K meet c i Indicates a K In polling, the delay is compared with the delay of the i-th position record in the second record unit, ∧ means "and", a K =max(a j |a j meetc i ) means a K Equal to the maximum value of all delays compared with the delay recorded at the i-th position. Formula for calculating a K Probability of being recorded.

[0211] Assume a K meet c i The probability is 1 / n, that is, a K The probability of encountering each number in C is the same. In fact, the difference between the number of delays encountered at different positions in the above polling method is at most 1. Therefore, it is assumed here that a K meet c i The probability of 1 / n is small. Based on this assumption, then:

[0212]

[0213] but

[0214]

[0215] but

[0216]

[0217] Among them, it represents c i >a N*p Indicates that the delay recorded at the i-th position is greater than the p-th percentile tail delay. i >a N*p )) represents the mathematical expectation of the number of delays greater than the p-percentile tail delay among the above n delays, E(Num(c i ≤a N*p )) represents the mathematical expectation of the number of delays whose n delays are not greater than the p-percentile tail delay, that is, in theory, the expectation of the p-percentile tail delay is the number of delays in the n delays. Large recording delay, so w can be n(1-1 / n) N(1-p) .

[0218] In one possible implementation, when the number of packets in the first flow is very large, the n delays recorded in the second recording unit are randomly replaced with smaller delays every certain number of packets to reduce the final estimation error. The accompanying tail delay estimation process can be described as follows:

[0219] Assume that the number of first-stream data packets received by the first network device within the estimation window is N, where N is an integer greater than or equal to n. Then, the above w can be n(1-q / n) N(1-p) , that is, the order of the above n delays can be n(1-q / n) N(1-p) The largest delay is used to estimate the p-th tail delay of the first stream in the estimation window, and the nth (1-q / n) N(1-p) The maximum delay is an estimated value of the N*pth maximum delay among the N delays corresponding to the N data packets in the first flow passing through the above-mentioned target device, where q is a real number greater than 0 and less than 1. The specific estimation process can be exemplified by the following calculation formula:

[0220]

[0221] The meanings of the relevant characters can be found in the relevant description in the above formula (1), which will not be repeated here.

[0222] Similarly, assuming a K meet c i The probability is 1 / n, that is, a K The probability of encountering each number in C is the same. In fact, the difference between the number of delays encountered at different positions in the above polling method is at most 1. Therefore, it is assumed here that a K meet c i The probability of 1 / n is small. The precise analysis of this problem is related to the timing of the delay arrival. The solution is complex and difficult to accurately solve. Therefore, it can be assumed that the method of randomly replacing the original large delay with a small delay every certain number of packets is independent of the default recording method of the second recording unit. That is, it is assumed that the default recording method of the second recording unit is executed with a probability of q each time a polling comparison is performed, and the method of randomly replacing the original large delay with a small delay is executed with a probability of 1-q, where q is a set real number greater than 0 and less than 1. Based on this assumption, then:

[0223]

[0224] but

[0225]

[0226] but

[0227]

[0228] Wherein, q represents the default recording mode of executing the second recording unit with a probability of q, and q is a set real number greater than 0 and less than 1. The meaning of the relevant characters can be found in the relevant description in the above formula (1), which will not be repeated here.

[0229] In one possible implementation, regarding the processing of information recorded in the multiple first recording units mapped to the first flow after the end of the above-mentioned estimation window, please refer to the corresponding description in the case where the number of packets is less than the threshold. Regarding the multiple second recording units associated with the multiple first recording units, for the second recording units without hash conflicts, the first network device can release the storage space of these second recording units for subsequent recording and storage of other information.

[0230] Based on the tail delay estimation method described above, the first network device can estimate the tail delay of the first flow passing through the target device within the estimation window. If the estimation window is the preset time window, the transmission process of the first flow may include multiple preset time windows. The first network device can estimate the tail delay of the first flow passing through the target device within each of the multiple preset time windows. The specific estimation process can be found in the above description. For each estimated tail delay of the first flow passing through the target device within a preset time window, the first network device can send this tail delay to the controller for subsequent summary analysis and processing. Alternatively, after estimating the tail delay of the first flow passing through the target device within the multiple preset time windows, the first network device can send all the tail delays to the controller for subsequent summary analysis and processing. If the estimation window is the entire transmission time of the flow, the first network device can send the estimated tail delay of the first flow passing through the target device to the controller after estimating the tail delay. The controller can then perform summary analysis and processing based on this tail delay information.

[0231] In addition to receiving the tail delay information of the first flow passing through the above-mentioned target device from the first network device, the controller can also receive the tail delay information of the first flow passing through the above-mentioned target device from other network devices through which the first flow passes in the data plane. The estimation and processing operations of the tail delay of the first flow passing through the above-mentioned target device by each of the other network devices are the same as those of the first network device and will not be repeated here.

[0232] After the controller receives the tail delay information of the first flow sent by the above-mentioned first network device and the above-mentioned other network devices through the above-mentioned target device, it can be summarized in time sequence. For example, the tail delay estimation data under a longer time window (up to the life cycle of the flow) can be obtained by using the number of packets as the weight and weighted quantile method. Then, based on these tail delay estimation data, according to the needs of network operation and maintenance, tail delay mutation warning can be performed to detect anomalies such as congestion, and network space mapping of tail delay can be performed to obtain delay-based network slicing analysis and network bottleneck detection. Based on the obtained full-flow tail delay data, a variety of network measurement tools for users and operation and maintenance personnel can be deployed.

[0233] In a possible implementation, the operation of estimating the tail delay of the first stream in the corresponding estimation window based on the information recorded in the first recording unit, or the first recording unit and the second recording unit, can be implemented by a controller. Specifically, the first network device records relevant information in the first recording unit, or the first recording unit and the second recording unit of the first stream mapping within the estimation window. For the specific recorded information, please refer to the above description and will not be repeated here. Then, after the estimation window ends, the first network device sends the information recorded in the first recording unit, or the first recording unit and the second recording unit to the controller. Optionally, the information in the index recording unit that records the index information between the first recording unit and the second recording unit is also sent to the controller. The controller estimates the tail delay of the first stream passing through the target device within the corresponding estimation window based on this information. The specific estimation method can refer to the estimation method of the first network device mentioned above and will not be repeated here.

[0234] Optionally, in the case where only the above-mentioned first delay and number of packets are recorded in the first recording unit, when the first network device sends the information in the first recording unit to the controller, it can send the flow information of the flow mapped to the first recording unit to the controller, so that the controller can associate the information recorded in the first recording unit with the corresponding flow.

[0235] In order to better understand the communication delay estimation method introduced above, further examples are given below with reference to the accompanying drawings.

[0236] See also Figure 6 , Figure 6 The schematic diagram of the processing flow of the first network device is shown as an example. Figure 6 The whole process involves Bloom filter, stream encoder, long flow inspection unit, long flow delay recording unit and information recording unit.

[0237] Specifically, the first network device hashes the flow information of the first flow into multiple bits in a bitmap through a Bloom filter, and multiple first recording units in the information recording unit mapped by the first flow can be determined through the multiple bits.

[0238] The first network device performs the aforementioned XOR operation on the stream information through the stream encoder, and records the result of the XOR operation in the first recording unit, or records the stream information of the first stream in the first recording unit. In addition, the delay information, the number of packets and the number of streams can be recorded in multiple first recording units mapped by the first stream, and the number of packets recorded in each of the multiple first recording units is returned.

[0239] The first network device checks through the above-mentioned long stream inspection unit whether there is a number of packets returned by the stream encoder that is equal to the above-mentioned threshold value. If so, then a second recording unit is allocated to the first recording unit corresponding to the number of packets equal to the threshold value, and an index recording unit is allocated to record the index information between the first recording unit and the second recording unit. In addition, information on the specific position of the second recording unit in the information recording unit is returned.

[0240] If the number of packets recorded in the first recording unit corresponding to the number of packets equal to the threshold is greater than the threshold, the first network device, through the long flow delay recording unit, records the delay of the larger packet passing through the target device in the second recording unit in the polling manner described above. Optionally, the first network device, through the long flow delay recording unit, may also randomly select a delay from the delays recorded in the second recording unit and replace it with a smaller delay every certain number of packets, as described above.

[0241] For the specific implementation of the operations performed by the first network device through the Bloom filter, the stream encoder, the long flow checking unit and the long flow delay recording unit, reference may be made to the aforementioned related descriptions, which will not be repeated here.

[0242] exist Figure 6In the figure, it can also be seen that the total storage size allocated to all first recording units in the information recording unit is k*(9+x) bytes (B), where k is the number of bits in the bitmap. Since each bit map has a first recording unit, k is an integer greater than 1. In addition, each first recording unit can include four items of information: the XOR result of the flow information, the number of flows, the number of packets, and the first delay. For example, 1 byte can be used to record the number of flows, 4 bytes can be used to record the number of packets, 4 bytes can be used to record the first delay, and x bytes can be used to record the XOR result of the flow information. The x can be configured as needed. It can be calculated that the total storage size allocated to all first recording units is k*(9+x) bytes. In addition, the total storage size allocated to all index recording units in the information recording unit is k*log2(M) / 8 bytes, and the total storage size allocated to all second recording units is M*n*4 bytes. Among them, M is the number of second recording units, which can be estimated based on the number of concurrent flows that may exceed the threshold. Specifically, it can be estimated based on the existing flow size distribution and network slicing results. n is the maximum number of delay information that can be recorded by each second recording unit. The size of each second recording unit is n*4 bytes, and each delay information is recorded with 4 bytes.

[0243] It should be noted that the specific number of bytes given here is only an example and does not constitute a limitation to the embodiments of the present application. In specific implementations, other sizes of bytes may be used to record this information.

[0244] See also Figure 7 ,exist Figure 7 In the embodiment, each network device can be the first network device mentioned above, then each network device can implement the operation implemented by the first network device, or the first network device is the Figure 7 Any of the network devices shown in . Specifically, a control information channel is first established between each network device and the controller. The controller sets controller information such as the estimation window, quantile, and monitoring threshold. This control information is then sent to each network device via the control information channel. Then, a data transmission channel is established between each network device and the controller. The controller sends a data signal of the tail delay measurement of the flow to each network device. Based on this data signal, each network device begins recording information related to the received flow delay. For specific recorded information, refer to the information recorded in the first recording unit and the second recording unit.

[0245] After each network device has recorded the relevant information in the above-mentioned first recording unit, or the above-mentioned first recording unit and the above-mentioned second recording unit, in one possible implementation, each network device sends the information recorded by the above-mentioned recording unit to the controller through its own device control surface, and the controller estimates the tail delay of the flow based on this information. Then, the controller summarizes and analyzes the estimated tail delay and uses the analyzed data for related processing needs such as operation and maintenance. The specific estimation process and summary analysis can be found in the above-mentioned related descriptions and will not be repeated here. Regarding the process of sending the information recorded in the recording unit to the controller, you can refer to the example of Figure 8 .

[0246] See also Figure 8 , Figure 8 As shown, each network device in the data plane can perform the operations performed by the first network device mentioned above, so each network device records the relevant information of the data packet delay. Taking the first network device as an example, after the corresponding estimation window ends, the first network device transfers the information recorded by the recording unit to its own device control plane. Specifically, the device control plane can obtain the information in these recording units through a pull operation, or the first network device can push the register group composed of these recording units to its own device control plane. Specifically, these recording units may include the information recorded in the first recording unit, index recording unit and second recording unit of the first flow mapping mentioned above, Figure 8 In the example, the number of the first recording units mapped by the first stream is three, and the number of the corresponding second recording units is also three. However, in the specific implementation, the number of the first and second recording units is not limited.

[0247] in addition, Figure 8 As shown in the figure, assuming that the tail delay estimation of the first flow passing through the above-mentioned target device is performed by the controller, then the device control plane of the first network device can send the information of the above-mentioned recording unit obtained in the form of a data packet to the controller through the above-mentioned established data transmission channel. The specific format of the data packet sent can be seen in Figure 8 The format shown in , the data packet includes a network device identifier (sid), an estimation window identifier (tw), a data type (type), and information recorded in a record unit. The network device identifier is used to identify the network device that sends the data packet, the estimation window identifier is used to identify the information in which estimation window the information in the sent record unit is recorded, and the data type is used to identify whether the load data in the data packet is the information in the first record unit, the information in the index record unit, or the information in the second record unit. Figure 8For example, if the data type is 1, it indicates that the payload data in the data packet is the information in the first record unit; if the data type is 2, it indicates that the payload data in the data packet is the information in the index record unit; and if the data type is 3, it indicates that the payload data in the data packet is the information in the second record unit.

[0248] In another possible implementation, after the corresponding estimation window ends, the recorded information can be sent to its own device control plane for corresponding tail delay estimation. For specific estimation, please refer to the description of the first network device estimating the tail delay of the first flow passing through the above-mentioned target device. Then, each network device sends the estimated tail delay information of the flow to the controller, which summarizes and analyzes it and uses the analyzed data for operation and maintenance and other related processing needs. For example, still taking the first network device as an example, the specific way in which the first network device transfers the information recorded by the recording unit to its own device control plane can be referred to the above description. Figure 8 The corresponding description in will not be repeated here. After the device control plane of the first network device obtains this information, it estimates the tail delay of the first flow in the corresponding estimation window after passing through the above-mentioned target device, and can send the tail delay to the controller through the data transmission channel established above. Specifically, the tail delay is also encapsulated in a data packet and sent. In order to identify that the data in the data packet is a data packet with tail delay, a special value of the TCP reserved field can be set in the packet header of the data packet. In this way, after the controller receives the data, it finds that the special value of the TCP reserved field exists in the data packet header, and can determine that the data packet is a data packet including the tail delay of the flow. In addition, the data packet also includes the above-mentioned network device identifier (sid), estimated window identifier (tw) and data type (type), and can also include information such as flow information of the first flow and the number of packets of the first flow received in the corresponding estimation window.

[0249] Optionally, in the case where the number of packets of the target flow received by the first network device within the estimation window is less than the above threshold, since the first record unit mapped by the target flow has no associated second record unit, the information transferred by the first network device to its own device control plane does not include information of the second record unit and the index record unit, and the remaining information is still sent. Please refer to the above description for details.

[0250] In the embodiments introduced above, although most of them are introduced using the first flow and the first data packet as examples, the above-mentioned first network device can record the information of each flow passing through itself in the manner described above, thereby realizing the estimation of the tail delay of each flow, that is, realizing the tail delay estimation of the entire flow.

[0251] In summary, in the embodiments of the present application, it is possible to measure tail delay information on a flow-by-flow basis and obtain tail delay information on a flow-by-flow basis for the entire flow at a fine granularity. Furthermore, it is possible to support the measurement of the 99th percentile delay of a 1M flow without exceeding 30MB. At the same time, the data plane logic of the embodiments of the present application can be deployed within a programmable data plane, for example, on a Barefoot Tofino switch, which is a preset capability of the network device and has good deployability. In addition, when the number of packets is greater than the threshold, the information of the second recording unit is updated by the above-mentioned polling method and by randomly selecting the delay in the second recording unit and replacing it with a smaller delay every certain number of packets. Compared with the random sketch algorithm with the same space overhead, the measurement accuracy of the tail delay of the flow is significantly improved. For example, the simulation experiment uses the NS3 simulation platform, the network topology is the fat-tree topology commonly found in data centers, the flow distribution data uses open source data from large data centers, the network load is set at 50%, and the flows at the same end meet Poisson arrival requirements. The second recording unit has n=10, and the random sketch algorithm uses the same storage space as the embodiments of the present application under this condition to sample and store the flow delay. The estimation window is set to 100ms, and other network settings are also reasonable values. In this experimental environment, Figure 9 The simulation results are shown in Figure 2. In the simulation results, for the 99th percentile tail delay, the measurement error is reduced by 96.5%; and for the 95th percentile tail delay, the measurement error is reduced by 80.7%.

[0252] Compared with the existing tail delay measurement scheme, in the embodiment of the present application, by collecting the relevant information of the tail delay of the flow within the estimation window and using the above-mentioned various recording units to record the information, the recording of unnecessary information is reduced, so that valid information is finally obtained, thereby effectively reducing the reporting of repeated or invalid information and greatly reducing the additional bandwidth load generated.

[0253] In summary, the embodiments of the present application can reduce the resource overhead of network devices during the tail delay measurement process of each flow, so that network devices can support the flow-by-flow tail delay measurement task of a large number of concurrent flows in a large-scale network within limited storage resources, without generating huge additional load, and achieving good measurement accuracy.

[0254] The above mainly introduces the communication delay estimation method provided in the embodiment of the present application. It can be understood that, in order to implement the corresponding functions mentioned above, each device includes a hardware structure and / or software module corresponding to the execution of each function. In combination with the units and steps of the various examples described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0255] The embodiment of the present application can divide the functional modules of the device according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. In actual implementation, there may be other division methods.

[0256] In the case of dividing each functional module into corresponding functional modules, Figure 10 A schematic diagram of a possible logical structure of an apparatus is shown. The apparatus may be the first network device described in the above method embodiment, or may be a chip in the first network device, or may be a processing system in the first network device. The apparatus 1000 includes a processing unit 1001 and, optionally, may further include a sending unit 1002 and / or a receiving unit 1003.

[0257] The processing unit 1001 is configured to:

[0258] Obtain flow information from the received first data packet, the flow information being identification information of the first flow to which the first data packet belongs; the processing unit 1001 can be used to implement the above Figure 3 The operation of obtaining flow information in step 301;

[0259] Determine the first recording unit of the first flow mapping based on the flow information; the first recording unit includes a first delay and a number of packets, the first delay is the maximum delay of a single data packet of the target flow passing through the target device, and the number of packets is the number of data packets of the target flow received by the network device; the target flow includes the flow mapped to the first recording unit; the processing unit 1001 can be used to implement the above Figure 3 The operation of determining the first recording unit of the first stream mapping in step 302;

[0260] Update the number of packets, and if the delay of the first data packet passing through the target device is greater than the first delay, update the first delay to the delay of the first data packet passing through the target device; the processing unit 1001 can be used to implement the above Figure 3 In the case where the number of packets is less than the threshold, the first delay is used to estimate the tail delay of the first flow through the target device.

[0261] Optionally, the target device is the network device; or, the network device is a device on a data plane in a network, the network device is the last hop device of the target flow in the data plane, and the target device includes multiple devices that the single data packet passes through in the data plane.

[0262] In one possible implementation, the above-mentioned network device is a device on a data plane in a network, and the network device is the last-hop device of the data packet of the target flow in the data plane. The target device includes multiple devices through which the single data packet passes in the data plane; the delay of the first data packet passing through the target device is the delay of the first data packet passing through the multiple devices; the first data packet includes the delay of the first data packet passing through other devices, and the other devices are other network devices among the multiple devices except the network device.

[0263] In one possible embodiment, the above-mentioned network device is a device on the data plane in the network, and the network device is the last-hop device of the data packet of the target flow in the data plane. The target device includes multiple devices through which the single data packet passes in the data plane; the delay of the first data packet passing through the target device is the delay of the first data packet passing through the multiple devices; the first data packet includes the timestamp of the entry of the first-hop device through which the first data packet passes in the data plane, and the timestamp is used to calculate the delay of the first data packet passing through the multiple devices.

[0264] In one possible implementation, the data packets of the target flow received by the above-mentioned network device are data packets received by the network device within an estimation window, and the estimation window is a time window used to estimate the tail delay of the first flow passing through the above-mentioned target device; the first delay is the maximum delay of a single data packet of the target flow passing through the target device, including: the first delay is the maximum delay of a single data packet of the target flow passing through the target device within the time window; the tail delay of the first flow passing through the above-mentioned target device is the tail delay of the first flow passing through the target device within the time window.

[0265] In one possible implementation, there are multiple first recording units mapped to the first flow; the processing unit 1001 is also used to: after the estimation window ends, if the number of packets recorded by at least one of the multiple first recording units is less than the threshold, determine that the first delay recorded in the first unit is the estimated value of the tail delay of the first flow passing through the target device within the estimation window, and send the first delay recorded in the first unit to the controller; the first unit is the unit with the smallest first delay recorded among the multiple first recording units.

[0266] In a possible implementation, the first recording unit of the first stream mapping is multiple; the network device further includes a sending unit 1002, which is used to send the information recorded in the multiple first recording units to the controller when the estimation window ends.

[0267] In a possible implementation, when the number of packets recorded in the first recording unit is greater than or equal to the threshold, the processing unit 1001 is further configured to:

[0268] A second recording unit is allocated to the first recording unit; the second recording unit is used to record the delay of each of n data packets passing through the target device, where the n data packets are data packets of the target flow received by the network device after the first data packet, and n is an integer greater than 1; the n delays are used to estimate the tail delay of the first flow passing through the above-mentioned target device.

[0269] In a possible implementation, the network device further includes a receiving unit 1003, configured to receive a second data packet, where the second data packet is a data packet of the target flow;

[0270] The above-mentioned processing unit 1001 is also used to obtain the second delay among the n delays of the second recording unit according to a polling manner; when the delay of the second data packet passing through the target device is greater than the second delay, the second delay in the second recording unit is updated to the delay of the second data packet passing through the target device.

[0271] In one possible implementation, the receiving unit 1003 is further configured to receive a third data packet, where the third data packet is a data packet of the target flow, and before the third data packet and after the second data packet, the receiving unit 1003 receives m data packets of the target flow, where m is an integer greater than 1.

[0272] The above-mentioned processing unit 1001 is also used to randomly obtain a third delay from the n delays of the second recording unit; when the delay of the third data packet passing through the target device is less than the third delay, the third delay in the second recording unit is updated to the delay of the third data packet passing through the target device.

[0273] In a possible implementation, the processing unit 1001 is further configured to: when the estimation window ends, estimate the tail delay of the first flow passing through the target device within the estimation window based on the n delays in the second recording unit.

[0274] Optionally, the tail delay of the first stream passing through the above-mentioned target device is the p-percentile tail delay, and p is greater than 80 and less than 100; the above-mentioned processing unit 1001 estimates the tail delay of the first stream passing through the above-mentioned target device based on the n delays in the second recording unit, including: determining the w-th delay among the n delays as the estimated tail delay value of the first stream passing through the target device within the estimation window, the w-th delay is the delay ranked w among the n delays sorted from small to large, and w is an integer greater than 0 and less than or equal to n.

[0275] In a possible implementation, the network device further includes a sending unit 1002, configured to send the information recorded in the first recording unit and the second recording unit to a controller when the estimation window ends.

[0276] In one possible implementation, the processing unit 1001 determines the first record unit of the first flow mapping based on the flow information, including: mapping the flow information to multiple bits in a bitmap through multiple hash functions; a bit in the bitmap is associated with one of the first record units; and the network device determines the first record unit of the first flow mapping based on the multiple bits.

[0277] In a possible implementation, the first recording unit further includes an XOR result of the number of streams and the stream information, where the number of streams is the number of target streams, and the XOR result of the stream information is a result obtained by performing an XOR operation on the stream information of the target stream.

[0278] Figure 10 The specific operation and beneficial effects of each unit in the device 1000 can be found in the above Figure 3 The corresponding description in possible method embodiments will not be repeated here.

[0279] In the case of dividing each functional module into corresponding functional modules, Figure 11 A possible logical structure diagram of an apparatus is shown. The apparatus may be the first network device described in the above method embodiment, or may be a chip in the first network device, or may be a processing system in the first network device. The apparatus 1100 includes a processing unit 1101 and a receiving unit 1102.

[0280] The processing unit 1101 is configured to:

[0281] Obtain flow information from the first data packet received by the receiving unit 1102, the flow information being identification information of the first flow to which the first data packet belongs; the processing unit 1001 can be used to implement the above Figure 3 The operation of obtaining flow information in step 301;

[0282] Determine the first recording unit of the first flow mapping based on the flow information; the first recording unit includes a first delay and a number of packets, the first delay is the maximum delay of a single data packet of the target flow passing through the target device, and the number of packets is the number of data packets of the target flow received by the network device; the target flow includes the flow mapped to the first recording unit; the processing unit 1001 can be used to implement the above Figure 3 The operation of determining the first recording unit of the first stream mapping in step 302;

[0283] Update the number of packets, and if the delay of the first data packet passing through the target device is greater than the first delay, update the first delay to the delay of the first data packet passing through the target device; the processing unit 1001 can be used to implement the above Figure 3 The operation updated in step 303;

[0284] When the number of packets is greater than or equal to a threshold, a second recording unit is allocated to the first recording unit; the second recording unit is used to record the delay of each of the n data packets passing through the target device, where the n data packets are part of the data packets of the target flow received by the network device after the first data packet, and n is an integer greater than 1; the n delays are used to estimate the tail delay of the first flow passing through the above-mentioned target device.

[0285] In the case of dividing each functional module into corresponding functional modules, Figure 12 A possible logical structure diagram of a device is shown. The device may be the controller described in the above method embodiment, or may be a chip in the controller, or may be a processing system in the controller. The device 1200 includes a receiving unit 1201 and a processing unit 1202.

[0286] Receiving unit 1201, configured to receive information recorded in a plurality of first recording units, wherein the information recorded in each of the first recording units includes a first delay and a number of packets, wherein the first delay is a maximum delay for a single data packet of a target flow to pass through a target device, and the number of packets is a number of data packets of the target flow received by the network device; the target flow includes a flow mapped to the first recording unit, and the target flow includes a first flow;

[0287] The processing unit 1202 is configured to estimate a tail delay of the first flow passing through the target device based on the first delays recorded in the multiple first recording units when the number of packets is less than a threshold.

[0288] Optionally, the target device is the network device; or, the network device is a device on a data plane in a network, the network device is the last hop device of the target flow in the data plane, and the target device includes multiple devices that the single data packet passes through in the data plane.

[0289] In one possible implementation, the receiving unit 1201 is further configured to receive an identifier of an estimation window, where the identifier indicates that the information recorded in the plurality of first recording units is recorded within the estimation window, where the estimation window is a time window for estimating a tail delay of the first flow passing through the target device.

[0290] The above-mentioned processing unit 1202 estimates the tail delay of the first flow passing through the above-mentioned target device based on the first delay recorded in the multiple first recording units, including: determining that the first delay recorded in the first unit is the estimated value of the tail delay of the first flow passing through the target device within the estimation window; the first unit is the unit with the smallest first delay recorded in the multiple first recording units.

[0291] In the case of dividing each functional module into corresponding functional modules, Figure 13 A possible logical structure diagram of a device is shown. The device may be the controller described in the above method embodiment, or may be a chip in the controller, or may be a processing system in the controller. The device 1300 includes a receiving unit 1301 and a processing unit 1302.

[0292] A receiving unit 1301 is configured to receive information recorded in a first recording unit and a second recording unit, wherein the information recorded in the first recording unit includes a first delay and a number of packets, wherein the first delay is a maximum delay for a single data packet of the first flow to pass through a target device, and the number of packets is a number of data packets of the first flow received by the network device; the second recording unit is associated with the first recording unit, and the information recorded in the second recording unit includes a delay for each of n data packets in the first flow to pass through the target device, where n is an integer greater than 1;

[0293] The processing unit 1302 is configured to estimate a tail delay of the first flow passing through the target device based on the n delays.

[0294] In one possible implementation, the receiving unit 1301 is further configured to receive an identifier of an estimation window, where the identifier indicates that the information recorded in the first recording unit and the second recording unit is recorded within the estimation window, where the estimation window is a time window for estimating the tail delay of the first flow passing through the target device.

[0295] The tail delay of the first flow passing through the target device is a p-percentile tail delay, where p is greater than 80% and less than 100%. The processing unit 1302 estimates the tail delay of the first flow passing through the target device based on the n delays, including:

[0296] The wth delay among the n delays is determined as the estimated tail delay value of the first flow passing through the target device within the estimation window, the wth delay is the delay ranked w among the n delays sorted from small to large, and w is an integer greater than 0 and less than or equal to n.

[0297] Figure 14 The figure shows a possible hardware structure diagram of an apparatus according to an embodiment of the present application. The apparatus may be the first network device described in the above embodiment, or may be a chip in the first network device, or may be a processing system in the first network device. The apparatus 1400 includes a processor 1401, a memory 1402, and a communication interface 1403. The processor 1401, the communication interface 1403, and the memory 1402 may be interconnected or connected to each other via a bus 1404.

[0298] Exemplarily, the memory 1402 is used to store computer programs and data of the device 1400. The memory 1402 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or portable read-only memory (CD-ROM).

[0299] The communication interface 1403 includes a sending interface and a receiving interface. There may be multiple communication interfaces 1403 , which are used to support the device 1400 to communicate, such as receiving or sending data or messages.

[0300] Exemplarily, the processor 1401 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like. The processor 1401 may be used to read the program stored in the memory 1402, so that the apparatus 1400 executes the above-mentioned Figure 3 The operations performed by the first network device in any one of the communication delay estimation methods described in its possible embodiments.

[0301] In one possible implementation, the processor 1401 may be configured to perform the following operations: obtaining flow information from a received first data packet, the flow information being identification information of a first flow to which the first data packet belongs; determining a first recording unit for mapping the first flow based on the flow information; the first recording unit including a first delay and a number of packets, the first delay being the maximum delay for a single data packet of the target flow to pass through a target device, and the number of packets being the number of data packets of the target flow received by the network device; the target flow including a flow mapped to the first recording unit; updating the number of packets, and updating the first delay to the delay for the first data packet to pass through the target device when the delay for the first data packet to pass through the target device is greater than the first delay; and when the number of packets is less than a threshold, the first delay is used to estimate the tail delay for the first flow to pass through the above-mentioned target device.

[0302] Figure 14 The specific operation and beneficial effects of each unit in the device 1400 can be found in the above Figure 3 The corresponding description in possible method embodiments will not be repeated here.

[0303] Figure 15 The figure shows a possible hardware structure diagram of an apparatus according to an embodiment of the present application. The apparatus may be the first network device described in the above embodiment, or may be a chip in the first network device, or may be a processing system in the first network device. The apparatus 1500 includes a processor 1501, a memory 1502, and a communication interface 1503. The processor 1501, the communication interface 1503, and the memory 1502 may be interconnected or connected via a bus 1504.

[0304] Exemplarily, the memory 1502 is used to store computer programs and data of the device 1500. The memory 1502 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or portable read-only memory (CD-ROM).

[0305] The communication interface 1503 includes a sending interface and a receiving interface. There may be multiple communication interfaces 1503 , which are used to support the device 1500 to communicate, such as receiving or sending data or messages.

[0306] Exemplarily, the processor 1501 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like. The processor 1501 may be used to read the program stored in the memory 1502, so that the apparatus 1500 executes the above-mentioned Figure 3 The operations performed by the first network device in any one of the communication delay estimation methods described in its possible embodiments.

[0307] In one possible implementation, the processor 1501 may be configured to perform the following operations: obtaining flow information from the received first data packet, the flow information being used to indicate that the first data packet is a data packet in a first flow; determining a first recording unit to which the first flow is mapped based on the flow information; the first recording unit being used to record a first delay and a number of packets, the first delay being the maximum delay for a single data packet of the target flow to pass through a target device, and the number of packets being the number of data packets of the target flow received by the network device; the target flow including the flow mapped to the first recording unit; updating the number of packets, and if the delay for the first data packet to pass through the target device is greater than the first delay, updating the first delay to the delay for the first data packet to pass through the target device; if the number of packets is greater than or equal to a threshold, allocating a second recording unit to the first recording unit; the second recording unit being used to record the delay for each of n data packets to pass through the target device, the n data packets being part of the data packets of the target flow received by the network device after the first data packet, where n is an integer greater than 1; the n delays being used to estimate a tail delay for the first flow to pass through the target device.

[0308] Figure 15 The specific operation and beneficial effects of each unit in the device 1500 can be found in the above Figure 3 The corresponding description in possible method embodiments will not be repeated here.

[0309] Figure 16 The figure shows a possible hardware structure diagram of a device according to an embodiment of the present application. The device can be the controller described in the above embodiment, or a chip within the controller, or a processing system within the controller. The device 1600 includes a processor 1601, a memory 1602, and a communication interface 1603. The processor 1601, the communication interface 1603, and the memory 1602 can be interconnected or connected via a bus 1604.

[0310] Exemplarily, the memory 1602 is used to store computer programs and data of the device 1600. The memory 1602 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or portable read-only memory (CD-ROM).

[0311] The communication interface 1603 includes a sending interface and a receiving interface. There may be multiple communication interfaces 1603 , which are used to support the device 1600 to communicate, such as receiving or sending data or messages.

[0312] Exemplarily, the processor 1601 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like. The processor 1601 may be used to read the program stored in the memory 1602, so that the device 1600 executes the above-mentioned program. Figure 3 The operations performed by the controller in any communication delay estimation method described in its possible embodiments.

[0313] In one possible implementation, the processor 1601 may be used to perform the following operations: receiving information recorded in a plurality of first recording units, the information recorded in each of the first recording units including a first delay and a number of packets, the first delay being the maximum delay for a single data packet of the target flow to pass through a target device, and the number of packets being the number of data packets of the target flow received by the network device; the target flow including a flow mapped to the first recording unit, the target flow including a first flow; and when the number of packets is less than a threshold, estimating the tail delay of the first flow passing through the above-mentioned target device based on the first delay recorded in the plurality of first recording units.

[0314] Figure 16 The specific operation and beneficial effects of each unit in the device 1600 can be found in the above Figure 3 The corresponding description in possible method embodiments will not be repeated here.

[0315] Figure 17 The figure shows a possible hardware structure diagram of a device according to an embodiment of the present application. The device may be the controller described in the above embodiment, or may be a chip within the controller, or may be a processing system within the controller. The device 1700 includes a processor 1701, a memory 1702, and a communication interface 1703. The processor 1701, the communication interface 1703, and the memory 1702 may be interconnected or connected via a bus 1704.

[0316] Exemplarily, the memory 1702 is used to store computer programs and data of the device 1700. The memory 1702 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or portable read-only memory (CD-ROM).

[0317] The communication interface 1703 includes a sending interface and a receiving interface. There may be multiple communication interfaces 1703 , which are used to support the device 1700 to communicate, such as receiving or sending data or messages.

[0318] Exemplarily, the processor 1701 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like. The processor 1701 may be used to read the program stored in the memory 1702, so that the apparatus 1700 executes the above-mentioned Figure 3 The operations performed by the controller in any communication delay estimation method described in its possible embodiments.

[0319] In one possible implementation, the processor 1701 may be used to perform the following operations: receiving information recorded in a first recording unit and a second recording unit, the information recorded in the first recording unit including a first delay and a number of packets, the first delay being the maximum delay for a single data packet of the first flow to pass through a target device, and the number of packets being the number of data packets of the first flow received by the network device; the second recording unit being associated with the first recording unit, the information recorded in the second recording unit including the delay for each of n data packets in the first flow to pass through the target device, where n is an integer greater than 1; and estimating the tail delay of the first flow passing through the above-mentioned target device based on the n delays.

[0320] Figure 17 The specific operation and beneficial effects of each unit in the device 1700 can be found in the above Figure 3 The corresponding description in possible method embodiments will not be repeated here.

[0321] The present invention also provides a computer-readable storage medium that stores a computer program. The computer program is executed by a processor to implement the above Figure 3 The operations performed by the first network device in the method described in any embodiment of the possible method embodiments.

[0322] The present invention also provides a computer-readable storage medium that stores a computer program. The computer program is executed by a processor to implement the above Figure 3 and the operations performed by the controller in the method described in any embodiment of the possible method embodiments.

[0323] The present application also provides a computer program product. When the computer program product is read and executed by a computer, the above Figure 3 The operations performed by the first network device in any of the methods described in any of the possible method embodiments will be executed.

[0324] The present application also provides a computer program product. When the computer program product is read and executed by a computer, the above Figure 3 The operations performed by the controller in any of the methods described in the possible method embodiments will be executed.

[0325] In summary, the embodiments of the present application can reduce the resource overhead of the network device during the tail delay measurement process of each flow, so that the network device can measure and estimate the tail delay of all flows passing through the network device based on existing storage and processing resources.

[0326] In this application, the terms "first", "second", etc. are used to distinguish between identical or similar items having substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limit on quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the various described examples, the first delay can be referred to as the second delay, and similarly, the second delay can be referred to as the first delay. Both the first delay and the second delay can be delays, and in some cases, can be separate and different delays.

[0327] It should also be understood that in each embodiment of the embodiments of the present application, the size of the serial number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0328] It will also be understood that the term “comprise” (also known as “includes,” “including,” “comprises,” and / or “comprising”) when used in this specification specifies the presence of stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0329] It should also be understood that references throughout this specification to "one embodiment," "an embodiment," or "one possible implementation" mean that specific features, structures, or characteristics associated with that embodiment or implementation are included in at least one embodiment of the present application. Therefore, the appearance of "in one embodiment," "in an embodiment," or "one possible implementation" throughout this specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0330] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A communication delay estimation method, characterized in that: The method comprises: The network device obtains flow information from the received first data packet, where the flow information is identification information of a first flow to which the first data packet belongs; The network device determines a first recording unit of the first flow mapping based on the flow information; the first recording unit includes a first delay and a number of packets, the first delay being a maximum delay for a single data packet of the target flow to pass through the target device, and the number of packets being the number of data packets of the target flow received by the network device; the target flow includes the flow mapped to the first recording unit; The network device updates the number of packets, and when a delay of the first data packet passing through the target device is greater than the first delay, updates the first delay to the delay of the first data packet passing through the target device; When the number of packets is smaller than a threshold, the first delay is used to estimate a tail delay of the first flow passing through the target device.

2. The method according to claim 1, characterized in that The target device is the network device; or, The network device is a device on a data plane in a network, the network device is a last-hop device of the target flow in the data plane, and the target device includes multiple devices that the single data packet passes through in the data plane.

3. The method according to claim 1 or 2, characterized in that The network device is a device on a data plane in a network, the network device is the last hop device of the data packet of the target flow in the data plane, and the target device includes multiple devices that the single data packet passes through in the data plane; The delay of the first data packet passing through the target device is the delay of the first data packet passing through the multiple devices; The first data packet includes a time delay of the first data packet passing through other devices, and the other devices are other network devices among the multiple devices except the network device.

4. The method according to claim 1 or 2, characterized in that The network device is a device on a data plane in a network, the network device is the last hop device of the data packet of the target flow in the data plane, and the target device includes multiple devices that the single data packet passes through in the data plane; The delay of the first data packet passing through the target device is the delay of the first data packet passing through the multiple devices; The first data packet includes a timestamp of an entry of a first-hop device that the first data packet passes through in the data plane, and the timestamp is used to calculate a delay of the first data packet passing through the multiple devices.

5. The method according to claim 1 or 2, characterized in that The data packets of the target flow received by the network device are data packets received by the network device within an estimation window, where the estimation window is a time window for estimating the tail delay of the first flow passing through the target device; The first delay is the maximum delay of a single data packet of the target flow passing through the target device, including: the first delay is the maximum delay of a single data packet of the target flow passing through the target device within the time window; The tail delay of the first flow passing through the target device is the tail delay of the first flow passing through the target device within the time window.

6. The method according to claim 5, characterized in that The first recording unit of the first stream mapping is multiple; and the method further includes: After the estimation window ends, if the number of packets recorded by at least one of the multiple first recording units is less than the threshold, the network device determines that the first delay recorded in the first unit is the estimated value of the tail delay of the first flow passing through the target device within the estimation window, and sends the first delay recorded in the first unit to the controller; the first unit is the unit with the smallest first delay recorded among the multiple first recording units.

7. The method according to claim 5, characterized in that The first recording unit of the first stream mapping is multiple; and the method further includes: When the estimation window ends, the network device sends the information recorded in the plurality of first recording units to a controller.

8. The method according to claim 5, characterized in that In a case where the number of packets recorded in the first recording unit is greater than or equal to the threshold, the method further includes: The network device allocates a second recording unit to the first recording unit; the second recording unit is used to record the delay of each of n data packets passing through the target device, the n data packets are data packets in the data packets of the target flow received by the network device after the first data packet, and n is an integer greater than 1; the n delays are used to estimate the tail delay of the first flow passing through the target device.

9. The method according to claim 8, characterized in that The method further comprises: The network device receives a second data packet, where the second data packet is a data packet of the target flow; The network device obtains the second delay among the n delays of the second recording unit in a polling manner; In a case where the delay of the second data packet passing through the target device is greater than the second delay, the network device updates the second delay in the second recording unit to the delay of the second data packet passing through the target device.

10. The method according to claim 8, characterized in that The method further comprises: The network device receives a third data packet, where the third data packet is a data packet of the target flow. Before the third data packet and after the second data packet, the network device receives m data packets of the target flow, where m is an integer greater than 1. The network device randomly obtains a third delay from the n delays in the second recording unit; In a case where the delay of the third data packet passing through the target device is less than the third delay, the network device updates the third delay in the second recording unit to the delay of the third data packet passing through the target device.

11. The method according to claim 9 or 10, characterized in that The method further comprises: When the estimation window ends, the network device estimates a tail delay of the first flow passing through the target device within the estimation window based on the n delays in the second recording unit.

12. The method according to claim 11, characterized in that The tail delay of the first flow passing through the target device is p-percentile tail delay, where p is greater than 80 and less than 100; The network device estimating the tail delay of the first flow passing through the target device based on the n delays in the second recording unit, including: The network device determines the wth delay among the n delays as the estimated tail delay of the first flow passing through the target device within the estimation window, the wth delay being the delay ranked w among the n delays sorted from small to large, and w being an integer greater than 0 and less than or equal to n.

13. The method according to claim 9 or 10, characterized in that The method further comprises: When the estimation window ends, the network device sends the information recorded in the first recording unit and the second recording unit to a controller.

14. The method according to claim 1 or 2, characterized in that The network device determines, based on the flow information, a first recording unit of the first flow mapping, including: The network device maps the flow information into a plurality of bits in a bitmap using a plurality of hash functions; a bit in the bitmap is associated with a first recording unit; The network device determines the first recording unit of the first stream map based on the plurality of bits.

15. The method according to claim 1 or 2, characterized in that The first recording unit also includes the number of streams and the XOR result of the stream information, where the number of streams is the number of the target streams, and the XOR result of the stream information is the result obtained by performing an XOR operation on the stream information of the target stream.

16. A communication delay estimation method, characterized in that: The method comprises: The network device obtains flow information from the received first data packet, where the flow information is used to indicate that the first data packet is a data packet in a first flow; The network device determines a first recording unit of the first flow mapping based on the flow information; the first recording unit is used to record a first delay and a number of packets, the first delay being a maximum delay for a single data packet of the target flow to pass through the target device, and the number of packets being the number of data packets of the target flow received by the network device; the target flow includes the flow mapped to the first recording unit; The network device updates the number of packets and, if a delay of the first data packet passing through the target device is greater than the first delay, updates the first delay to a delay of the first data packet passing through the target device; When the number of packets is greater than or equal to a threshold, the network device allocates a second recording unit to the first recording unit; the second recording unit is used to record the delay of each of n data packets passing through the target device, where the n data packets are part of the data packets of the target flow received by the network device after the first data packet, and n is an integer greater than 1; the n delays are used to estimate the tail delay of the first flow passing through the target device.

17. A communication delay estimation method, characterized in that: The method comprises: The controller receives information recorded in a plurality of first recording units from a network device, wherein the information recorded in each of the first recording units includes a first delay and a number of packets, wherein the first delay is a maximum delay for a single data packet of a target flow to pass through a target device, and the number of packets is a number of data packets of the target flow received by the network device; the target flow includes a flow mapped to the first recording unit, and the target flow includes a first flow; When the number of packets is smaller than a threshold, the controller estimates a tail delay of the first flow passing through the target device based on the first delays recorded in the plurality of first recording units.

18. The method according to claim 17, characterized in that The method further includes: the controller further receiving an identifier of an estimation window, the identifier indicating that the information recorded in the plurality of first recording units is recorded within the estimation window, the estimation window being a time window for estimating a tail delay of the first flow passing through the target device; The controller estimates the tail delay of the first flow passing through the target device based on the first delay recorded in the multiple first recording units, including: the controller determines that the first delay recorded in the first unit is the estimated tail delay value of the first flow passing through the target device within the estimation window; the first unit is the unit with the smallest first delay recorded in the multiple first recording units.

19. A communication delay estimation method, characterized in that: The method comprises: The controller receives information recorded in a first recording unit and a second recording unit from a network device, where the information recorded in the first recording unit includes a first delay and a number of packets, where the first delay is a maximum delay for a single data packet of a first flow to pass through a target device, and the number of packets is the number of data packets of the first flow received by the network device; The second recording unit is associated with the first recording unit, and information recorded in the second recording unit includes a delay of each of n data packets in the first flow passing through the target device, where n is an integer greater than 1; The controller estimates a tail delay of the first flow passing through the target device based on the n delays.

20. The method according to claim 19, characterized in that The method further includes: the controller further receiving an identifier of an estimation window, the identifier indicating that the information recorded in the first recording unit and the second recording unit is recorded within the estimation window, the estimation window being a time window for estimating a tail delay of the first flow passing through the target device; The tail delay of the first flow passing through the target device is a p-percentile tail delay, where p is greater than 80% and less than 100%. The controller estimates the tail delay of the first flow passing through the target device based on the n delays, including: The controller determines the wth delay among the n delays as the estimated tail delay of the first flow passing through the target device within the estimation window, the wth delay is the delay ranked w among the n delays sorted from small to large, the delays ranked before w are less than or equal to the p-percentile tail delay, and w is an integer greater than 0 and less than or equal to n.

21. The method according to any one of claims 19-20, characterized in that The target device is the network device; or, The network device is a device on a data plane in a network, the network device is a last-hop device of the target flow in the data plane, and the target device includes multiple devices that the single data packet passes through in the data plane.

22. A network device, characterized in that: The network device comprises a unit for performing the method according to any one of claims 1-15.

23. A network device, characterized in that: The network device comprises means for performing the method of claim 16.

24. A controller, characterized in that: The controller comprises means for performing the method of claim 17 or 18.

25. A controller, characterized in that: The controller comprises means for performing the method of claim 19 or 20.

26. A network device, characterized in that: The network device includes a processor and a memory; wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the network device performs the method according to any one of claims 1 to 15; or, the network device performs the method according to claim 16.

27. A controller, characterized in that: The controller includes a processor and a memory; wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the controller executes the method according to claim 17 or 18; or, causes the network device to execute the method according to claim 19 or 20.

28. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 15; or the computer program is executed by a processor to implement the method according to claim 16.

29. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to claim 17 or 18; or the computer program is executed by a processor to implement the method according to claim 19 or 20.

30. A system, characterized in that The system includes a network device and a controller, wherein the network device is the network device according to claim 22, and the controller is the controller according to claim 24 or 25; or, the network device is the network device according to claim 23, and the controller is the controller according to claim 25; or, the network device is the network device according to claim 26, and the controller is the controller according to claim 27.

Images