A network measurement method, device, system, apparatus and storage medium

Abstract

A network measurement method, device, system, equipment and storage medium are disclosed, and relate to the field of network measurement. Based on a plurality of sketch counters adopting a hierarchical data structure for network flow statistics, the compression of the sketch counters is realized, and the memory resource consumption is reduced. By constructing a linear equation system and solving a least square problem, the approximate solution of the overflow value of each counter segment can be calculated in order from high to low according to the level, and the recovery value is obtained based on the overflow value until the recovery value of each counter segment of the lowest layer is obtained as the counting value of each sketch counter. In this way, by constructing a linear equation system and solving a least square problem, the approximate solution closest to the actual overflow value is obtained, the accuracy of calculating the overflow value is improved, thereby ensuring the accuracy of calculating the counting value of the counter, and the measurement accuracy of the network measurement task on the exchange equipment with scarce memory resources is ensured.

Description

Technical Field

[0001] This application relates to the field of network measurement technology, and in particular to a network measurement method, apparatus, system, device and storage medium. Background Technology

[0002] As data centers evolve towards ultra-high speeds and full-stack programmability, data center switching equipment needs to simultaneously support multiple tasks such as network measurement, packet processing, and state storage. These tasks face memory resource contention, and different network measurement tasks also compete for memory resources. Furthermore, as the concurrent traffic that switching equipment needs to handle increases, the memory resources required for network measurement tasks also increase. Therefore, the limited memory resources available in switching equipment for implementing network measurement tasks necessitate reducing the measurement accuracy of these tasks in exchange for their feasibility.

[0003] Therefore, how to ensure the measurement accuracy of network measurement tasks on switching devices with scarce memory resources has become an urgent technical problem to be solved. Summary of the Invention

[0004] This application provides a network measurement method, apparatus, system, device, and storage medium that can ensure the measurement accuracy of network measurement tasks on programmable switching devices with scarce memory resources.

[0005] Firstly, a network measurement method is provided, comprising: acquiring the count values ​​of counter segments of multiple sketch counters used for statistical network flow; the multiple sketch counters include m levels, each level including one or more counter segments. Following a descending order of levels, a system of linear equations from level 1 to level m-1 is constructed sequentially based on the overflow count of the lower-level counter segment and the recovered value of the upper-level counter segment in every two adjacent levels of the multiple sketch counters. The overflow count indicates the number of times the value of each lower-level counter segment overflows to each upper-level counter segment in every two adjacent levels; the recovered value is the count value recovered based on the count value of the upper-level counter segment and the overflow value; the recovered value of the highest-level counter segment is the count value. The system of linear equations from level 1 to level m-1 is solved sequentially until the recovered value of each counter segment in the lowest level is obtained. In this system, the recovered value of the upper-level counter segment in the j-th linear equation system is equal to the recovered value corresponding to the least squares solution of the (j-1)-th linear equation system. The recovered value corresponding to the least squares solution is determined based on the least squares solution that serves as the overflow value of the lower-level counter segment in the (j-1)-th linear equation system. The measurement result of the network flow is determined based on the recovered values ​​of each counter segment in the lowest layer, where the recovered value of each counter segment in the lowest layer is the count value of each sketch counter.

[0006] Based on the aforementioned network measurement method, this application uses multiple sketch counters with a hierarchical data structure to statistically analyze network flows, thereby compressing the sketch counters and reducing the memory resources required. By constructing a system of linear equations and solving a least-squares problem, approximate solutions to the overflow values ​​of each counter segment can be calculated sequentially from high to low hierarchy. The recovered values ​​of each counter segment are then calculated based on these overflow values ​​until the recovered values ​​of the lowest-level counter segments are obtained as the count value for each sketch counter. Thus, compared to existing schemes that estimate the numerical range of the overflow values ​​of counter segments, this application, by constructing a system of linear equations and solving a least-squares problem, can obtain approximate solutions that are closer to the actual overflow values ​​of each counter segment, improving the accuracy of the determined overflow values. This ensures the accuracy of the count values ​​determined based on the compressed counters, guaranteeing the measurement accuracy of network measurement tasks on memory-scarce switching devices.

[0007] As one possible implementation, n is obtained by calculating the number of times the value of the q-th counter segment at the i-th level overflows to the p-th counter segment at the (i+1)-th level. i+1 line n i The element value in the p-th row and q-th column of the matrix; the matrix represents the overflow relationship between the counter segment of the i-th level and the counter segment of the (i+1)-th level; n i+1 n represents the number of counter segments contained in the (i+1)th level. i This represents the number of counter segments contained in the i-th level. Based on the matrix and the value vector Y composed of the recovered values ​​of each counter segment in the (i+1)-th level, the j-th linear equation system is obtained. Here, the values ​​of i range from m-1 to 1, with i being 1 indicating the i-th level is the lowest level, and the sum of i and j is m; the vector expression of the j-th linear equation system is Y = AX. Thus, the j-th linear equation system is constructed using the overflow relationship between the counter segments in the i-th and (i+1)-th levels, and the value vector composed of the recovered values ​​of each counter segment in the (i+1)-th level. When the values ​​of i range from m-1 to 1, linear equation systems from the 1st to the (m-1)th levels are constructed sequentially, thereby enabling the determination of the overflow value of each counter segment in each level based on the linear equation system.

[0008] Optionally, based on the overflow flag value of the qth counter segment of the i-th level and the number of times the qth segment is mapped to the pth counter segment of the (i+1)-th level by the hash function of the i-th level, the element value in the p-th row and q-th column of the matrix is ​​determined, thereby determining the matrix representing the overflow relationship between the counter segments of the i-th level and the counter segments of the (i+1)-th level. The accuracy of determining the overflow value of each counter segment is further improved by using the matrix.

[0009] One possible implementation involves determining the j-th least squares solution of the j-th linear equation system, where the j-th least squares solution is the overflow value of each counter segment in the i-th level. Based on the overflow value and count value of each counter segment in the i-th level, the recovered value of each counter segment in the j-th level is determined. This process is repeated with j ranging from 1 to m-1, sequentially obtaining the recovered values ​​for each level from the (m-1)-th level downwards. Then, the next linear equation system is constructed, and the above steps are repeated to obtain the recovered values ​​for the next level. This cycle continues until the recovered value of each counter segment in the first level is obtained. Since overflow occurs at each level, the overflow value flows sequentially to the next level. Therefore, by constructing and solving linear equation systems sequentially downwards from the (m-1)-th level, the recovered values ​​of the counter segments at each level can be accurately obtained, improving the accuracy of network measurements.

[0010] Optionally, determine the value vector X when the L2 norm of Y-AX is minimized, where the value vector X is a value vector composed of the overflow values ​​of each counter segment in the i-th level. This is based on the value vector X of the i-th level and the bit values ​​w of each counter segment in the i-th level. i The value vector C is composed of the count values ​​of each counter segment in the i-th level, and the value vector C' is composed of the recovered values ​​of each counter segment in the i-th level, where... In this way, by determining the value vector X when the L2 norm of Y-AX is minimized, the overflow value of each counter segment in the i-th level can be accurately determined, and the recovered value of each counter segment in the i-th level after overflow value recovery can be determined based on the overflow value, thereby improving the accuracy of network measurement.

[0011] One possible implementation involves determining the number of layers *m* of multiple sketch counters and the number of segments in each layer. Based on *m* and the number of segments in each layer, the multiple sketch counters are divided into counter segments for each of the *m* layers. The number of segments in the *i*-th layer is greater than the number of segments in the (i+1)-th layer above the *i*-th layer. The first layer is the lowest layer of the *m* layers, and the number of counter segments in the first layer is equal to the number of multiple sketch counters. The *i*-th layer includes a hash function used to determine the identifier of at least one counter segment in the (i+1)-th layer to which the overflow value of each counter segment in the *i*-th layer carries over. Each counter segment includes an overflow flag indicating whether a numerical overflow has occurred. In this way, the multiple sketch counters are divided into counter segments across *m* layers, achieving sketch counter compression and reducing the memory resources required for the counters.

[0012] As one possible implementation, the number of layers *m* is determined based on the size of schedulable resources; the number of counters with different counting ranges is determined based on historical traffic characteristics; the counting range corresponding to each layer after dividing multiple sketch counters into *m* layers is determined; based on the *i*-th counting range of the *i*-th layer, the number of target counters corresponding to the counting ranges that intersect with the *i*-th counting range is determined as the number of basic fragments for the *i*-th layer. The number of fragments for the *1*-th layer is determined as the number of basic fragments for the *1*-th layer, and the number of fragments for the *1*-th layer is the number of multiple sketch counters. Based on the number of basic fragments for each layer in the *m* layers other than the *1*-th layer, the number of fragments for each layer in the other layers is determined; the number of fragments for each layer is greater than the number of basic fragments. Thus, by determining the number of counters for different counting ranges based on historical traffic characteristics, and determining the number of basic segments required for each level, the number of segments in the first level is equal to the number of multiple sketch counters. In the m levels, the number of segments in all levels except the first level is greater than the corresponding number of basic segments, so that the lower-level counter segments can share more upper-level counter segments. The resulting linear equation system can further improve the accuracy of determining the overflow value.

[0013] Optionally, the number of segments in each of the other levels is determined to be the sum of the base number of segments for each level and a preset number of segments. In this way, the number of segments in each level is increased by the preset number of segments compared to the base number, increasing the number of upper-level counter segments that lower-level counter segments can share, further improving the accuracy of network measurements while reducing the memory resources required for the counters.

[0014] Optionally, the carry rate of the i-th level is determined to be the ratio of the number of basic segments in the (i+1)-th level to the number of basic segments in the i-th level; the carry rate of the i-th level is determined to be the first product of a preset parameter value; the preset parameter value is greater than 1; the first product is determined to be the second product of the number of segments in the i-th level; the number of segments in the (i+1)-th level is determined to be an integer greater than or equal to the second product; where the value of i is from 1 to m-1. Thus, the ratio of the number of segments in the (i+1)-th level to the number of segments in the i-th level is greater than the first product, which is the product of the carry rate of the i-th level and the preset parameter value. This results in the number of segments in each level being greater than the number of basic segments, increasing the number of upper-level counter segments that lower-level counter segments can share, and further improving the accuracy of network measurements while reducing the memory resources required by the counter.

[0015] Optionally, the i-th level includes multiple hash functions, and each level in the multiple levels includes the same number of hash functions; the number of hash functions in each level is inversely proportional to the size of the preset parameter value. That is, the more hash functions there are, the closer the preset parameter value is to 1, thereby reducing the memory resources required by the counter while ensuring the accuracy of network measurements.

[0016] Optionally, the preset parameter value c k The functional relationship between the number of hash functions k in each level and the given information is as follows: in e is a natural constant.

[0017] Secondly, a network measurement method is provided, which is applied to a network measurement system including control plane nodes and data plane nodes that are communicatively connected. The method includes: the data plane nodes acquiring count values ​​of counter segments from multiple sketch counters used for statistical analysis of network flow. The multiple sketch counters include m levels, and each level includes one or more counter segments.

[0018] The data plane nodes, arranged in descending order of hierarchy, construct a system of linear equations based on the overflow count of the counter segment at level i and the recovered value of the counter segment at level (i+1). The data plane nodes solve the system of linear equations using the least-squares solution, which serves as the overflow value for each counter segment at level i. Based on the count value and overflow value of each counter segment, the recovered value for each counter segment at level i is obtained. The overflow count indicates the number of times the value of each counter segment at level i overflows into each counter segment at level (i+1). Level 1 is the lowest level, and level m is the highest level; the recovered value of the counter segment at the highest level is the count value.

[0019] The data plane node repeats the above steps in order of i, from m-1 to 1, until the recovered value of each counter segment in the lowest layer is obtained. The data plane node sends the recovered value of each counter segment in the lowest layer to the control plane node. The control plane node determines the measurement result of the network flow based on the recovered value of each counter segment in the lowest layer; the recovered value of each counter segment in the lowest layer is the count value of each sketch counter.

[0020] As one possible implementation, the data plane node obtains n based on the number of times the value of the q-th counter segment at the i-th level overflows to the p-th segment at the (i+1)-th level. i+1 line n i The matrix represents the element value in the p-th row and q-th column of the matrix; the matrix represents the overflow relationship between the counter segment of the i-th level and the counter segment of the (i+1)-th level. The data plane node obtains the j-th linear equation system based on the matrix and the value vector Y composed of the recovered values ​​of each counter segment of the (i+1)-th level; where the value of i ranges from m-1 to 1, and when i is 1, the i-th level is the lowest level; the sum of i and j is m; the vector expression of the j-th linear equation system is Y = AX.

[0021] Optionally, the data plane node determines the element value in the p-th row and q-th column of the matrix based on the overflow flag value of the q-th counter segment in the i-th level and the number of times the q-th counter segment is mapped to the p-th segment in the (i+1)-th level by the hash function of the i-th level.

[0022] As one possible implementation, the data plane node determines the j-th least squares solution of the j-th linear equation system; where the j-th least squares solution is the overflow value of each counter segment in the i-th level. Based on the overflow value and count value of each counter segment in the i-th level, the data plane node determines the recovered value of each counter segment in the i-th level. The data plane node repeats the above steps in the order of j from 1 to m-1 until the recovered value of each counter segment in the first level is obtained.

[0023] Optionally, the data plane node determines the value vector X when the L2 norm of Y-AX is minimized, where the value vector X is a value vector composed of the overflow values ​​of each counter segment at the i-th level. The data plane node determines this value vector X based on the value vector X at the i-th level and the number of bits w contained in each counter segment. i The value vector C is composed of the count values ​​of each counter segment in the i-th level, and the value vector C' is composed of the recovered values ​​of each counter segment in the i-th level, where...

[0024] As one possible implementation, the data plane node determines the number of layers *m* of multiple sketch counters and the number of segments in each layer. The control plane node divides the multiple sketch counters into counter segments for each of the *m* layers based on the number of layers *m* and the number of segments in each layer. The control plane node then sends the counter segments for each of the *m* layers to the data plane node.

[0025] In this system, the number of segments in the i-th level of the m levels is greater than the number of segments in the (i+1)-th level; the first level is the lowest level of the m levels, and the number of counter segments included in the first level is equal to the number of multiple sketch counters; the i-th level includes a hash function, which is used to determine the identifier of at least one counter segment in the (i+1)-th level to which the overflow value of each counter segment in the i-th level is carried over; each counter segment includes an overflow flag bit, which is used to indicate whether a numerical overflow has occurred in the counter segment.

[0026] Optionally, the control plane node determines the number of layers *m* based on the schedulable resources of the data plane node, determines the number of counters within a certain counting range based on historical traffic characteristics, and determines the counting range corresponding to each level after dividing the multiple sketch counters into *m* levels. Based on the *i*-th counting range of the *i*-th level, the number of target counters corresponding to the counting ranges that intersect with the *i*-th counting range is determined as the number of basic fragments for the *i*-th level. The control plane node determines the number of fragments for the *1*-th level as the number of basic fragments for the *1*-th level, and the number of fragments for the *1*-th level is the number of multiple sketch counters. Based on the number of basic fragments for each level in the *m* levels other than the *1*-th level, the control plane node determines the number of fragments for each level in the other levels; the number of fragments for each level is greater than the number of basic fragments.

[0027] Optionally, the control plane node determines the number of segments for each level in other levels as the sum of the basic number of segments for each level and the preset number of segments.

[0028] Optionally, the control plane node determines the carry rate of the i-th level as the ratio of the number of basic segments in the (i+1)-th level to the number of basic segments in the i-th level; determines the first product of the carry rate of the i-th level and a preset parameter value; the preset parameter value is greater than 1; the control plane node determines the second product of the first product and the number of segments in the i-th level; the control plane node determines the number of segments in the (i+1)-th level as an integer greater than or equal to the second product; where the value of i is from 1 to m-1.

[0029] Optionally, the i-th level includes multiple hash functions, and each level in the multiple levels includes the same number of hash functions; the number of hash functions included in each level is inversely proportional to the preset parameter value.

[0030] Optionally, the preset parameter value c k The functional relationship between the number of hash functions k in each level and the given information is as follows: in e is a natural constant.

[0031] The technical effects of the above-mentioned network measurement methods can be referred to the technical effects of the network measurement methods described in the first aspect, and will not be repeated here.

[0032] One possible implementation involves the control plane node sending counter segments for each level and manually compiled measurement code to the data plane node. Before acquiring the count values ​​of the counter segments used for statistical analysis of the network flow, the data plane node measures the network flow based on the measurement code. If the statistical value of the network flow is v, the e-th counter segment in the first level corresponding to the network flow is incremented by the count value v. If the e-th counter segment overflows, the data plane node sets the overflow flag of the e-th counter segment to 1 and carries over the overflow value of the e-th counter segment to the upper-level counter segment according to the hash function of each level, until the counter segment no longer overflows. In this way, the data plane node measures the network flow based on the layered counter segments. Since the counter segments of each level share the upper-level counter segments, the data plane node carries over the overflow value to the upper-level counter segments, thereby accurately recovering the count value of each counter based on the count values ​​of the counter segments of each level, improving the measurement accuracy of the network measurement.

[0033] As one possible implementation, the data plane nodes include direct measurement nodes and auxiliary measurement nodes. The control plane nodes send counter fragments and measurement codes of the first level to the direct measurement nodes, and send counter fragments of all levels except the first level, and measurement codes of each of the other m levels, to the auxiliary measurement nodes. In this way, the sketch counters are distributed across the direct and auxiliary measurement nodes, reducing the computational and storage load on individual nodes and improving the utilization efficiency of node resources in the network measurement system.

[0034] As one possible implementation, the direct measurement node measures the network flow based on the measurement code of the first level. When the statistical value of the network flow is v, it increments the count value v in the e-th counter segment of the corresponding first level. If the direct measurement node experiences a value overflow in the e-th counter segment of the first level, it sets the overflow flag bit of the e-th counter segment to 1, determines the target identifier of the overflow value of the e-th counter segment to be carried over to the counter segment of the second level according to the hash function of the first level, and sends the overflow value and target identifier to the auxiliary measurement node.

[0035] The auxiliary measurement node receives the overflow value and the target identifier; it adds the overflow value to the counter segment corresponding to the target identifier; if the counter segment corresponding to the target identifier overflows, it sets the overflow flag bit of the counter segment corresponding to the target identifier to 1; according to the hash function of the second level, it determines that the overflow value of the counter segment corresponding to the target identifier is carried over to the counter segment of the third level, and continues to carry over upwards in the event of a numerical overflow until the counter segment no longer overflows.

[0036] Based on the above implementation, the direct measurement node is responsible for measuring the first level of counter segments that are accessed more frequently, while the auxiliary measurement node is responsible for measuring the counter segments of other levels that are accessed less frequently. This utilizes the hot and cold characteristics of the layered sketch counters to reduce the communication overhead of different nodes in distributed deployment.

[0037] The data plane nodes also include counter recovery nodes. Direct measurement nodes send counter segments from the first level to the counter recovery node; auxiliary measurement nodes send counter segments from all levels except the first level out of the m levels to the counter recovery node; the counter recovery node obtains the count value of the counter segment from each of the m levels. In this way, the counter recovery node obtains the count value of the counter segment from each level and recovers the counter's count value, thus assigning the computational task to a single node and improving the accuracy of the determined counter count value.

[0038] Thirdly, a network measurement apparatus is provided, the apparatus comprising modules for performing the network measurement method of the first aspect or any possible implementation thereof.

[0039] The network measurement device described in the third aspect can be a terminal device or a network device, or it can be a chip (system), network card or other component or assembly that can be set in a terminal device or a network device, or it can be a device that includes a terminal device or a network device. This application does not limit it in this regard.

[0040] Furthermore, the technical effects of the network measurement device described in the third aspect can be referred to the technical effects of the network measurement method described in the first aspect, and will not be repeated here.

[0041] Fourthly, a network device cluster is provided, including at least one network device, each electronic device including a processor and a memory; the processor is coupled to the memory; the memory stores computer instructions, which are loaded and executed by the processor to enable the network device cluster to perform the operational steps of the method described in any possible implementation of the first or second aspect above.

[0042] Fifthly, a computer-readable storage medium is provided, comprising: computer software instructions; when the computer software instructions are executed in a network device cluster, causing the computer to perform operational steps of the method as described in any possible implementation of the first or second aspect.

[0043] Sixthly, embodiments of this application provide a chip system. The chip system includes a memory and at least one processor. The memory stores a set of computer instructions, which, when executed by the processor, perform the operational steps of the method described in any possible implementation of the first or second aspect.

[0044] In a seventh aspect, a computer program product is provided that, when run on a network device cluster, causes the network device cluster to perform the operational steps of the method as described in any possible implementation of the first or second aspect.

[0045] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of a counter compression method;

[0047] Figure 2 This is a schematic diagram illustrating the execution of an update operation where a fixed flag bit version exists in the counter;

[0048] Figure 3 This is a schematic diagram of a shared hash unit.

[0049] Figure 4 This is a schematic diagram illustrating pipeline sharing in a multi-sketch, multi-machine deployment.

[0050] Figure 5 This is a schematic diagram of the architecture of a network measurement system provided in an embodiment of this application;

[0051] Figure 6 This is a schematic diagram of the architecture of another network measurement system provided in an embodiment of this application;

[0052] Figure 7 A flowchart illustrating a network measurement method provided in an embodiment of this application;

[0053] Figure 8 A flowchart illustrating another network measurement method provided in an embodiment of this application;

[0054] Figure 9 A schematic diagram of a sketch counter with a hierarchical data structure provided in an embodiment of this application;

[0055] Figure 10 A schematic diagram illustrating the recovery of an overflow value of a counter, provided in an embodiment of this application;

[0056] Figure 11 A flowchart illustrating a network measurement method for centralized deployment of multiple sketch counters provided in this application embodiment;

[0057] Figure 12 An interactive schematic diagram of a network measurement system 600 in the measurement preparation stage provided in an embodiment of this application;

[0058] Figure 13 An interactive schematic diagram of a direct measurement node 621 and an auxiliary measurement node 622 during a measurement stage provided in an embodiment of this application;

[0059] Figure 14 An interactive schematic diagram of a network measurement system 600 in the post-measurement stage provided in an embodiment of this application;

[0060] Figure 15 A schematic diagram of a network measurement device provided in an embodiment of this application;

[0061] Figure 16 This application provides a schematic diagram of the structure of a network device according to an embodiment of the present application.

[0062] Figure 17 This is a schematic diagram of the structure of a network device cluster provided in an embodiment of this application;

[0063] Figure 18 This is a schematic diagram illustrating a network connection structure between network devices, as provided in an embodiment of this application. Detailed Implementation

[0064] The embodiments of this application can be applied to existing network measurement, data centers, switching equipment, programmable switching equipment, programmable switching chips, etc., and can also be applied to future network measurement, data centers, switching equipment, programmable switching equipment, programmable switching chips, etc., and the embodiments of this application do not limit them.

[0065] To facilitate understanding, the relevant terms involved in the embodiments of this application will be introduced below.

[0066] (1) Network Measurement

[0067] In data center networks, network measurement plays a crucial role in data acquisition and analysis.

[0068] Network measurement provides data support for security protection, resource scheduling, and business decisions by sensing the characteristics of large amounts of traffic in real time.

[0069] Typical applications of network measurement include: detecting distributed denial-of-service (DDoS) attacks through traffic mutations, implementing traffic engineering optimization based on link load data, and analyzing user behavior characteristics by statistically analyzing video service traffic.

[0070] (2) Control plane

[0071] In a network, a plane is an abstract concept representing the location where a specific process occurs.

[0072] The control plane is the part of the system responsible for management, coordination, and decision-making. It is used to formulate rules, strategies, and path selection, but does not directly handle actual data transmission.

[0073] The control plane is used to control how data packets are forwarded in a network.

[0074] (3) Data plane

[0075] The data plane is the part of the system responsible for the actual data transmission and forwarding, and it executes specific operations according to the rules issued by the control plane.

[0076] The data plane is used for the actual forwarding of data packets.

[0077] (4) Sketch algorithm

[0078] The sketch algorithm counts the frequency of a certain element in a network flow, reflecting the characteristics of the network flow. It does not actually store the elements in the network flow, but only their cardinality.

[0079] The basic principle is that each cell of the array maintains a counter. When an element in the data stream hashes to a certain position in the array, the counter at that position is incremented. To query the frequency of an element, simply hash to the corresponding counter. Since different elements may be indexed to the same position, the count value obtained by this method is generally too large.

[0080] (5) Network flow

[0081] A network flow is a collection of packets with corresponding header fields.

[0082] For example, a header field can be a 5-tuple.

[0083] The 5-tuple refers to the five fields in the message header: source Internet Protocol (IP) address, destination IP address, source port number, destination port number, and transport layer protocol number.

[0084] To facilitate understanding of the technical issues in the background section, the existing network measurement methods will be explained below.

[0085] In data center networks, network measurement systems often need to process high-speed processing streams under microsecond-level latency constraints. This places stringent requirements on the computational efficiency and resource overhead of network measurement algorithms, namely, ensuring computational efficiency with minimal resource overhead.

[0086] This paper provides a brief overview of network measurement algorithms using the existing sketch algorithm.

[0087] The sketch algorithm uses a sampling probabilistic data structure to compress and represent the entire traffic flow. It can approximate the calculation of traffic statistics in a sublinear memory space, offering advantages in efficiency and universality. Sublinear memory space typically describes the sublinear growth rate of memory required by an algorithm or system when processing data. The sketch algorithm maps the identifying features of network flows or packets within a network flow to an array of counters using a hash function, and estimates metrics such as network flow size and frequency of occurrence through a multi-hash stacking mechanism. Currently, mainstream variants of the sketch algorithm include Bloom filters for existence detection, count-min sketches supporting frequency statistics, and cardinality estimation methods such as HyperLogLog.

[0088] However, as data center networks evolve towards ultra-high speed and full-stack programmability, the sketch algorithm faces significant memory resource bottlenecks during deployment. Ultra-high speed refers to east-west traffic at the terabits per second (Tbps) level, which refers to the traffic exchanged between servers within the data center's memory, also known as lateral traffic. Full-stack programmability refers to switching devices where both the data plane and control plane can be flexibly programmed.

[0089] Because the programmable switching chips in data center switching equipment need to support multiple tasks simultaneously, such as network measurement, packet processing, and state storage, there is memory resource contention between different tasks; furthermore, different network measurement tasks also compete for memory resources. As the concurrent traffic that switching equipment needs to handle increases, the memory resources required for network measurement tasks also increase. Therefore, in data center switching equipment, the memory resources available for implementing high-precision sketch algorithms for network measurement tasks are insufficient, necessitating a reduction in measurement accuracy to maintain feasibility.

[0090] To address the aforementioned technical issues, existing switching devices employ a counter compression method based on the sketch algorithm. This method reduces the memory overhead required to deploy the sketch algorithm on a single switching device by compressing the sketch algorithm's counter. For example, in this method, the switching device divides a 32-bit counter into multiple 8-bit counter segments, and these segments are arranged in a hierarchical structure based on carry relationships. Higher-level counter segments store the overflow values ​​of lower-level counter segments, and more lower-level counter segments share fewer higher-level counter segments, thereby achieving counter compression.

[0091] Figure 1 This is a schematic diagram of a counter compression method, as shown below. Figure 1 As shown, the counter is divided into a two-layer data structure. The first layer includes five counter segments, and the second layer includes only three counter segments. A total of six network flows are measured by the counters in the two-layer data structure.

[0092] After performing network measurements using the compressed counters described above, the switching equipment needs to infer the value of each counter based on the count values ​​of the higher-layer and lower-layer counter segments. For example, if the overflow values ​​of multiple lower-layer counters in the switching equipment carry over to a higher-layer counter segment, the range of overflow values ​​for the lower-layer counter segment can be determined based on the fact that the overflow values ​​of multiple lower-layer counter segments are less than the count value of the higher-layer counter segment. The overflow values ​​of other lower-layer counter segments can also have their ranges determined using the aforementioned method, thus obtaining the range of overflow values ​​for a single counter segment. Alternatively, if the overflow value of a lower-layer counter carries over to multiple higher-layer counter segments, the switching equipment can take the minimum value among the count values ​​of the multiple higher-layer counter segments as the overflow value. Based on the inferred overflow value of each counter segment, the switching equipment determines the recovered count value of each counter segment and ultimately determines the value of each counter.

[0093] Therefore, since the overflow value of each counter segment is determined by estimation, the inaccuracy of the determined overflow value leads to inaccuracy of the final value of each counter, resulting in a large measurement error when performing network measurements based on the compressed counters.

[0094] Figure 2 This is a schematic diagram illustrating an update operation performed using a counter with a fixed flag bit version. For example... Figure 2 As shown, the counter counts elements with key value k. The counter consists of two parts: a flag bit and a count bit. The flag bit stores the multiplier, and the estimated value of the counter is obtained by calculating the value of the multiplier and the value of the count bit.

[0095] When a counter's flag is 0, counting simply involves incrementing the counter by 1. This continues until the counter reaches its upper limit, at which point the flag is incremented by 1 and the counter is set to 0. From then on, for each count, the counter is incremented by 1 with a probability equal to the reciprocal of m. If the counter reaches its upper limit again, the process is repeated: incrementing the flag by 1 and setting the counter to 0.

[0096] In the above Figure 2 In the counter shown, when calculating the counter's value, if the value of the flag bit is 0, then the value of the counter bit is the query value. If the flag bit is not 0, the query value is divided into two parts: one part is the query value of the counter bit, which, mathematically speaking, increases the counter bit by 1 for every m insertions, so it is equal to the expansion coefficient m multiplied by the value of the counter bit. The other part is the value represented by the flag bit.

[0097] Given that there are (nx) counter bits, according to the update steps described above, every time the counter bits increase by 2^(nx), the flag bit increases by 1, and the counter bits return to 0. Therefore, it takes 2^(nx) counts to increase the flag bit from 0 to 1. Once the flag bit is not 0, mathematically speaking, every m counts can increase the counter bit by 1. Since increasing the flag bit by 1 requires increasing the counter bit by 2^(nx), the required number of counts is m * 2^(nx). Therefore, if the value of the flag bit is y, its count value is 2^(nx) + m * 2^(nx) * (y - 1).

[0098] Thus, with the counter size remaining unchanged, the upper limit of the count is significantly increased, thereby reducing the memory resources required by the counter. However, switching devices using... Figure 2 When determining the counter value after network measurement, the count value of the counter bits with a flag greater than 1 can only be a multiple of m, which leads to the problem of sparse counting. Therefore, a large variance will be introduced when determining the counter value, resulting in a large error in network measurement.

[0099] Existing technologies also employ hardware resource optimization methods across multiple sketches. The basic idea is to improve resource utilization by sharing resources between different sketches. Specifically, this involves sharing hardware resources across tasks based on optimization rules, where the optimization rules are predefined heuristic rules.

[0100] Figure 3 This is a schematic diagram illustrating a shared hash unit. (Example) Figure 3As shown, there are three sketch data structures to be deployed: S1, S2, and S3. S1 requires three hash operations, represented by h11(A), h12(A), and h13(A). S2 requires two hash operations, represented by h21(A) and h22(A). S3 requires one hash operation, represented by h31(A). Therefore, the three sketches require six hash units, where A represents the network flow.

[0101] The results of the first two hash operations of S1 can be shared with S2 and S3. That is, the two hash operations of S2 can use the hash operation results of h11(A) and h12(A), and the hash operation result of S3 can use the hash operation result of h11(A). Thus, the three sketches need to use three hash units. Sharing hash units improves the utilization rate of hash units.

[0102] However, the above hash unit sharing method can only achieve resource sharing of hash units on a single switching device, and does not achieve memory optimization.

[0103] Figure 4 This is a schematic diagram illustrating pipeline sharing in a multi-sketch, multi-machine deployment. For example... Figure 4 As shown, measurement tasks a and b are divided into basic units composed of match action tables (MATs). Measurement task a includes three processing nodes: a1, a2, and a3, while measurement task b includes five processing nodes: b1, b2, b3, b4, and b5. Measurement tasks a and b are topologically sorted. The actual execution flow of the processing nodes in measurement task a is a1, a2, a3, and the actual execution flow of the processing nodes in measurement task b is b1, b2, b3, b4, b5. Since a1 and b1 can be placed on one pipeline of the switching equipment, a2 and b3 can be placed on another pipeline, and a3 and b4 can be placed on yet another pipeline, merging a1 and b1 yields c1, merging a2 and b3 yields c2, and merging a3 and b4 yields c3. Therefore, by merging processing nodes, the number of pipeline stages and communication overhead of the switching equipment can be reduced.

[0104] While the methods described above consider scenarios with multiple sketches and multiple machines, they can only be deployed on switching devices within a pipelined architecture, thus limiting their deployment scenarios. Furthermore, these pipeline sharing methods can only optimize the number of pipeline stages and communication overhead, and cannot achieve memory optimization.

[0105] To ensure the measurement accuracy of the sketch algorithm on memory-scarce switching devices, this application provides a network measurement method, particularly a method based on compressed counters. Multiple sketch counters using a hierarchical data structure for statistical analysis of network flows are compressed, reducing the memory resources required for the counters. By constructing a system of linear equations and solving a least-squares problem, approximate solutions to the overflow values ​​of each counter segment are calculated sequentially from high to low hierarchy. The recovered values ​​of each counter segment are then calculated based on the overflow values ​​until the recovered values ​​of each counter segment at the lowest level are obtained as the count value for each sketch counter. Thus, compared to existing schemes that estimate the numerical range of the overflow values ​​of counter segments, this application, by constructing a system of linear equations and solving a least-squares problem, obtains approximate solutions that are closer to the actual overflow values ​​of each counter segment, improving the accuracy of the determined overflow values. This ensures the accuracy of the count values ​​determined based on the compressed counters, thereby guaranteeing the measurement accuracy of network measurement tasks on memory-scarce switching devices.

[0106] The following is in conjunction with the appendix Figure 5 To be continued Figure 14 The implementation methods of the embodiments of this application will be described.

[0107] Figure 5 This is a schematic diagram of the architecture of a network measurement system provided in an embodiment of this application.

[0108] like Figure 5 As shown, the network measurement system 500 includes a control plane node 510 and a data plane node 520. The network measurement system 500 is divided into a control plane and a data plane. The control plane node 510 is located in the control plane. The control plane node 510 can be a single server, a server cluster consisting of multiple servers, a virtual machine after server virtualization, or software running on a server; this embodiment does not impose any limitations on this. The data plane node 520 is located in the data plane. The data plane node 520 can be a forwarding device or a device cluster containing forwarding devices. Forwarding devices can be, for example, switching devices, routers, etc. The data plane node 520 can also be software running on forwarding devices or device clusters for implementing network measurement; this embodiment does not impose any limitations on this.

[0109] During the measurement preparation phase, the control plane node 510 receives input historical traffic characteristics and the sketch algorithm to determine the number of layers m and the number of segments per layer for multiple counters. Based on the number of layers m and the number of segments per layer, the multiple counters are divided into m layers of counter segments per layer. Among them, the historical traffic characteristics are the frequency and other features of elements appearing in the network flow measured in the historical network.

[0110] The i-th level includes n i There are n counter segments. i Each counter segment in the counter segments has the same number of bits, i is greater than or equal to 1 and less than or equal to m.

[0111] As one possible implementation, the control plane node 510 determines the number of layers m of multiple counters in the sketch algorithm based on the schedulable resources of the switching device corresponding to the data plane node 520, and determines the number of fragments in each of the m layers based on historical traffic characteristics.

[0112] Optionally, when the data plane node 520 is a programmable switching device with a pipelined architecture, the control plane node 510 determines the number of pipelines contained in the programmable switching device as the number of layers m.

[0113] Optionally, the control plane node 510 determines the required number of layers m that is less than the schedulable resources based on the schedulable resources of the switching equipment corresponding to the data plane node 520 and the preset correspondence between the number of layers and the required resources.

[0114] Optionally, the control plane node 510 divides the multiple sketch counters into m levels and determines the counting range corresponding to each level. The number of bits contained in each level can be the same or different; this embodiment does not impose any restrictions. Based on the i-th counting range of the i-th level, the control plane node 510 determines the number of target counters corresponding to the counting ranges that intersect with the i-th counting range as the number of basic fragments for the i-th level.

[0115] For example, control plane node 510 divides the 12 sketch counters into two levels. Each counter segment in the first level includes 5 bits, and the counting range for the first level is 0 to 32. Each counter segment in the second level includes 3 bits, and the counting range for the second level is 33 to 256. Based on historical traffic characteristics, control plane node 510 determines that the required number of counters with a counting range of 0-16 is 3, the number of counters with a counting range of 0-64 is 4, and the number of counters with a counting range of 0-256 is 5.

[0116] Control plane node 510 determines that the number of counters intersecting with the counting range (0-32) of the first level is 12, and the number of counters intersecting with the counting range (33-256) of the second level is 9. Control plane node 510 determines that the number of segments in the first level is 12 and the number of segments in the second level is 9.

[0117] Control plane node 510 sends counter fragments and manually compiled measurement codes for each layer to data plane node 520.

[0118] In this system, the number of counter segments in layer i of layer m is greater than the number of counter segments in layer i+1 above layer i. Layer 1 is the lowest layer of layer m, and the number of segments in layer 1 is equal to the number of counters. Layer i includes a hash function, which is used to determine the identifier of at least one counter segment in layer i+1 to which the overflow value of each counter segment in layer i is carried over. Each counter segment includes an overflow flag bit, which is used to indicate whether a numerical overflow has occurred in the counter segment.

[0119] The identifier of the counter segment is a unique identifier for that counter segment. For example, the identifier of the counter segment can be the counter number, or it can be the position of the counter segment among multiple counter segments at the corresponding level, etc., and this application embodiment does not limit this.

[0120] The overflow flag can be 1 or 0. When the overflow flag is 1, it indicates that the counter segment has overflowed. When the overflow flag is 0, it indicates that the counter segment has not overflowed.

[0121] During the measurement phase, data plane node 520 measures the network flow based on the measurement code, obtaining the value of the counter segment for each layer after measurement.

[0122] In the post-measurement phase, data plane node 520, from layer m-1 to layer 1, constructs a system of linear equations for adjacent layers and solves a least-squares problem based on the number of times the value of each counter segment in the lower layer overflows into at least one counter segment in the upper layer, and the recovered overflow value of each counter segment in the upper layer. This yields the overflow value for each layer from layer m-1 to layer 1, and calculates the recovered overflow value of each counter segment based on the overflow value of each layer, until the recovered overflow value of each counter segment in layer 1 is obtained. The recovered overflow value of each counter segment in layer 1 is the value of each counter. Data plane node 520 then sends the value of each counter to control plane node 510.

[0123] The control plane node 510 receives the values ​​of each counter sent by the data plane node 520 and determines the measurement results of the network flow.

[0124] As one possible implementation, the control plane node 510 determines the measurement result of the measurement index of the element to be measured based on the value of each counter and the counter corresponding to the element to be measured in the network flow.

[0125] The above describes the system architecture of a network measurement system that centrally deploys the sketch algorithm. The following section will combine... Figure 6 The system architecture of a network measurement system with a distributed deployment of the sketch algorithm is described. Figure 6 This is a schematic diagram of the architecture of another network measurement system provided in an embodiment of this application.

[0126] like Figure 6 As shown, the network measurement system 600 includes a control plane node 610 and a data plane node 620. The data plane node includes a direct measurement node 621, an auxiliary measurement node 622, and a counter recovery node 623.

[0127] The direct measurement node 621 can be a high-load programmable forwarding device in the data plane, and the direct measurement node 621 deploys the lowest-level counter segment of the hierarchical data structure counter. For example, when the counter is divided into m layers, the direct measurement node 621 deploys the first-level counter segment of layer m.

[0128] The auxiliary measurement node 622 can be a low-load programmable forwarding device in the data plane. The auxiliary measurement node 622 deploys upper-layer counter segments of the hierarchical data structure counter, that is, counter segments of other layers in the hierarchical counter except for the lowest layer counter segment. For example, when the counter is divided into m layers, the auxiliary measurement node deploys counter segments of all layers of layer m except for layer 1.

[0129] The counter recovery node 623 can be any device in the data plane or a server with idle computing resources. For example, the counter recovery node 623 can be a server with idle central processing unit (CPU) computing resources.

[0130] During the measurement preparation phase, control plane node 610 sends a layer 1 counter segment and a layer 1 measurement code to direct measurement node 621. Control plane node 620 sends counter segments of all layers except layer 1 and measurement codes of each layer in layer m to auxiliary measurement node 622.

[0131] During the measurement phase, the direct measurement node 621 measures the network flow based on the Layer 1 measurement code, obtaining the value of each counter segment in Layer 1. If a counter segment x overflows, the overflow flag bit of the counter segment is set to 1. Based on the Layer 1 hash function, the overflow value of counter segment x is carried over to the target identifier of the Layer 2 counter segment. The direct measurement node 621 then sends the overflow value and the corresponding target identifier to the indirect measurement nodes.

[0132] The auxiliary measurement node 622 receives the overflow value and target identifier sent by the direct measurement node 621, and increments the overflow value in the counter segment corresponding to the target identifier. If a value overflow occurs in the counter segment corresponding to the target identifier, the auxiliary measurement node 622 sets the overflow flag bit of the counter segment to 1, determines the carry-over value of the counter segment corresponding to the target identifier to the counter segment at level 3 based on the hash function of level 2, and continues to carry-over upwards until the counter segment no longer overflows if a value overflow occurs in the counter segment at level 3.

[0133] In the post-measurement phase, the direct measurement node 621 sends the first-layer counter segment to the counter recovery node 623. The auxiliary measurement node 622 sends counter segments from all layers m except the first layer to the counter recovery node 623. The counter recovery node 623 determines the value of each counter based on the values ​​of the counter segments from each layer in the m layers. The counter recovery node 623 then sends the value of each counter to the control plane node 610.

[0134] The process by which counter recovery node 623 determines the value of each counter based on the value of the counter segment in each layer of m can be found in the above text. Figure 5 and the following text Figure 7 The relevant descriptions in the embodiments of this application will not be repeated here.

[0135] The above text Figure 6 In the described network measurement system 600, the control plane node 610 distributes the counters of the sketch algorithm across multiple measurement nodes of the data plane node 620. Frequently accessed lower-level counter segments are deployed on the high-load direct measurement nodes 621 through which the entire network flow passes, while less frequently accessed upper-level counter segments are deployed on the less-loaded auxiliary measurement nodes 622. The counter recovery node 623, with idle computing resources, recovers the value of each counter based on the values ​​of the counter segments. This fully utilizes the resources of various types of nodes in the network measurement system, reduces the computational and storage load of individual nodes, and minimizes communication overhead between different measurement nodes.

[0136] Next Figure 5 Taking any node in the network measurement system 500 as an example, combined with Figure 7 The network measurement method according to the embodiments of this application will be described. Figure 7 This is a flowchart illustrating a network measurement method provided in an embodiment of this application. (About...) Figure 7 The network measurement system 500 mentioned above can be referenced. Figure 5As described, the measurement node can be any network device in the network measurement system 500, or a cluster of network devices in the network measurement system 500, or any network device or software for network measurement running on a network device. The embodiments of this application will not be elaborated here.

[0137] like Figure 7 As shown, the network measurement method includes steps 701 to 703.

[0138] Step 701: The measurement node obtains the count values ​​of the counter segments of multiple sketch counters used for statistics.

[0139] The measurement node is deployed with multiple sketch counters, which are layered and compressed using a hierarchical data structure. These sketch counters comprise m levels, and each level includes one or more counter segments. After the multiple sketch counters perform statistical analysis on the network flow, the measurement node obtains the count value of the counter segment from each of the m levels.

[0140] In this system, the number of fragments in the i-th level of the m levels is greater than the number of fragments in the (i+1)-th level above the i-th level. The first level is the lowest level of the m levels, and the number of counter fragments in the first level is equal to the number of multiple sketch counters. The i-th level includes a hash function, which is used to determine the identifier of at least one counter fragment in the (i+1)-th level to which the overflow value of each counter fragment in the i-th level is carried over. Each counter fragment includes an overflow flag, which indicates whether a numerical overflow has occurred in the counter fragment.

[0141] For examples of multiple sketch counters using a hierarchical data structure, please refer to the above text. Figure 1 The relevant descriptions are not repeated here in the embodiments of this application.

[0142] As one possible implementation, before the measurement node obtains the count values ​​of the counter segments of multiple sketch counters used for network flow statistics, it also needs to determine the number of layers m of the multiple sketch counters and the number of segments in each layer, and divide the multiple sketch counters into counter segments of each of the m layers according to the number of layers m and the number of segments in each layer.

[0143] Optionally, the measurement node can either acquire a pre-saved number of layers m and the number of segments in each layer, or it can receive the input number of layers m and the number of segments in each layer. The input number of layers m and the number of segments in each layer can be determined manually.

[0144] Optionally, the measurement node can also determine the number of layers m based on the size of schedulable resources, and determine the number of segments at each layer based on historical traffic characteristics. For specific implementation details, please refer to the above text. Figure 5 The relevant description in the document.

[0145] Optionally, the measurement node can also use the number of segments determined based on historical traffic characteristics as the base number of segments for each level. The measurement node determines the number of segments in the first level as the base number of segments in the first level, and the number of segments in the first level is the number of multiple sketch counters; the measurement node determines the number of segments in each of the other levels in the m levels based on the base number of segments in each level except the first level; the number of segments in each level is greater than the base number of segments.

[0146] For example, if the number of basic segments in the first level is 5, the measurement node determines that the number of segments in the first level is 5. In other levels, if the number of basic segments in the second level is 2, the measurement node can determine that the number of segments in the second level is 3 or 4. That is, the number of segments in the second level is greater than the number of basic segments in the second level (2), and the number of segments in the first level is greater than the number of segments in the second level.

[0147] Optionally, the measurement node determines the number of segments for each level in other levels as the sum of the base number of segments for each level and a preset number of segments, where the preset number of segments for each level can be the same or different. For example, the preset number of segments for each level is 1.

[0148] Optionally, the measurement node determines the carry rate of the i-th level as the ratio of the number of basic segments in the (i+1)-th level to the number of basic segments in the i-th level. The measurement node determines the first product of the carry rate of the i-th level and a preset parameter value, determines the second product of the first product and the number of segments in the i-th level, and determines the number of segments in the (i+1)-th level as an integer greater than or equal to the second product.

[0149] The preset parameter value is greater than 1. For example, the preset parameter value can be 1.09, 1.089, 1.05, 1.03, 1.024, etc., and this application does not impose any restrictions on this. The i-th layer includes multiple hash functions. Each layer in the multiple layers includes the same number of hash functions. The number of hash functions included in each layer is inversely proportional to the preset parameter value, that is, the more hash functions there are, the closer the preset parameter value is to 1.

[0150] Among them, the preset parameter value c k The functional relationship between the number of hash functions k in each level and the given information is as follows: in e is a natural constant.

[0151] Here, the value of i ranges from 1 to m-1. That is, when i equals 1, the measurement node can obtain the number of segments in the second level based on the carry rate of the first level, the number of segments in the first level, and the preset parameter value. When i equals m-1, the measurement node can obtain the number of segments in the m-th level based on the carry rate of the (m-1)th level, the number of segments in the (m-1)th level, and the preset parameter value.

[0152] Step 702: The measurement nodes are arranged in descending order of hierarchy. Based on the overflow count of the lower-level counter segment and the recovery value of the upper-level counter segment in each of the multiple sketch counters, the first to the (m-1)th linear equation system is constructed sequentially. The first to the (m-1)th linear equation system is solved until the recovery value of each counter segment in the lowest level is obtained.

[0153] The process of determining the count value of each counter by the measurement node includes m-1 steps, where the m-1 steps include the j-th step, which includes the following:

[0154] The measurement node constructs the j-th linear equation system based on the overflow count of the counter segment in the i-th level and the recovery value of the counter segment in the (i+1)-th level, where the sum of i and j is m, and the first level when i is 1 is the lowest level.

[0155] The measurement node solves the j-th linear equation system to obtain the least-squares solution of the j-th linear equation system, and determines the recovered value corresponding to the least-squares solution of the j-th linear equation system. Here, the j-th linear equation system corresponds to the i-th level and the (i+1)-th level; the recovered value corresponding to the least-squares solution is determined by the measurement node based on the least-squares solution that serves as the overflow value of each counter segment in the i-th level.

[0156] Among them, the (j-1)th linear equation system corresponds to the (i+1)th level and the (i+2)th level. The recovered value corresponding to the least squares solution of the (j-1)th linear equation system is the recovered value of each counter segment in the (i+1)th level of the j-th linear equation system.

[0157] The (j+1)th linear equation system corresponds to the (i-1)th level and the ith level. The recovered value corresponding to the least squares solution of the jth linear equation system is the recovered value of each counter segment in the (j+1)th linear equation system that serves as the ith level above.

[0158] The measurement nodes, following the order of j from 1 to m-1, repeat the above steps until the recovered value of each counter segment in the lowest layer is obtained. The order of i is from m-1 to 1.

[0159] As one possible implementation, this application embodiment does not restrict the form of the constructed linear equation system; the linear equation system can be in vector form.

[0160] The measurement node calculates n based on the number of times the value of the q-th counter segment at the i-th level overflows to the p-th counter segment at the (i+1)-th level. i+1 line n i The value of the element in the p-th row and q-th column of the matrix, based on n i+1 line n i The value of each element in each row and column of the column forms n. i+1 line n i The matrix represents the overflow relationship between the counter segments at level i and level (i+1). The measurement node obtains the j-th linear equation system based on the matrix and the value vector Y composed of the recovered values ​​of each counter segment at level (i+1), where the vector expression of the j-th linear equation system is Y = AX.

[0161] The measurement node, based on the number of segments contained in the i-th level, determines the overflow count of the q-th counting segment from the first counting segment in the i-th level to the last counting segment in the i-th level, thus obtaining the element value in the q-th column of the first row. The measurement node, based on the number of segments contained in the (i+1)-th level, determines the overflow count of the p-th counting segment from the first counting segment in the i-th level to the last counting segment in the i-th level, thus obtaining the element value in the first column of the p-th row, thereby obtaining the matrix.

[0162] Optionally, the measuring node determines the element value in the p-th row and q-th column of the matrix based on the overflow flag value of the q-th counting segment in the i-th level and the number of times the q-th counter segment is mapped to the p-th counting segment in the (i+1)-th level by the hash function of the i-th level.

[0163] Specifically, the measurement node determines the element value of the overflow flag bit of the q-th counting segment at the i-th level as 0 if the flag value indicates that the q-th counter segment has not overflowed. If the flag value indicates that the q-th counter segment has overflowed, the element value of the p-th row and q-th column of the matrix is ​​determined by the number of times the q-th counter segment is mapped to the p-th counting segment at the (i+1)-th level by the hash function of the i-th level.

[0164] For example, the measurement node uses the overflow flag value r of the q-th counter segment at the i-th level. qThe number of times N is the q-th counter segment mapped by the hash function of the i-th level to the p-th segment of the (i+1)-th level. i (q), determine the value a of the element in the p-th row and q-th column of the matrix. pq a pq =r q *N i (q); where the flag value r is triggered when the value of the q-th counter segment overflows. q When the value of the q-th counter segment is 1, the flag value r is set to 1. q The value is 0.

[0165] As one possible implementation, the measurement node determines the j-th least squares solution of the j-th linear equation system, where the j-th least squares solution is the overflow value of each counter segment in the i-th level. Based on the overflow value and count value of each counter segment in the i-th level, the measurement node determines the recovered value of each counter segment in the i-th level. The measurement node repeats the above steps in the order of j from 1 to m-1 until the recovered value of each counter segment in the first level is obtained.

[0166] The measurement node solves the least squares problem for the j-th linear equation system to obtain the j-th least squares solution. The j-th least squares solution is the overflow value of each counter segment in the i-th and (i+1)-th levels corresponding to the j-th linear equation system, which serves as the lower level of the i-th level. For each counter segment in the i-th level, the measurement node calculates the overflow value of the counter segment and the number of bits w in the counter segment. i Determine the value of w in case 2. i The product of the power and the overflow value is the count value after the overflow value is restored to the counter segment. The measurement node determines the current count value of the counter segment, the sum of the count value after the overflow value is restored to the counter segment, and determines it as the restored value of the counter segment.

[0167] The measurement nodes determine the recovery values ​​of the counter segments from the (m-1)th level to the first level in sequence according to the value of j from 1 to m-1, until the recovery value of each counter segment in the first level is obtained.

[0168] Optionally, the measurement node determines the value vector X when the L2 norm of Y-AX is minimized, based on the value vector X of the i-th level and the bits w contained in each counter segment of the i-th level. i The value vector C is composed of the count values ​​of each counter segment in the i-th level, and the value vector C' is composed of the recovered values ​​of each counter segment in the i-th level, where...

[0169] Here, the value vector X is a value vector composed of the overflow values ​​of each counter segment in the i-th level. When solving the least squares problem, the measurement node can determine the value vector X when the L2 norm of Y-AX is minimized, based on the vector expression Y=AX of the j-th linear equation system. Therefore, using the value vector X as the approximate solution to the j-th linear equation system Y=AX yields the highest accuracy.

[0170] The measurement node is based on the value vector X and the bit w. i The value vector C, composed of the count values ​​of each counter segment in the i-th level, determines the value vector composed of the recovered values ​​of each counter segment in the i-th level.

[0171] Step 703: The measurement node determines the measurement result of the network flow based on the recovered value of each counter segment in the lowest layer.

[0172] As one possible implementation, the recovered value of each counter segment in the lowest layer is the count value of each sketch counter. The measurement node determines the measurement result of the network flow based on the count value of each sketch counter and the sketch counter of each metric in the network flow. Each metric can be the flow size, the frequency of occurrence of flow elements, the number of occurrences of flow elements, etc.

[0173] As one possible implementation, the measurement node obtains the measurement results of each measurement metric of the network flow by solving a least-squares problem through a system of linear equations based on the count value of each sketch counter and the multiple sketch counters to which each measurement metric is mapped by a hash function.

[0174] Based on the above Figure 7The description of steps 701 to 703 in the text describes how the measurement node compresses multiple sketch counters using a hierarchical data structure based on statistical analysis of network flows, reducing the memory resources required by the counters. By constructing a system of linear equations and solving a least-squares problem, the measurement node can sequentially calculate approximate solutions for the overflow values ​​of each counter segment in descending order of hierarchy, and calculate the recovered values ​​of each counter segment based on the overflow values, until the recovered values ​​of each counter segment at the lowest level are obtained as the count value of each sketch counter. Thus, compared to existing schemes that estimate the numerical range of the overflow values ​​of counter segments, the measurement node in this application, by constructing a system of linear equations and solving a least-squares problem, can obtain approximate solutions close to the actual overflow values ​​of each counter segment, improving the accuracy of the determined overflow values. This ensures the accuracy of the count values ​​determined based on the compressed counters, guaranteeing the measurement accuracy of network measurement tasks on memory-scarce switching devices.

[0175] The above describes the overall process of network measurement by measurement nodes. This application does not limit the entity executing the network measurement method. The following will... Figure 5 The following explanation uses the control plane node 510 and data plane node 520 in the network measurement system 500 as an example to illustrate network measurement. Figure 8 This is a flowchart illustrating another network measurement method provided in an embodiment of this application.

[0176] like Figure 8 As shown, the network measurement method includes steps 801 to 809.

[0177] Step 801: Control plane node 510 to determine the number of layers m of multiple sketch counters and the number of fragments in each layer.

[0178] The specific implementation of controlling the plane node 510 to determine the number of layers m and the number of segments at each layer can be found in the above text. Figure 7 The relevant description in step 701 is not limited in this embodiment.

[0179] Step 802: Control plane node 510 divides multiple sketch counters into counter segments for each of the m levels based on the number of layers m and the number of segments in each level.

[0180] Control plane node 510 divides multiple sketch counters into m levels. Each sketch counter contains the same number of bits, and each counter segment within each level contains the same number of bits. Control plane node 510 retains a corresponding number of counter segments within each level, based on the number of segments in each level.

[0181] Step 803: Control plane node 510 sends counter fragments for each of the m levels to data plane node 520.

[0182] Step 804: Data plane node 520 receives counter segments for each of the m levels sent by control plane node 510.

[0183] Step 805: Data plane node 520 obtains the count values ​​of counter segments of multiple sketch counters used for statistical network flow.

[0184] The specific implementation of step 805 is the same as described above. Figure 7 Step 701 is similar to that in the previous embodiment, and will not be described in detail here.

[0185] Step 806: Data plane node 520 constructs the first to the (m-1)th linear equation system in descending order of hierarchy, based on the overflow count of the lower-level counter segment and the recovery value of the upper-level counter segment in each of the multiple sketch counters; solves the first to the (m-1)th linear equation system until the recovery value of each counter segment in the lowest level is obtained.

[0186] The specific implementation of step 806 is the same as described above. Figure 7 Step 702 is similar to that in the previous embodiment, and will not be described in detail here.

[0187] Step 807: Data plane node 520 sends the recovery values ​​of each counter segment in the lowest layer to control plane node 510.

[0188] Step 808: Control plane node 510 receives the recovered values ​​of each counter segment in the lowest layer sent by data plane node 520.

[0189] Step 809: Control plane node 510 determines the measurement results of network flow based on the recovered values ​​of each counter segment in the lowest layer.

[0190] The specific implementation of step 809 is the same as described above. Figure 7 Step 703 is similar to that in the previous embodiment, and will not be described in detail here.

[0191] The above describes the overall process of network measurement performed by the control plane node 510 and data plane node 520 in the network measurement system 500. The following describes the process of network measurement performed by the control plane node 510 and data plane node 520 through a specific embodiment.

[0192] Control plane node 510 determines the number of layers m for the 6 sketch counters to be 3, and determines the carry rate p for the first layer based on historical traffic characteristics. l The carry rate for the second level is p2.

[0193] Control plane node 510 determines the number of segments n1 in the first level to be 6. Control plane node 510 then determines the number of segments in each level according to the required functional relationship n. i+1 / n i ≥1.09*p i We determine that n2 / n1 ≥ 1.09 * p1 and n3 / n2 ≥ 1.09 * p2. Control plane node 510, based on the number of segments n1 and the carry rate p1 of the first level, determines the number of segments n2 of the second level to be 3. Control plane node 510, based on the number of segments n2 and the carry rate p2 of the second level, determines the number of segments n3 of the third level to be 1.

[0194] Control plane node 510 divides the six sketch counters into three levels: the first level has 6 fragments, the second level has 3 fragments, and the third level has 1 fragment. Each level's counter fragment includes an additional single-bit overflow flag, indicating whether an overflow has occurred. The i-th level includes k corresponding hash functions h... i,1 h i,2 , ..., h i,k k hash functions are used to carry over the overflow value of the counter segment at level i to the k counter segments at level (i+1). Different hash functions may carry over the overflow value of the same counter segment to different counter segments, and different hash functions may also carry over the overflow value of the same counter segment to the same counter segment, that is, the same overflow value may be carried over to the same counter segment multiple times.

[0195] Figure 9 This is a schematic diagram of a sketch counter with a hierarchical data structure provided in an embodiment of this application. Figure 9As shown, the sketch counter is divided into 3 levels. The first level has 6 counter segments, the second level has 3 counter segments, and the third level has 1 counter segment. When the overflow value of the first counter segment from left to right in the first level is carried over to the first counter segment from left to right in the second level, the overflow value of the second counter segment in the first level is carried over to the second counter segment in the second level, the overflow value of the third counter segment in the first level is carried over to the first and second counter segments in the second level, the overflow value of the fourth counter segment in the first level is carried over to the third counter segment in the second level, the overflow value of the fifth counter segment in the first level is carried over to the second and third counter segments in the second level, the overflow value of the sixth counter segment in the first level is carried over to the third counter segment in the second level, and the overflow values ​​of the first to third counter segments in the second level are carried over to the first counter segment in the third level.

[0196] Control plane node 510 sends counter segments from each of the three levels to data plane node 520. Data plane node 520 measures the network flow. When a data packet from a network flow arrives at data plane node 520, assuming the number of bytes needed to be increased is v, and the counter number for that network flow is e, data plane node 520 first increments the e-th counter segment of the first level by the value v. If no overflow occurs, the process terminates; otherwise, the flag of the e-th counter segment is set to 1. Data plane node 520 then calculates the overflow value of the e-th counter segment and carries it over to the k counter segments of the second level, whose numbers are h respectively. 1,1 (e), h 1,2 (e), ..., h 1,k (e). Data plane node 520 adds the overflow value to the counter segment corresponding to the above number in the second level. If the counter segment corresponding to the above number in the second level still overflows, the count value of the counter segment in the second level is updated until no more overflows occur.

[0197] Data plane node 520 recovers overflow values ​​in descending order of hierarchy. It begins recovery from the second level, using the overflow values ​​of the three counter segments in the second level as the carry-over to the overflow count of the counter segments in the second level, and the count value of the third level. A first system of linear equations is then constructed, and the least-squares solution is obtained as the overflow value for each counter segment in the second level. Data plane node 520 determines the recovered value for each counter segment in the second level based on the overflow value and the count value of each counter segment.

[0198] Then, data plane node 520 performs overflow value recovery for the first level. Based on the overflow values ​​of the six counter segments in the first level, it carries over the overflow count of each counter segment in the second level and the recovered value of the second level. It then constructs a second system of linear equations and solves it using the least-squares solution as the overflow value for each counter segment in the first level. Data plane node 520 determines the recovered value for each counter segment in the first level based on the overflow value and the count value of each counter segment.

[0199] Taking the overflow count of the counter segment in the first level and the recovery value in the second level as an example, to construct the second system of linear equations, Figure 10 This is a schematic diagram illustrating the recovery of an overflow value from a counter, as provided in an embodiment of this application. Figure 10 As shown, the overflow flags of the second and sixth counter segments in the first level are 0, while the overflow flags of the first, third, fourth, and fifth counter segments are 1.

[0200] Data plane node 520 determines the second linear equation system as shown in Formula 1 based on the identifier value of the identifier bit of each counter segment in the first level, the number of overflows of each counter segment in the first level mapped to each counter segment in the second level by the hash function of the first level, and the count values ​​y1, y2 and y3 of each counter segment in the second level.

[0201] Formula 1: Where x1, x3, x4, and x5 represent the overflow values ​​of the first, third, fourth, and fifth counter segments of the first level, respectively. y1, y2, and y3 represent the recovered values ​​of the overflow values ​​of the first, second, and third counter segments of the second level, respectively.

[0202] The vector expression for the second system of linear equations is Y = AX.

[0203] Data plane node 520 solves the least squares solution of the second linear equation system, that is, data plane node 520 solves the Min... X ‖Y-AX‖2, yielding the value vector X when the L2 norm of Y-AX is minimized. The value vector X is composed of the overflow values ​​of each counter segment in the first level.

[0204] Data plane node 520 determines the recovery value of each counter segment in the first level based on the overflow value of each counter segment in the first level and the count value of each counter segment.

[0205] The above describes the network measurement process performed by the control plane node 510 and data plane node 520 in the network measurement system 500. This application embodiment does not limit the deployment method of multiple sketch counters; multiple sketch counters can be deployed centrally or in a distributed manner. For the network measurement method when multiple sketch counters are deployed centrally, please refer to the above description. Figures 7 to 10 The relevant description. Next, we will combine... Figure 11 This paper describes the network measurement method for a distributed deployment of multiple sketch counters. Figure 11 This is a flowchart illustrating a network measurement method for centralized deployment of multiple sketch counters, as provided in an embodiment of this application. (About...) Figure 11 The network measurement system 600 mentioned above can be referenced. Figure 6 The description of the embodiments in this application will not be repeated here.

[0206] like Figure 11 As shown, the network measurement method includes steps 1101 to 1118.

[0207] Step 1101: Control plane node 610 to determine the number of layers m of multiple sketch counters and the number of fragments in each layer.

[0208] The specific implementation of controlling the plane node 610 to determine the number of layers m and the number of segments at each layer can be found in the above text. Figure 7 The relevant description in step 701 is not limited in this embodiment.

[0209] Step 1102: Control plane node 610 divides multiple sketch counters into counter segments for each of the m levels based on the number of layers m and the number of segments in each level.

[0210] The specific implementation method of step 1102 is the same as above. Figure 8 The type of step 802 in this embodiment will not be described in detail here.

[0211] Step 1103: Control plane node 610 to receive manually compiled measurement code.

[0212] Step 1104: Control plane node 610 sends the first-level counter segment and the first-level measurement code to direct measurement node 621.

[0213] Among them, the direct measurement node 621 is a high-load programmable forwarding device in the data plane. The direct measurement node 621 is a topology key point through which all data passes directly. For example, the direct measurement node 621 is a top of rack (ToR) switching device or a gateway switching device.

[0214] The number of direct measurement nodes 621 can be one or more, and this embodiment does not limit this. When there are multiple direct measurement nodes 621, the control plane node 610 sends a partial counter fragment of the first level and a measurement code of the first level to each direct measurement node 621. Each direct measurement node 621 deploys the partial counter fragment of the first level to obtain all the counter fragments of the first level.

[0215] Step 1105: Direct measurement node 621 receives the first-level counter segment and the first-level measurement code sent by control plane node 610.

[0216] Step 1106: Control plane node 610 sends counter fragments of all levels except the first level and measurement codes of each level in the other m levels to auxiliary measurement node 622.

[0217] The auxiliary measurement node 622 is a low-load programmable forwarding device in the data plane. The number of auxiliary measurement nodes 622 can be one or more; this embodiment does not impose any limitation. When there are multiple auxiliary measurement nodes 622, the control plane node 610 sends partial counter fragments from other levels and measurement codes for each level to each auxiliary measurement node. The partial counter fragments from the same level deployed by each auxiliary measurement node 622 are combined to obtain all counter fragments for the same level.

[0218] Step 1107: Auxiliary measurement node 622 receives counter fragments of all levels except the first level and measurement codes of each level in the m levels sent by control plane node 610.

[0219] Step 1108: Direct measurement node 621 measures the network flow based on the measurement code of the first level, and determines the overflow value of each counter segment of the first level to the target identifier of the counter segment of the second level.

[0220] When the statistical value of the network flow is v, the direct measurement node 621 increments the count value v in the e-th counter segment of the first level corresponding to the network flow; when the e-th counter segment of the first level overflows, the overflow flag bit of the e-th counter segment is set to 1, and the overflow value of the e-th counter segment is carried over to the target flag of the counter segment of the second level according to the hash function of the first level.

[0221] In this process, direct measurement node 621 measures network flows. When a data packet of a network flow arrives at direct measurement node 621, assuming the number of bytes required to be added to the packet is v, and the counter identifier for this network flow is e, direct measurement node 621 first increments the value v of the e-th counter segment at the first level. If no overflow occurs, the process terminates; otherwise, the overflow flag bit of the e-th counter segment is set to 1. Direct measurement node 621 then calculates the carry-over of the overflow value of the e-th counter segment to the identifiers h of the k counter segments at the second level. 1,1 (e), h 1,2 (e), ..., h 1,k (e).

[0222] Step 1109: Direct measurement node 621 sends the overflow value and target identifier to auxiliary measurement node 622.

[0223] Step 1110: The auxiliary measurement node 622 receives the overflow value and target identifier sent by the direct measurement node 621.

[0224] Step 1111: Auxiliary measurement node 622 adds an overflow value to the counter segment corresponding to the target identifier, and continues to carry over until the counter segment no longer overflows when a numerical overflow occurs.

[0225] When the counter segment corresponding to the target identifier overflows, the auxiliary measurement node 622 sets the overflow flag bit of the counter segment corresponding to the target identifier to 1; according to the hash function of the second level, it determines that the overflow value of the counter segment corresponding to the target identifier is carried over to the counter segment of the third level, and continues to carry over upwards when a value overflow occurs until the counter segment no longer overflows.

[0226] The auxiliary measurement node 622 receives the overflow value of the e-th counter segment from the first level and the identifier of the counter segment whose overflow value is carried over from the second level. It increments the overflow value in the counter segment corresponding to the identifier. If no overflow occurs, the process terminates; otherwise, it sets the overflow identifier of the counter segment that overflowed to 1. The auxiliary measurement node 622 calculates the carry-over of the overflow value of the counter segment that overflowed to the identifier of the k counter segments in the third level based on the k hash functions of the second level. If a counter segment overflows in the third level, the carry-over continues up the hierarchy until no more counter segments overflow.

[0227] Step 1112: Direct measurement node 621 sends the first-level counter segment to counter recovery node 623.

[0228] Among them, counter recovery node 623 is a host server with idle CPU computing resources.

[0229] Step 1113: Auxiliary measurement node 622 sends counter segments of all levels except the first level out of m levels to counter recovery node 623.

[0230] Step 1114: Counter recovery node 623 obtains the count value of the counter segment of each level in m levels.

[0231] The specific implementation method of step 1114 is the same as above. Figure 7 Step 701 is similar to that in the previous embodiment, and will not be described in detail here.

[0232] Step 1115: Counter recovery node 623 determines the recovery value of each counter segment in the lowest layer.

[0233] The specific implementation method of step 1115 is the same as above. Figure 7 Step 702 is similar to that in the previous embodiment, and will not be described in detail here.

[0234] Step 1116: Counter recovery node 623 sends the recovery value of each counter segment in the lowest layer to control plane node 610.

[0235] Step 1117: Control plane node 610 receives the recovery values ​​of each counter segment in the lowest layer sent by counter recovery node 623.

[0236] Step 1118: Control plane node 610 determines the measurement results of network flow based on the recovered values ​​of each counter segment in the lowest layer.

[0237] The specific implementation method of step 1118 is the same as above. Figure 7 Step 703 is similar to that in the previous embodiment, and will not be described in detail here.

[0238] The above describes the overall process of network measurement in the network measurement system 600, including the control plane node 610, direct measurement node 621, auxiliary measurement node 622, and counter recovery node 623. The following section will combine... Figures 12 to 14 The process of network measurement of control plane node 610, direct measurement node 621, auxiliary measurement node 622 and counter recovery node 623 when multiple sketch counters are deployed in a distributed manner is described through a specific embodiment.

[0239] A complete measurement cycle for network measurement using the Network Measurement System 600 consists of three phases: the measurement preparation phase, the measurement phase, and the post-measurement phase.

[0240] Figure 12 This is an interactive schematic diagram of a network measurement system 600 in the measurement preparation stage provided in an embodiment of this application.

[0241] like Figure 12 As shown, the control plane node 610 receives multiple sketch counters and historical traffic features from the sketch algorithm. The control plane node 610 determines the number of layers m and the number of segments in each layer based on the historical traffic features, divides the multiple sketch counters into counter segments in each of the m layers, and receives manually compiled measurement code.

[0242] Based on the network topology characteristics of the network measurement system 600, the control plane node 610 selects a switching device as the direct measurement node 621 and the auxiliary measurement node 622. The control plane node 610 sends low-level counter fragments and low-level measurement codes to the direct measurement node 621, and sends high-level counter fragments and high-level measurement codes to the auxiliary measurement node 622.

[0243] Figure 13 This is a schematic diagram illustrating the interaction between a direct measurement node 621 and an auxiliary measurement node 622 during a measurement phase, as provided in an embodiment of this application.

[0244] like Figure 13 As shown, after the measurement begins, the network flow first passes through the direct measurement node 621. The direct measurement node 621 updates the count value of the underlying counter segment based on the received data packets. When a value overflow occurs in a certain underlying counter segment, the direct measurement node 621 determines the segment number of the upper-level counter segment to which the overflow value is carried over. The direct measurement node 621 saves the segment number and the overflow value as a tuple in a cache. When the tuple information saved in the cache reaches a preset size, it is packaged and sent to the auxiliary measurement node 622.

[0245] The auxiliary measurement node 622 receives the binary information sent by the direct measurement node 621 and updates the count value of the upper-layer counter segment according to the segment number and overflow value in the binary information.

[0246] Figure 14 This is an interactive schematic diagram of a network measurement system 600 in the post-measurement stage provided in an embodiment of this application.

[0247] like Figure 14 As shown, the direct measurement node 621 sends the underlying counter fragment to the counter recovery node 623. The auxiliary measurement node 622 sends the upper-level counter recovery node to the counter recovery node 623. The counter recovery node 623 performs a counter recovery algorithm to obtain the count value of each sketch counter and sends the count value of each sketch counter to the control plane node 610.

[0248] The control plane node 610 obtains the measurement results of the network flow within the statistical period based on the count value of each sketch counter, and calculates the characteristics of the current flow.

[0249] Based on the above Figures 11 to 14 The relevant description states that when multiple sketch counters are deployed in a distributed manner, frequently accessed lower-level counter segments are placed on direct measurement nodes through which the full data stream can pass, while less frequently accessed upper-level counter segments are placed on lighter-loaded auxiliary measurement nodes. Host servers with sufficient CPU resources act as counter recovery nodes, responsible for the computationally expensive counter recovery algorithms. The central server is responsible for control plane management and data analysis. By rationally allocating the various parts of the data structure, the resources of various nodes in the network are fully utilized, further reducing the computational and storage load of individual devices. At the same time, by utilizing the hot and cold characteristics of each part of the data structure, the communication overhead between different nodes is reduced, thereby improving the scalability of the entire system.

[0250] The network measurement method provided in the embodiments of this application has been described in detail above. The following will combine... Figure 15 This describes the network measurement device 1500 provided in this embodiment. The network measurement device 1500 can be used to implement the above-described... Figures 7-14 The network measurement method described herein includes the functions of the measurement node, network measurement system 500, or network measurement system 600.

[0251] Figure 15 This is a schematic diagram of a network measurement device provided in an embodiment of this application, such as... Figure 15 As shown, the network measurement device 1500 includes an acquisition module 1501, a processing module 1502, a count value recovery module 1503, and a determination module 1504.

[0252] Acquisition module 1501 is used to acquire the count values ​​of counter segments from multiple sketch counters used for network flow statistics; the multiple sketch counters include m levels, and each level includes one or more counter segments. For example, acquisition module 1501 is used to perform... Figure 7 Step 701 in the process.

[0253] Processing module 1502 is used to construct linear equations from the first to the (m-1)th level in descending order of hierarchy, based on the overflow counts of the lower-level counter segments and the recovered values ​​of the upper-level counter segments in every two adjacent levels of the sketch counter. The overflow count indicates the number of times the value of each lower-level counter segment overflows into each upper-level counter segment in every two adjacent levels. The recovered value is a count value recovered based on the count value of the upper-level counter segment and the overflow value. The recovered value of the highest-level counter segment is the count value. For example, processing module 1502 is used to execute... Figure 7 Step 702 in the process.

[0254] The count value recovery module 1503 is used to solve the first to m-1 linear equation systems sequentially until the recovered value of each counter segment in the lowest level is obtained. The recovered value of the upper-level counter segment in the j-th linear equation system is equal to the recovered value corresponding to the least-squares solution of the (j-1)-th linear equation system. The recovered value corresponding to the least-squares solution is determined based on the least-squares solution used as the overflow value of the lower-level counter segment in the (j-1)-th linear equation system. For example, the count value recovery module 1503 is used to execute... Figure 7 Step 702 in the process.

[0255] The determination module 1504 is used to determine the measurement results of the network flow based on the recovered values ​​of each counter segment in the lowest layer, where the recovered value of each counter segment in the lowest layer is the count value of each sketch counter. For example, the determination module 1504 is used to perform... Figure 7 Step 703 in the process.

[0256] As one possible implementation, processing module 1502 is specifically used to obtain n based on the number of times the value of the q-th counter segment of the i-th level overflows to the p-th counter segment of the (i+1)-th level. i+1 line n i The element value in the p-th row and q-th column of the matrix; the matrix represents the overflow relationship between the counter segment of the i-th level and the counter segment of the (i+1)-th level; n i+1 n represents the number of counter segments contained in the (i+1)th level. i This represents the number of counter segments contained in the i-th level; based on the matrix and the value vector Y composed of the recovered values ​​of each counter segment in the (i+1)-th level, the j-th linear equation system is obtained; where the value of i is in the order from m-1 to 1, when i is 1, the i-th level is the lowest level, and the sum of i and j is m; the vector expression of the j-th linear equation system is Y = AX.

[0257] As one possible implementation, the processing module 1502 is specifically used to determine the element value in the p-th row and q-th column of the matrix based on the overflow flag value of the q-th counter segment of the i-th level and the number of times the q-th counter segment is mapped to the p-th counter segment of the (i+1)-th level by the hash function of the i-th level.

[0258] As one possible implementation, the count value recovery module 1503 is specifically used to determine the j-th least squares solution of the j-th linear equation system; where the j-th least squares solution is the overflow value of each counter segment in the i-th level; based on the overflow value of each counter segment in the i-th level and the count value of each counter segment in the i-th level, the recovered value of each counter segment in the i-th level is determined; the above steps are repeated in the order of j from 1 to m-1 until the recovered value of each counter segment in the first level is obtained.

[0259] As one possible implementation, the count value recovery module 1503 is specifically used to determine the value vector X when the L2 norm of Y-AX is minimized, where the value vector X is a value vector composed of the overflow values ​​of each counter segment at the i-th level; based on the value vector X at the i-th level and the bits w contained in each counter segment at the i-th level... i The value vector C is composed of the count values ​​of each counter segment in the i-th level, and the value vector C' is composed of the recovered values ​​of each counter segment in the i-th level, where...

[0260] As one possible implementation, the determining module 1504 is further configured to, before the obtaining module 1501 obtains the count values ​​of the counter segments of the multiple sketch counters used for network flow statistics, determine the number of layers m of the multiple sketch counters and the number of segments in each layer; based on the number of layers m and the number of segments in each layer, divide the multiple sketch counters into counter segments in each of the m layers; wherein, the number of segments in the i-th layer of the m layers is greater than the number of segments in the (i+1)-th layer above the i-th layer; the first layer is the lowest layer of the m layers, and the number of counter segments included in the first layer is equal to the number of multiple sketch counters; the i-th layer includes a hash function, which is used to determine the identifier of at least one counter segment in the (i+1)-th layer to which the overflow value of each counter segment in the i-th layer is carried over; each counter segment includes an overflow flag bit, which is used to indicate whether a numerical overflow has occurred in the counter segment.

[0261] As one possible implementation, module 1504 is specifically used to determine the number of layers m based on the size of schedulable resources; determine the number of counters with different counting ranges based on historical traffic characteristics; determine the counting range corresponding to each layer after dividing multiple sketch counters into m layers; based on the i-th counting range of the i-th layer, determine the number of target counters corresponding to the counting ranges that intersect with the i-th counting range as the number of basic fragments of the i-th layer; determine the number of fragments of the 1st layer as the number of basic fragments of the 1st layer, and the number of fragments of the 1st layer is the number of multiple sketch counters; based on the number of basic fragments of each layer in the m layers other than the 1st layer, determine the number of fragments in each of the other layers; the number of fragments in each layer is greater than the number of basic fragments.

[0262] As one possible implementation, module 1504 is specifically used to determine the number of segments in each of the other levels as the sum of the basic number of segments in each level and the preset number of segments.

[0263] As one possible implementation, module 1504 is specifically used to determine the carry rate of the i-th level as the ratio of the number of basic segments of the (i+1)-th level to the number of basic segments of the i-th level; determine the first product of the carry rate of the i-th level and a preset parameter value; the preset parameter value is greater than 1; determine the second product of the first product and the number of segments of the i-th level; determine the number of segments of the (i+1)-th level as an integer greater than or equal to the second product; where the value of i is from 1 to m-1.

[0264] As one possible implementation, the i-th level includes multiple hash functions, and each level in the multiple levels includes the same number of hash functions; the number of hash functions included in each level is inversely proportional to the size of the preset parameter value.

[0265] As one possible implementation, the preset parameter value c k The functional relationship between the number of hash functions k in each level and the given information is as follows: in e is a natural constant.

[0266] Network measurement device 1510 is used as a data plane node in the network measurement system, and network measurement device 1520 is used as a control plane node in the network measurement system. Network measurement device 1510 and network measurement device 1520 are communicatively connected, and either network measurement device 1510 or network measurement device 1520 is used to implement the functions of network measurement device 1500.

[0267] As one possible implementation, network measurement device 1511 is applied to a direct measurement node in the data plane, and network measurement device 1512 is applied to an auxiliary measurement node in the data plane. Network measurement device 1520 sends a counter fragment of the first level and a measurement code of the first level to network measurement device 1511, and sends counter fragments of all levels except the first level and measurement codes of each level in the m levels to network measurement device 1512.

[0268] As one possible implementation, the network measurement device 1511 measures the network flow based on the measurement code of the first layer. When the statistical value of the network flow is v, the count value v is incremented on the e-th counter segment in the first layer corresponding to the network flow. If the e-th counter segment overflows, the overflow flag bit of the e-th counter segment is set to 1, and the target identifier of the overflow value of the e-th counter segment is determined according to the hash function of the first layer. The overflow value and the target identifier are then sent to the network measurement device 1512.

[0269] The network measurement device 1512 receives the overflow value and the target identifier; it adds the overflow value to the counter segment corresponding to the target identifier; if the counter segment corresponding to the target identifier overflows, it sets the overflow flag bit of the counter segment corresponding to the target identifier to 1; according to the hash function of the second level, it determines that the overflow value of the counter segment corresponding to the target identifier is carried over to the counter segment of the third level, and continues to carry over upwards if a numerical overflow occurs until the counter segment no longer overflows.

[0270] Network measurement device 1513 is applied to the counter recovery node in the data plane node. Network measurement device 1511 sends the counter segment of the first level to network measurement device 1513; network measurement device 1512 sends the counter segments of the other j levels among the m levels excluding the first level to network measurement device 1513; network measurement device 1513 obtains the count value of the counter segment of each of the m levels.

[0271] The acquisition module 1501, processing module 1502, count value recovery module 1503, and determination module 1504 can all be implemented in software or in hardware. For example, the implementation of the acquisition module 1501 will be described below. Similarly, the implementation of the processing module 1502, count value recovery module 1503, and determination module 1504 can refer to the implementation of the acquisition module 1501.

[0272] As an example of a software functional unit, module 1501 may include code running on a computing instance. The computing instance may include at least one of a physical host (network device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, module 1501 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0273] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0274] As an example of a hardware functional unit, the acquisition module 1501 may include at least one network device, such as a server. Alternatively, the acquisition module 1501 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0275] The multiple network devices included in the acquisition module 1501 can be distributed in the same region or in different regions. Similarly, the multiple network devices included in the acquisition module 1501 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple network devices included in the acquisition module 1501 can be distributed in the same VPC or in multiple VPCs. These multiple network devices can be any combination of network devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0276] It should be noted that, in other embodiments, any one of the acquisition module 1501, processing module 1502, count value recovery module 1503, and determination module 1504 can be used to execute any step in the network measurement method. The steps implemented by the acquisition module 1501, processing module 1502, count value recovery module 1503, and determination module 1504 can be specified as needed. By implementing different steps in the network measurement method through the acquisition module 1501, processing module 1502, count value recovery module 1503, and determination module 1504, all functions of the network measurement device 1500 can be realized.

[0277] This application also provides a network device 1600. Figure 16 This is a schematic diagram of the structure of a network device provided in an embodiment of this application, such as... Figure 16 As shown, network device 1600 includes: bus 1602, processor 1604, memory 1606, and communication interface 1608. Processor 1604, memory 1606, and communication interface 1608 communicate with each other via bus 1602. Network device 1600 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in network device 1600.

[0278] The 1602 bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 16 The bus 1602 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 1602 may include a path for transmitting information between various components of the network device 1600 (e.g., memory 1606, processor 1604, communication interface 1608).

[0279] Processor 1604 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0280] The memory 1606 may include volatile memory, such as random access memory (RAM). The processor 1604 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0281] The memory 1606 stores executable program code, which the processor 1604 executes to implement the functions of the aforementioned acquisition module 1501, processing module 1502, count value recovery module 1503, and determination module 1504, thereby realizing the network measurement method. In other words, the memory 1606 stores instructions for executing the network measurement method.

[0282] The communication interface 1608 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the network device 1600 and other devices or communication networks.

[0283] This application also provides a network device cluster. The network device cluster includes at least one network device. The network device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the network device can also be a terminal device such as a desktop computer, laptop computer, or smartphone. The network device can also be a switching device, such as a gateway switch, a programmable switch, or a pipelined switch.

[0284] Figure 17 This is a schematic diagram of the structure of a network device cluster provided in an embodiment of this application, such as... Figure 17 As shown, the network device cluster includes at least one network device 1600. The memory 1606 of one or more network devices 1600 in the network device cluster may store the same instructions for performing network measurement methods.

[0285] In some possible implementations, the memory 1606 of one or more network devices 1600 in the network device cluster may also store partial instructions for executing network measurement methods. In other words, a combination of one or more network devices 1600 can jointly execute instructions for performing network measurement methods.

[0286] It should be noted that the memory 1606 in different network devices 1600 within the network device cluster can store different instructions, which are used to execute some functions of the network measurement device 1500. That is, the instructions stored in the memory 1606 of different network devices 1600 can implement the functions of one or more modules among the acquisition module 1501, processing module 1502, count value recovery module 1503, and determination module 1504.

[0287] In some possible implementations, one or more network devices in a network device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 18 One possible implementation method is shown. Figure 18 This application provides a schematic diagram of a network device connection structure, as shown in the embodiment of the present application. Figure 18 As shown, two network devices 1600A and 1600B are connected via a network. Specifically, they are connected to the network through the communication interfaces in each network device. In this possible implementation, the memory 1606 in network device 1600A stores instructions for executing the functions of the acquisition module 1501. Simultaneously, the memory 1606 in network device 1600B stores instructions for executing the functions of the processing module 1502, the count value recovery module 1503, and the determination module 1504.

[0288] Figure 18 The connection method between the network device clusters shown can be such that, considering the network measurement method provided in this application requires data calculation, the functions implemented by the processing module 1502, the count value recovery module 1503, and the determination module 1504 are delegated to the network device 1600B for execution.

[0289] It should be understood that Figure 18 The functions of network device 1600A shown can also be performed by multiple network devices 1600. Similarly, the functions of network device 1600B can also be performed by multiple network devices 1600.

[0290] This application also provides another network device cluster. The connection relationships between the network devices in this network device cluster can be similarly referred to... Figure 17 and Figure 18The connection method of the network device cluster is different in that the memory 1606 of one or more network devices 1600 in the network device cluster can store the same instructions for executing network measurement methods.

[0291] In some possible implementations, the memory 1606 of one or more network devices 1600 in the network device cluster may also store partial instructions for executing network measurement methods. In other words, a combination of one or more network devices 1600 can jointly execute instructions for performing network measurement methods.

[0292] It should be noted that the memory 1606 in different network devices 1600 within the network device cluster can store different instructions for executing some functions of the network measurement system. That is, the instructions stored in the memory 1606 of different network devices 1600 can implement the functions of one or more nodes in the network measurement system.

[0293] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions capable of running on a network device cluster or stored on any available medium. When the computer program product is run on at least one network device, it causes the at least one network device to perform a network measurement method.

[0294] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a network device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the network device to perform a network measurement method.

[0295] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A network measurement method, characterized in that, The method includes: Obtain the count values ​​of counter segments from multiple sketch counters used for network flow statistics; the multiple sketch counters include m levels, and each level includes one or more counter segments; Following the order of hierarchy from high to low, a system of linear equations from the first to the (m-1)th is constructed sequentially based on the overflow count of the lower-level counter segment and the recovery value of the upper-level counter segment in each of the multiple sketch counters; wherein, the overflow count indicates the number of times the value of each lower-level counter segment overflows to each upper-level counter segment in each of the two adjacent levels, and the recovery value is the count value recovered based on the count value of the upper-level counter segment and the overflow value; Solve the first to m-1 linear equation systems in the order described above until the recovered value of each counter segment in the lowest level is obtained; wherein, the recovered value of the upper counter segment in the j-th linear equation system is equal to the recovered value corresponding to the least squares solution of the (j-1)-th linear equation system, and the recovered value corresponding to the least squares solution is determined based on the least squares solution that serves as the overflow value of the lower counter segment in the (j-1)-th linear equation system; The measurement result of the network flow is determined based on the recovered value of each counter segment in the lowest layer, where the recovered value of each counter segment in the lowest layer is the count value of each sketch counter.

2. The method according to claim 1, characterized in that, The system of linear equations, from highest to lowest level, is constructed sequentially based on the overflow count of the lower-level counter segment and the recovery value of the upper-level counter segment in every two adjacent levels of the multiple sketch counters. This includes: Based on the number of times the value of the q-th counter segment in the i-th level overflows to the p-th counter segment in the (i+1)-th level, we obtain n. i+1 line n i The element value in the p-th row and q-th column of the matrix; the matrix represents the overflow relationship between the counter segment of the i-th level and the counter segment of the (i+1)-th level; n i+1 n represents the number of counter segments contained in the (i+1)th level. i This represents the number of counter segments contained in the i-th level; Based on the matrix and the value vector Y composed of the recovered values ​​of each counter segment of the (i+1)th level, the j-th linear equation system is obtained; where the value of i is in the order from m-1 to 1, when i is 1, the i-th level is the lowest level, and the sum of i and j is m; the vector expression of the j-th linear equation system is Y = AX.

3. The method according to claim 2, characterized in that, The sequential solution of the first to m-1 linear equation systems, until the recovered values ​​of each counter segment in the lowest layer are obtained, includes: Determine the j-th least squares solution of the j-th linear equation system; wherein the j-th least squares solution is the overflow value of each counter segment of the i-th level; The value vector X, composed of the overflow values ​​of each counter segment in the i-th level, and the bits w contained in each counter segment in the i-th level. i The value vector C is composed of the count values ​​of each counter segment in the i-th level, and the value vector C' is composed of the recovered values ​​of each counter segment in the i-th level, where... Repeat the above steps in the order of j from 1 to m-1 until the recovered value of each counter segment in the first level is obtained.

4. The method according to any one of claims 1-3, characterized in that, Before obtaining the count values ​​of the counter segments used for statistical network flow calculations, the method further includes: Determine the number of layers m of the plurality of sketch counters and the number of fragments at each layer; Based on the number of layers m and the number of segments in each layer, the plurality of sketch counters are divided into counter segments for each of the m layers; Wherein, the number of fragments in the i-th level of the m levels is greater than the number of fragments in the (i+1)-th level above the i-th level; the first level is the lowest level of the m levels, and the number of counter fragments included in the first level is equal to the number of the plurality of sketch counters; the i-th level includes a hash function, which is used to determine the identifier of at least one counter fragment in the (i+1)-th level to which the overflow value of each counter fragment in the i-th level is carried over; each counter fragment includes an overflow flag bit, which is used to indicate whether the counter fragment has experienced a numerical overflow.

5. The method according to claim 4, characterized in that, Determining the number of layers m of the plurality of sketch counters and the number of fragments at each layer includes: The number of layers m is determined based on the size of the schedulable resources; The number of basic segments for each level is determined based on historical traffic characteristics; The number of fragments in the first level is determined as the base number of fragments in the first level, and the number of fragments in the first level is the number of the plurality of sketch counters; The number of segments in each of the other m levels is determined based on the number of basic segments in each level other than the first level; the number of segments in each level is greater than the number of basic segments.

6. The method according to claim 5, characterized in that, The step of determining the number of segments in each of the other levels based on the number of basic segments in each level other than the first level among the m levels includes: The number of segments in each of the other levels is determined to be the sum of the basic number of segments in each level and the preset number of segments.

7. The method according to claim 5, characterized in that, The step of determining the number of segments in each of the other levels based on the number of basic segments in each level other than the first level among the m levels includes: The carry rate of the i-th level is determined to be the ratio of the number of basic fragments in the (i+1)-th level to the number of basic fragments in the i-th level. Determine the first product of the carry rate of the i-th level and a preset parameter value; the preset parameter value is greater than 1; Determine the first product and the second product of the number of segments at the i-th level. The number of segments in the (i+1)th level is determined to be an integer greater than or equal to the second product; where the value of i is from 1 to m-1.

8. The method according to claim 7, characterized in that, The i-th level includes multiple hash functions, and each level includes the same number of hash functions; the number of hash functions included in each level is inversely proportional to the size of the preset parameter value.

9. The method according to claim 8, characterized in that, Preset parameter value c k The functional relationship between the number of hash functions k included in each level and the given number of hash functions is as follows: in e is a natural constant.

10. The method according to any one of claims 1-9, characterized in that, The method is applied to a network measurement system, wherein the network measurement system includes a control plane node and a data plane node that are communicatively connected; the data plane node is used to execute the method of any one of claims 1-3, and the control plane node is used to execute the method of any one of claims 4-9.

11. The method according to claim 10, characterized in that, The data plane nodes include direct measurement nodes and auxiliary measurement nodes; after dividing the plurality of sketch counters into counter fragments for each of the m levels according to the number of layers m and the number of fragments in each level, the method further includes: The control plane node sends the counter fragment of the first level and the measurement code of the first level to the direct measurement node, and sends the counter fragments of the other levels of the m levels excluding the first level and the measurement code of each of the other levels to the auxiliary measurement node.

12. The method according to claim 11, characterized in that, Before obtaining the count values ​​of the counter segments used for statistical analysis of network flows, the method further includes: The direct measurement node measures the network flow based on the measurement code of the first level. When the statistical value of the network flow is v, the count value v is incremented by the e-th counter segment in the first level corresponding to the network flow. When the e-th counter segment in the first level overflows, the direct measurement node sets the overflow flag bit of the e-th counter segment to 1, and determines the target identifier of the overflow value of the e-th counter segment to be carried over to the counter segment in the second level according to the hash function of the first level. The direct measurement node sends the overflow value and the target identifier to the auxiliary measurement node; The auxiliary measurement node receives the overflow value and the target identifier; The auxiliary measurement node adds the overflow value to the counter segment corresponding to the target identifier. If the counter segment corresponding to the target identifier overflows, the overflow flag bit of the counter segment corresponding to the target identifier is set to 1. According to the hash function of the second level, the overflow value of the counter segment corresponding to the target identifier is carried over to the counter segment of the third level, and the carry-over continues until the counter segment no longer overflows.

13. The method according to claim 11 or 12, characterized in that, The data plane node further includes a counter recovery node; the step of obtaining the count value of a counter segment for multiple sketch counters used to count network flows includes: The direct measurement node sends the first-level counter segment to the counter recovery node; The auxiliary measurement node sends counter fragments of the other j levels among the m levels, excluding the first level; The counter recovery node obtains the count value of the counter segment of each of the m levels.

14. A network measurement device, characterized in that, The device includes: The acquisition module is used to acquire the count values ​​of counter segments of multiple sketch counters used for network flow statistics; the multiple sketch counters include m levels, and each level includes one or more counter segments; The processing module is used to construct a system of linear equations from the first to the (m-1)th, in descending order of hierarchy, based on the overflow counts of the lower-level counter segments and the recovered values ​​of the upper-level counter segments in every two adjacent levels of the plurality of sketch counters. The overflow counts indicate the number of times the value of each lower-level counter segment overflows into each upper-level counter segment in every two adjacent levels. The recovered value is a count value recovered based on the count value and overflow value of the upper-level counter segments. The recovered value of the highest-level counter segment is the count value. The counter value recovery module is used to solve the first to m-1 linear equation systems in the order stated therein until the recovered value of each counter segment in the lowest layer is obtained; wherein, the recovered value of the upper counter segment in the j-th linear equation system is equal to the recovered value corresponding to the least squares solution of the (j-1)-th linear equation system, and the recovered value corresponding to the least squares solution is determined based on the least squares solution which serves as the overflow value of the lower counter segment in the (j-1)-th linear equation system; The determination module is used to determine the measurement result of the network flow based on the recovered value of each counter segment in the lowest layer, wherein the recovered value of each counter segment in the lowest layer is the count value of each sketch counter.

15. A network device cluster, characterized in that, It includes at least one network device, each network device including a processor and memory; The processor of the at least one network device is configured to execute instructions stored in the memory of the at least one network device to cause the network device cluster to perform the steps of the method as described in any one of claims 1-13.

16. A computer program product containing instructions, characterized in that, When the instruction is executed by the network device cluster, the network device cluster performs the steps of the method as described in any one of claims 1-13.

17. A computer-readable storage medium, characterized in that, It includes computer program instructions, which, when executed by a network device cluster, perform the steps of the method as described in any one of claims 1-13.

Images