Collaborative network measurement methods, devices, electronic equipment, and media involving three parties.

Abstract

This invention provides a three-party collaborative network measurement method, apparatus, electronic device, and medium, comprising: acquiring a data stream to be sent, the data stream including data packets and a collaborative measurement protocol; adding measurement point information to the data packets through a data sender; sending the data packets to acquire the network status of the network measurement system; performing measurements on the data packets through a switch based on the network status; and sending the measured data packets to a data receiver. The advantages of this invention are: a data plane collaborative measurement protocol, decoupling the plane processing mechanism from specific measurement tasks, allowing for flexible expansion to support multiple measurement tasks without affecting normal network operation, i.e., compatibility with traditional protocols while performing measurement tasks; and a lightweight data plane collaborative measurement processing mechanism that minimizes measurement memory overhead.

Description

Technical Field

[0001] This invention relates to the field of computer and data security technology, and in particular to a three-party collaborative network measurement method, device, electronic device, and medium. Background Technology

[0002] Network traffic measurement plays a crucial role in traffic engineering and load balancing. With the development of Software-Defined Networking (SDN) technology, many cooperative measurement frameworks have been proposed and implemented in production networks, enabling any data flow to be processed by a specific network measurement point (NMP). Mainstream measurement frameworks primarily utilize SDN controllers and switches to implement various cooperative measurement strategies. The core idea is that the controller decides where and how to allocate measurement tasks based on its global view of traffic, while the switches check and record upcoming traffic according to the controller's rules.

[0003] Typical controller-switch cooperative measurement frameworks include OpenWatch, CSAMP, DCM, NSPA, and CFS.

[0004] The above technical solutions all have potential drawbacks, including:

[0005] First, switches must maintain match-action rules by following controller instructions, which incurs significant space overhead. For example, CSAMP and DCM maintain a table and two Bloom filters on each switch to check whether every flow of traffic should be measured by that switch. Specifically, the table introduced in CSAMP contains n^2 elements (n is the number of edge switches in the network), while the Bloom filters in DCM have at least 1 bit per element to ensure a low false alarm rate. However, commercial switches only have on-chip memory on the order of 10MB.

[0006] Second, before measurement, switches require a significant amount of processing time. As network traffic increases, the computational and memory access tasks consume substantial amounts of switch computing resources. For example, in CSAMP, the time complexity of finding a matching cooperative measurement protocol with N rules for each switch is O(logN). Using TCAM can reduce this space complexity to O(1), but TCAM is expensive. NSPA does not require maintaining cooperative measurement protocols on the switches; instead, switches on the forwarding path record the packets that flow with probability p. This method is simple but may lead to an unbalanced measurement load among switches.

[0007] Third, these frameworks only consider the collaboration between the controller and the switch, which to some extent limits the design flexibility of the measurement mechanism. Intuitively, switches operate at the network L2 or L3 layer, and therefore cannot support more complex operations such as header customization, packet encapsulation, source routing, and data encoding. Summary of the Invention

[0008] The main objective of this invention is to propose a three-party collaborative network measurement method, device, electronic device, and medium, which improves the compatibility of three-party collaborative networks and reduces measurement costs.

[0009] One aspect of the present invention provides a three-party collaborative network measurement method for a network measurement system, the network measurement system comprising a data sender, a data receiver, a controller, and a switch, characterized in that:

[0010] In response to a network traffic measurement request, a data stream to be sent is acquired. The data stream to be sent includes data packets and a cooperative measurement protocol. The cooperative measurement protocol is used to characterize the measurement point information of the data packets.

[0011] The data sender adds the measurement point information to the data packet and then sends the data packet.

[0012] The network status of the network measurement system is obtained, and the data packets are measured through the switch according to the network status. The network status includes either a known traffic matrix or an unknown traffic matrix.

[0013] The measured data packet is sent to the data receiver.

[0014] According to the aforementioned three-party collaborative network measurement method, the collaborative measurement protocol includes a measurement point number and a payload protocol number. The measurement point number is used to characterize the sequence number of the forwarding path of the switch in the network measurement system, and the payload protocol number is used to characterize the protocol type of the payload.

[0015] According to the aforementioned three-party collaborative network measurement method, the process of adding the measurement point information to the data packet by the data sender and sending the data packet includes:

[0016] The data sender queries the measurement point number in the data packet;

[0017] If no measurement point number is found, the five-tuple of the data packet is obtained and sent to the controller. The controller then generates the optimal measurement point number and updates the cooperative measurement protocol.

[0018] According to the aforementioned three-party collaborative network measurement method, acquiring the network status of the network measurement system and performing measurements on the data packets through the switch based on the network status includes:

[0019] When the network measurement system is in a state where the traffic matrix is ​​known, it uses a tabu search algorithm with neighborhood pruning to perform network measurement allocation for the switches based on the switches, switch measurement workload, and data flow, thereby obtaining the optimal measurement point for each data flow, where each measurement point is associated with a corresponding switch.

[0020] When the network measurement system is in a state where the traffic matrix is ​​unknown, it obtains the historical load records of the switch and the optimal measurement point number through the controller, and uses one of the online greedy algorithm and the online random two-choice algorithm to allocate the optimal measurement point for the data flow.

[0021] According to the aforementioned three-party collaborative network measurement method, a tabu search algorithm with neighborhood pruning is used to perform network measurement allocation for the switches, obtaining the optimal measurement point for each data stream, including:

[0022] Based on the switches, switch measurement workload, and data flow, network measurement task allocation and constraints are determined. These constraints include a first constraint, a second constraint, a third constraint, and a fourth constraint. The first constraint characterizes the switch measurement workload as not exceeding an upper limit or falling below a lower limit, which are determined based on the switch's physical characteristics. The second constraint characterizes the total number of data packets measured on the switch during load measurement. The third constraint characterizes the maximum error between the defined measured value and the actual value. The fourth constraint also characterizes the requirement that each data flow is measured by at least one switch on its forwarding path.

[0023] A greedy algorithm is used to traverse each data stream. For each large data stream whose size exceeds a preset threshold, the switch with the smallest current measurement workload on its forwarding path is selected as the measurement point.

[0024] According to the aforementioned three-party collaborative network measurement method, the allocation of optimal measurement points for the data flow is performed using either an online greedy algorithm or an online random binary selection algorithm, including:

[0025] The online greedy algorithm is used to check the historical measurement load of each switch on the forwarding path through the controller when a large flow measurement point selection request is received from the data sender. The switch with the smallest historical measurement load is selected as the measurement point, and the sequence number of the measurement point on the forwarding path is used to reply to the data sender.

[0026] The online random two-choice algorithm is used to check the historical measurement load of each switch on the forwarding path through the controller when a measurement point selection request is received. It finds the two switches with the smallest historical measurement load and randomly selects one of them as the measurement point to reply to the data sender.

[0027] According to the aforementioned three-party collaborative network measurement method, the measurement performed by the switch includes:

[0028] Upon receiving the data packet, the cooperative measurement protocol of the data packet is analyzed to obtain the measurement point number in the cooperative measurement protocol;

[0029] Perform the following processing based on the measurement point number:

[0030] If the measurement point number is a preset value, and the data packet has already been measured, then the data packet is forwarded.

[0031] If the measurement point number is not a preset value, update the value of the measurement point number. If the updated value of the measurement point number is not a preset value, forward the data packet. If the updated value of the measurement point number is a preset value, perform the measurement processing of the data packet.

[0032] Another aspect of this invention provides a three-party collaborative network measurement device, comprising:

[0033] The protocol module is used to obtain a data stream to be sent based on a network traffic measurement request. The data stream to be sent includes data packets and a cooperative measurement protocol. The cooperative measurement protocol is used to characterize the measurement point information of the data packets.

[0034] The collaborative processing module is used to add the measurement point information to the data packet through the data sender, and then send the data packet.

[0035] The collaborative allocation module is used to obtain the network status of the network measurement system and perform measurement on the data packets through the switch according to the network status. The network status includes one of known traffic matrix and unknown traffic matrix.

[0036] The sending module is used to send the measured data packet to the data receiver.

[0037] Another aspect of the present invention provides an electronic device, including a processor and a memory;

[0038] The memory is used to store programs;

[0039] The processor executes the program to implement the method as described above.

[0040] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the methods described above.

[0041] The beneficial effects of this invention are as follows: the data plane cooperative measurement protocol decouples the plane processing mechanism from the specific measurement task, which can be flexibly extended to support multiple measurement tasks without affecting the normal operation of the network, that is, it is compatible with traditional protocols while performing measurement tasks; the lightweight data plane cooperative measurement processing mechanism minimizes measurement memory overhead.

[0042] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0043] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0044] Figure 1 This is a schematic diagram of the network measurement system structure according to an embodiment of the present invention.

[0045] Figure 2 This is a flowchart illustrating the three-party collaborative network measurement method according to an embodiment of the present invention.

[0046] Figure 3 This is a schematic diagram of a collaborative measurement protocol according to an embodiment of the present invention.

[0047] Figure 4 This is a schematic diagram of the data sending process according to an embodiment of the present invention.

[0048] Figure 5a , Figure 5b and Figure 5c These are schematic diagrams of the switch processing flow, logic diagrams, and data plane processing algorithms according to embodiments of the present invention.

[0049] Figure 6a and Figure 6b These are schematic diagrams of the data sender and the data packet sender execution logic algorithm, respectively, according to embodiments of the present invention.

[0050] Figure 7 This is a schematic diagram of the process of assigning measurement tasks based on the network status of the network measurement system according to an embodiment of the present invention.

[0051] Figure 8a and Figure 8bThese are, respectively, a flowchart illustrating the offline algorithm for allocating measurement tasks using a known traffic matrix, and a diagram illustrating the tabu search algorithm for neighborhood pruning, according to embodiments of the present invention.

[0052] Figure 9a , Figure 9b and Figure 9c These are, respectively, a flowchart illustrating the online algorithm for allocating measurement tasks when the traffic matrix is ​​unknown, a flowchart illustrating the online greedy algorithm, and a flowchart illustrating the online random two-choice algorithm according to embodiments of the present invention.

[0053] Figure 10 This is a schematic diagram of a three-party collaborative network measurement framework according to an embodiment of the present invention.

[0054] Figure 11 This is a diagram of a three-party collaborative network measurement and analysis device according to an embodiment of the present invention. Detailed Implementation

[0055] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings. Throughout the description, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions. In the following description, suffixes such as "module," "part," or "unit" used to denote elements are used only for the purpose of illustrative purposes and have no specific meaning in themselves. Therefore, "module," "part," or "unit" can be used interchangeably. Terms such as "first," "second," etc., are used only to distinguish technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the sequential relationship of the indicated technical features. In the following description, the consecutive reference numerals for method steps are for ease of review and understanding. Adjusting the implementation order of steps, in conjunction with the overall technical solution of the present invention and the logical relationship between the various steps, will not affect the technical effect achieved by the technical solution of the present invention. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0056] refer to Figure 1 , Figure 1 This is a schematic diagram of a network measurement system, which includes a data sender 100, a controller 200, a switch 300, and a data receiver 400.

[0057] The data sender 100 obtains the data stream to be sent according to the network traffic measurement request. The data stream includes data packets and a cooperative measurement protocol, whereby the cooperative measurement protocol is used to characterize the measurement point information of the data packets. The data sender 100 adds the measurement point information to the data packets and sends the data packets. The controller 200 obtains the network status of the network measurement system and performs measurements on the data packets through the switch 30 according to the network status. The network status includes either a known traffic matrix or an unknown traffic matrix. The measured data packets are then sent to the data receiver 400.

[0058] In some embodiments, the switch 300 is a programmable switch, wherein the measurement point information includes a measurement point number and the switch corresponding to the measurement point, wherein the measurement point number represents the number of the forwarding path.

[0059] In some embodiments, the data stream consists of multiple data packets, and the data stream is divided into small streams and large streams according to the number of its data packets. Small streams account for the majority of the traffic in the network measurement system but occupy less bandwidth, while large streams are a minority but occupy the majority of bandwidth.

[0060] refer to Figure 2 , Figure 2 This is a flowchart illustrating a three-party collaborative network measurement method, which includes, but is not limited to, steps S100 to S400:

[0061] S100, in response to a network traffic measurement request, acquires a data stream to be sent, which includes data packets and a cooperative measurement protocol. The cooperative measurement protocol is used to characterize the measurement point information of the data packets.

[0062] In some embodiments, the cooperative measurement protocol represents measurement task allocation information to reduce the proportion of measurement fields in the packet header, thereby improving bandwidth utilization efficiency.

[0063] For example, refer to Figure 3 This diagram illustrates the Cooperative Measurement Protocol (CMP), where ETH represents the data link layer protocol, IP represents the network layer protocol, and TCP / UDP represents the transport layer protocol. The CMP protocol is positioned between the network and transport layers. The CMP consists of two 8-bit fields: Measurement Point Number (MPN) and Payload Protocol Number (PPN). The MPN field is set by the sender. The sender determines which switch should measure the packet by looking up its local MPN table and then writes the sequence number of that switch on the forwarding path into the MPN field of the flow. The PPN field displays the protocol type of the packet's payload.

[0064] In some embodiments, if the network does not use the IPv4 Diffserv field, the Diffserv field can be reused to record the MPN value. If the Diffserv field is used, CMP does not introduce any additional fields in the packet header. After being encapsulated by the CMP protocol, the packet is sent into the network. By setting the PPN field for the upper-layer transport layer protocol type, the communicating end and the programmable data plane device can obtain the upper-layer transport layer protocol number, thus enabling cooperative measurement protocols to be compatible with various protocols without affecting network operation and performance. Switches can determine whether to perform a measurement task by checking the MPN field in the packet's CMP protocol, thereby saving extra memory resources that would otherwise be stored on the switch.

[0065] S200 adds measurement point information to the data packet through the data sender and then sends the data packet.

[0066] In some embodiments, reference Figure 4 The diagram shown illustrates the measurement process performed by the switch, which includes, but is not limited to, steps S210 to S220:

[0067] S210, the data sender queries the measurement point number in the data packet;

[0068] S220: If no measurement point number is found, the five-tuple of the data packet is obtained and sent to the controller. The controller then generates the optimal measurement point number and updates the cooperative measurement protocol. It is understood that in this embodiment, when a measurement point number is found, the cooperative measurement protocol is directly updated based on that number.

[0069] Further, refer to the following: Figure 5a , Figure 5b and Figure 5c The diagram shows the switch processing flow, logic diagram, and data plane processing algorithm diagram.

[0070] in Figure 5a Including but not limited to steps S230 to S240:

[0071] S230, upon receiving a data packet, analyze the cooperative measurement protocol of the data packet and obtain the measurement point number in the cooperative measurement protocol;

[0072] S240, perform the following processing based on the measurement point number: if the measurement point number is a preset value, the data packet has been measured, then forward the data packet; if the measurement point number is not a preset value, update the value of the measurement point number; if the updated value of the measurement point number is not a preset value, then forward the data packet; if the updated value of the measurement point number is a preset value, then perform the measurement processing of the data packet.

[0073] For example, such as Figure 5b This diagram illustrates the processing logic of a switch. The switch needs to know whether the data packets passing through it should be measured. For example... Figure 5b As shown, upon receiving a data packet, the switch first reads its MPN field. If this field is 0, it means the current data packet has already been processed by another switch, so it is forwarded directly; otherwise, the MPN field in the current data packet is decremented by 1. If the MPN field is 0 after decrementing by 1, it means the flow to which the packet belongs needs to be measured by this switch, and the switch performs the measurement task; if the MPN field is not 0 after decrementing by 1, or the measurement task has been completed, the switch will forward the data packet. See the data plane processing procedure for details. Figure 5c The algorithm for data plane processing is shown. This data plane processing mechanism is decoupled from specific measurement tasks, allowing for flexible expansion to support various measurement tasks without affecting the normal operation of the network.

[0074] In some embodiments, the three-party collaborative network measurement of the present invention further includes a data sender, wherein the data sender refers to the party that sends the data packet, and in the embodiments of the present invention, both sender and data packet sender refer to the data sender.

[0075] For example, refer to Figure 6a and Figure 6b These figures illustrate the data sender and the data packet sender's execution logic algorithm. First, referring to Figure 6, unlike switches, data packet senders at the network edge may have sufficient memory, abundant computing resources, and data packet encapsulation capabilities. Therefore, this paper argues that introducing the data sender into the measurement framework and exploring new cooperative measurement mechanisms will revolutionize network measurement. On one hand, the matching rule table can be migrated from the switch to the sender, freeing up the limited memory space occupied by matching tables and filters on the switch, similar to those in cSamp and DCM. On the other hand, the sender can define new fields in the packet header and encode the necessary information into the data packet to implement customized measurement strategies.

[0076] Taking the TPC architecture as an example, in the TPC architecture, the packet sender is responsible for:

[0077] 1) Maintain a table to record measurement rules from the controller;

[0078] 2) Embed information about where the data packet will be measured in the packet header;

[0079] 3) Send the encapsulated data packet to the network.

[0080] To this end, the sender maintains a measurement point number (MPN) table locally to support the measurement distribution of traffic (i.e., where the flow should be measured). This table uses the hash of the 5-tuple of a large flow as the key and the optimal MPN value assigned by the controller as the value. Since network traffic distribution is typically highly uneven (large flows are few in number but occupy the vast majority of bandwidth), the TPC only assigns measurement tasks to large flows along the routing path, while measurements of small flows are performed by their ingress switches.

[0081] refer to Figure 6a When the sender wants to send a new stream F', it first checks the MPN table. If an entry for F' exists in the table, it finds the entry, sets the CMP.MPN field in the F' header to the best MPN value in the entry, and then forwards it directly. If the stream F' does not exist in the MPN table, it requests the best MPN value from the controller. After the controller returns, it sets the MPN field of F' to the best MPN value returned by the controller, forwards it, and adds an entry for F' to the MPN table. The key is the hash value calculated from the 5-tuple of F', and the value is the best MPN value returned by the controller. The execution logic of the data sender is as follows: Figure 6b The algorithm shown.

[0082] S300 acquires the network status of the network measurement system and performs measurements on data packets through the switch based on the network status. The network status includes either a known traffic matrix or an unknown traffic matrix.

[0083] In some embodiments, reference Figure 7 The flowchart shown is a network status measurement task allocation diagram of the network measurement system, which includes, but is not limited to, steps S310 to S320:

[0084] S310, Obtain the network status of the network measurement system;

[0085] S320, when the network measurement system is in a state where the traffic matrix is ​​known, it uses a tabu search algorithm with neighborhood pruning to perform network measurement allocation for the switches based on the switches, switch measurement workload, and data flow, obtaining the optimal measurement point for each data flow, where each measurement point is associated with a corresponding switch; when the network measurement system is in a state where the traffic matrix is ​​unknown, it obtains the historical load records of the switches and the optimal measurement point number through the controller, and uses either an online greedy algorithm or an online random two-choice algorithm to perform the allocation of the optimal measurement point for the data flow.

[0086] In some embodiments, a control plane cooperative network measurement task allocation method with low measurement load utilizes the sender's storage and computing resources, the measurement point's packet processing capabilities, and the controller's global awareness capabilities to allocate network measurement tasks, designing a control plane cooperative network measurement task allocation method that integrates the advantages of all three parties.

[0087] In some embodiments, in offline scenarios, the traffic matrix is ​​known and the task placement problem is not sensitive to latency. Therefore, the task placement problem is modeled as an integer programming problem. Solving this problem can yield the global optimal solution of the task placement method, which minimizes the maximum workload of the measurement points.

[0088] For example, refer to Figure 8a The flowchart shown illustrates the offline algorithm for assigning measurement tasks when the flow matrix is ​​known, including but not limited to steps S3211 to S3212:

[0089] S3211, based on the switch, the switch measurement workload, and the data flow, determine the network measurement task allocation and constraints. These constraints include a first constraint, a second constraint, a third constraint, and a fourth constraint. The first constraint characterizes the switch measurement workload as not exceeding an upper limit or falling below a lower limit, which are determined based on the switch's physical characteristics. The second constraint characterizes the total number of data packets measured on the switch during load measurement. The third constraint characterizes the maximum error between the defined measured value and the actual value. The fourth constraint characterizes the maximum error between the measured value and the actual value. The fifth constraint characterizes the maximum error between each data flow and the measurement workload.

[0090] S3212 uses a greedy algorithm to traverse each data stream. For each large data stream whose size exceeds a preset threshold, the switch with the smallest current measurement workload on its forwarding path is selected as the measurement point.

[0091] For example, refer to Figure 8b The diagram illustrates a tabu search algorithm for neighborhood pruning. To efficiently distribute flow measurement tasks among measurement points, embodiments of this invention assume the task allocation method must satisfy the following principles: each data packet is processed at most once; each flow is assigned to at least one measurement point on the forwarding path; the workload of each measurement point is the number of data packets being measured at each measurement point. Define a set of switches S; a set of flows I within a measurement period; and the measurement workload L of switch k. k This refers to the number of packets measured by switch k within a given period; the maximum measurement workload L. ub Minimum measurement workload L lb Measurement task allocation parameter b i,k This indicates whether the switch is responsible for measuring flow i, b during the measurement period. i,k ∈{0,1}; define the size of the stream as f.i ε is the number of packets contained in flow i; the maximum average measurement error (number of packets) per switch is ε; and the set of forwarding paths T for flow i is ε. i Let t be a path for forwarding flow i, where t∈T i The collaborative network measurement task allocation problem is defined as follows:

[0092] minmax{L k ,s k ∈S}

[0093]

[0094] The matrix to be solved, b i,k This serves as the basis for the controller to issue optimal measurement points. The first constraint means that the measurement workload cannot exceed the upper limit or be less than the lower limit. The upper limit is determined by the physical characteristics of the measurement point device, and the lower limit defaults to 0. The second constraint provides the mathematical definition and calculation basis for the measurement workload, namely, in the switch s k The first constraint is the total number of packets measured; the second constraint limits the maximum error between the measured and actual values; and the third constraint states that each flow is measured by at least one switch on the forwarding path.

[0095] When the flow matrix is ​​known, the above problem is a minimax problem in 0 / 1 integer programming, which is an NP-complete problem. The optimal solution algorithm for this problem degenerates to an enumeration-level time complexity, but an approximate optimal solution can be quickly obtained using industry-standard solvers (such as Gurobi). However, as the amount of data increases, the time required to solve this problem increases exponentially, making it difficult to meet the low-latency requirements of measurement task allocation.

[0096] Therefore, embodiments of the present invention design a neighborhood pruning tabu search algorithm ( Figure 5c The algorithm shown (for data plane processing) can be used to solve the task allocation problem described above. The initial solution state of the task allocation problem can be given using a greedy algorithm: traverse each flow sequentially, and for each large flow whose size exceeds a preset threshold, select the switch with the smallest current measurement workload on its forwarding path as the measurement point.

[0097] Neighborhood state transition actions are defined as moving a measurement point of a specific flow from one switch to another. The search algorithm is based on the assumption that the larger the flow, the greater the impact of flow movement on the maximum measurement workload. Therefore, starting from the initial state, the algorithm further selects a set of larger flows (called "mega-flows") within the large flow using a threshold `huge_flow_threshold`, and searches for possible movement operations for these mega-flows. This significantly reduces the number of neighborhoods to be searched, thereby further reducing the time complexity of the search algorithm.

[0098] After generating the initial state, the algorithm iterates through each massive flow. Let the massive flow f be the current state. i Already assigned to switch s i cand Its measurement workload is f i The switch with the lowest measurement workload on the forwarding path is s. i min _ wld Its measurement workload is if Then the action Add it to the candidate set.

[0099] After the candidate set is generated, a certain number of actions are randomly selected each time to generate neighborhoods and form a neighborhood set called neighbors. The evaluation function of the neighborhood, func(neighbor), is defined as the maximum measurement workload of the switch. The smaller the value of func(neighbor), the closer the neighborhood is to the optimal solution.

[0100] After obtaining the initial state, iterate through each neighborhood and calculate its evaluation function. Select the neighborhood that is better than the current state and has the minimum measurement load. A Set it as the next state and execute its corresponding set of movement actions. If the optimal neighborhood is taboo, use the suboptimal neighborhood. A' Update the current state, and so on; if all neighbors are taboo, then pardon the best neighbor. A Finally, use neighbor. A Update the current status.

[0101] After each state update, the tabu table must be updated by decrementing the tabu length of each entry by 1, i.e., tabu_table[neighbor] = tabu_table[neighbor] - 1, and entries with a tabu length of 0 must be cleared.

[0102] The following is a brief discussion. Figure 5c The algorithm for data plane processing is shown below. Assume there are *s* switches in the network topology, the algorithm iterates *k* times, the dataset contains *n* large flows, the number of neighborhoods in the candidate set is *p* in each iteration, generating each neighborhood requires selecting *a* movement actions, and the tabu list has a length of *l*. It is easy to see that the greedy algorithm for generating the initial state has a time complexity of O(ns). Figure 5cThe time complexity of the data plane processing algorithm shown is O(ns+k(n+plogp+a+l)). Since a, l, and p are all preset small constants less than 100, Figure 5c The time complexity of the data plane processing algorithm shown can be further simplified to O(n(s+k)). Therefore... Figure 5c The time complexity of the data plane processing algorithm shown is related to the number of large flows in the network, the number of switches, and the number of algorithm iterations. Experimental results show that when the number of iterations k = 50, Figure 5c The algorithm for processing the data plane shown can achieve good results. Therefore, the time complexity of this algorithm largely depends on the time complexity of the initial state generation algorithm.

[0103] Figure 5c The data plane processing algorithm shown compresses the solution space by properly pruning the neighborhood. Figure 5c The data plane processing algorithm shown solves the cooperative measurement task allocation problem, obtaining the optimal measurement point to which each flow should be assigned. Combined with the flow forwarding path, the switch can obtain the optimal MPN value for each flow and respond when the data sender makes a request.

[0104] In some embodiments, in network measurement scenarios where the traffic matrix is ​​unknown, task placement is latency-sensitive. Therefore, an online control plane cooperative measurement task allocation method is designed to achieve real-time allocation of measurement tasks and minimize the maximum workload of measurement points.

[0105] In some embodiments, reference Figure 9a , Figure 9b and Figure 9c These are flowcharts illustrating the online algorithm for allocating measurement tasks when the flow matrix is ​​unknown, the online greedy algorithm, and the online random two-choice algorithm.

[0106] in, Figure 9a The flowchart shown illustrates the online algorithm for assigning measurement tasks when the flow matrix is ​​unknown, including but not limited to steps S3221 to S3222:

[0107] S3221, Determine whether the measurement request is from either a data sender or a measurement point.

[0108] S3222, the online greedy algorithm is used when a large flow measurement point selection request is received from the data sender. It checks the historical measurement load of each switch on the forwarding path through the controller, selects the switch with the smallest historical measurement load as the measurement point, and uses the sequence number of the measurement point on the forwarding path to reply to the data sender. The online random two-choice algorithm is used when a measurement point selection request is received. It checks the historical measurement load of each switch on the forwarding path through the controller, finds the two switches with the smallest historical measurement load, and randomly selects one of them as the measurement point to reply to the data sender.

[0109] It is understood that, in this embodiment of the invention, for latency-sensitive scenarios, the controller no longer directly calculates time-consuming optimization problems. Instead, it quickly selects a measurement point by executing a simple online collaborative measurement algorithm based on historical measurement load records within short time slices. Regardless of the online algorithm used by the controller, when the controller receives a request from the data sender for the optimal MPN value for a specified flow, it will run the online algorithm and return the calculation result to the data sender.

[0110] refer to Figure 9b The online greedy algorithm shown in the diagram, when receiving a request from the packet sender to select a high-volume measurement point, the controller checks the historical measurement load of each switch on the forwarding path, selects the switch with the lowest historical measurement load as the measurement point, and uses the sequence number of this measurement point on the forwarding path to reply to the packet sender.

[0111] refer to Figure 9c The online random two-choice algorithm shown is as follows: When a measurement point selection request is received, the controller also checks the historical measurement load of each switch on the forwarding path, finds the two switches with the smallest historical measurement load, and randomly selects one of them as the measurement point to reply to the data packet sender.

[0112] S400 sends the measured data packets to the data receiver.

[0113] refer to Figure 10 , Figure 10 This is a schematic diagram of a collaborative network measurement framework involving three parties. (For example...) Figure 10 As shown, for a large flow F, the sender first looks up its local MPN table; if not found, the sender sends the flow's 5-tuple to the controller (step 1). Then, the controller calculates the sequence number of the switch with the lowest workload on the forwarding path (step 2) and sends it back to the sender (step 3). Upon receiving the controller's reply, the sender updates its MPN table, sets the MPN field in the packet header, and then sends the packet out (step 4).

[0114] When the switch receives a data packet, it checks the MPN value.

[0115] 1) When a data packet enters the switch, if the MPN value is greater than 0, the switch will decrement the MPN value by 1. After decrementing the MPN value, if the MPN value is not equal to 0, the data packet will be forwarded directly (steps 5 and 6); if the MPN value is 0, the measurement task will be performed (step 7).

[0116] 2) When a data packet with MPN=0 enters the switch, the switch will forward it directly (step 8).

[0117] Because the MPN value of a small flow will be set to 1, the small flow will be measured by its corresponding ingress switch. At the end of a time slice, the controller reads the measurement data from each switch.

[0118] refer to Figure 11 , Figure 11 This is a diagram of a three-way collaborative network measurement and analysis device. The device includes a protocol module 1110, a collaborative processing module 1120, a collaborative allocation module 1130, and a transmission module 1140.

[0119] The system includes the following modules: a protocol module, which acquires a data stream to be sent based on a network traffic measurement request. The data stream includes data packets and a cooperative measurement protocol, which characterizes the measurement point information of the data packets; a collaborative processing module, which adds the measurement point information to the data packets through the data sender and then sends the data packets; a collaborative allocation module, which acquires the network status of the network measurement system and performs measurements on the data packets through the switch based on the network status, which includes either a known traffic matrix or an unknown traffic matrix; and a sending module, which sends the measured data packets to the data receiver.

[0120] Exemplarily, with the cooperation of the protocol module, collaborative processing module, collaborative allocation module, and sending module in the device, the embodiment device is applied to a network measurement system. The embodiment device can implement any of the aforementioned three-party collaborative network measurement methods, namely, responding to a network traffic measurement request, acquiring a data stream to be sent, which includes data packets and a collaborative measurement protocol, the collaborative measurement protocol being used to characterize the measurement point information of the data packets; adding the measurement point information to the data packets through the data sender, and sending the data packets; acquiring the network status of the network measurement system, and performing measurements on the data packets through a switch according to the network status, the network status including either a known traffic matrix or an unknown traffic matrix; and sending the measured data packets to the data receiver. The beneficial effects of the present invention are: a data plane collaborative measurement protocol, the plane processing mechanism is decoupled from the specific measurement task, which can be flexibly extended to support multiple measurement tasks without affecting the normal operation of the network, that is, it is compatible with traditional protocols while performing measurement tasks; a lightweight data plane collaborative measurement processing mechanism minimizes measurement memory overhead.

[0121] This invention also provides an electronic device, which includes a processor and a memory;

[0122] The memory stores the program;

[0123] The processor executes a program to perform the aforementioned three-party collaborative network measurement method; the electronic device has the function of carrying and running the software system for three-party collaborative network measurement provided in the embodiments of the present invention.

[0124] This invention also provides a computer-readable storage medium storing a program that is executed by a processor to implement the three-party collaborative network measurement method described above.

[0125] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0126] This invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the aforementioned three-way collaborative network measurement method.

[0127] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the described functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0128] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0129] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0130] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0131] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0132] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0133] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0134] The above is a detailed description of the preferred embodiments of the present invention, but the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A three-party collaborative network measurement method for a network measurement system, the network measurement system comprising a data sender, a data receiver, a controller, and a switch, characterized in that: In response to a network traffic measurement request, the data sender obtains a data stream to be sent, which includes data packets and a cooperative measurement protocol. The cooperative measurement protocol is used to characterize the measurement point information of the data packets. The cooperative measurement protocol includes a measurement point number and a payload protocol number. The measurement point number is used to characterize the sequence number of the forwarding path of the switch in the network measurement system, and the payload protocol number is used to characterize the protocol type of the payload. The data sender queries the measurement point number in the data packet and searches the local measurement point number table. If no measurement point number is found, the sender obtains the five-tuple of the data packet and sends it to the controller, which then generates the optimal measurement point number. The step of generating the optimal measurement point number through the controller includes: the controller acquiring the network status of the network measurement system, and performing measurements on the data packets through the switch according to the network status, wherein the network status includes either a known traffic matrix or an unknown traffic matrix; when the network measurement system status is a known traffic matrix, the controller performs network measurement allocation for the switch using a neighborhood pruning tabu search algorithm based on the switch, the switch measurement workload, and the data flow, to obtain the optimal measurement point for each data flow, wherein the measurement point is associated with a corresponding switch; when the network measurement system status is an unknown traffic matrix, the controller acquires the historical load records of the switch and the optimal measurement point number, and performs the allocation of the optimal measurement point for the data flow using either an online greedy algorithm or an online random two-choice algorithm. The data sender adds the optimal measurement point number replied by the controller as measurement point information to the data packet, updates the cooperative measurement protocol, and sends the data packet. The switch performs measurements on the data packets, including: upon receiving the data packet, analyzing the cooperative measurement protocol of the data packet to obtain the measurement point number in the cooperative measurement protocol; and performing the following processing based on the measurement point number: if the measurement point number is a preset value, the data packet has been measured, and the data packet is forwarded; if the measurement point number is not a preset value, the value of the measurement point number is updated; if the updated value of the measurement point number is not a preset value, the data packet is forwarded; if the updated value of the measurement point number is a preset value, the measurement processing of the data packet is performed. The switch sends the measured data packets to the data receiver.

2. The three-party collaborative network measurement method according to claim 1, characterized in that, The tabu search algorithm employing neighborhood pruning is used to perform network measurement allocation for the switch, obtaining the optimal measurement point for each data stream, including: Based on the switches, switch measurement workload, and data flow, network measurement task allocation and constraints are determined. These constraints include a first constraint, a second constraint, a third constraint, and a fourth constraint. The first constraint characterizes the switch measurement workload as not exceeding an upper limit or falling below a lower limit, which are determined based on the switch's physical characteristics. The second constraint characterizes the total number of data packets measured on the switch during load measurement. The third constraint characterizes the maximum error between the defined measured value and the actual value. The fourth constraint characterizes each data flow as being measured by at least one switch on the forwarding path. A greedy algorithm is used to traverse each data stream. For each large data stream whose size exceeds a preset threshold, the switch with the smallest current measurement workload on its forwarding path is selected as the measurement point.

3. The three-party collaborative network measurement method according to claim 1, characterized in that, The allocation of optimal measurement points for executing the data stream using either an online greedy algorithm or an online random binary selection algorithm includes: The online greedy algorithm is used to check the historical measurement load of each switch on the forwarding path through the controller when a large flow measurement point selection request is received from the data sender. The switch with the smallest historical measurement load is selected as the measurement point, and the sequence number of the measurement point on the forwarding path is used to reply to the data sender. The online random two-choice algorithm is used to check the historical measurement load of each switch on the forwarding path through the controller when a measurement point selection request is received. It finds the two switches with the smallest historical measurement load and randomly selects one of them as the measurement point to reply to the data sender.

4. A three-party collaborative network measurement device, characterized in that, For a network measurement system, the network measurement system includes a data sender, a data receiver, a controller, and a switch, and the cooperative network measurement device includes: The protocol module is used by the data sender to obtain a data stream to be sent according to a network traffic measurement request. The data stream to be sent includes data packets and a cooperative measurement protocol. The cooperative measurement protocol is used to characterize the measurement point information of the data packets. The cooperative measurement protocol includes a measurement point number and a payload protocol number. The measurement point number is used to characterize the sequence number of the forwarding path of the switch in the network measurement system. The payload protocol number is used to characterize the protocol type of the payload. The collaborative processing module is used by the data sender to query the measurement point number in the data packet, search the local measurement point number table, and if no measurement point number is found, obtain the five-tuple of the data packet and send it to the controller. The collaborative allocation module is used to generate optimal measurement point numbers through the controller, including: the controller acquiring the network status of the network measurement system, and performing measurements on the data packets through the switch according to the network status, wherein the network status includes either a known traffic matrix or an unknown traffic matrix; when the network measurement system status is a known traffic matrix, the controller performs network measurement allocation for the switch using a neighborhood pruning tabu search algorithm based on the switch, the switch measurement workload, and the data flow, to obtain the optimal measurement point for each data flow, wherein the measurement point is associated with a corresponding switch; when the network measurement system status is an unknown traffic matrix, the controller acquires the historical load records of the switch and the optimal measurement point number, and performs the allocation of the optimal measurement point for the data flow using either an online greedy algorithm or an online random two-choice algorithm. The sending module is used for the data sender to add the optimal measurement point number replied by the controller as measurement point information to the data packet, update the cooperative measurement protocol, and send the data packet; the switch performs measurement on the data packet and sends the measured data packet to the data receiver; wherein, the switch performs measurement on the data packet, including: when receiving the data packet, analyzing the cooperative measurement protocol of the data packet to obtain the measurement point number in the cooperative measurement protocol; performing the following processing according to the measurement point number: when the measurement point number is a preset value, the data packet has been measured, and the data packet is forwarded; when the measurement point number is not a preset value, the value of the measurement point number is updated; when the updated value of the measurement point number is not a preset value, the data packet is forwarded; when the updated value of the measurement point number is a preset value, the measurement processing of the data packet is performed.

5. An electronic device, characterized in that, Including the processor and memory; The memory is used to store programs; The processor executes the program to implement the three-party collaborative network measurement method as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The storage medium stores a program that is executed by a processor to implement the three-party collaborative network measurement method as described in any one of claims 1-3.

Images