A distributed data plane resource optimization method and system

Abstract

This invention discloses a distributed data plane resource optimization method and system, which fully utilizes merging and splitting algorithms to optimize data plane program resources and solve the resource limitation problem of switches in programmable networks. First, the input programs are classified. Then, a merging algorithm is used for resource-sparse programs, and a splitting algorithm is used for resource-intensive programs. The merging and splitting algorithms yield a set of subtasks, and finally, the deployment scheme of each task in the subtask set in the underlying network is obtained. Compared with existing technologies, this invention can improve the speed and effectiveness of program merging, effectively reduce program redundancy, and reduce the granularity of program placement and scheduling overhead between devices by splitting programs. It also provides a deployment scheme with higher resource utilization, demonstrating the practicality of a distributed data plane resource optimization method and system.

Description

Technical Field

[0001] This invention belongs to the field of programmable networks, and specifically relates to a distributed data plane resource optimization method and system. Background Technology

[0002] Compared to traditional data planes, existing programmable data planes offer greater flexibility, provide an abstract language framework, simplify the development of stateful network functions, and are currently a widely accepted resource optimization solution. Programmable data planes can be deployed on programmable switches using programmable switch compilers; however, limited switch resources severely hinder the development of existing technologies.

[0003] A single switch typically has only tens of MB of total memory. For high-hit-rate detection measurement tasks, network administrators need to limit the number of counters in the switch to ensure deployment within the resource constraints of a single switch. This increases the complexity of the network administrator's work and is prone to counting errors. With the advent of programmable switches and the P4 programming language, running multiple P4 programs simultaneously on the data plane can fully utilize switch resources. Current research has also been conducted on this topic, but certain issues remain.

[0004] When a program is executed in a programmable network, it relies on a P4 program that can run on a switch. The P4 program is target-independent; it runs on highly specialized hardware-based switches or software-specific software switches. After compilation, the P4 program generates an intermediate representation (IR), which represents the P4 program in graph form. The nodes are Match Action Tables (MATs), which are crucial for the P4 program to function as a new domain-specific language. A MAT contains a match field for matching packet headers, and the actions defined in the action table perform corresponding operations on the match results. At runtime, based on dependencies and hardware resource requirements, the corresponding MAT is placed in the switch's ASIC pipeline to execute the measurement tasks defined by the P4 program. This paper focuses on the data plane program in the measurement domain, treating the measurement tasks implemented by the network administrator as corresponding P4 programs as a single task.

[0005] To address the resource constraints of switches, existing methods primarily optimize in two ways: reducing resource consumption and implementing distributed solutions. SPEED adopts the idea of ​​reducing data plane resource redundancy by merging the longest common subsequence (LCS) of the intermediate representations generated by the P4 front-end compiler. On two identical P4 programs, the TDGs of both programs are first topologically sorted, and the longest common subsequence is calculated, with this subsequence used as the merging node. In terms of distributed deployment, SPEED employs the OBS concept, simplifying the complexity of the underlying program deployment.

[0006] To reduce redundancy in the data plane, SPEED uses the longest common subsequence (LCS) merging method. On two identical P4 programs, SPEED first performs a topological sort on the two TDG graphs, then obtains their longest common subsequence based on the two topological orders, using this subsequence as the merging node. However, during the topological sorting process, a graph may generate multiple topological orders, and SPEED struggles to account for various scenarios. Therefore, merging on some TDGs may not always achieve the optimal solution.

[0007] In the research of distributed deployment, SPEED involves the problem of linear programming deployment. When deploying tasks to the real network, SPEED adopts the OBS concept, which simplifies the complexity of the underlying network deployment. However, it inevitably ignores the dependencies between programs. To address this issue, SPEED uses an inserted routing module to carry all metadata to ensure the normal operation of the two programs before and after. However, this introduces new overhead and latency problems. Summary of the Invention

[0008] To overcome the shortcomings of existing technologies, this invention provides a distributed data plane resource optimization method and system. It fully utilizes merging and splitting algorithms to optimize task resources and reduce program redundancy, thereby saving programmable switch resources. First, input programs are categorized based on their stage usage and the fixed stage of the programmable switch. Then, a splitting algorithm is used for resource-intensive programs, and a merging algorithm is used for resource-sparse programs. Finally, an integer linear programming model is used to calculate the final deployment scheme of the subtask set (SST) obtained from the merging and splitting algorithms. This invention overcomes the shortcomings of existing technologies, such as difficulty in achieving optimal program merging and long task processing times. Under the premise of satisfying task constraints, switch resource constraints, and global constraints, it aims to minimize resource stage usage, ensure balanced resource allocation across programmable switches, complete the mapping relationship between the switch's underlying layer and tasks, improve task processing time and efficiency, and increase the utilization rate of the switch's scarce memory resources.

[0009] To make the content of this invention clearer, the concepts in programmable switches will first be explained:

[0010] pipeline: The pipeline of a programmable switch, such as... Figure 1 As shown, each pipeline consists of multiple stages. The program is placed after the switch and is executed in strict order, starting from the first stage and proceeding backward.

[0011] stage: Each pipeline has a limited number of stages, and each stage has limited memory (SRAM and TCAM) and computing resources.

[0012] The technical solution of this invention is as follows:

[0013] A distributed data plane resource optimization method includes the following steps:

[0014] Step 1: Compile the input program at the front end to generate a TDG table dependency graph, and add the TDG table dependency graph to the original task set; at the same time, divide the program into resource-intensive and resource-sparse types. The resource-intensive program is the input program that occupies more stage resources than the fixed stage of the programmable switch, and the resource-sparse program is the input program that occupies less stage resources than the fixed stage of the programmable switch.

[0015] Step 2, optimize the input program's task resources according to the program type, specifically including:

[0016] The resource-sparse program is used with a merging algorithm: multiple resource-sparse programs are merged pairwise, and the merged subtasks are obtained by minimizing the number of nodes. The merged subtasks are then added to the subtask set SST.

[0017] The resource-intensive program is split using a splitting algorithm: by deleting edges in the TDG table dependency graph of the resource-intensive program, the split subtasks with the minimum stage occupancy and the minimum difference between the stages occupied by each subtask are obtained, and the split subtasks are added to the subtask set SST.

[0018] Step 3: Obtain the final deployment scheme of each task in the subtask set SST described in Step 2 on the underlying network.

[0019] Furthermore, the original task set and the subtask set SST are maintained to preserve the mapping relationship between the input program and the programs in the subtask set SST, the corresponding flow path information, and the resource usage, ensuring that the original dependency relationship can still be maintained after task resource optimization.

[0020] Furthermore, the TDG table dependency graph is specifically as follows:

[0021] The TDG table dependency graph is a directed acyclic graph, represented as: G = (T, E);

[0022] Wherein, G represents the TDG table dependency graph, the node T of the graph is the matching action table MAT, and the edge E of the graph is the dependency relationship between the matching action tables MAT; the attributes of the matching action table MAT represented by each node T include: matching field, action field, and table size; the attribute values ​​of the dependency relationship represented by each edge E include: no dependency, matching dependency, behavior dependency, inverted dependency, and successor dependency.

[0023] Furthermore, the merging algorithm specifically includes the following sub-steps:

[0024] Step 2.1.1: The adjacency matrix D is used to describe the dependency relationships between all n nodes in the TDG table dependency graph. If D[i,j] = 1, it means that there is a dependency path between the two nodes; if D[i,j] = 0, it means that there is no dependency path between the two nodes.

[0025] The dependency graph G of the TDG table of the two resource-sparse programs that need to be merged is used. a and G b Generate adjacency matrix D a and D b And the following conditions must be met: the nodes containing the same matching action table MAT in different TDG table dependency graphs are represented by the same subscript; the merged TDG table dependency graph has no loops; the dependency relationships between TDG table dependency graphs remain unchanged.

[0026] Step 2.1.2, merge candidate sets T p , used to represent the set of all deduplicated nodes in a dependency graph of multiple TDG tables; edge set E p The following relationship must be satisfied:

[0027]

[0028] Where i and j represent the subscripts of the matrix, i.e., the node numbers in the TDG table dependency graph;

[0029] According to the merge candidate set T p and edge set E p Construct an undirected graph G p =(T p E p The weight set W and the degree set of the points are combined to form a weighted undirected graph G. u =(T p E p ,W,D);

[0030] Step 2.1.3, the weighted undirected graph G u Input the data into the DTSingle algorithm and the DTTwo algorithm to remove surrounding nodes and obtain the S1 point set;

[0031] The removed surrounding nodes include: surrounding nodes of a single node whose weight is greater than the sum of the weights of all its neighboring nodes, and surrounding nodes of the two nodes whose distance is 2 and whose weights are greater than the sum of the weights of all their neighboring nodes.

[0032] Step 2.1.4: Using the processed point set S1, obtain a weighted undirected graph G1 based on the weighted undirected graph Gu, denoted as G1 = (S1, G... u );

[0033] Step 2.1.5: Traverse the nodes in the S1 point set;

[0034] Define ST as a set array, where ST[i] represents the largest weighted independent set in the set from the first point to the i-th point and including the i-th point;

[0035] Define WT as an array, where WT[i] represents the sum of weights of the largest weighted independent set in the set, from the first point to the i-th point and including the i-th point;

[0036] Let bestSet represent the optimal maximum weighted independent set; bestWeight represent the sum of the weights of the optimal maximum weighted independent set.

[0037] Let the current node be u. Let the weight of the largest weighted independent set from the first node to the node u and including the node u be the initial value of WT[u], which is the weight of u, and express it as WT[u] = W[u]. The initial value of the largest weighted independent set ST[u] is {u}, which is expressed as ST[u] = ST[u] ∪ {u}.

[0038] Simultaneously, traverse the S1 point set from index 1 to i, and traverse to the current node v. If after adding node u to the maximum weighted independent set ST[v] of node v, the obtained ST[u] set is an independent set, and WT[u] is larger, then update the maximum weighted independent set of the current node and the weight sum of the maximum weighted independent set to ST[u] and WT[u].

[0039] Continue until the entire S1 point set has been traversed, obtaining the maximum weighted independent set ST[i] and the weights WT[i] of the maximum weighted independent set;

[0040] Step 2.1.6: Assign the maximum weighted independent set ST[i] and the weight sum WT[i] of the maximum weighted independent set to the bestSet and the bestWeight to obtain the optimal maximum weighted independent set and the optimal weight sum of the maximum weighted independent set;

[0041] Step 2.1.7, define the resource sharing set T. share , used to represent a set of nodes that are repeated in the dependency graph of multiple TDG tables and can be used to merge;

[0042] Output the nodes in the optimal maximum weighted independent set bestSet, and add the nodes to the resource-sharing set T. share Update T share Obtain the TDG table dependency graph G of the merged subtasks. m =(T m E m );

[0043] Where T m =T p E m For T share Traversing G a and G b Updated dependencies;

[0044] Step 2.1.8, derive the dependency graph G from the TDG table of the merged subtasks. m Add the subtask set SST to complete the merging of resource-sparse programs.

[0045] Furthermore, the splitting algorithm specifically includes the following sub-steps:

[0046] Step 2.2.1: Input the TDG table dependency graph G = (T, E) of the resource-intensive program to be split;

[0047] Step 2.2.2: Initialize two subgraphs, sub1 and sub2; initialize the first node pointers, firstNode and subNode, of the two subgraphs; initialize the stage number difference, d.

[0048] Step 2.2.3: For each node t in the TDG table dependency graph G of the resource-intensive program to be split, make a judgment:

[0049] If the attribute value of the dependency relationship between two nodes is no dependency, calculate the number of stages in the two subgraphs before and after the edge between the two nodes; if the actual difference in the number of stages between the two subgraphs, dis, is less than the difference in the number of stages, then an update is performed. The specific update operation includes: setting d = dis, and making the subNode pointer point to the node t.

[0050] If the attribute value of the dependency relationship between two nodes is not "no dependency", then the splitting is not performed, and the next node is judged until all nodes in the TDG table dependency graph G of the resource-intensive program to be split have been traversed.

[0051] Step 2.2.4: After the traversal is completed, determine whether the two subgraphs obtained by splitting are the same;

[0052] If the two subgraphs are not completely identical, the two subgraphs obtained from the split will be added to the subtask set SST as two subtasks to complete the splitting of the resource-intensive program.

[0053] If the two subgraphs are completely identical, the splitting will fail, indicating that there is an unreasonable aspect to the program code.

[0054] Furthermore, the specific calculation process for the deployment scheme is as follows:

[0055] The load is balanced according to the objective function to ensure that the number of levels occupied by tasks in all the subtask sets SST is minimized, the number of switches occupied is minimized, and the resource usage of the occupied switches is balanced.

[0056] The formula for the objective function is as follows:

[0057]

[0058]

[0059] Where PS represents the set of programmable switches in the network, and j represents the j-th programmable switch in PS. Indicates whether the j-th switch in PS is in use, x i,j It is a relational variable indicating whether task i is placed on switch j;

[0060] Constraint analysis, the constraints include: task-specific constraints, switch resource constraints, and global constraints;

[0061] Task self-constraints: Task i must be placed on the switch through which its flowpath passes. The formula for task self-constraints is as follows:

[0062]

[0063] Where, path i,j Indicates whether task i flows through programmable switch j, x i,j It is a relational variable indicating whether task i is placed on switch j;

[0064] Switch resource constraints: These ensure that the resources used by a task placed on the switch are less than the switch's resources. The resources used by the task include TCAM resources, SRAM resources, and stage resources. The formula for switch resource constraints is as follows:

[0065]

[0066]

[0067]

[0068] in, This indicates the TCAM resources required for task i. This indicates the SRAM resources required by task i. x represents the stage resources required for task i. i,j It is a relational variable indicating whether task i is placed on switch j;

[0069] Global constraint: Used to ensure that the resource consumption of each switch tends to be balanced, and that the variance of the amount of resources consumed by a single switch is minimized among all switches with deployed tasks.

[0070]

[0071]

[0072] Where m represents the number of switches, and n represents the number of tasks. x represents the stage resources required for task i. i,j It is a relational variable indicating whether task i is placed on switch j;

[0073] Based on the objective function and the constraints, an integer linear programming model is established to obtain the final deployment scheme of each task in the subtask set SST in the underlying network.

[0074] A distributed data plane resource optimization system, the system comprising:

[0075] Classifier: Used to perform front-end compilation of the input program to generate a TDG table dependency graph, and to perform program resource consumption analysis and program classification;

[0076] Integrated compiler: used to execute the merging algorithm or the splitting algorithm, and to optimize the input program for task resources;

[0077] Integer linear programming model: The integer linear programming model includes the objective function and the constraints, and is used to obtain the final deployment scheme of each task in the subtask set SST in the underlying network.

[0078] A distributed data plane resource optimization system, the system comprising:

[0079] Classifier: Used to perform front-end compilation of the input program to generate a TDG table dependency graph, and to perform program resource consumption analysis and program classification;

[0080] Integrated compiler: used to execute the merging algorithm or the splitting algorithm, and to optimize the input program for task resources;

[0081] Deployment strategy module: used to obtain the final deployment scheme of each task in the subtask set SST in the underlying network.

[0082] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0083] This invention designs an integrated compiler for P4C. By classifying resource-sparse and resource-intensive programs, it proposes targeted merging and splitting algorithms to optimize program resources in a targeted manner within the integrated compiler. Specifically, the merging algorithm reduces redundancy caused by program heterogeneity, altering resource consumption at the program level; the program splitting algorithm reduces the resource granularity of the program, improving the flexibility of program resource allocation.

[0084] This invention designs a network-wide program deployment scheme. It receives data plane programs optimized by an integrated compiler and inputs them into an integer linear programming model established by an objective function and defined constraints to calculate the program deployment scheme. This integrates the resources of the entire data plane to deploy the program, which can efficiently utilize switch memory resources and better meet the needs of programmable data plane constraints. Attached Figure Description

[0085] Figure 1 This is a pipeline diagram of a programmable switch as described in this invention specification.

[0086] Figure 2 This is a flowchart of a distributed data plane resource optimization system according to an embodiment of the present invention.

[0087] Figure 3 This is a schematic diagram of the merging algorithm in the experimental example of this invention.

[0088] Figure 4 This is a schematic diagram of the adjacency matrix in the experimental example of this invention.

[0089] Figure 5 This is a schematic diagram of the merging algorithm results in the experimental example of this invention.

[0090] Figure 6This is a comparison chart showing the efficiency of merging nodes between the method proposed in this invention and the P4visor algorithm in the comparative example of this invention.

[0091] Figure 7 This is a comparative diagram showing the speed of merging node execution between the method proposed in this invention and the P4visor algorithm in the comparative example of this invention.

[0092] Figure 8 This is an example diagram of the SPEED merged node in the comparative example of this invention.

[0093] Figure 9 This is a comparative diagram showing the merging effect of the method proposed in this invention and the SPEED algorithm in the comparative example of this invention.

[0094] Figure 10 This is a variance comparison chart between the method proposed in this invention and the SPEED algorithm in the comparative example of this invention. Detailed Implementation

[0095] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0096] Example

[0097] like Figure 2 The diagram shows a flowchart of a distributed data plane resource optimization system. A distributed data plane resource optimization method is implemented by applying a distributed data plane resource optimization system.

[0098] A distributed data plane resource optimization method includes the following steps:

[0099] Step 1: Preprocess the program;

[0100] Specifically, the program is input into the classifier, and the front-end compiles it to generate a table dependency graph (TDG). The TDG is a directed acyclic graph G = (T, E). Node T represents the matching action table MAT, and E represents the edges existing between matching action tables MAT. In the TDG, the matching action table MAT is the vertex Vt, and the dependencies between tables are the edges. Therefore, for each node T, there are the following attributes: match fields, action fields, and table size; for each edge E, there are the following attributes: no dependency nop, match dependency match dep, action dependency action dep, reverse dependency reverse dep, and successor dependency successor dep.

[0101] The resources used by the program are analyzed, and the input program is divided into resource-intensive and resource-sparse programs according to whether its stage usage exceeds the fixed stage of the programmable switch. Different processing algorithms are used for different types of programs when optimizing resources.

[0102] Step 2: Optimize task resources using an integrated compiler;

[0103] Different processing methods are used for different types of programs, including merging algorithms and splitting algorithms. The merging algorithm is for multiple resource-sparse programs, and its main purpose is to reduce program redundancy; the splitting algorithm is for a single resource-intensive program, and it is used to solve programs where the resources of a single switch are insufficient to support the normal operation of the program.

[0104] Option 1, Merging Algorithm: Merging can be used to merge multiple programs. The following description uses the merging of two programs as an example.

[0105] Merging two P4 programs essentially involves merging two Directed Acyclic Graphs (DAGs) formed by the control flow graphs of the two P4 programs, combining them into a single weighted DAG. The goal of the merge is to minimize the number of nodes, merge the two DAGs, and maximize their overlap. A TDG (Directed Acyclic Graph) is a type of DAG. The specific process is as follows:

[0106] First, the adjacency matrix D is used to describe the dependencies between all n nodes in the TDG table dependency graph. If D[i,j] = 1, it means there is a dependency path between the two nodes; if D[i,j] = 0, it means there is no dependency path between the two nodes. Based on the TDG table dependency graph G of the two programs... a and G b Generate adjacency matrix D a and D b , used to describe dependencies, i.e. G a =(T a E a The corresponding adjacency matrix D a G b =(T b E b The corresponding adjacency matrix D b And the following conditions must be met: In G a and G b The nodes containing the same matching action table MAT in the merged TDG table dependency graph have the same index; the dependency graph of the merged TDG table has no loops; and G a and G b The dependencies remain unchanged.

[0107] Secondly, merge candidate sets T p , used to represent the set of all deduplicated nodes in a dependency graph of multiple TDG tables; edge set Ep The following relationship must be satisfied:

[0108]

[0109] Where i and j represent the subscripts of the matrix, i.e., the node numbers in the TDG table dependency graph;

[0110] Construct an undirected graph G p =(T p E p The weight set W and degree set D of the points together form a weighted undirected graph G. u =(T p E p ,W,D).

[0111] Then, input the undirected graph G. u The DTSingle and DTTwo algorithms are used to obtain the S1 point set. During this process, nodes are removed if: the weight of a single node is greater than the sum of the weights of all its neighbors; or if the distance between two nodes is 2 and the sum of the weights of these two nodes is greater than the sum of the weights of all their surrounding neighbors.

[0112] Next, the processed S1 is used in G. u Based on this, we obtain a weighted undirected graph G1, denoted as G1 = (S1, G... u );

[0113] Next, the nodes in the S1 point set are traversed;

[0114] ST is an array of sets, where ST[i] represents the largest weighted independent set from point 1 to point i, including point i. WT is an array, where WT[i] represents the sum of weights of the largest weighted independent sets from point 1 to point i, including point i. bestSet represents the desired optimal largest weighted independent set. bestWeight represents the sum of weights of the desired optimal largest weighted independent set.

[0115] Let the current point be u. Let the weight of the largest weighted independent set from the first point to node u and including u be the weight of WT, and let the initial value of the set ST[u] be {u}. At the same time, traverse S1 from index 1 to i. If the current node is v, and if adding point u to the set ST[v] makes the set an independent set and WT[u] larger, then update ST[u] and WT[u].

[0116] Next, iterate through S1, find the largest WT[i], and assign the corresponding WT[i] and ST[i] to bestSet and bestWeight.

[0117] Then, define the resource sharing set T. share , is used to represent a set of nodes that are repeated in the dependency graph of multiple TDG tables and can be used to merge.

[0118] Output the nodes in bestSet to complete the merge operation. Add the bestSet nodes to the shared resource set T. share Establish the TDG table dependency graph G for the merged subtasks. m =(T m E m ); where T m =T p E m According to T share Traversing G a and G b Update.

[0119] Finally, G m Add to the subtask set SST.

[0120] Solution 2, Splitting Algorithm: Splitting reduces the granularity of program placement while minimizing scheduling overhead between devices. Selecting appropriate splitting nodes and edges is crucial. This can be described as a mathematical problem to solve. For the generated TDG table dependency graph, due to the different edge attributes, the TDG table dependency graph can be converted into a weighted directed acyclic graph. Appropriate edges are selected and deleted from the graph to minimize the sum of stage occupancy in the two resulting subgraphs and minimize the difference in stage occupancy between them. This solution is applicable to multi-node splitting; a two-node splitting example is used here, with the specific steps as follows:

[0121] First, the TDG table of the task to be split depends on graph G = (T, E) as input.

[0122] Next, initialize subgraphs sub1 and sub2; initialize the pointers firstNode and subNode of the first node of the subgraph; initialize the stage number difference d, which is used to determine the actual difference in the number of stages between sub1 and sub2.

[0123] Next, for each node t in G, we need to check whether the node's dependency attribute value t.dep is no_dep. If it is, we calculate the number of stages in the two subgraphs before and after the edge. If the actual difference in the number of stages in the two subgraphs dis is less than d, we update the node. The specific update operation is d = dis and subNode points to the node t.

[0124] Finally, after the traversal is complete, if two subgraphs are not completely identical, the subgraph is added to the subtask set SST; otherwise, the splitting fails, and an error message is displayed indicating that the task code is not properly written.

[0125] Step 3: After optimizing the program at the compilation level, deploy it to the underlying network in a distributed manner using the deployment strategy module. First, describe how to place the program in the underlying network, then use an integer linear programming model to provide the implementation details of task deployment in a real-world environment.

[0126] Existing layout solutions focus on the placement of a single switch or do not consider the inherent constraints between the measurement task and the switch; or they consider task constraints but suffer from uneven resource allocation. Therefore, this invention estimates the resource usage of the measurement task, performs preprocessing of program resources, and then obtains the corresponding flow paths and mapping relationships within the network through the measurement task information, performing task layout across the entire network to optimize global resource utilization.

[0127] Step 3.1: In the task resource optimization phase of the previous step, two arrays are maintained simultaneously. These arrays record the mapping relationship between tasks and subtasks, the corresponding flow path information, and resource usage. This ensures that the original dependencies can be maintained after task resource optimization. Next, the program will be deployed on switches across the entire network based on task constraints, switch resource constraints, and global constraints.

[0128] Step 3.2: Given the obtained set of subtasks SST, model it as an integer linear programming problem (ILP).

[0129] Establish an integer linear programming model based on the objective function and constraints:

[0130] Step 3.2.1: Switches have various types of interdependent resources, and load balancing must also be considered. In an extreme case, deploying all tasks on a single switch is clearly undesirable. Therefore, the distributed approach minimizes the number of stages occupied by all tasks, the number of switches used, and the resource allocation across the switches. The objective function is constructed as shown in formulas (2) and (3):

[0131]

[0132]

[0133] Where PS represents the set of programmable switches in the network, and j represents the j-th programmable switch in PS. Indicates whether the j-th switch in PS is in use, x i,j Whether the task is placed on switch j.

[0134] Step 3.2.2: Constraints:

[0135] (1) Task self-constraints: Task i must be placed on the switch through which its flowpath passes. The task self-constraints are shown in formula (4):

[0136]

[0137] Where, path i.j Indicates whether task i flows through switch j;

[0138] (2) Switch resource constraints: The resources used by a task placed on the switch must be less than the switch's resources (including TCAM, SRAM, and stage usage). This indicates the TCAM resources required for task i. This indicates the SRAM resources required by task i. This represents the stage resources required for task i. The switch resource constraints are shown in formulas (5), (6), and (7):

[0139]

[0140]

[0141]

[0142] (3) Global Constraints: The resource consumption of each switch tends to be balanced. Among all the switches with deployed tasks, the variance of the resource consumption of a single switch is minimized. The global constraints are shown in formulas (8) and (9):

[0143]

[0144]

[0145] Step 3.2.3: Based on the above steps, establish an integer linear programming model, input the subtask set SST into the integer linear programming model to calculate the final deployment scheme of each program.

[0146] Experimental Example

[0147] For the given three measurement tasks P4 program a, b, and c, the specific steps are as follows:

[0148] Preprocessing the program: Compile the program into a BMV2 front-end in the classifier to generate a TDG table dependency graph;

[0149] The program resource assessment classification shows that the resource assessment of programs a and b is less than the switch's own stage, while the resource assessment of program c is greater than the switch's stage.

[0150] In the integrated compiler of this invention, task resource optimization is performed:

[0151] Option 1: Based on the results of the previous step, process program a and program b using a merging algorithm, such as... Figure 3 As shown, based on the TDG table dependency graph of the two programs, G a and G b Generate adjacency matrix D a and D b Adjacency matrix D is used to represent dependencies. Given a TDG table dependency graph, the dependencies of all involved nodes are represented by an adjacency matrix D, where D[i,j]=1 indicates that there is a dependency path between the two nodes; otherwise, there is no dependency path.

[0152] That is, G a =(T a E a The corresponding adjacency matrix D a G b =(T b E b The corresponding adjacency matrix D b The following conditions must be met: In G a and G b In the MAT table, identical matching actions are represented by the same index; the dependency graph of the merged TDG table is free of loops; and G a and G b The dependencies remain unchanged.

[0153] Define the candidate set T for merging p , including G a and G b Define the set of edges E, which contains all unique nodes in the set. p The relationship shown in formula (1) must be satisfied:

[0154]

[0155] Where i and j represent the subscripts of the matrix, i.e., the node numbers in the TDG table dependency graph;

[0156] Construct an undirected graph G p =(T p E p ), Figure 4 For G p The adjacency matrix is ​​used. The type of edge dependency is used as the edge weight, and combined with the vertex weight set W and vertex degree set D, a weighted undirected graph G is constructed. u =(T p E p ,W,D).

[0157] Input undirected graph G uUse the DTSingle and DTTwo algorithms to obtain the S1 point set. During this process, remove nodes that are: a single node whose weight is greater than the sum of the weights of all its neighbors; or two nodes whose distance is 2 and whose weights are greater than the sum of the weights of all their surrounding neighbors.

[0158] The processed S1 point set is used in the weighted undirected graph G u Based on this, we obtain a weighted undirected graph G1, denoted as G1 = (S1, G... u ).

[0159] ST is an array of sets, where ST[i] represents the largest weighted independent set in the set from point 1 to point i, including point i. WT is an array, where WT[i] represents the sum of weights of the largest weighted independent set in the set from point 1 to point i, including point i. bestSet represents the largest weighted independent set to be found. bestWeight represents the sum of weights of the largest weighted independent set to be found.

[0160] Iterate through the nodes in S1, and let the current node be u. Set the weight of the largest weighted independent set from the first node to node u, including u, and initialize WT to the weight of u. Initialize the set ST[u] to {u}. Simultaneously, iterate through S1 from index 1 to i, with the current node being v. If adding point u to set ST[v] results in an independent set and WT[u] being larger, then update ST[u] and WT[u].

[0161] Iterate through S1, find the largest WT[i], and assign the corresponding WT[i] and ST[i] to bestSet and bestWeight. In this experimental example, bestSet is {0}.

[0162] Output the nodes of bestSet to complete the merge operation. Add the bestSet nodes to the shared resource set T. share Establish the merged graph G m =(T m E m ), where T m =T p E m According to T share To traverse G a and G b This will update and create the TDG table dependency graph for the merged tasks. The merged result will look like this. Figure 5 As shown:

[0163] G m Add to the subtask set SST.

[0164] Option 2: Split program C.

[0165] The TDG table dependency graph of the task to be split, with G = (T, E) as input.

[0166] Initialize the subgraph sets sub1 and sub2; initialize the pointers firstNode and subNode of the first node of the subgraph; initialize the difference in number d to the stage of the graph.

[0167] For each node t in G, we need to check whether the node's dependency attribute value t.dep is no_dep. If it is, we calculate the number of stages in the two subgraphs before and after the edge. If the difference in the number of stages in the two subgraphs is less than d, we update the node. The specific update operation is d = dis and subNode points to the node t.

[0168] After the traversal is complete, if the two subgraphs obtained by program c are not completely identical, then the subgraphs are added to the subtask set SST.

[0169] By merging or splitting the scheme to process multiple programs, a subtask set SST is finally generated. Based on the objective function and constraint function set in the deployment strategy module, the subtask set SST is passed into an integer linear programming model, along with the topology of the underlying network, to calculate the final deployment scheme.

[0170] Comparative Example

[0171] Under the same merging effect, the method proposed in this invention can complete the merging operation in less time than P4visor, improving speed by two orders of magnitude. Figure 6 , Figure 7 As shown in the figure, when the efficiency of merging nodes is similar, the execution speed of the method proposed in this invention is much faster than that of P4visor. This is because P4visor uses a heuristic algorithm, which is more time-consuming in node selection.

[0172] An example of a SPEED merge node is as follows: Figure 3 As shown, Figure 3 (a) is and Figure 3 (b) are two TDG graphs, where each node is a one-to-one correspondence and can be merged. Figure 3 The topological order of (a) is 1324, while Figure 3 (b) The topological order is 4321 or 4231. If SPEED takes 4321 as... Figure 3If we follow the topological order of (b), then the number of merged nodes is 1, but the maximum number of merged nodes should be 2. In this example, there are two nodes of equal status (BC), so the sequence should consider 2! cases instead of just a single case. Similarly, if there are n nodes of equal status, then n! cases should be considered, and finding the optimal solution would require an additional O(n!) time complexity.

[0173] In comparison with SPEED in terms of the number of merged nodes, this invention demonstrates superior performance, reducing the number of merged nodes by 58.64% compared to SPEED. It also shows better results in reducing program redundancy. The comparison results are as follows: Figure 9 As shown.

[0174] In a network-wide deployment, the resource utilization rate of this invention is higher than that of similar deployment schemes. Using the variance of switch resource usage in the global network as an evaluation metric, the method proposed in this invention is compared with SPEED, revealing that the variance of the proposed method is 5 times smaller than the other methods, effectively achieving resource balance. This demonstrates that the method proposed in this invention can achieve balanced resource usage in distributed deployments. The variance comparison results are as follows... Figure 10 As shown.

[0175] This invention has used specific examples to illustrate its principles and implementation schemes. The above description of the embodiments is only for the purpose of helping to understand the method and core ideas of this invention. Furthermore, those skilled in the art will recognize that, based on the ideas of this invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for distributed data plane resource optimization, the method comprising: The steps include the following: Step 1: Compile the input program at the front end to generate a TDG table dependency graph, and add the TDG table dependency graph to the original task set; at the same time, divide the program into resource-intensive and resource-sparse types. The resource-intensive program is the input program that occupies more stage resources than the fixed stage of the programmable switch, and the resource-sparse program is the input program that occupies less stage resources than the fixed stage of the programmable switch. Step 2, optimize the input program's task resources according to the program type, specifically including: The resource-sparse program is used with a merging algorithm: multiple resource-sparse programs are merged pairwise, and the merged subtasks are obtained by minimizing the number of nodes. The merged subtasks are then added to the subtask set SST. The resource-intensive program is split using a splitting algorithm: by deleting edges in the TDG table dependency graph of the resource-intensive program, the split subtasks with the minimum stage occupancy and the minimum difference between the stages occupied by each subtask are obtained, and the split subtasks are added to the subtask set SST. Step 3: Obtain the final deployment scheme of each task in the subtask set SST described in Step 2 on the underlying network; The specific calculation process for the deployment scheme is as follows: The load is balanced according to the objective function to ensure that the number of levels occupied by tasks in all the subtask sets SST is minimized, the number of switches occupied is minimized, and the resource usage of the occupied switches is balanced. The formula for the objective function is as follows: where PS denotes the set of programmable switches in the network, denotes the i-th switch in PS, denotes the i-th switch in PS, is a relation variable denoting whether a task is placed on a switch or not. Constraint analysis, the constraints include: task-specific constraints, switch resource constraints, and global constraints; Task self constraint: task Must be placed on the switch through which the flowpath flows, the task self constraint formula is as follows: wherein, represents a task whether it flows through a programmable switch , is a relationship variable that represents a task whether it is placed on a switch ; Switch resource constraints: These ensure that the resources used by a task placed on the switch are less than the switch's resources. The resources used by the task include TCAM resources, SRAM resources, and stage resources. The formula for switch resource constraints is as follows: wherein, represents a task TCAM resources, represents a task SRAM resources, represents a task stage resources, is a relationship variable representing a task whether placed on a switch on a switch; Global constraint: Used to ensure that the resource consumption of each switch tends to be balanced, and that the variance of the amount of resources consumed by a single switch is minimized among all switches with deployed tasks. where m represents the number of switches, n represents the number of tasks, represents a task the stage resources needed, represents a task whether placed on the switch a relationship variable; Based on the objective function and the constraints, an integer linear programming model is established to obtain the final deployment scheme of each task in the subtask set SST in the underlying network.

2. The method of claim 1, wherein, The original task set and the subtask set SST are maintained to preserve the mapping relationship between the input program and the programs in the subtask set SST, the corresponding flow path information, and the resource usage, so as to ensure that the original dependency relationship can still be maintained after task resource optimization.

3. The method of claim 1, wherein, The TDG table dependency graph is specifically as follows: The TDG table dependency graph is a directed acyclic graph, denoted as: ; wherein, TDG table dependency graph, a node of the graph a match action table MAT, an edge of the graph a dependency relationship between match action tables MAT; each node attributes of the match action table MAT represented include: match field, action field, size of the table; each edge attribute values of the dependency relationship represented include: no dependency, match dependency, behavior dependency, reverse dependency, and replacement dependency.

4. The method of claim 3, wherein, The merging algorithm specifically includes the following sub-steps: Step 2.1.1, Adjacency Matrix For describing the dependency between all n nodes in a TDG table dependency graph, if denotes that there is a dependency path between two nodes; if denotes that there is no dependency path between two nodes; TDG table dependency graph of two resource-sparse programs to be merged and generate an adjacency matrix and ; and satisfy the following conditions: the same matching action table MAT in different TDG table dependency graphs is represented by the same subscript; the merged TDG table dependency graph is loop-free; the dependency relationship between the TDG table dependency graphs remains unchanged; Step 2.1.2, merging candidate set , for representing a set of all nodes after deduplication in the plurality of TDG dependency graphs; edge set satisfy the following relationship: in, The subscript represents the matrix index, i.e., the node number in the TDG table dependency graph; According to the merge candidate set and edge set Construct an undirected graph The weight set of the junction point Degree set of points This forms a weighted undirected graph. ; Step 2.1.3, the weighted undirected graph Enter to Algorithms and The algorithm removes surrounding nodes to obtain Point set; The removed surrounding nodes include: surrounding nodes of a single node whose weight is greater than the sum of the weights of all its neighboring nodes, and surrounding nodes of the two nodes whose distance is 2 and whose weights are greater than the sum of the weights of all their neighboring nodes. Step 2.1.4, using the processed result... The point set in the weighted undirected graph Obtain a weighted undirected graph based on , represented as ); Step 2.1.5, regarding the... Traverse the nodes in the point set; definition For collection arrays, Represents the range from the first point to the second point in the set. The point and containing the first point The largest weighted independent set of points; definition It is an array. This represents the sequence from the first point to the second point in the set. The point and containing the first point The sum of the weights of the maximum weighted independent set of points; use This represents the optimal maximum weighted independent set. This represents the sum of weights of the optimal maximum weighted independent set. Let the current node be u, and let the sum of the weights of the largest weighted independent set in the set from the first node to node u, including node u, be the sum of the weights of the sets. The initial value is the weight of u. , represented as Maximum weighted independent set The initial value is {u}, denoted as ; Simultaneously traverse from index 1 to i For each node set, traverse up to the current node v. If adding node u to the maximum weighted independent set ST[v] of node v results in an independent set ST[u] and WT[u] being larger, then update the maximum weighted independent set of the current node and the sum of the weights of the maximum weighted independent set to ST[u] and WT[u]. Until the entire process is completed Point set, obtain the maximum weighted independent set The sum of the weights of the largest weighted independent set, WT[i]; Step 2.1.6, the maximum weighted independent set The weights WT[i] of the maximum weighted independent set are assigned to the bestSet and the bestWeight to obtain the optimal maximum weighted independent set and the optimal weights of the maximum weighted independent set; Step 2.1.7, Define the resource sharing set , used to represent a set of nodes that are repeated in the dependency graph of multiple TDG tables and can be used to merge; Output the nodes in the optimal maximum weighted independent set bestSet, and add the nodes to the resource-sharing set. ,renew Obtain the TDG table dependency graph of the merged subtasks. ; in , for During traversal and Updated dependencies; Step 2.1.8: Calculate the TDG table dependency graph of the merged subtasks. Add the subtask set SST to complete the merging of resource-sparse programs.

5. The distributed data plane resource optimization method according to claim 3, characterized in that, The splitting algorithm specifically includes the following sub-steps: Step 2.2.1: Input the TDG table dependency graph of the resource-intensive program to be split. ; Step 2.2.2: Initialize two subgraphs, sub1 and sub2; initialize the first node pointers, firstNode and subNode, of the two subgraphs; initialize the stage number difference, d. Step 2.2.3, for the TDG table dependency graph of the resource-intensive program to be split. Each node in Make a judgment: If the attribute value of the dependency relationship between two nodes is no dependency, calculate the number of stages in the two subgraphs before and after the edge between the two nodes; if the difference in the actual number of stages in the two subgraphs, dis, is less than the difference in number, then an update is performed. The specific update operation includes: setting d=dis, and making the subNode pointer point to the node t. If the dependency attribute value between two nodes is not "no dependency", then the split is not performed, and the process continues to check the next node until the dependency graph of the TDG table of the resource-intensive program to be split has been traversed. All nodes in; Step 2.2.4: After the traversal is completed, determine whether the two subgraphs obtained by splitting are the same; If the two subgraphs are not completely identical, the two subgraphs obtained from the split will be added as two subtasks to the subtask set SST to complete the splitting of the resource-intensive program. If the two subgraphs are completely identical, the splitting will fail, indicating that there is an unreasonable aspect to the program code.

Images