An online routing and scheduling method for mixed traffic in data center networks

Abstract

The present invention proposes an online routing and scheduling mechanism for mixed traffic in a data center network. The present invention formalizes the mixed flow scheduling problem into an online optimization problem, and realizes the path selection and rate allocation of the mixed flow by solving the optimization problem. The specific transmission mechanism is: 1) The sending host selects the transmission path and allocates the rate for the mixed flow according to the flow information and network information. 2) The switch on the transmission path allocates bandwidth to the arriving flow and saves the link congestion information in the data packet header. 3) The receiving host saves the path link congestion situation in the confirmation data packet header and sends it back to the sending host. 4) The flow with a determined path does not change its route during the transmission process, and more bandwidth is allocated to the deadline flow that is closer to the deadline and the non-deadline flow with less remaining unsent data. The present invention combines routing and scheduling, effectively reducing the transmission time of the deadline flow and the deadline miss rate of the non-deadline flow.

Description

Technical Field

[0001] The present invention belongs to the field of data center networks, and in particular to an online routing and scheduling mechanism for mixed traffic in a data center network. Background Art

[0002] Traffic scheduling is an effective way to improve the data transmission performance of data center networks. When faced with flows with different priorities and performance requirements, traffic scheduling aims to optimize the performance indicators of flows. Traditional scheduling mechanisms usually use the earliest deadline priority rule to reduce the deadline miss rate of deadline flows. For online systems that can submit requests at any time, the minimum remaining time priority rule is usually used to minimize the flow completion time of non-deadline flows. However, data center network environments are usually shared by various applications that generate both types of traffic at the same time. Therefore, the data transmitted in data center networks is often a mixture of traffic with different performance requirements. Therefore, traffic scheduling mechanisms for mixed traffic scenarios have broad application prospects.

[0003] The existing scheduling mechanism provides many ideas for optimizing the two traffic performance indicators, but does not consider the possibility of routing to improve data transmission performance. The data center network is not unblocked, and uneven routing often leads to insufficient utilization of links within the network. Therefore, the path selection within the data center network often allows the scheduling mechanism to play a greater role. Summary of the invention

[0004] The object of the present invention is to provide an online routing and scheduling mechanism for mixed traffic in a data center network.

[0005] The technical solution to achieve the purpose of the present invention is:

[0006] An online routing and scheduling mechanism for mixed traffic in a data center network, characterized in that the mechanism mainly includes the following steps:

[0007] Step 1: Formalize the mixed traffic scheduling problem as a constrained online optimization problem: for deadline flows, the performance indicator is the deadline miss rate; for non-deadline flows, the performance indicator is the flow completion time. Mixed flow scheduling should reduce the deadline miss rate of deadline flows on the one hand, and reduce the flow completion time of non-deadline flows on the other hand. Combining the above two performance indicators and the effective role of routing selection in the scheduling mechanism, an online optimization problem is constructed.

[0008] Step 2: The sending host selects the transmission path and allocates the rate for the mixed flow according to the priority level based on the flow information and network congestion information: The end host solves the optimization problem in step 1 locally, and determines the respective transmission paths and sending rates for the mixed flows arriving at the end host based on the principle that deadline flows take precedence over non-deadline flows. The flows with determined routes do not change their routes during transmission.

[0009] Step 3: The switch on the transmission path allocates link bandwidth to the flows arriving at the switch based on the principle that deadline flows take precedence over non-deadline flows, and saves the path link congestion information in the packet header: the switch allocates more bandwidth to deadline flows that are closer to the deadline and non-deadline flows with less remaining unsent data.

[0010] Step 4: The receiving host saves the link congestion status on the transmission path in the confirmation packet header and sends it back to the sending host: the flow specific information, rate information and link congestion information are transmitted between the end host and the switch through the forward data transmission and feedback mechanism.

[0011] Compared with the prior art, the present invention has the following significant advantages:

[0012] (1) The online routing and scheduling mechanism for mixed traffic in a data center network proposed in the present invention can simultaneously and well optimize the performance indicators of both traffic types in a mixed traffic environment. The mechanism allocates more bandwidth to deadline flows that are closer to their deadlines and non-deadline flows with less remaining unsent data, effectively reducing the deadline miss rate of deadline flows and the flow completion time of non-deadline flows.

[0013] (2) When constructing a scheduling mechanism in a mixed traffic scenario of a data center network, the present invention can combine route selection and traffic scheduling, select a better transmission path for traffic according to the overall network congestion situation, and make scheduling play a better role;

[0014] (3) The hybrid traffic online routing and scheduling mechanism proposed in the present invention can better adapt to the dynamic changes of traffic in the data center network and has high practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a data center network topology model.

[0016] Figure 2 It is the overall architecture diagram of the online routing and scheduling mechanism of the present invention. DETAILED DESCRIPTION

[0017] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0018] Figure 1It is a data center network topology model that includes hosts, switches, and communication links. The data center network is abstracted into an undirected graph, where hosts or switches are nodes, each edge represents a full-duplex communication link, and each link has its own link bandwidth, that is, the maximum transmission rate of the link. Figure 2 , a data center network hybrid traffic online routing and scheduling mechanism of the present invention comprises the following steps:

[0019] Step S1, formalize the hybrid traffic scheduling problem as a constrained online optimization problem:

[0020] Data center network environments are usually shared by various services that generate traffic with different performance indicators. Most user interactive services generate deadline flows with strict deadlines, while data parallel services generate non-deadline flows, which tend to reduce their transmission time. For deadline flows, the performance indicator is the deadline miss rate; for non-deadline flows, the performance indicator is the flow completion time. The hybrid traffic scheduling mechanism in data center networks should reduce the deadline miss rate of deadline flows on the one hand, and minimize the flow completion time of non-deadline flows on the other hand. Considering the two performance requirements of hybrid traffic and combining the effective role of routing selection on the scheduling mechanism, this step formalizes the hybrid flow scheduling problem into an online constraint problem for solution.

[0021] Step S101, give the objective function and constraints:

[0022]

[0023] Subject to:

[0024]

[0025]

[0026]

[0027]

[0028] Among them, F D is the set of deadline flows, F N is the set of non-deadline flows, b f (t) is the bandwidth allocated to each flow f at time t, D f (t) is the remaining deadline time of the deadline flow at time t, T f (t) is the amount of remaining unsent data in the non-deadline flow at time t, is the routing decision variable representing the kth transmission path selected for flow f, C lis the bandwidth of link l. Constraint 1a indicates that the sum of the rates of flows transmitted on link l at any time t should not be greater than the bandwidth of the link. Constraints 1b and 1c require that a flow can only choose one route for transmission. Constraint 1d indicates that the bandwidth allocated to the flow at any time t should be non-negative.

[0029] For deadline flows, their deadlines are usually known when online requests are transmitted because of their small data size. In practice, neither user interaction services nor data parallel services generate very long flows. They arrive at once, so we assume that the deadlines and sizes of all flows are known when they arrive. The earliest deadline priority rule can minimize the deadline miss rate of deadline flows, so the first term in the objective function means that when deadline flows are transmitted on a specific route, the scheduling mechanism will allocate more bandwidth to deadline flows that are closer to the deadline. For non-deadline flows, the shortest remaining processing time rule prioritizes scheduling flows with less remaining unsent data. Short flows have higher priority than long flows, which can better minimize the flow completion time. Therefore, when the second term in the objective function is for non-deadline flow scheduling, when flows are transmitted on a specific transmission path, the scheduling mechanism will prioritize allocating more bandwidth to non-deadline flows with less remaining unsent data.

[0030] Step S102, converting the objective function and constraints into a solvable linear programming problem:

[0031] In step S101, both the objective function and the constraints contain binary variables. This will make the optimization problem difficult to solve in a large-scale system, so we further relax the binary variable constraints and transform constraint 1b into the following constraints:

[0032]

[0033] But at this time, the objective function and constraints still contain two variables, b f (t) and The nonlinear relationship between variables makes this optimization problem still a difficult non-convex optimization problem. To solve this problem, a new variable is introduced replace The original problem is transformed into a solvable linear programming problem as follows:

[0034]

[0035] Subject to:

[0036]

[0037]

[0038] Step S2: The sending host selects a transmission path and allocates a rate for the mixed stream according to the priority level based on the stream information and network congestion information:

[0039] The sending host solves the linear programming problem in step S102 locally to determine the route. The value of determines the transmission path of each mixed flow. The above objective function and constraints satisfy the basic form of decomposition, and the basic idea of ​​decomposition is to decompose the original problem into multiple sub-problems that can be solved in a distributed manner, and these sub-problems are coordinated by the main problem through cost operators or direct resource allocation. Most of the existing decomposition methods can be divided into primal decomposition methods and dual decomposition methods. The former is based on the decomposition of the original problem, while the latter is based on the decomposition of the Lagrangian dual problem. When the original problem contains coupling constraints, the dual decomposition method is appropriate. After the original problem is relaxed using the Lagrangian operator, the generated main dual problem can be decoupled into multiple sub-problems. The objective function in step S102 can be seen as a function of the variables The utility function That is, the simple sum of the utilities of each flow, constraint 2a is a coupling constraint, so the optimization problem satisfies the characteristics of dual decomposition. First, the generalized Lagrangian function is constructed through the Lagrangian operator:

[0040]

[0041] Lagrangian operator λ l (t) represents the congestion level of the internal links in the network, and is defined as It is expressed as the sum of the congestion levels of all links on each specific route of the mixed flow. Therefore, the original optimization problem can be written as the following main dual problem and corresponding sub-problems:

[0042] Main question:

[0043]

[0044] Sub-questions:

[0045]

[0046] Indicates that the mixed flow is at a certain λ s The advantage of using the dual decomposition method is that after the constraints are relaxed, any type of flow can be optimized independently without affecting other flows. The combination of the optimal solutions for each flow is the global optimal solution. In this case, each sending host can optimize the flow it sends locally. The sending host solves the subproblem based on the local mixed flow set and obtains the transmission path k for each flow. Since the solution to the subproblem can be a decimal, the value of k is determined using the following rules:

[0047]

[0048] After determining the route for each flow, the sending host returns to problem (1) to determine the sending rate for each flow. Substituting the solved variables After that, problem (1) becomes a problem about the unique variable b f (t) is an easy-to-solve convex optimization problem, which can also be solved by the Lagrangian dual decomposition method. The problem can be restated as follows:

[0049]

[0050]

[0051]

[0052] Among them, F l (t) represents the set of all flows passing through link l at time t. When allocating routes and flow rates, the sender sets deadline flows over non-deadline flows. For deadline flows, the sender allocates rates from near to far according to the deadline. If a deadline flow cannot be transmitted within the deadline, the transmission of the flow is terminated; for non-deadline flows, the rate is allocated from small to large according to the amount of remaining unsent data, so as to reduce the transmission time of non-deadline flows. Whenever a new flow arrives at the sender host, the sender host will re-route and re-allocate the rates for all local mixed flows, but the flow that has been determined and sent will not change its transmission path during the transmission process.

[0053] Step S3: The switch on the transmission path allocates link bandwidth to the flow arriving at the switch and saves the path link congestion information in the data packet header:

[0054] The switch allocates the outbound link bandwidth to the flows arriving at the switch based on the specific information of the flows and the rate information assigned by the sending host. The bandwidth allocation is set to prioritize the bandwidth of flows with deadlines over flows without deadlines. The switch allocates more bandwidth to flows with deadlines closer to the deadline and flows without deadlines that have less remaining unsent data. After allocating bandwidth to a flow, the remaining link bandwidth and link congestion information are updated, and the updated congestion information is stored in the packet header and sent to the next-hop switch or receiving host on the path.

[0055] Step S4: The receiving host saves the transmission path link congestion status in the confirmation data packet header and sends it back to the sending host:

[0056] The receiving host writes the congestion information of each flow arriving at the host into the corresponding confirmation packet header and sends it back to the sending host for the next round of iterative calculation. Through the forward data transmission and feedback mechanism, the flow specific information, rate information and link congestion information are transmitted between the end host and the switch, so as to optimize the overall performance of the network.

Claims

1. A data center network mixed traffic online routing and scheduling method, characterized in that The method comprises the following steps: Step 1: Formalize the mixed traffic scheduling problem as a constrained online optimization problem: for deadline flows, the performance indicator is the deadline miss rate; for non-deadline flows, the performance indicator is the flow completion time; mixed flow scheduling should reduce the deadline miss rate of deadline flows on the one hand, and reduce the flow completion time of non-deadline flows on the other hand; comprehensively consider the deadline miss rate of deadline flows and the flow completion time of non-deadline flows, and combine the effective role of routing selection on the scheduling mechanism to construct an online optimization problem; For the formalized mixed traffic scheduling problem, the specific steps are as follows: Step S101, give the objective function and constraints: Maximize: Subject to: Among them, F D is the set of deadline flows, F N is the set of non-deadline flows, b f (t) is the bandwidth allocated to each flow f at time t, D f (t) is the remaining deadline time of the deadline flow at time t, T f (t) is the amount of remaining unsent data in the non-deadline flow at time t, is the routing decision variable representing the kth transmission path selected for flow f, C l is the bandwidth of link l; constraint 1a indicates that the sum of the rates of flows transmitted on link l at any time t should not be greater than the bandwidth of the link; constraints 1b and 1c require that a flow can only choose one route for transmission; constraint 1d indicates that the bandwidth allocated to the flow at any time t should be non-negative; Step S102, converting the objective function and constraints into the following solvable linear programming problem: Maximize: Subject to: The original objective function and constraints contain b f (t) and Two unknown variables, introducing variables After substitution, the original problem is converted into a linear programming problem; Step 2: The sending host selects the transmission path and allocates the rate for the mixed flow according to the priority level based on the flow information and network congestion information: The end host solves the optimization problem in step 1 locally, and determines the respective transmission paths and sending rates for the mixed flows arriving at the end host according to the principle that the deadline flow takes precedence over the non-deadline flow. The flow with the determined transmission path does not change its route during the transmission process; The sending host selects the flow transmission path and allocates the flow rate according to the network congestion information, and the steps are as follows: each sending host solves the optimization problem of step S102 locally, and after solving the flow transmission path, solves the original problem again to obtain the optimal allocation of the flow rate; at the sending end, set the path and rate of the deadline flow to be superior to the non-deadline flow; for the deadline flow, the rate is allocated from near to far according to the deadline, and if a deadline flow cannot be completed within the deadline, the transmission of the flow is terminated; for the non-deadline flow, the rate is allocated from small to large according to the amount of remaining unsent data, and the minimum remaining time priority rule is used to reduce the transmission time of the non-deadline flow; whenever a new flow arrives at the sending host, the end host will re-route and re-allocate the rate of all local mixed flows, but the flow that has determined the transmission path and sent will not change its transmission path during the transmission process; Step 3: The switch on the transmission path allocates link bandwidth to the flows arriving at the switch according to the principle that deadline flows take precedence over non-deadline flows, and saves the link congestion information in the packet header: the switch allocates more bandwidth to deadline flows that are closer to the deadline and non-deadline flows with less remaining unsent data; In this step, the switch on the transmission path allocates link bandwidth to the flow arriving at the switch, and saves the path link congestion information in the data packet header. The steps are as follows: the switch allocates the export link bandwidth to the flow arriving at the switch according to the specific information of the flow and the rate information allocated by the sending host; bandwidth allocation is set to prioritize the deadline flow over the non-deadline flow, and the switch allocates more bandwidth to the deadline flow that is closer to the deadline and the non-deadline flow with less remaining unsent data; after each bandwidth allocation of a flow, the link bandwidth remaining situation and link congestion information are updated, and the updated congestion information is stored in the data packet header and sent to the next hop switch or the receiving host on the path; Step 4: The receiving host saves the link congestion status on the transmission path in the confirmation packet header and sends it back to the sending host: the flow specific information, rate information and link congestion information are transmitted between the end host and the switch through the forward data transmission and feedback mechanism; In this step, the receiving host saves the transmission path link congestion status in the confirmation data packet header and sends it back to the sending host. The steps are as follows: the receiving host writes the congestion information in the transmission path of each flow arriving at the host into its corresponding confirmation data packet header and sends it back to its sending host for the next round of iterative calculation.

Images