Data flow scheduling method and device

Abstract

The present application discloses a data stream scheduling method and network device for hierarchical scheduling of data stream groups based on two dimensions: transmission time and bandwidth consumption of the data stream group within a certain time period, thereby effectively reducing the average transmission time of the data stream group and improving the efficiency of big data processing. The network device determines the transmission time of the first data stream group and the second data stream group; then, the network device determines the first scheduling priority of the first data stream group and the second data stream group based on the expected transmission time; if the first scheduling priority of the first data stream group and the second data stream group are the same, the network device determines the second scheduling priority of the first data stream group and the second data stream group based on the bandwidth consumption of the first data stream group and the second data stream group within a certain time period; finally, the network device determines the scheduling order between the first data stream group and the second data stream group based on the second scheduling priority of the first data stream group and the second data stream group.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data stream scheduling method and device. Background Art

[0002] With the rapid development of Internet and Internet of Things technologies, a large number of devices and applications are constantly generating data. These data are stored in data centers, and big data analysis and processing of these data has become a current research hotspot.

[0003] Typically, a large data processing task transmits a large amount of data over the network. Among them, it is generally necessary to process many parallel data streams during the data transmission process, and these data streams can be abstracted into an aggregate flow (coflow, also referred to as a data flow group in this application). Since in most large data processing tasks, transmitting coflow often takes a lot of time. Therefore, the completion time of the large data processing task depends to a large extent on the time it takes for the coflow transmission to be completed.

[0004] Data centers often execute multiple big data processing tasks simultaneously. The goal of big data processing is to complete all of them in the shortest possible time. Currently, improving big data processing efficiency is primarily achieved by reducing the average coflow transmission time. However, existing data flow scheduling methods are not able to effectively reduce the average coflow transmission time, resulting in low big data processing efficiency. Summary of the Invention

[0005] The present application provides a data stream scheduling method, which determines a first scheduling priority of a data stream group based on the transmission time of the data stream group, and when there are multiple data stream groups with the same first scheduling priority, determines the second scheduling priority of the multiple data stream groups by comparing the bandwidth consumption of the multiple data stream groups; by hierarchically scheduling data stream groups based on the two dimensions of transmission time and bandwidth consumption of the data stream groups, data stream groups with short transmission time and small bandwidth consumption can be prioritized for scheduling, shortening the waiting time for transmission of most data stream groups, thereby effectively reducing the average transmission time of the data stream groups and improving the efficiency of big data processing.

[0006] The first aspect of the present application provides a data flow scheduling method. A network device determines a first transmission time of a first data flow group and a second transmission time of a second data flow group, wherein the first data flow group and the second data flow group can specifically be two different coflows to be scheduled, the first data flow group includes one or more first data flows, and the second data flow group includes one or more second data flows. Then, the network device determines a first scheduling priority of the first data flow group and a first scheduling priority of the second data flow group based on the first transmission time and the second transmission time. For example, the data flow with a smaller transmission time has a higher scheduling priority. If the first scheduling priority of the first data flow group and the first scheduling priority of the second data flow are the same, the network device determines a second scheduling priority of the first data flow and a second scheduling priority of the second data flow based on the first bandwidth consumption of the first data flow group and the second bandwidth consumption of the second data flow group, wherein the first bandwidth consumption refers to the bandwidth consumed by transmitting the first data flow group within a preset time period, for example, the bandwidth consumed by transmitting the first data flow group per unit time, and the second bandwidth consumption is the bandwidth consumed by transmitting the second data flow group within the preset time period. Finally, the network device determines a scheduling order between the first data flow group and the second data flow group based on the second scheduling priority of the first data flow group and the second scheduling priority of the second data flow group. For example, the scheduling order of the data flow with a higher second scheduling priority is first, and the scheduling order of the data flow with a lower second scheduling priority is later.

[0007] In the present application, the first scheduling priority of the data stream group is determined according to the transmission time of the data stream group, and when there are multiple data stream groups with the same first scheduling priority, the second scheduling priority of the multiple data stream groups is determined by comparing the bandwidth consumption between the multiple data stream groups; by hierarchically scheduling the data stream groups from the two dimensions of transmission time and bandwidth consumption of the data stream group, the data stream groups with short transmission time and small bandwidth consumption can be scheduled first, shortening the waiting time for most data stream groups to be transmitted, thereby effectively reducing the average transmission time of the data stream group and improving the efficiency of big data processing. Optionally, before the network device determines the second scheduling priorities corresponding to the first data stream group and the second data stream group respectively, the network device can first determine the first bandwidth consumption of the first data stream group and the second bandwidth consumption of the second data stream group.

[0008] Optionally, the network device determines the first bandwidth consumption of the first data stream group and the second bandwidth consumption of the second data stream group, including: the network device determines that the total amount of data transmitted by the first transmission port group within a preset time period is the first bandwidth consumption, and the first transmission port group is the input port group corresponding to the first data stream group or the output port group corresponding to the first data stream group, that is, the total amount of data flowing into or out of the network device by the first data stream group during the preset time period can be determined as the first bandwidth consumption; the network device determines that the total amount of data transmitted by the second transmission port group within the preset time period is the second bandwidth consumption, and the second transmission port group is the input port group corresponding to the second data stream group or the output port group corresponding to the second data stream group, that is, the total amount of data flowing into or out of the network device by the second data stream group during the preset time period can be determined as the second bandwidth consumption.

[0009] Optionally, the first transmission time includes a first expected transmission time, and the second transmission time includes a second expected transmission time; determining the first transmission time of the first data stream group and the second transmission time of the second data stream group includes: determining a third transmission port group corresponding to the first data stream group and a fourth transmission port group corresponding to the second data stream group, the third transmission port group including N transmission ports for transmitting the first data stream group, the fourth transmission port group including M transmission ports for transmitting the second data stream group, the N being greater than or equal to 1, and the M being greater than or equal to 1; determining the expected transmission times corresponding to the N transmission ports in the third transmission port group and the expected transmission times corresponding to the M transmission ports in the fourth transmission port group, that is, respectively determining the time when the N transmission ports are expected to transmit the first data stream group, and respectively determining the time when the M transmission ports are expected to transmit the second data stream group; determining the first expected transmission time of the first data stream group according to the expected transmission times corresponding to the N transmission ports, and determining the second expected transmission time of the second data stream group according to the expected transmission times corresponding to the M transmission ports.

[0010] Optionally, determining the expected transmission time corresponding to each of the N transmission ports in the third transmission port group, and the expected transmission time corresponding to each of the M transmission ports in the fourth transmission port group, includes: determining the expected transmission time corresponding to the transmission port based on the amount of data to be transmitted of each transmission port in the N transmission ports and the bandwidth of the transmission port; determining the expected transmission time corresponding to the transmission port based on the amount of data to be transmitted corresponding to each transmission port in the M transmission ports and the bandwidth of the transmission port; wherein, the expected transmission time corresponding to a transmission port is the ratio of the amount of data to be transmitted of the transmission port to the bandwidth of the transmission port.

[0011] Optionally, determining the first estimated transmission time of the first data stream group based on the estimated transmission times corresponding to the N transmission ports, and determining the second estimated transmission time of the second data stream group based on the estimated transmission times corresponding to the M transmission ports, includes: determining the first estimated transmission time of the first data stream group based on the estimated transmission time corresponding to the transmission port with the largest estimated transmission time among the N transmission ports; and determining the second estimated transmission time of the second data stream group based on the estimated transmission time corresponding to the transmission port with the largest estimated transmission time among the M transmission port groups. For example, if transmission port 1 has the largest estimated transmission time among the N transmission ports, then the estimated transmission time corresponding to transmission port 1 can be determined as the first estimated transmission time of the first data stream group.

[0012] Optionally, the first transmission time includes a first transmitted time, and the second transmission time includes a second transmitted time; determining the first transmitted time of the first data stream group and the second transmitted time of the second data stream group includes: determining a third transmission port group corresponding to the first data stream group and a fourth transmission port group corresponding to the second data stream group, the third transmission port group including N transmission ports for transmitting the first data stream group, the fourth transmission port group including M transmission ports for transmitting the second data stream group, the N being greater than or equal to 1, and the M being greater than or equal to 1; determining the transmitted times corresponding to the N transmission ports in the third transmission port group, and the transmitted times corresponding to the transmission ports in the fourth transmission port group; determining the first transmitted time of the first data stream group according to the transmitted times corresponding to the N transmission ports, and determining the second transmitted time of the second data stream group according to the transmitted times corresponding to the M transmission ports.

[0013] Optionally, determining the transmitted time corresponding to each of the N transmission ports in the third transmission port group, and the transmitted time corresponding to each of the M transmission ports in the fourth transmission port group, includes: determining the transmitted time corresponding to the transmission port based on the amount of transmitted data of each transmission port in the N transmission ports and the bandwidth of the transmission port; determining the transmitted time corresponding to the transmission port based on the amount of transmitted data of each transmission port in the M transmission ports and the bandwidth of the transmission port; wherein, the transmitted time corresponding to a transmission port is the ratio of the amount of transmitted data of the transmission port to the second bandwidth.

[0014] Optionally, determining the first transmitted time of the first data stream group based on the transmitted time corresponding to the N transmission ports, and determining the second transmitted time of the second data stream group based on the transmitted time corresponding to the M transmission ports, includes: determining the first transmitted time of the first data stream group based on the transmitted time corresponding to the transmission port with the largest transmitted time among the N transmission ports; determining the second transmitted time of the second data stream group based on the transmitted time corresponding to the transmission port with the largest transmitted time among the M transmission ports.

[0015] Optionally, the method further includes: if the first transmission time is greater than the second transmission time, determining that the first scheduling priority of the second data stream group is higher than the first scheduling priority of the first data stream group; if the first transmission time is less than the second transmission time, determining that the first scheduling priority of the first data stream group is higher than the first scheduling priority of the second data stream group; if the first transmission time is equal to the second transmission time, determining that the first scheduling priority of the first data stream group is the same as the first scheduling priority of the second data stream group. In other words, the smaller the transmission time of a data stream group, the higher its first scheduling priority.

[0016] Optionally, the method further includes: if the first bandwidth consumption is greater than the second bandwidth consumption, determining that the second scheduling priority of the second data stream group is higher than the second scheduling priority of the first data stream group; if the first bandwidth consumption is less than the second bandwidth consumption, determining that the second scheduling priority of the first data stream group is higher than the second scheduling priority of the second data stream group; if the first bandwidth consumption is greater than the second bandwidth consumption, determining that the second scheduling priority of the second data stream group is the same as the second scheduling priority of the first data stream group. In other words, the smaller the bandwidth consumption of a data stream group, the higher its second scheduling priority.

[0017] According to a second aspect of the present application, a network device is provided. The network device includes: a processing unit; the processing unit is configured to: determine a first transmission time of a first data stream group and a second transmission time of a second data stream group, the first data stream group and the second data stream group being data stream groups to be scheduled, the first data stream group including one or more first data streams, and the second data stream group including one or more second data streams; determine a first scheduling priority of the first data stream group and a first scheduling priority of the second data stream group based on the first transmission time and the second transmission time; if the first scheduling priority of the first data stream group and the first scheduling priority of the second data stream group are the same, determine a second scheduling priority of the first data stream group and a second scheduling priority of the second data stream group based on a first bandwidth consumption of the first data stream group and a second bandwidth consumption of the second data stream group, the first bandwidth consumption being the bandwidth consumed by transmitting the first data stream group in a preset time period, and the second bandwidth consumption being the bandwidth consumed by transmitting the second data stream group in the preset time period; and determine a scheduling order between the first data stream group and the second data stream group based on the second scheduling priority of the first data stream group and the second scheduling priority of the second data stream group.

[0018] Optionally, the processing unit is further configured to determine a first bandwidth consumption of the first data stream group and a second bandwidth consumption of the second data stream group.

[0019] Optionally, the processing unit is also used to: determine the total amount of data transmitted by the first transmission port group within a preset time period as the first bandwidth consumption, the first transmission port group is the input port group corresponding to the first data stream group or the output port group corresponding to the first data stream group; determine the total amount of data transmitted by the second transmission port group within the preset time period as the second bandwidth consumption, the second transmission port group is the input port group corresponding to the second data stream group or the output port group corresponding to the second data stream group.

[0020] Optionally, the first transmission time includes a first expected transmission time, and the second transmission time includes a second expected transmission time; the processing unit is also used to: determine a third transmission port group corresponding to the first data stream group and a fourth transmission port group corresponding to the second data stream group, the third transmission port group including N transmission ports for transmitting the first data stream group, the fourth transmission port group including M transmission ports for transmitting the second data stream group, the N being greater than or equal to 1, and the M being greater than or equal to 1; determine the expected transmission times corresponding to the N transmission ports in the third transmission port group, and the expected transmission times corresponding to the M transmission ports in the fourth transmission port group; determine the first expected transmission time of the first data stream group according to the expected transmission times corresponding to the N transmission ports, and determine the second expected transmission time of the second data stream group according to the expected transmission times corresponding to the M transmission ports.

[0021] Optionally, the processing unit is further used to: determine the expected transmission time corresponding to the transmission port based on the amount of data to be transmitted of each transmission port in the N transmission ports and the bandwidth of the transmission port; determine the expected transmission time corresponding to the transmission port based on the amount of data to be transmitted of each transmission port in the M transmission ports and the bandwidth of the transmission port; wherein the expected transmission time corresponding to a transmission port is the ratio of the amount of data to be transmitted of the transmission port to the bandwidth of the transmission port.

[0022] Optionally, the processing unit is also used to: determine the first expected transmission time of the first data stream group based on the expected transmission time corresponding to the transmission port with the largest expected transmission time among the N transmission ports; determine the second expected transmission time of the second data stream group based on the expected transmission time corresponding to the transmission port with the largest expected transmission time among the M transmission port groups.

[0023] Optionally, the first transmission time includes a first transmitted time, and the second transmission time includes a second transmitted time; the processing unit is also used to: determine a third transmission port group corresponding to the first data stream group and a fourth transmission port group corresponding to the second data stream group, the third transmission port group includes N transmission ports for transmitting the first data stream group, and the fourth transmission port group includes M transmission ports for transmitting the second data stream group, wherein N is greater than or equal to 1, and M is greater than or equal to 1; determine the transmitted times corresponding to the N transmission ports in the third transmission port group, and the transmitted times corresponding to the transmission ports in the fourth transmission port group; determine the first transmitted time of the first data stream group according to the transmitted times corresponding to the N transmission ports, and determine the second transmitted time of the second data stream group according to the transmitted times corresponding to the M transmission ports.

[0024] Optionally, the processing unit is further used to: determine the transmitted time corresponding to the transmission port based on the amount of transmitted data of each transmission port in the N transmission ports and the bandwidth of the transmission port; determine the transmitted time corresponding to the transmission port based on the amount of transmitted data of each transmission port in the M transmission ports and the bandwidth of the transmission port; wherein the transmitted time corresponding to a transmission port is the ratio of the amount of transmitted data of the transmission port to the second bandwidth.

[0025] Optionally, the processing unit is also used to: determine the first transmission time of the first data stream group based on the transmission time corresponding to the transmission port with the largest transmission time among the N transmission ports; determine the second transmission time of the second data stream group based on the transmission time corresponding to the transmission port with the largest transmission time among the M transmission ports.

[0026] The third aspect of the present application provides a network device, which includes: a processor and a memory; the memory is used to store instructions; the processor is used to execute the instructions in the memory, so that the network device performs any method as described in the first aspect above.

[0027] In a fourth aspect, the present application provides a computer storage medium, which may be non-volatile; the computer storage medium stores computer-readable instructions, which, when executed by a processor, may implement any one of the methods in the first aspect.

[0028] A fifth aspect of the present application provides a computer program product comprising instructions, which, when executed on a computer, causes the computer to execute any one of the methods in the first aspect.

[0029] The embodiments of the present application can be combined with each other without conflict.

[0030] The present application provides a data stream scheduling method and network device, which determine the first scheduling priority of a data stream group based on the transmission time of the data stream group, and when there are multiple data stream groups with the same first scheduling priority, determine the second scheduling priority of the multiple data stream groups by comparing the bandwidth consumption of the multiple data stream groups; by hierarchically scheduling the data stream groups based on the two dimensions of transmission time and bandwidth consumption of the data stream groups, data stream groups with short transmission time and low bandwidth consumption can be scheduled first, shortening the waiting time for transmission of most data stream groups, thereby effectively reducing the average transmission time of the data stream groups and improving the efficiency of big data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of a data transmission process provided in an embodiment of the present application;

[0032] Figure 2 A schematic diagram of an application scenario of the data stream scheduling method provided in an embodiment of the present application;

[0033] Figure 3 A flowchart of a data stream scheduling method 300 provided in an embodiment of the present application;

[0034] Figure 4 A schematic diagram of a scheduling data flow provided in an embodiment of the present application;

[0035] Figure 5 Another schematic diagram of a scheduling data flow provided in an embodiment of the present application;

[0036] Figure 6 A schematic diagram of a scheduling data flow provided in an embodiment of the present application;

[0037] Figure 7 A schematic diagram of the structure of a network device 70 provided in an embodiment of the present application;

[0038] Figure 8 A schematic diagram of the structure of another network device 80 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of this application more clear, the embodiments of this application are described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only embodiments of a part of this application, rather than all embodiments. It is known to those skilled in the art that with the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0040] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the descriptions used in this way can be interchangeable where appropriate so that the embodiments can be implemented in a sequence other than that illustrated or described in this application. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or that are inherent to these processes, methods, products or devices. The naming or numbering of steps in this application does not mean that the steps in the method flow must be executed in the time / logical sequence indicated by the naming or numbering. The process steps that have been named or numbered can be changed in the execution order according to the technical purpose to be achieved, as long as the same or similar technical effects can be achieved. The division of units in this application is a logical division. In actual application, there may be other division methods. For example, multiple units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, and the indirect coupling or communication connection between units can be electrical or other similar forms, which are not limited in this application. Moreover, the units or sub-units described as separate components may or may not be physically separated, may or may not be physical units, or may be distributed into multiple circuit units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this application.

[0041] Currently, specialized big data processing applications are often used to process the large amounts of data stored in data centers. Hadoop, a distributed operating platform for big data applications, has been widely used as a mainstream operating platform for big data applications.

[0042] On Hadoop, multiple big data applications (called jobs) can be run at the same time, and each job can perform one or more big data processing tasks. Generally, a job contains a set of mapping calculation tasks (called map tasks) and a set of merging calculation tasks (called reduce tasks). These map tasks and reduce tasks can be distributed on multiple computing nodes (such as servers). During the running of the job, the map task runs first and generates some intermediate results after running; the intermediate results generated by the map task are transmitted to the reduce task through the network, that is, the intermediate results are the input for the reduce task to run. After all the reduce tasks are completed, a job is considered to be completed. That is to say, during the running of a job, there is a large amount of network data transmission between all map tasks and all reduce tasks. For details, please refer to Figure 1 , Figure 1 A schematic diagram of a data transmission process provided in an embodiment of the present application. Figure 1 In a job, map tasks on different compute nodes transmit data to reduce tasks on other compute nodes over the network. Generally speaking, the data flow from all map tasks to all reduce tasks in a job is called a coflow. In other words, a coflow is essentially the entire data flow used to transmit intermediate results within a job.

[0043] On big data platforms, a large number of jobs typically run simultaneously. Some higher-priority jobs monopolize platform resources by reserving them during operation. Most lower-priority jobs, on the other hand, compete for resources on the platform. These lower-priority jobs often have no priority differences. Therefore, the goal of big data processing is to complete all jobs in the shortest possible time—that is, to minimize the average completion time of these jobs. Generally speaking, job completion time consists of computation time and coflow completion time (CCT). Minimizing the average CCT is a common approach to achieving the shortest average job completion time (JCT).

[0044] In view of this, the present application provides a data stream scheduling method, which determines the first scheduling priority of a data stream group according to the transmission time of the data stream group, and when there are multiple data stream groups with the same first scheduling priority, determines the second scheduling priority of the multiple data stream groups by comparing the bandwidth consumption between the multiple data stream groups; by hierarchically scheduling the data stream groups from the two dimensions of transmission time and bandwidth consumption of the data stream groups, data stream groups with short transmission time and small bandwidth consumption can be scheduled first, shortening the waiting time for transmission for most data stream groups, thereby effectively reducing the average transmission time of the data stream groups and improving the efficiency of big data processing.

[0045] See also Figure 2 , Figure 2 Schematic diagram of application scenario of the data flow scheduling method provided in the embodiment of the present application. Figure 2 As shown in Figure 1, the architecture of this application scenario includes: compute node 1, network device, and compute node 2. Compute node 1 sends one or more coflows to compute node 2 via the network device. Upon receiving multiple coflows from compute node 1, the network device can schedule the coflows to determine the order in which they are transmitted. Generally, coflows with higher priorities are transmitted to compute node 2 first, while coflows with the same priority can be randomly selected and transmitted to compute node 2.

[0046] It should be understood that computing node 1 can be a node that performs a certain stage task in a big data processing task. After performing this stage task, computing node 1 can generate an intermediate result and send the intermediate result to computing node 2 via a network device. Computing node 2 can be a node that performs the next stage task in the big data processing task. It completes its stage task by receiving the intermediate result transmitted by the network device and using the intermediate result as input. Computing node 1 and computing node 2 can be single computing node devices or a device group including multiple computing node devices.

[0047] The network device may be a switch, router, gateway, hub, or other device capable of forwarding data streams. Furthermore, the network device may also be a server used to perform traffic forwarding under software-defined networking (SDN) technology, i.e., a server that forwards data streams through software. For ease of description, the data stream scheduling method provided in the embodiments of the present application will be described in detail below using a switch as an example.

[0048] See Figure 3 , Figure 3 A flowchart of a data stream scheduling method 300 provided in an embodiment of the present application.

[0049] In step 301, a first transmission time of a first data stream group and a second transmission time of a second data stream group are determined, where the first data stream group and the second data stream group are data stream groups to be scheduled, the first data stream group includes one or more first data streams, and the second data stream group includes one or more second data streams.

[0050] Specifically, the first data flow group and the second data flow group may be two different coflows to be scheduled, and each data flow group may include one or more data flows; for example, the first data flow group and the second data flow group may be coflows sent by the same computing node to the switch, or the first data flow group and the second data flow group may be coflows sent by different computing nodes to the switch. During operation, the switch may receive one or more coflows, and the first data flow group and the second data flow group may be any two coflows among the multiple coflows received by the switch.

[0051] The first data stream group and the second data stream group may or may not arrive at the switch at the same time. When the switch receives the first data stream group and the second data stream group, it may schedule the first data stream group and the second data stream group to determine the transmission order between the first data stream group and the second data stream group. For example, when the first data stream group and the second data stream group arrive at the switch at the same time, the switch first determines the transmission order between the first data stream group and the second data stream group, and then transmits the first data stream group or the second data stream group according to the determined transmission order. For another example, when the first data stream group arrives at the switch first and the second data stream group arrives at the switch later, the switch may first transmit the first data stream group. When the second data stream group arrives at the switch, the switch determines the transmission order between the first data stream group and the second data stream group. If the transmission order of the second data stream group is earlier than that of the first data stream group, the switch may stop transmitting the first data stream group and give priority to transmitting the second data stream group.

[0052] It should be understood that a switch typically has multiple transmission ports, through which data streams can flow into or out of the switch. Since the transmission ports on a switch typically have a fixed bandwidth, meaning that the amount of data that a particular transmission port can transmit per unit time is fixed, if the data volume of a data stream group is constant, the time it takes for that data stream to transmit through a particular transmission port on the switch is also fixed. In other words, the transmission time of a data stream group can be determined based on the data volume of the data stream group and the bandwidth of the transmission port through which the data stream group flows.

[0053] In step 302, a first scheduling priority of the first data stream group and a first scheduling priority of the second data stream group are determined according to the first transmission time and the second transmission time.

[0054] In this embodiment, the first scheduling priority corresponding to the first data stream group and the second data stream group is determined by comparing the transmission times; wherein, the data stream with the shorter expected transmission time has a higher scheduling priority.

[0055] Specifically, if the first transmission time of the first data stream group is greater than the second expected transmission time of the second data stream group, it is determined that the first scheduling priority of the second data stream group is higher than the first scheduling priority of the first data stream group, that is, the second data stream group is scheduled first; if the first expected transmission time of the first data stream group is less than the second expected transmission time of the second data stream group, it is determined that the first scheduling priority of the first data stream group is higher than the first scheduling priority of the second data stream group, that is, the first data stream group is scheduled first; if the first expected transmission time of the first data stream group is equal to the second expected transmission time of the second data stream group, it is determined that the first scheduling priority of the first data stream group is the same as the first scheduling priority of the second data stream group.

[0056] In this embodiment, the scheduling priority of the data flow is determined based on the transmission time, so that the data flow with a shorter transmission time can be transmitted first, and the waiting time for the data flow with a lower scheduling priority can be shortened as much as possible, thereby reducing the average transmission time of the data flow.

[0057] This embodiment uses the first and second data flow groups in a switch as examples to describe the process of determining the first scheduling priority of each data flow group. In actual applications, a switch may simultaneously have multiple data flow groups to be scheduled. The switch can similarly first determine the transmission time corresponding to each data flow group and then sort the data flow groups in ascending order based on their transmission time to determine the first scheduling priority of each data flow group. Specifically, the data flow group with the shorter transmission time has a higher first scheduling priority.

[0058] In step 303, if the first scheduling priority of the first data stream group is the same as the first scheduling priority of the second data stream group, the second scheduling priority of the first data stream group and the second scheduling priority of the second data stream group are determined according to the first bandwidth consumption of the first data stream group and the second bandwidth consumption of the second data stream group.

[0059] In this embodiment, when the first scheduling priority of the first data stream group and the first scheduling priority of the second data stream group are the same, the second scheduling priorities corresponding to the first data stream group and the second data stream group can be further determined based on the first bandwidth consumption of the first data stream group and the second bandwidth consumption of the second data stream group. The first bandwidth consumption refers to the bandwidth consumed (i.e., required) for transmitting the first data stream group in a preset time period, and the second bandwidth consumption refers to the bandwidth consumed (i.e., required) for transmitting the second data stream group in the preset time period.

[0060] In a possible implementation, determining the first bandwidth consumption of the first data stream group and the second bandwidth consumption of the second data stream group may specifically include: determining the total amount of data transmitted by the first transmission port group within a preset time period as the first bandwidth consumption, where the first transmission port group is the ingress port group corresponding to the first data stream group or the egress port group corresponding to the first data stream group, that is, determining the total amount of data transmitted corresponding to all ingress ports or all egress ports through which the first data stream group flows within the preset time period as the first bandwidth consumption; determining the total amount of data transmitted by the second transmission port group within the preset time period as the second bandwidth consumption, where the second transmission port group is the ingress port group corresponding to the second data stream group or the egress port group corresponding to the second data stream group, that is, determining the total amount of data transmitted corresponding to all ingress ports or all egress ports through which the second data stream group flows within the preset time period as the first bandwidth consumption.

[0061] For example, assuming that a first data stream group enters the switch through ingress port 1, ingress port 2, and ingress port 3, and that the first data stream group exits the switch through egress port 1 and egress port 2, the total amount of data transmitted by ingress port 1, ingress port 2, and ingress port 3 during a preset time period can be determined as the first bandwidth consumption; or the total amount of data transmitted by egress port 1 and egress port 2 during a preset time period can be determined as the first bandwidth consumption. Because the switch does not actually transmit data streams when determining the first bandwidth consumption and the second bandwidth consumption, the first bandwidth consumption and the second bandwidth consumption in this application are both theoretical bandwidth consumption within the preset time period.

[0062] The preset time period can be determined or set based on actual conditions. For example, the preset time period can be a unit time, that is, the first bandwidth consumption can specifically be the bandwidth consumed per unit time after the first data stream group is transmitted, and the second bandwidth consumption can specifically be the bandwidth consumed per unit time after the second data stream group is transmitted. For example, the preset time period can also be an estimated transmission time, that is, the first bandwidth consumption can specifically be the bandwidth consumed per unit time after the first data stream group is transmitted, and the second bandwidth consumption can specifically be the bandwidth consumed per unit time after the second data stream group is transmitted. In other words, the first bandwidth consumption is equal to the data volume of the first data stream group, and the second bandwidth consumption is equal to the data volume of the second data stream group.

[0063] In this embodiment, if the first bandwidth consumption is greater than the second bandwidth consumption, the second scheduling priority of the second data stream group is determined to be higher than the second scheduling priority of the first data stream group; if the first bandwidth consumption is less than the second bandwidth consumption, the second scheduling priority of the first data stream group is determined to be higher than the second scheduling priority of the second data stream group; if the first bandwidth consumption is greater than the second bandwidth consumption, the second scheduling priority of the second data stream group is determined to be the same as the second scheduling priority of the first data stream group. Simply put, the smaller the bandwidth consumption of a data stream group, the higher its second scheduling priority. That is, if the transmission time of the data stream groups is the same, the data stream with smaller bandwidth consumption is scheduled first.

[0064] It can be understood that when the first scheduling priority of the first data stream group and the second data stream group is the same, the data stream group with smaller bandwidth consumption can be transmitted first, so that the data stream group with smaller bandwidth resource consumption can be transmitted in parallel as much as possible, and the data stream group with higher bandwidth resource consumption can be executed at the end, thereby reducing the average transmission time of the data stream.

[0065] In step 304, a scheduling order between the first data flow group and the second data flow group is determined according to the second scheduling priority of the first data flow group and the second scheduling priority of the second data flow group.

[0066] In this embodiment, after determining the second scheduling priorities of the first data flow group and the second data flow group, the first data flow group and the second data flow group can be sorted according to the second scheduling priority of each data flow group. The data flow group with a higher second scheduling priority is scheduled first, and the data flow group with a lower second scheduling priority is scheduled last.

[0067] In addition, if the second scheduling priority of the first data flow group and the second data flow group is the same, one data flow group can be randomly selected between the first data flow group and the second data flow group for transmission first; if the switch can support the simultaneous transmission of the first data flow group and the second data flow group, the first data flow group and the second data flow group can also be transmitted simultaneously.

[0068] It should be noted that when the switch determines the scheduling order of multiple data flow groups, the switch prioritizes determining the scheduling order of the data flow groups based on the first scheduling priority of the data flow groups; for multiple data flow groups with the same first scheduling priority, the switch further determines the scheduling order between the multiple data flow groups based on the second scheduling priority of the multiple data flow groups, thereby obtaining the final scheduling order.

[0069] For example, for data flow groups 1, 2, 3, and 4, the first scheduling priorities corresponding to data flow groups 1, 2, 3, and 4 are represented by A1, A2, A3, and A4, respectively. Assume that data flow group 1 has the highest first scheduling priority, data flow groups 2 and 3 have the same first scheduling priority, and data flow group 4 has the lowest first scheduling priority, i.e., A1 > A2 = A3 > A4. If data flow groups 2 and 3 have the same first scheduling priority, the second scheduling priorities of data flow groups 2 and 3 are determined, respectively. Assume that the second scheduling priorities of data flow groups 2 and 3 are B2 and B3, respectively, and that the second scheduling priority of data flow group 3 is higher than the second scheduling priority of data flow group 2, i.e., B3 > B2. Therefore, data flow group 3 is scheduled before data flow group 2. Therefore, based on the first and second scheduling priorities, the scheduling order of data flow groups 1 to 4 can be determined as: data flow group 1 > data flow group 3 > data flow group 2 > data flow group 4.

[0070] In this embodiment, the first scheduling priority of the data stream group is determined based on the transmission time of the data stream group, and when there are multiple data stream groups with the same first scheduling priority, the second scheduling priority of the multiple data stream groups is determined by comparing the bandwidth consumption between the multiple data stream groups; by hierarchically scheduling the data stream groups from the two dimensions of transmission time and bandwidth consumption of the data stream groups, data stream groups with short transmission time and small bandwidth consumption can be scheduled first, shortening the waiting time for transmission for most data stream groups, thereby effectively reducing the average transmission time of the data stream groups and improving the efficiency of big data processing.

[0071] Optionally, in the above step 301, the first transmission time of the first data stream group and the second transmission time of the second data stream group may be determined in different ways.

[0072] When the data amount of the data stream group is known, determining the first transmission time of the first data stream group and the second transmission time of the second data stream group can specifically be determining the first expected transmission time of the first data stream group and the second expected transmission time of the second data stream group, wherein the first transmission time includes the first expected transmission time and the second transmission time includes the second expected transmission time.

[0073] In one possible embodiment, the corresponding expected transmission time can be determined by the amount of data corresponding to the data stream group and the bandwidth of the transmission port through which it flows. Specifically, determining the first expected transmission time of the first data stream group and the second expected transmission time of the second data stream group can include: determining a third transmission port group corresponding to the first data stream group and a fourth transmission port group corresponding to the second data stream group, the third transmission port group including N transmission ports for transmitting the first data stream group, the fourth transmission port group including M transmission ports for transmitting the second data stream group, the N being greater than or equal to 1, and the M being greater than or equal to 1; determining the expected transmission time corresponding to each of the N transmission ports in the third transmission port group and the expected transmission time corresponding to each of the M transmission ports in the fourth transmission port group; determining the first expected transmission time of the first data stream group based on the expected transmission time corresponding to each of the N transmission ports, and determining the second expected transmission time of the second data stream group based on the expected transmission time corresponding to each of the M transmission ports.

[0074] The N transmission ports included in the third transmission port group include ingress ports and egress ports, and the M transmission ports included in the fourth transmission port group also include ingress ports and egress ports. In other words, the transmission ports included in the third transmission port group and the fourth transmission port group are all transmission ports through which the first data stream group and the second data stream group flow, respectively.

[0075] Optionally, determining the expected transmission time corresponding to each of the N transmission ports in the third transmission port group, and the expected transmission time corresponding to each of the M transmission ports in the fourth transmission port group, includes: determining the expected transmission time corresponding to the transmission port based on the amount of data to be transmitted of each transmission port in the N transmission ports and the bandwidth of the transmission port; determining the expected transmission time corresponding to the transmission port based on the amount of data to be transmitted corresponding to each transmission port in the M transmission ports and the bandwidth of the transmission port; wherein, the expected transmission time corresponding to a transmission port is the ratio of the amount of data to be transmitted of the transmission port to the bandwidth of the transmission port.

[0076] It should be understood that a switch typically includes an ingress port and an egress port. Data stream groups can flow into the switch from the ingress port and out of the switch from the egress port. Since the bandwidths corresponding to the ingress and egress ports through which the data stream group flows may be different, and the amount of data flowing through the ingress and egress ports may also be different, the actual transmission time of the data stream group through the switch can be determined by the port with the largest preset transmission time among the ingress and egress ports through which the data stream group flows. For example, if the estimated transmission time of transmission port 1 is the largest among N transmission ports, the estimated transmission time corresponding to transmission port 1 can be determined as the first estimated transmission time of the first data stream group.

[0077] The switch may determine the data volume of the data flow to be scheduled in a variety of ways.

[0078] In one possible implementation, since most jobs within a data center are executed periodically, and the amount of intermediate results generated by the same job during execution is often the same, if the job corresponding to the data flow to be scheduled has already been executed, the amount of data generated by the data flow during the execution of the job can be used to determine the amount of data in the data flow to be scheduled.

[0079] In another possible implementation, because the data streams generated within the data center have a certain periodicity, the data volume of the data stream to be scheduled can be predicted using an artificial intelligence (AI) algorithm based on the data volume of the data streams generated within the data center over a period of time. Alternatively, the data volume of the data stream to be scheduled can be directly predicted based on the data volume corresponding to the data streams of the same period in history. This embodiment does not limit the specific method for predicting the data volume of the data stream.

[0080] In addition, when the amount of data of the data stream group is unknown, determining the first transmission time of the first data stream group and the second transmission time of the second data stream group can specifically be determining the first transmitted time of the first data stream group and the second transmitted time of the second data stream group, wherein the first transmission time includes the first transmitted time and the second transmission time includes the second transmitted time.

[0081] In a possible implementation, the switch may record the transmission time of the data stream in real time during the data stream transmission process, so that the switch can obtain the corresponding transmission time of the first data stream group and the second data stream group to be scheduled.

[0082] In another possible embodiment, the corresponding transmitted time can also be determined by the amount of data transmitted by the data stream group and the bandwidth of the transmission port through which it flows. Specifically, determining the first transmitted time of the first data stream group and the second transmitted time of the second data stream group can include: determining a third transmission port group corresponding to the first data stream group and a fourth transmission port group corresponding to the second data stream group, the third transmission port group including N transmission ports for transmitting the first data stream group, the fourth transmission port group including M transmission ports for transmitting the second data stream group, the N being greater than or equal to 1, and the M being greater than or equal to 1; determining the transmitted time corresponding to each of the N transmission ports in the third transmission port group and the transmitted time corresponding to each of the transmission ports in the fourth transmission port group; determining the first transmitted time of the first data stream group based on the transmitted time corresponding to each of the N transmission ports, and determining the second transmitted time of the second data stream group based on the transmitted time corresponding to each of the M transmission ports.

[0083] Specifically, the above-mentioned determination of the transmitted time corresponding to the N transmission ports in the third transmission port group and the transmitted time corresponding to the M transmission ports in the fourth transmission port group includes: determining the transmitted time corresponding to the transmission port based on the amount of transmitted data of each transmission port in the N transmission ports and the bandwidth of the transmission port; determining the transmitted time corresponding to the transmission port based on the amount of transmitted data of each transmission port in the M transmission ports and the bandwidth of the transmission port; wherein the transmitted time corresponding to a transmission port is the ratio of the amount of transmitted data of the transmission port to the second bandwidth.

[0084] It is understood that during data transmission, the switch can record the amount of data transmitted by each transmission port, thereby obtaining the amount of data transmitted by each transmission port. Specifically, for a first data flow group, the switch can obtain the amount of data transmitted corresponding to each transmission port that transmits the first data flow group. Then, based on the amount of data transmitted corresponding to the transmission port and the bandwidth of the transmission port itself, the switch can determine the time at which the transmission port has transmitted the first data flow group, thereby obtaining the elapsed transmission time.

[0085] Determining the first transmitted time of the first data stream group based on the transmitted times corresponding to the N transmission ports, and determining the second transmitted time of the second data stream group based on the transmitted times corresponding to the M transmission ports, includes: determining the first transmitted time of the first data stream group based on the transmitted time corresponding to the transmission port with the largest transmitted time among the N transmission ports; and determining the second transmitted time of the second data stream group based on the transmitted time corresponding to the transmission port with the largest transmitted time among the M transmission ports. For example, if transmission port 1 has the largest transmitted time among the N transmission ports, the transmitted time corresponding to transmission port 1 can be determined as the first transmitted time of the first data stream group. For ease of understanding, the data stream scheduling method provided in this embodiment will be described in detail below with reference to specific examples.

[0086] See Figure 4 , Figure 4 Schematic diagram of the scheduling data flow provided in the embodiment of the present application. Figure 4 As shown in the figure, there are two data flow groups to be scheduled in the switch, namely coflow1 and coflow2 (hereinafter referred to as cf1 and cf2). The data volume of cf1 and cf2 is known: cf1 has a data volume of 3 data units, and cf2 has a data volume of 4 data units. cf1 and cf2 enter the switch through ingress port 1 and ingress port 2, respectively, and the available bandwidth of ingress port 1 and ingress port 2 is 1 data unit and 4 data units, respectively. cf1 and cf2 have the same egress port, and the available bandwidth of the egress port is 2 data units.

[0087] Since the data volume of cf1 and cf2 is known, using the data stream scheduling method provided in the embodiment of the present application, the estimated transmission time corresponding to cf1 can be obtained as 3 / 1=3, and the estimated transmission time corresponding to cf2 can be obtained as 4 / 2=2, that is, the estimated transmission time of cf2 is less than the estimated transmission time of cf. In other words, it can be determined that the first scheduling priority of cf2 is higher than the first scheduling priority of cf1, and cf2 can be transmitted first.

[0088] Therefore, when cf2 is scheduled first, the average CCT is:

[0089] Avg CCT=(CCT2+CCT1) / 2=((4 / 2)+(4 / 2+2 / 2+1 / 2)) / 2=2.75

[0090] Here, CCT2 is the time required for cf2 to complete transmission, and CCT1 is the time required for cf1 to complete transmission. Specifically, because cf2 is transmitted first, it takes two time units, or 4 / 2, from the start of cf2 transmission to its completion. Meanwhile, during these two time units, two data units from cf1 arrive at the egress port, and these two data units are received by the egress port within the next time unit. Therefore, it takes one time unit for the first two data units from cf1 to be received by the egress port. During this time unit, the third data unit from cf1 arrives at the destination port. The destination port receives this third data unit in 1 / 2 of a time unit, so CCT1 is 4 / 2 + 2 / 2 + 1 / 2.

[0091] If the data stream scheduling method provided in the embodiment of the present application is not adopted, but data streams are scheduled according to the size of the data streams, for example, data streams with small data volumes are scheduled preferentially, then when cf1 is scheduled preferentially, the average CCT is specifically:

[0092] Avg CCT=(CCT1+CCT2) / 2=((3 / 1)+(3+1 / 2)) / 2=3.25

[0093] Specifically, cf1 enters the switch through ingress port 1. Only one data unit enters the switch per time unit, and this data unit reaches the egress port and is received. Simultaneously with cf1 entering the switch, cf2 enters the switch through ingress port 2 and effectively reaches the egress port. Since the egress port's receiving capacity is two data units per time unit, the egress port receives one data unit from cf1 and one data unit from cf2 in each of the first three time units. After the first three time units have elapsed, cf1 has been completely received at the egress port, resulting in CCT1 = 3 / 1. However, cf2 still has one data unit left to be received at the egress port, resulting in CCT2 = 3 + 1 / 2.

[0094] This shows that if we only focus on the overall data volume of a coflow without considering the bandwidth bottlenecks that the dataflow may encounter, the resulting average CCT will not be optimal if a coflow with a small data volume encounters a bandwidth bottleneck after passing through a congestion point. In other words, in this embodiment, by determining the scheduling order of data flows based on their transmission time, data flows with shorter transmission times can be prioritized, shortening the waiting time for other data flows and effectively reducing the average transmission time of the data flows.

[0095] See Figure 5 , Figure 5 This is another schematic diagram of the scheduling data flow provided by the embodiment of the present application. Figure 5As shown in the figure, there are three data flow groups to be scheduled in the switch: coflow3, coflow4, and coflow5 (hereinafter referred to as cf3, cf4, and cf5), with corresponding data volumes of 4 data units, 2 data units, and 2 data units, respectively. cf3 receives 2 data units from ingress port 1 and is sent to egress port 1, and also receives 2 data units from ingress port 2 and is sent to egress port 2. cf4 enters from ingress port 1 and exits from egress port 1. cf5 enters from ingress port 2 and exits from egress port 2. The available bandwidth of ingress port 1, ingress port 2, egress port 1, and ingress port 2 is 1 data unit.

[0096] First, the first level scheduling is performed, that is, the first scheduling priority of the coflow is determined according to the expected transmission time corresponding to the coflow.

[0097] Specifically, for any coflow, coflow can be expressed as cf k , the amount of data flowing into the i-th input port of the switch is The available bandwidth of the ingress port is cf k The amount of data flowing out of the i-th output port of the switch is The available bandwidth of the outbound port is Due to cf k The expected transmission time of is affected by the bottleneck bandwidth of all ports it passes through, so cf k Estimated transfer time For all coflows, you can use cfbt k Sort in ascending order, cfbt k The smaller the coflow, the higher its first scheduling priority.

[0098] Combine Figure 5 In the corresponding embodiment, the estimated transmission time corresponding to cf3 to cf5 can be obtained. Specifically, the calculation results of the estimated transmission time are shown in Table 1.

[0099] Table 1 Calculation results of estimated transmission time

[0100]

[0101] It can be seen from Table 1 that the estimated transmission times of cf3 to cf5 are the same, that is, the first scheduling priorities of cf3 to cf5 are the same.

[0102] Therefore, we can perform second-level scheduling, which determines the second scheduling priority of coflows with the same CBT based on the total bandwidth resource consumption they will incur after transmission. For example, the CBTs of cf3 and cf5 are the same. Transmitting cf3 will result in a total bandwidth resource consumption of 4; transmitting cf4 and cf5 will result in total bandwidth resource consumption of 2 and 2, respectively. Therefore, we can determine that the second scheduling priority of cf4 and cf5 is the same and higher than the second scheduling priority of cf3. In other words, cf4 and cf5 are scheduled first, and cf3 is scheduled last.

[0103] Therefore, in the scheduling order of prioritizing cf4 and cf5 and finally scheduling cf3, the average CCT is:

[0104] Avg CCT=(CCT4+CCT5+CCT3) / 3=((2 / 1)+(2 / 1)+(2+2 / 1)) / 3=8 / 3

[0105] However, if the scheduling order is to prioritize cf3 and then schedule cf4 and cf5, the average CCT is:

[0106] Avg CCT=(CCT1+CCT2+CCT3) / 3=((2 / 1)+(2+2 / 1)+(2+2 / 1)) / 3=10 / 3

[0107] Obviously, 8 / 3<10 / 3, that is, using the data stream scheduling method provided in this embodiment to schedule data streams can effectively reduce the average transmission time of the data streams.

[0108] Simply put, in this embodiment, when the expected transmission time of the data streams is the same, the scheduling order of the data streams is further determined according to the bandwidth consumption of the data streams, so that the tasks that consume less bandwidth resources within a certain period of time can be executed first as much as possible, that is, the data streams that consume less bandwidth resources are transmitted in parallel as much as possible, and the data streams that consume more bandwidth resources are transmitted at the end, thereby achieving the purpose of reducing the average transmission time of the data streams.

[0109] The above description is about scheduling of data flow groups when the data volume of the data flow groups is known; the following will describe in detail how to schedule data flow groups when the data volume of the data flow groups is unknown with reference to specific examples.

[0110] See Figure 6 , Figure 6 Schematic diagram of the scheduling data flow provided in the embodiment of the present application. Figure 6As shown in the figure, there are two data flows to be scheduled in the switch, coflow6 and coflow7 (hereinafter referred to as cf6 and cf7). cf6 arrives at time 4, enters the switch through the ingress port, and exits the switch through egress port 1. cf7 arrives at time 1, enters the switch through the ingress port, and exits the switch through egress port 2. The available bandwidth of the ingress port, egress port 1, and egress port 2 is 2 data units, 1 data unit, and 4 data units, respectively. The actual data volume of cf6 and cf7 is 6 data units and 7 data units, respectively, but the switch has no information about the data volume of cf6 and cf7.

[0111] First, the first level scheduling is performed, that is, the third scheduling priority of the coflow is determined according to the transmission time corresponding to the coflow.

[0112] Since we cannot know the data size corresponding to the coflow, we only know how much data the coflow has sent. Therefore, the data volume passing through the port is calculated using the amount of data sent. Specifically, for any coflow, the coflow can be expressed as cf k , the amount of data flowing into the i-th input port of the switch is The available bandwidth of the ingress port is cf k The amount of data flowing out of the i-th output port of the switch is The available bandwidth of the outbound port is Due to cf k The transmission time of is affected by the bottleneck bandwidth of all ports it passes through, so cf k Transferred time For all coflows, you can use cfdt k Sort in ascending order, cfdt k The smaller the coflow, the higher its third scheduling priority.

[0113] First, during the first three moments, cf6 has not yet arrived, while cf7 has sent six data units in the first three moments. At the beginning of moment 4, cf6 arrives. At this point, cfdt6 = 0 and cfdt7 = 3. Clearly, the elapsed time corresponding to cf6 is less than that corresponding to cf7. Therefore, cf6's third scheduling priority is higher than cf7's. Based on the third scheduling priority, we can see that from moment 4, cf6 sends data until the end of moment 6, and cfdt6 also increases to 3. At this point, since cfdt6 = cfdt7, second-level scheduling is performed.

[0114] Second-level scheduling is performed. This determines the fourth scheduling priority for coflows with the same cfdt based on the bandwidth consumption per unit of time after the coflow is transmitted. cf6 consumes one data unit of bandwidth per unit of time, while cf7 consumes two data units. Therefore, the fourth scheduling priority for cf6 is higher than that for cf7. Therefore, at the beginning of time 7, the switch continues to send data for one time unit from cf6 until the end of time 7.

[0115] Starting at time 8, because cfdt6 > cfdt7, the switch sends cf7. After 0.5 time (at time 8.5), cf7 is completely sent. The switch then sends the remaining two data units of cf6. Finally, at time 10.5, both cf6 and cf7 are completely sent.

[0116] Specifically, combined Figure 6 Based on the above analysis, we can obtain the scheduling basis calculation results corresponding to cf6 and cf7 at each time, as shown in Table 2.

[0117] Table 2 Scheduling basis calculation results

[0118]

[0119]

[0120] In order to implement the above embodiment, the present application also provides a network device 70. Figure 7 , Figure 7A schematic diagram of the structure of a network device 70 provided in an embodiment of the present application. The network device 70 includes a processing unit 701. The processing unit is configured to: determine a first transmission time of a first data stream group and a second transmission time of a second data stream group, where the first data stream group and the second data stream group are data stream groups to be scheduled, the first data stream group including one or more first data streams, and the second data stream group including one or more second data streams; determine a first scheduling priority of the first data stream group and a first scheduling priority of the second data stream group based on the first transmission time and the second transmission time; if the first scheduling priority of the first data stream group and the first scheduling priority of the second data stream group are the same, determine a second scheduling priority of the first data stream group and a second scheduling priority of the second data stream group based on a first bandwidth consumption of the first data stream group and a second bandwidth consumption of the second data stream group, where the first bandwidth consumption is the bandwidth consumed by transmitting the first data stream group during a preset time period, and the second bandwidth consumption is the bandwidth consumed by transmitting the second data stream group during the preset time period; and determine a scheduling order between the first data stream group and the second data stream group based on the second scheduling priority of the first data stream group and the second scheduling priority of the second data stream group.

[0121] Optionally, the processing unit is further configured to determine a first bandwidth consumption of the first data stream group and a second bandwidth consumption of the second data stream group.

[0122] Optionally, the processing unit is also used to: determine the total amount of data transmitted by the first transmission port group within a preset time period as the first bandwidth consumption, the first transmission port group is the input port group corresponding to the first data stream group or the output port group corresponding to the first data stream group; determine the total amount of data transmitted by the second transmission port group within the preset time period as the second bandwidth consumption, the second transmission port group is the input port group corresponding to the second data stream group or the output port group corresponding to the second data stream group.

[0123] Optionally, the first transmission time includes a first expected transmission time, and the second transmission time includes a second expected transmission time; the processing unit is also used to: determine a third transmission port group corresponding to the first data stream group and a fourth transmission port group corresponding to the second data stream group, the third transmission port group including N transmission ports for transmitting the first data stream group, the fourth transmission port group including M transmission ports for transmitting the second data stream group, the N being greater than or equal to 1, and the M being greater than or equal to 1; determine the expected transmission times corresponding to the N transmission ports in the third transmission port group, and the expected transmission times corresponding to the M transmission ports in the fourth transmission port group; determine the first expected transmission time of the first data stream group according to the expected transmission times corresponding to the N transmission ports, and determine the second expected transmission time of the second data stream group according to the expected transmission times corresponding to the M transmission ports.

[0124] Optionally, the processing unit is further used to: determine the expected transmission time corresponding to the transmission port based on the amount of data to be transmitted of each transmission port in the N transmission ports and the bandwidth of the transmission port; determine the expected transmission time corresponding to the transmission port based on the amount of data to be transmitted of each transmission port in the M transmission ports and the bandwidth of the transmission port; wherein the expected transmission time corresponding to a transmission port is the ratio of the amount of data to be transmitted of the transmission port to the bandwidth of the transmission port.

[0125] Optionally, the processing unit is also used to: determine the first expected transmission time of the first data stream group based on the expected transmission time corresponding to the transmission port with the largest expected transmission time among the N transmission ports; determine the second expected transmission time of the second data stream group based on the expected transmission time corresponding to the transmission port with the largest expected transmission time among the M transmission port groups.

[0126] Optionally, the first transmission time includes a first transmitted time, and the second transmission time includes a second transmitted time; the processing unit is also used to: determine a third transmission port group corresponding to the first data stream group and a fourth transmission port group corresponding to the second data stream group, the third transmission port group includes N transmission ports for transmitting the first data stream group, and the fourth transmission port group includes M transmission ports for transmitting the second data stream group, wherein N is greater than or equal to 1, and M is greater than or equal to 1; determine the transmitted times corresponding to the N transmission ports in the third transmission port group, and the transmitted times corresponding to the transmission ports in the fourth transmission port group; determine the first transmitted time of the first data stream group according to the transmitted times corresponding to the N transmission ports, and determine the second transmitted time of the second data stream group according to the transmitted times corresponding to the M transmission ports.

[0127] Optionally, the processing unit is further used to: determine the transmitted time corresponding to the transmission port based on the amount of transmitted data of each transmission port in the N transmission ports and the bandwidth of the transmission port; determine the transmitted time corresponding to the transmission port based on the amount of transmitted data of each transmission port in the M transmission ports and the bandwidth of the transmission port; wherein the transmitted time corresponding to a transmission port is the ratio of the amount of transmitted data of the transmission port to the second bandwidth.

[0128] Optionally, the processing unit is also used to: determine the first transmission time of the first data stream group based on the transmission time corresponding to the transmission port with the largest transmission time among the N transmission ports; determine the second transmission time of the second data stream group based on the transmission time corresponding to the transmission port with the largest transmission time among the M transmission ports.

[0129] The network device in the embodiment of the present application may have some units (or devices) implemented by hardware circuits and other units (or devices) implemented by software. It is also possible that all units (or devices) are implemented by hardware circuits, and it is also possible that all units (or devices) are implemented by software.

[0130] Figure 8 A structural diagram of another network device 80 provided in an embodiment of the present application is shown as follows: Figure 8As shown, the network device 80 includes a processor 801 and a memory 802. The memory 802 can be independent of the processor and located outside the network device 80 (Memory #3), or located within the processor 801 (Memory #2), or located outside the memory 801 but within the network device 80 (Memory #1). The memory 802 can be a physically independent unit, or it can be a storage space on a cloud server or a network hard drive.

[0131] The memory 802 is used to store computer-readable instructions (or computer program codes).

[0132] The processor 801 is configured to read computer-readable instructions stored in the memory 802 to implement the functions of the network device in the aforementioned embodiments.

[0133] Optionally, the memory 802 (Memory# 1 ) is located within the network device 80 .

[0134] Optionally, the memory 802 (Memory#2) is integrated with the processor.

[0135] Optionally, the memory 802 (Memory# 3 ) is located outside the network device 80 .

[0136] Optionally, the network device 80 further includes a communication interface 803 for receiving and sending data.

[0137] Optionally, the processor 801 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. A processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like. Furthermore, the memory 802 may include: volatile memory, such as random-access memory (RAM); non-volatile memory, such as flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), cloud storage, network attached storage, network drives, and the like; and may include a combination of the aforementioned types of memory or any other medium or product with storage functionality.

[0138] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0139] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0140] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0141] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0142] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk.

Claims

1. A data flow scheduling method, characterized in that: include: Determining a first transmission time of a first data stream group and a second transmission time of a second data stream group, where the first data stream group and the second data stream group are data stream groups to be scheduled, the first data stream group includes one or more first data streams, and the second data stream group includes one or more second data streams; Determine a first scheduling priority of the first data stream group and a first scheduling priority of the second data stream group according to the first transmission time and the second transmission time; If the first scheduling priority of the first data stream group and the first scheduling priority of the second data stream group are the same, the second scheduling priority of the first data stream group and the second scheduling priority of the second data stream group are determined according to the first bandwidth consumption of the first data stream group and the second bandwidth consumption of the second data stream group, the first bandwidth consumption is the bandwidth consumed by transmitting the first data stream group in a preset time period, the second bandwidth consumption is the bandwidth consumed by transmitting the second data stream group in the preset time period, the first bandwidth consumption is inversely proportional to the second scheduling priority of the first data stream group, and the second bandwidth consumption is inversely proportional to the second scheduling priority of the second data stream group; A scheduling order between the first data stream group and the second data stream group is determined according to the second scheduling priority of the first data stream group and the second scheduling priority of the second data stream group.

2. The data flow scheduling method according to claim 1, characterized in that: The method further comprises: A first bandwidth consumption of the first data flow group and a second bandwidth consumption of the second data flow group are determined.

3. The data flow scheduling method according to claim 2, characterized in that: The determining a first bandwidth consumption of the first data stream group and a second bandwidth consumption of the second data stream group includes: Determining a total amount of data transmitted by a first transmission port group within a preset time period as the first bandwidth consumption, where the first transmission port group is an ingress port group corresponding to the first data stream group or an egress port group corresponding to the first data stream group; The total amount of data transmitted by the second transmission port group within the preset time period is determined as the second bandwidth consumption, and the second transmission port group is the ingress port group corresponding to the second data flow group or the egress port group corresponding to the second data flow group.

4. The data flow scheduling method according to any one of claims 1 to 3, characterized in that: The first transmission time includes a first expected transmission time, and the second transmission time includes a second expected transmission time; Determining a first transmission time of the first data stream group and a second transmission time of the second data stream group includes: Determine a third transmission port group corresponding to the first data stream group and a fourth transmission port group corresponding to the second data stream group, the third transmission port group including N transmission ports for transmitting the first data stream group, the fourth transmission port group including M transmission ports for transmitting the second data stream group, where N is greater than or equal to 1, and M is greater than or equal to 1; Determine the estimated transmission times corresponding to the N transmission ports in the third transmission port group and the estimated transmission times corresponding to the M transmission ports in the fourth transmission port group; A first estimated transmission time of the first data stream group is determined according to the estimated transmission times corresponding to the N transmission ports, and a second estimated transmission time of the second data stream group is determined according to the estimated transmission times corresponding to the M transmission ports.

5. The data flow scheduling method according to claim 4, characterized in that: The determining the estimated transmission times respectively corresponding to the N transmission ports in the third transmission port group and the estimated transmission times respectively corresponding to the M transmission ports in the fourth transmission port group includes: Determining an estimated transmission time corresponding to each transmission port according to the amount of data to be transmitted of each transmission port among the N transmission ports and the bandwidth of the transmission port; Determining an estimated transmission time corresponding to each transmission port according to an amount of data to be transmitted corresponding to each transmission port of the M transmission ports and a bandwidth of the transmission port; The estimated transmission time corresponding to a transmission port is the ratio of the amount of data to be transmitted of the transmission port to the bandwidth of the transmission port.

6. The data flow scheduling method according to claim 4, characterized in that: The determining a first estimated transmission time of the first data stream group according to the estimated transmission times corresponding to the N transmission ports, and determining a second estimated transmission time of the second data stream group according to the estimated transmission times corresponding to the M transmission ports, comprises: determining a first estimated transmission time of the first data stream group according to the estimated transmission time corresponding to the transmission port with the largest estimated transmission time among the N transmission ports; The second estimated transmission time of the second data stream group is determined according to the estimated transmission time corresponding to the transmission port with the largest estimated transmission time in the M transmission port groups.

7. The data flow scheduling method according to any one of claims 1 to 3, characterized in that: The first transmission time includes a first transmitted time, and the second transmission time includes a second transmitted time; The determining of a first transmitted time of the first data stream group and a second transmitted time of the second data stream group comprises: Determine a third transmission port group corresponding to the first data stream group and a fourth transmission port group corresponding to the second data stream group, the third transmission port group including N transmission ports for transmitting the first data stream group, the fourth transmission port group including M transmission ports for transmitting the second data stream group, where N is greater than or equal to 1, and M is greater than or equal to 1; Determining the elapsed transmission time corresponding to each of the N transmission ports in the third transmission port group and the elapsed transmission time corresponding to each of the transmission ports in the fourth transmission port group; A first transmitted time of the first data stream group is determined according to the transmitted times corresponding to the N transmission ports, and a second transmitted time of the second data stream group is determined according to the transmitted times corresponding to the M transmission ports.

8. The data flow scheduling method according to claim 7, characterized in that: The determining the elapsed transmission times respectively corresponding to the N transmission ports in the third transmission port group and the elapsed transmission times respectively corresponding to the M transmission ports in the fourth transmission port group includes: Determining an elapsed transmission time corresponding to each transmission port according to an amount of transmitted data of each transmission port among the N transmission ports and a bandwidth of the transmission port; Determining an elapsed transmission time corresponding to each transmission port according to an amount of transmitted data of each transmission port among the M transmission ports and a bandwidth of the transmission port; The transmitted time corresponding to a transmission port is a ratio of the amount of data transmitted by the transmission port to the second bandwidth.

9. The data flow scheduling method according to claim 7, characterized in that: The determining a first transmitted time of the first data stream group according to the transmitted times corresponding to the N transmission ports, and determining a second transmitted time of the second data stream group according to the transmitted times corresponding to the M transmission ports, comprises: determining a first elapsed transmission time of the first data stream group according to the elapsed transmission time corresponding to the transmission port with the largest elapsed transmission time among the N transmission ports; The second elapsed transmission time of the second data stream group is determined according to the elapsed transmission time corresponding to the transmission port with the largest elapsed transmission time among the M transmission ports.

10. A network device, characterized in that: include: A processing unit; the processing unit is used to: Determining a first transmission time of a first data stream group and a second transmission time of a second data stream group, where the first data stream group and the second data stream group are data stream groups to be scheduled, the first data stream group includes one or more first data streams, and the second data stream group includes one or more second data streams; Determine a first scheduling priority of the first data stream group and a first scheduling priority of the second data stream group according to the first transmission time and the second transmission time; If the first scheduling priority of the first data stream group and the first scheduling priority of the second data stream group are the same, the second scheduling priority of the first data stream group and the second scheduling priority of the second data stream group are determined according to the first bandwidth consumption of the first data stream group and the second bandwidth consumption of the second data stream group, the first bandwidth consumption is the bandwidth consumed by transmitting the first data stream group in a preset time period, the second bandwidth consumption is the bandwidth consumed by transmitting the second data stream group in the preset time period, the first bandwidth consumption is inversely proportional to the second scheduling priority of the first data stream group, and the second bandwidth consumption is inversely proportional to the second scheduling priority of the second data stream group; A scheduling order between the first data stream group and the second data stream group is determined according to the second scheduling priority of the first data stream group and the second scheduling priority of the second data stream group.

11. The network device according to claim 10, wherein: The processing unit is further configured to: A first bandwidth consumption of the first data flow group and a second bandwidth consumption of the second data flow group are determined.

12. The network device according to claim 11, wherein: The processing unit is further configured to: Determining a total amount of data transmitted by a first transmission port group within a preset time period as the first bandwidth consumption, where the first transmission port group is an ingress port group corresponding to the first data stream group or an egress port group corresponding to the first data stream group; The total amount of data transmitted by the second transmission port group within the preset time period is determined as the second bandwidth consumption, and the second transmission port group is the ingress port group corresponding to the second data flow group or the egress port group corresponding to the second data flow group.

13. The network device according to any one of claims 10 to 12, characterized in that: The first transmission time includes a first expected transmission time, and the second transmission time includes a second expected transmission time; The processing unit is further configured to: Determine a third transmission port group corresponding to the first data stream group and a fourth transmission port group corresponding to the second data stream group, the third transmission port group including N transmission ports for transmitting the first data stream group, the fourth transmission port group including M transmission ports for transmitting the second data stream group, where N is greater than or equal to 1, and M is greater than or equal to 1; Determine the estimated transmission times corresponding to the N transmission ports in the third transmission port group and the estimated transmission times corresponding to the M transmission ports in the fourth transmission port group; A first estimated transmission time of the first data stream group is determined according to the estimated transmission times corresponding to the N transmission ports, and a second estimated transmission time of the second data stream group is determined according to the estimated transmission times corresponding to the M transmission ports.

14. The network device according to claim 13, wherein: The processing unit is further configured to: Determining an estimated transmission time corresponding to each transmission port according to the amount of data to be transmitted of each transmission port among the N transmission ports and the bandwidth of the transmission port; Determining an estimated transmission time corresponding to each transmission port according to an amount of data to be transmitted corresponding to each transmission port of the M transmission ports and a bandwidth of the transmission port; The estimated transmission time corresponding to a transmission port is the ratio of the amount of data to be transmitted of the transmission port to the bandwidth of the transmission port.

15. The network device according to claim 13, wherein: The processing unit is further configured to: determining a first estimated transmission time of the first data stream group according to the estimated transmission time corresponding to the transmission port with the largest estimated transmission time among the N transmission ports; The second estimated transmission time of the second data stream group is determined according to the estimated transmission time corresponding to the transmission port with the largest estimated transmission time in the M transmission port groups.

16. The network device according to any one of claims 10 to 12, characterized in that: The first transmission time includes a first transmitted time, and the second transmission time includes a second transmitted time; The processing unit is further configured to: Determine a third transmission port group corresponding to the first data stream group and a fourth transmission port group corresponding to the second data stream group, the third transmission port group including N transmission ports for transmitting the first data stream group, the fourth transmission port group including M transmission ports for transmitting the second data stream group, where N is greater than or equal to 1, and M is greater than or equal to 1; Determining the elapsed transmission time corresponding to each of the N transmission ports in the third transmission port group and the elapsed transmission time corresponding to each of the transmission ports in the fourth transmission port group; A first transmitted time of the first data stream group is determined according to the transmitted times corresponding to the N transmission ports, and a second transmitted time of the second data stream group is determined according to the transmitted times corresponding to the M transmission ports.

17. The network device according to claim 16, wherein: The processing unit is further configured to: Determining an elapsed transmission time corresponding to each transmission port according to an amount of transmitted data of each transmission port among the N transmission ports and a bandwidth of the transmission port; Determining an elapsed transmission time corresponding to each transmission port according to an amount of transmitted data of each transmission port among the M transmission ports and a bandwidth of the transmission port; The transmitted time corresponding to a transmission port is a ratio of the amount of data transmitted by the transmission port to the second bandwidth.

18. The network device according to claim 16, wherein: The processing unit is further configured to: determining a first elapsed transmission time of the first data stream group according to the elapsed transmission time corresponding to the transmission port with the largest elapsed transmission time among the N transmission ports; The second elapsed transmission time of the second data stream group is determined according to the elapsed transmission time corresponding to the transmission port with the largest elapsed transmission time among the M transmission ports.

Images