Multi-subflow parallel forwarding scheduling method for new intelligent computing center network

Abstract

The application discloses a multi-subflow parallel forwarding scheduling method for a new intelligent computing center network. The whole communication forwarding process is divided into three stages of data flow cutting distribution, parallel transmission and receiving end sorting and recombination. The cooperation of the sending end data flow cutting distribution strategy and the receiving end out-of-order data packet sorting and recombination operation is comprehensively considered, the best shunt ratio and the corresponding optimal forwarding path are determined, and the parallel transmission of the multi-subflow is completed on the physical network. The application solves the problem that the traditional network scheduling method is invalid due to the new characteristics of the AI large model distributed training business data flow, the small number of flow and the large bandwidth of single flow in the new intelligent computing center.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer network data transmission, and in particular relates to a forwarding scheduling method based on multi-subflow parallel transmission for super large bandwidth service data flow in a new intelligent computing center network. BACKGROUND

[0002] A new intelligent computing center (NICC) for AI large model distributed training is a brand new data center that deploys a high-performance distributed computing cluster and is dedicated to artificial intelligence services, providing powerful computing power and storage resources. Traditional data centers perform computing tasks through CPUs, while the NICC network is mainly used to carry AI large model training services, and the GPU computing power has higher computing performance than the CPU. In the AI large model distributed training scenario, various parallel modes such as data parallelism, model parallelism, and tensor parallelism all require the computing nodes in the cluster to perform a large number of collective communication operations to synchronize the computing results to other nodes after completing the current computing task, in order to perform the next round of computation. Complex collective communication will generate a very large amount of communication data, significantly increasing the bandwidth demand for network forwarding, and the NICC network is facing unprecedented challenges.

[0003] As the scale of the new intelligent computing center continues to expand, the NICC network will develop towards the direction of integration of multiple network modalities, such as MTN, InfiniBand, and RoCE. Different network modalities are heterogeneous, and there are significant differences in communication performance and characteristics. The completion time of communication is determined by the completion time of the slowest node, and the coexistence of multiple network modalities is more likely to cause long tail latency problems, slowing down the task completion time. In the AI large model distributed training process, the communication time occupies a large proportion, and the services provided by the network cannot match the speed of high-performance computing resources, which will cause a large amount of computing resources to be in an idle waiting state, directly affecting the parameter update and new round of iteration calculation of the large model training. The network is becoming a bottleneck that limits the efficiency and effectiveness of large model training.

[0004] Unlike traditional data center traffic, the traffic of new intelligent computing center has the characteristics of few flows and large bandwidth per flow. To solve the elephant flow scheduling problem, the solution of single flow and single path scheduling can be summarized as follows: first, divide the elephant flow from the data flow, and then consider the available load and elephant flow distribution of the path to reroute the elephant flow. The above scheme indeed reduces the influence of elephant flow on network load balancing in traditional data center, but in the face of the business needs of new intelligent computing center with few flows and super large flows, the hash routing with flow granularity may lead to uneven network link utilization, causing network congestion and significantly increasing network forwarding delay. Another type of scheduling scheme is to split the elephant flow and use multi-path parallel transmission to complete the communication forwarding of the elephant flow. The existing multi-path scheduling and forwarding scheme based on packet as the operation unit can take advantage of multi-path parallel transmission to effectively shorten the transmission delay, but it will cause serious packet out-of-order problem and introduce additional sorting and reorganization delay. The out-of-order problem is difficult to solve, and the effect of multi-path parallel scheduling according to the packet may not be as good as the single path scheme.

[0005] In view of the gap between the network and business needs of new intelligent computing center, the multi-path parallel transmission scheduling scheme can make full use of the rich parallel path resources in the network of new intelligent computing center, but the existing elephant flow multi-path parallel scheduling method mainly has the following problems to be solved:

[0006] 1. Elephant flow usually contains a large amount of data, which needs to be split and transmitted in parallel through multiple paths. The selection of elephant flow splitting granularity has a significant impact on the efficiency of multi-path parallel transmission. If the splitting granularity is too large, it may not be able to fully utilize the parallel transmission capability of multiple paths, and it is more likely to cause congestion in the network during transmission. If the splitting granularity is too small, it may cause serious out-of-order problem, increase the complexity of data processing at the receiving end, and produce a large amount of additional sorting and reorganization overhead, reducing the transmission efficiency. Therefore, the selection of elephant flow splitting granularity is a key problem, and an explicit scheme is urgently needed to split the elephant flow.

[0007] 2. When elephant flow is transmitted in parallel through multiple paths, due to the differences in performance such as delay and bandwidth of multiple paths, data may take longer to reach the receiving end on some paths, resulting in long tail delay problem and slowing down the overall transmission efficiency. In addition, in the environment with large path differences, inappropriate path selection will cause a sharp increase in out-of-order data, exacerbate the overhead problem of out-of-order processing, and cause the performance of multi-path parallel transmission to not increase but decrease. In the scenario of coexistence of multiple network modalities in new intelligent computing center, how to select the optimal transmission path for the split elephant flow to reduce long tail delay and alleviate data out-of-order problem, the existing scheme does not support.

[0008] 3. The current multi-path parallel scheduling method focuses on avoiding disorder as much as possible, but the existing scheme cannot completely avoid the situation of packet disorder. Therefore, it is important to find the scheduling scheme that minimizes the completion time of the elephant flow by comprehensively considering the division and distribution strategy of the sending end and the sorting and reorganization scheme of the receiving end, and balancing the increase and decrease of the parallel transmission delay of the elephant flow on the link and the sorting and reorganization delay of the receiving end.

[0009] In summary, based on the new intelligent computing center ultra-wide bandwidth service data flow communication scene, it is urgent to design a new multi-path parallel forwarding efficient scheduling mechanism. SUMMARY

[0010] The purpose of the application is to provide a multi-subflow parallel forwarding scheduling method for a new intelligent computing center network, which solves the problem that the existing scheduling strategy cannot meet the communication demand when the new intelligent computing center network deals with AI large model distributed training few flows and ultra-large flows.

[0011] The technical solution for achieving the purpose of the application is: a multi-subflow parallel forwarding scheduling method for a new intelligent computing center network, which performs multi-subflow parallel forwarding scheduling on the AI large model distributed training service data flow in the new intelligent computing center, comprising:

[0012] In the data flow division and distribution stage, the service data flow is first divided, the data packets are sequentially placed in the pre-set length fixed-length container for standby, and a continuous sequence number is sequentially allocated to each fixed-length container as a unique identifier; the sending end distributes the divided fixed-length container to the corresponding subflow in a round-by-round manner according to the configuration information issued by the central controller;

[0013] In the parallel transmission stage, the mapping of the configuration information to the physical network is completed, the fixed-length container is taken as the smallest operation unit for path selection, and different fixed-length containers can select different subflow paths for forwarding to realize parallel transmission with subflow granularity;

[0014] In the receiving end sorting and reorganization stage, after each round of parallel transmission is completed, the receiving end sorts and reorganizes the out-of-order fixed-length containers in the cache and submits them to the upper layer; and feeds back information to the central controller; finally, the corresponding cache space is emptied to wait for the next round of multi-subflow parallel transmission.

[0015] Further, before the current service data flow communication forwarding, the central controller integrates and issues configuration information and cache management information, specifically including:

[0016] After receiving the service request, the sending end uploads the key information contained in the request to the central controller;

[0017] The central controller acquires physical network status in real time, determines the number of available sub-streams in the network for data forwarding from the sender to the receiver and the link performance indicators corresponding to each sub-stream path, including the current available physical bandwidth and propagation delay;

[0018] The acquired physical network status information is taken as input, and the optimal shunt ratio of the current service data stream and the corresponding optimal forwarding path are found to minimize the total delay of data stream communication forwarding and maximize the utilization of network resources, so as to realize the optimal balance of parallel transmission delay and ordering and reorganization delay, and integrate the configuration information and the cache management information into configuration information and cache management information, which are issued by the central controller to the data forwarding plane to control parallel forwarding scheduling.

[0019] Further, the data stream cutting and distribution stage includes:

[0020] The sender cuts the service data stream requesting communication forwarding, and the sending node records the data packets in order into a pre-set length fixed-length container under the premise of ensuring data packet integrity, and assigns a continuous sequence number as a unique identifier to each fixed-length container in order;

[0021] After the cutting operation is completed, the sending node determines the optimal load condition of each sub-stream according to the configuration information issued by the central controller, takes the maximum number of fixed-length containers that can be accommodated by the receiving end cache as the basic index, synchronizes the transmission completion time of multiple sub-streams in each round as much as possible, calculates the total number of fixed-length containers delivered in each round, and accordingly distributes the cut fixed-length containers to the corresponding sub-streams for parallel transmission.

[0022] Further, the parallel transmission stage includes:

[0023] The cut fixed-length containers are taken as the smallest operation unit for path selection, all data packets belonging to the same fixed-length container will be scheduled to the same sub-stream path for forwarding, ensuring that the data packets in the fixed-length container arrive at the receiving end in order; different fixed-length containers are mapped to the physical network according to the configuration information issued by the central controller, and can select different paths for transmission to realize multi-sub-stream parallel transmission.

[0024] Further, the receiving end ordering and reorganization stage includes:

[0025] After receiving the buffer management information issued by the central controller, the receiving end opens an independent space for the current service data stream to classify and buffer the out-of-order fixed-length containers; the out-of-order fixed-length containers arriving at the receiving end are buffered by the receiving end, and if a fixed-length container is the first fixed-length container arriving at the receiving end and needing to be buffered on the substream, the receiving end newly creates a buffer queue in the corresponding independent space for temporarily storing the fixed-length container; otherwise, the fixed-length container is directly placed in the corresponding buffer queue for temporary storage; after waiting for the last fixed-length container of the same round to arrive and being confirmed, the receiving end uniformly performs the sorting and reorganization operation;

[0026] After each round of parallel transmission ends, the receiving end sorts the out-of-order fixed-length containers in the buffer according to the sequence number as the key, and then submits the reorganized containers to the upper layer and feeds back information to the central controller; finally, the corresponding buffer space is emptied to wait for the next round of multi-substream parallel transmission;

[0027] The central controller informs the sending end that the next round of parallel transmission can be started, until the entire service data stream is completely forwarded.

[0028] Further, the key information in the service request includes: the sending end node , the receiving end node , the elephant flow length , and the maximum communication delay .

[0029] Further, a constrained multi-objective optimization algorithm is used to balance the increase and decrease of the parallel transmission delay and the sorting and reorganization delay, to determine the best substream ratio of the current service data stream and to find the corresponding optimal forwarding path;

[0030] The constrained multi-objective optimization algorithm specifically includes: generating row vectors each composed of integers and having a sum of a fixed value as an initial population, trying different substream division and forwarding path matching schemes in the multiple selection, crossover and mutation processes, and controlling the results after multiple iterations to gradually approach the optimal balance of the multi-substream parallel transmission delay and the sorting and reorganization delay, so as to find the best substream ratio and the corresponding optimal forwarding path of the current service data stream under the condition that the total communication forwarding delay of the service data stream is minimum and the network resource utilization rate is maximum, and finally determine the optimal configuration of the number of fixed-length containers on each virtual substream path;

[0031] Further, the processing method of the constraint specifically includes: defining the violation value of each individual; The calculation method of the violation value is , where is a row vector, and satisfies , Total number of fixed-length containers after service data stream is divided, Total number of fixed-length containers that can be transmitted by each substream, is the number of available substreams in the network for the sender to forward data to the receiver; if the individual does not meet the constraint condition, the constraint violation value is the absolute value of the real constraint function value; if the individual meets the constraint condition, the constraint violation value is 0; the constraint violation value is normalized to obtain the constraint violation degree, and the value range of the constraint violation degree is ; the multi-objective optimization evolutionary algorithm distinguishes between feasible solutions and infeasible solutions according to the constraint violation degree, and the individual with a constraint violation degree of 0 corresponds to a feasible solution, and the individual with a constraint violation degree not equal to 0 corresponds to an infeasible solution.

[0032] Further, the process of dividing the service data stream requested by the sender for communication forwarding is:

[0033] Sender node First, the service request information is transferred to the central controller, and the information needs to include the data stream waiting for communication forwarding , is defined as a four-tuple , wherein and represent the sender node and the receiver node of the data stream , respectively, represents the length of the data stream, is the longest communication forwarding time that the data stream can accept;

[0034] The sender node divides the original data of the data stream before distribution, and each data packet is sequentially placed into multiple fixed-length containers according to the position order in the original data stream; when the current fixed-length container is full or the remaining space is insufficient to place a complete data packet, switching to the next fixed-length container to place the data packet; each fixed-length container is sequentially assigned a continuous sequence number as a unique identifier; the length of the fixed-length container is set in advance , satisfying the condition .

[0035] Further, the specific implementation of distributing the divided fixed-length containers to the corresponding substreams for parallel transmission is:

[0036] The total number of fixed-length containers transmitted in each round is taken as a reference index for the receiver buffer , and as much as possible is arranged for multi-substream parallel transmission of fixed-length containers according to the distribution ratio in the configuration information under the premise that the buffer can accommodate, and the total number of data packets transmitted in each round is calculated by the following formula ,

[0037]

[0038]

[0039]

[0040] wherein represents the number of split fixed-length containers transmitted per available subflow, is the number of available subflows in the network available for the sender to forward data to the receiver, is a positive integer variable;

[0041] The sending end node determines the optimal load condition of each subflow in each round according to the configuration and deployment information issued by the central controller and , and distributes the split fixed-length containers to the corresponding subflow in rounds, preparing for subsequent mapping to the physical network for multi-subflow parallel transmission.

[0042] Compared with the existing scheduling scheme, the present application has the following advantages:

[0043] (1) The present application makes full use of the rich parallel paths inside the new intelligent computing center network. These paths can provide more efficient transmission services for elephant flows after balancing, meeting the communication needs of AI large model distributed training business. Multi-path parallel transmission can fully utilize the available bandwidth in the network, effectively shorten the data forwarding time delay on the physical link, although it will introduce additional time delay for the sorting and reorganization operation at the receiving end, but by controlling the subflow distribution, the optimal solution can be found by weighing the increase and decrease of the time delay, truly playing the advantages of parallel transmission, greatly shortening the total communication forwarding time delay of elephant flow. In addition, multi-path can avoid the network resource fragmentation caused by single flow and single path, and alleviate the problem of elephant flow queuing at the sending end due to the lack of complete sufficient residual bandwidth, effectively improving the utilization rate of network bandwidth resources.

[0044] (2) The present application proposes to realize elephant flow transmission by taking subflow as the scheduling granularity. Subflow scheduling solves the problems of traditional flow scheduling granularity being too large and not flexible enough, uneven network load, long communication time, and waste of network resource fragmentation. Compared with the existing packet scheduling mode, the subflow scheduling scheme has a larger granularity, which alleviates the problem of serious packet disordering and long reorganization time in parallel transmission. Elephant flow scheduling by subflow realizes the trade-off between elephant flow parallel transmission delay on the link and the sorting and reorganization delay at the receiving end, which can effectively shorten the communication completion time of elephant flow.

[0045] (3) The application introduces the idea of multi-modal fusion of future networks, considers that the network modes of different sub-flows may be different, and there is a large difference in performance such as delay, independently analyzes and calculates the delay of each sub-flow, and is widely applicable to the multiple network architectures that may exist in the intelligent computing center now and in the future. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 is a control plane parallel forwarding scheduling process flowchart of an embodiment of the application.

[0047] Figure 2 is a data forwarding plane parallel forwarding and sorting process flowchart of an embodiment of the application.

[0048] Figure 3 is a schematic diagram of the cooperation relationship of the splitting and distribution and sorting and reorganization strategy of an embodiment of the application.

[0049] Figure 4 is a central controller multi-objective evolutionary optimization algorithm flowchart of an embodiment of the application. DETAILED DESCRIPTION

[0050] In view of the new demand characteristics of elephant flow communication of intelligent computing services, the scheduling mechanism needs to comprehensively consider the cooperation of the sending end data flow splitting and distribution and the receiving end data packet sorting and reorganization operation, find the best splitting ratio of the elephant flow and map it to the physical network to complete the parallel transmission of multiple sub-flows according to the optimal forwarding path, realize the two core goals of minimizing the elephant flow communication completion time and maximizing the network resource utilization rate. Thus, the advantages of multi-path parallel forwarding are truly played, the delay of data through network communication forwarding during AI large model distributed training is effectively reduced, and the intelligent computing resource utilization rate is improved, thereby providing important network communication service guarantee for the continuous development of AI large models.

[0051] The application takes a new intelligent computing center network as an application scenario, optimizes the scheduling and forwarding work of AI large model distributed training service data flow, and proposes an efficient scheduling mechanism for multi-sub-flow parallel forwarding, which divides the entire communication forwarding process into three stages of data flow splitting and distribution, parallel transmission and receiving end sorting and reorganization. The cooperation of the sending end data flow splitting and distribution strategy and the receiving end out-of-order data packet sorting and reorganization operation is comprehensively considered, the best elephant flow splitting ratio is determined, and the parallel transmission of multiple sub-flows is completed on the physical network.

[0052] The multi-subflow parallel forwarding scheduling method provided by the application is to solve the problem of invalidation of the traditional network scheduling method caused by the new characteristics of the AI large model distributed training business data flow, the small number of flow, and the large bandwidth of single flow in the new intelligent computing center. The application makes full use of network resources, balances network load, effectively shortens network transmission delay, and alleviates the current network bottleneck of AI large model training. In order to deepen the understanding of the application, the application will be further described in conjunction with the drawings and examples of the specification.

[0053] Embodiment one

[0054] Referring to Figure 1 and Figure 2 , the embodiment of the application provides a whole architecture of a multi-subflow parallel forwarding scheduling mechanism in a new intelligent computing center network, which can effectively reduce the communication forwarding delay of elephant flow, improve the network resource utilization rate, and better meet the communication demand of the AI large model distributed training business data flow. To some extent, it makes up for the gap between the current new intelligent computing center network and business demand, improves the efficiency and effect of large model training, and helps the sustainable development of AI industry. In addition, the scheme has the ability to adapt to the future network multi-modal fusion.

[0055] In order to improve the effective bandwidth of the new intelligent computing center network, the core idea of the embodiment of the application is to realize efficient multi-subflow parallel transmission, and for this purpose, the whole elephant flow communication forwarding process is divided into three stages: data flow cutting and distribution stage, parallel transmission stage, and receiving end sorting and recombination stage. The specific operation of each stage is as follows.

[0056] In the data flow cutting and distribution stage, the sending end node is the main work object. First, the business request information needs to be handed over to the central controller, and the information needs to contain the data flow , defined as a four-tuple . Among them, and represent the sending end node and the receiving end node of the data flow , indicates the length of the data flow, is the longest communication forwarding time that the data flow can accept. The sending end node Before distribution, the raw data of data stream F needs to be segmented. Each data packet is sequentially placed into multiple fixed-length containers of the same length according to its position in the raw data stream. During this process, if the current fixed-length container is full or there is insufficient space to place a complete data packet, the process switches to placing the data packet into the next fixed-length container. Each fixed-length container is assigned a sequential sequence number as a unique identifier. This identifier is used by the receiving end to confirm the order and integrity, so as to correctly reassemble it into the original data stream. The data flow characteristics of the large-scale model training communication service in the intelligent computing center are: a small number of streams and a large bandwidth per stream, i.e. The data volume will be large, so a fixed-length container is chosen for segmentation to avoid significant space waste. Furthermore, compared to variable-length segmentation schemes, fixed-length containers facilitate sorting and reorganization at the receiving end, reducing unnecessary overhead and shortening the overall communication forwarding time. In addition, fixed-length containers effectively avoid the polarization problem caused by variable segmentation lengths, preventing extreme situations where some data blocks are very large while others are very small, thus helping to balance network load. The length of the fixed-length container is pre-set to [value missing]. The conditions are met. Based on the configuration and deployment information issued by the central controller, the sending node determines the optimal load for each sub-stream and distributes the pre-divided fixed-length containers to the corresponding sub-streams in rounds for parallel transmission.

[0057] During the parallel transmission phase, to facilitate the analysis of the data stream Multiple available independent sub-streams To reduce the latency of parallel transmission, we can model complex network topologies, such as CLOS, FatTree, and Dragonfly, as a unified directed graph. ,in Represents the network A set of terminal nodes This represents the set of available subflows among all nodes in the network. In the above expression... Defined as From sending node to receiving node of A set of available sub-streams. Considering that future intelligent computing centers are likely to have multiple network modes coexisting, the network modes of multiple different sub-streams used for a single data stream communication may be different, resulting in significant differences in performance such as latency. This directly manifests as different sub-streams being selected to transmit the same length of data. The latency of parallel transmission The differences can be significant, so it is essential to analyze and calculate the delay of each substream independently. Due to transmission delay and propagation delay It consists of two parts, and the formula is expressed as follows:

[0058]

[0059] Transmission delay is the time required for data to be transmitted within a network device, specifically from the first bit of data being transmitted until the last bit is sent. Transmission delay is determined by the data length and transmission rate. Propagation delay refers to the time required for data to travel a certain distance in the form of electromagnetic waves through a physical link. Propagation delay is closely related to the physical length of the channel and the speed of signal propagation on the physical medium, but is independent of the data size. For a given network, sub-streams can be considered... The corresponding propagation delay fluctuates within a certain range, using the symbol... express.

[0060] During the sorting and reorganization phase at the receiving end, the main task is performed by the receiving end node. Out-of-order fixed-length containers are sorted by sequence number and reassembled before being submitted to the central controller to complete communication forwarding. Although the sending nodes distribute fixed-length containers in sequence, the use of multiple sub-streams for parallel transmission means that the network modes of each sub-stream may differ, resulting in different latency characteristics and varying real-time congestion conditions. This can lead to disordered arrival order of fixed-length containers at the receiving end. Out-of-order fixed-length containers are buffered at the receiving end, waiting for the arrival and confirmation of the last fixed-length container in the same round before being uniformly sorted and reassembled. The buffering mechanism of the receiving nodes has a significant impact on the operation of the stream reassembly phase. Considering the periodic nature of large-scale distributed training services, the data streams that need to be forwarded for business requests often burst synchronously, and the receiving nodes in the intelligent computing center network often face many-to-one communication situations. In order to clearly distinguish the fixed-length containers after being segmented from different data streams forwarded by different sending nodes, the receiving end allocates an independent space for categorized caching for the corresponding data stream after receiving the buffer management information from the central controller. The detailed caching operation is as follows: if the data packet is the first data packet to arrive at the receiving end on the substream, the receiving end creates a new cache queue for it in the corresponding independent space for temporary storage; otherwise, the data packet is directly put into the corresponding cache queue for temporary storage.

[0061] The following is combined Figure 1 and Figure 2 Detailed explanation of the specific steps in the embodiments of the present invention:

[0062] Step 1: After receiving the Elephant Stream forwarding service request, the sender will include key information from the request, such as the sending node. Receiver node Elephant Flow Length Longest communication delay and upload to the central controller.

[0063] Step 2, the central controller acquires the physical network state in real time, determines the number of available sub-streams for the sender to send to the receiver in the network, and the current available physical bandwidth, propagation delay, etc. of the link corresponding to each sub-stream path.

[0064] Step 3, taking the physical network state information acquired in Step 2 as input, the central controller calculates the optimal shunting ratio of the current service request and configures the number of load-fixed containers on the corresponding virtual path, aiming to minimize the total data flow communication forwarding delay and maximize network resource utilization.

[0065] Step 4, the sender splits the original elephant flow, assembles the data packets into fixed-length containers of a pre-set length in order under the premise of ensuring data packet integrity, and assigns a continuous sequence number as a unique identifier to each fixed-length container in order.

[0066] Step 5, the sender node performs mapping deployment of the physical network according to the configuration information issued in Step 3, and uses a split-wheel method to perform parallel transmission with sub-streams as the granularity. The total number of fixed-length containers transmitted in each round is referenced by the receiver buffer, and as long as the buffer can accommodate, as many fixed-length containers as possible are distributed according to the shunting ratio in the configuration information for multi-sub-stream parallel transmission.

[0067] Step 6, the central controller issues buffer management information to the corresponding receiver, and the receiver receives the information and opens an independent space for the data flow to buffer the out-of-order fixed-length containers.

[0068] Step 7, after each round of parallel transmission is completed, the receiver sorts and recombines the out-of-order fixed-length containers in the buffer and submits them to the upper layer. The receiver needs to feed back information to the central controller, and the central controller receives the information and notifies the sender to start the next round of parallel transmission until the entire service data flow is completely forwarded.

[0069] The specific operation of Step 2 is described in detail below. In the embodiment of the present application, is used to represent an elephant flow requesting communication forwarding, which can be defined as a four-tuple . Among them, and represent the sender node and the receiver node of the data flow, represents the length of the data flow, is the longest communication forwarding time that the data flow can accept. Assuming that​ The central controller obtains the existing network topology information in the network when receiving the communication forwarding request The sending end node can send the data streams to the receiving end node The sending end node can send the data streams to the receiving end node The sending end node can send the data streams to the receiving end node The sending end node can send the data streams to the receiving end node The sending end node can send the data streams to the receiving end node The sending end node can send the data streams to the receiving end node The sending end node can send the data streams to the receiving end node The sending end node can send the data streams to the receiving end node The sending end node can send the data streams to the receiving end node The sending end node can send the data streams to the receiving end node The sending end node can send the data streams to the receiving end node The sending end node can send the data streams to the receiving end node The sending end node can send the data streams to the receiving end node The sending end node can send the data streams to the receiving end node In the scenario where multiple network modalities coexist in the intelligent computing center, the links of these sub-streams may be constructed using different network modalities, and they may have different performances in terms of delay, bandwidth, and other performance indicators. Therefore, the corresponding propagation delays of the links from to may be quite different. In order to better fit the actual situation of the intelligent computing center network, the model uses to estimate the propagation delay of the link , which is independent of the data allocated to the link and can be obtained from the feedback information of the receiving end.

[0070] In the traditional Ethernet, scheduling and forwarding are performed in the minimum unit of flow, and the classic algorithm is the ECMP equivalent multipath routing algorithm, which hashes the data streams with the same five-tuple to the same path for transmission. It is known that the business data streams of AI large model training in the new intelligent computing center have the characteristics of few flows and large single-flow bandwidth. In this scenario, flow-based scheduling is prone to polarization, that is, some large elephant flows are concentrated and hashed to the same path. This easily causes the traffic overload of a small number of paths in the whole network, while the link bandwidth of other paths is not effectively utilized, and the network load is severely uneven. Another existing scheduling method is to select a path for each data packet in the data stream, and balance in the granularity of packets. Different data packets of the same flow can be transmitted through different paths. This packet-based scheduling method theoretically has the smallest granularity and can fully utilize the available residual bandwidth in the network for forwarding, but it will cause the problem of out-of-order arrival of data packets at the receiving end. Severe out-of-order problem will cause a significant decrease in throughput and require a large amount of additional out-of-order reorganization work to consume cache and CPU resources. The out-of-order problem is an important reason why packet-based scheduling has poor performance and is difficult to support large-scale network traffic forwarding.

[0071] To solve the above problems, a more reasonable granularity is selected for scheduling forwarding, and an embodiment of the application designs a sub-flow scheduling scheme. First, step 4 splits the elephant flow, and the specific operation is as follows: the sending end node After receiving the service request, the data packets are recorded in the pre-set length fixed container in sequence on the premise of ensuring the integrity of the data packets, and when the length of the fixed container is exceeded, the data packets are recorded in the next fixed container. The sending end will sequentially assign a continuous sequence number to each fixed container as a unique identifier, and this identifier is used to confirm the sequence and integrity of the fixed container received by the receiving end, so as to correctly reassemble the original data stream. In the parallel transmission process, the fixed container is used as the smallest operation unit for path selection, and all data packets belonging to the same fixed container are scheduled to the same sub-flow path for forwarding, and the data packets in the fixed container arrive at the receiving end in sequence. Different fixed containers can select different paths for transmission according to the configuration information issued by the central controller, realize multi-sub-flow parallel transmission, and improve the effective bandwidth of the network. Due to the difference in transmission delay of different sub-flows, the receiving end may have disorder between different fixed containers, and only the fixed container needs to be sorted and reassembled, which can reduce unnecessary overhead compared to processing packet disorder, thereby shortening the overall communication forwarding time and truly playing the advantages of multi-path parallel transmission.

[0072] Considering that if the fixed containers are disordered, buffering and waiting for sorting and reassembling are needed at the receiving end, and once the number of buffered fixed containers is too large, it may cause serious buffer congestion, affecting the overall communication forwarding performance, we perform multi-sub-flow parallel transmission in rounds, and select an appropriate total number of fixed containers for transmission in each round, so that the receiving end can process the disordered fixed containers in the buffer in time after each round, and balance the additional calculation cost and time cost caused by too many rounds. In step 5, the maximum number of fixed data packets that can be accommodated by the receiving end buffer is used as a basic index to synchronize the transmission completion time of multiple sub-flows in each round as much as possible, and the total number of data packets transmitted in each round can be calculated by the following formula :

[0073]

[0074]

[0075]

[0076] Step 6 is the operation of the application embodiment design to cache the out-of-order fixed-length containers of the sub-flow scheduling scheme at the receiving end. The out-of-order fixed-length containers arriving at the receiving end will be cached by the receiving end, waiting for the last fixed-length container of the same round to arrive and be confirmed before performing the sorting and reorganization operation. The cache mechanism of the receiving end has a great influence on the operation of the stream reorganization stage. Considering the periodic characteristics of the large model distributed training business, the data stream of the business request needs to be forwarded often synchronously bursts, and the receiving end node in the intelligent computing center network often faces the situation of many-to-one communication. In order to clearly distinguish different data streams forwarded by different sending end nodes, the receiving end opens an independent space for each data stream according to the cache management information issued by the central controller for classified caching. The detailed cache operation is: if a fixed-length container is the first fixed-length container to arrive at the receiving end and needs to be cached, the receiving end creates a new cache queue in the corresponding independent space for temporary storage; otherwise, the fixed-length container is directly placed in the corresponding cache queue for temporary storage. After each round of parallel transmission ends, the receiving end sorts and reorganizes the out-of-order fixed-length containers in the cache, and then submits them to the upper layer, and feeds back information to the central controller, and finally clears the corresponding cache space to wait for the fixed-length containers of the next round of multi-sub-flow parallel transmission.

[0077] Embodiment two

[0078] Referring to Figure 2 , the application embodiment provides a sending end splitting and distribution and receiving end sorting and reorganization strategy cooperation method in a multi-sub-flow parallel forwarding scheduling mechanism. The multi-sub-flow parallel forwarding scheduling mechanism designed by the application is to provide efficient services for the new intelligent computing center network in the AI large model distributed training business background, and reducing the data transmission delay through the network between parallel computing devices is crucial for large model distributed training. The communication delay not only relates to the efficiency and cost of large model training, but also affects the synchronization of model parameters between nodes, and the lagging update of some nodes leads to the decline of the convergence speed and final quality of the large model. Therefore, how to perform optimal sub-flow division and reasonably distribute fixed-length container loads for each sub-flow to minimize the elephant flow communication forwarding delay is the most core optimization target in the multi-sub-flow parallel forwarding scheduling mechanism.

[0079] The application embodiment designs a multi-sub-flow parallel forwarding scheduling mechanism to optimize the communication forwarding performance of the network facing super large bandwidth business data streams, and the data stream The total communication delay is composed of three parts, which are the splitting and distribution delay , the parallel transmission delay and the sorting and reorganization delay , and the formula is described as . The splitting and distribution delay is less affected by different splitting operations, and the The influence of the data packet loss rate can also be basically ignored, so the parallel forwarding scheduling mechanism of the multiple sub-streams is optimized The decisive factors are the parallel transmission delay and the sorting and recombination delay . Because the multiple sub-streams and multiple paths can fully utilize the available bandwidth in the network, balance the network load and improve the network resource utilization, the network effective bandwidth is significantly improved, and the transmission delay of the data stream is also reduced. In addition, through the analysis of the calculation formula of the transmission delay , it can be roughly considered that the reasonable parallel transmission of the same data stream by using more sub-streams can further shorten the transmission delay of the data stream. The sorting and recombination delay is the time spent by the receiving end for sorting and recombining the received possibly out-of-order data packets into a correct and complete original data stream. Qualitative analysis shows that the more the total number of sub-streams actually used in the parallel transmission of the data stream, the more serious the out-of-order problem caused by the receiving end, and the receiving end will necessarily spend more time to do more complex sorting and recombination to obtain the original data stream, that is, the value of the sorting and recombination delay is also larger. In short, the sorting and recombination delay is a variable that is positively correlated with the total number of sub-streams. In practical applications, in addition to the number of sub-streams, the delay of the sorting and recombination stage will also be affected by other factors, but the model is mainly established for exploring the optimal scheduling strategy of the multiple sub-streams parallel transmission of the super-large bandwidth data stream, and other secondary factors are assumed to be the same to simplify the model.

[0080] In summary, for different sub-stream division schemes of the same data stream, the parallel transmission delay and the sorting and recombination delay are two factors that influence and restrict each other, and jointly determine the change of the total communication delay (minimize the total communication delay) of the requested forwarding service data stream . The embodiment of the application designs an optimization algorithm in the central controller, finds the optimal balance of the parallel transmission delay and the sorting and recombination delay by trying different sub-stream division schemes, explores the cooperation of the sending end splitting and distribution and the receiving end sorting and recombination strategy, and realizes the minimum total communication completion delay of the service data stream and the network resource utilization The biggest key decision target. It should be noted that the central controller optimization algorithm designed in the embodiment of the application allows the decision result to degenerate into a special case of single-path scheduling, that is, when the optimization result output by the algorithm minimizing the task completion time corresponds to a situation in which only one sub-flow path has a load in the configuration information, the network will directly route according to the configuration information and complete the communication forwarding of the service data flow in a single-flow single-path manner.

[0081] Embodiment three

[0082] Referring to Figure 3 , Figure 4 The multi-objective optimization method for determining the optimal split ratio of the central controller of the multi-sub-flow parallel forwarding scheduling mechanism provided in the embodiment of the application can determine the optimal number of sub-flows according to the number of available sub-flows that can be used by the sender to forward data to the receiver in the physical network and the current available physical bandwidth, propagation delay and other information of the link corresponding to each sub-flow path, and reasonably allocate the load of each sub-flow, effectively shorten the communication forwarding delay of the elephant flow, and improve the network resource utilization rate of the intelligent computing center. The constraint condition limits the total number of fixed-length containers for transmission of all sub-flows to be consistent with the number of fixed-length containers after the original elephant flow is split, which can ensure that the receiver receives complete and non-repeated data. The constrained NSGA-II algorithm specifically includes the following steps:

[0083] Step 1, input the preset data and the obtained network parameters, including the length of the fixed-length container , the length of the original elephant flow to be transmitted , the number of available sub-flows , etc.

[0084] Step 2, adjust the parameter settings of the optimization algorithm, including the number of iterations, the population size, the distribution index of the crossover and mutation operations, etc.

[0085] Step 3, initialize the population, that is, randomly generate an initial population with a size of .

[0086] Step 4, calculate the constraint violation degree of the solution, and distinguish between feasible solutions and infeasible solutions according to the rules

[0087] Step 5, perform fast non-dominated sorting, and follow the solution comparison principle of distinguishing between feasible solutions and infeasible solutions under the constraint condition.

[0088] Step 6, calculate the crowding distance of each individual in each non-dominated layer according to the sorting result, and select appropriate individuals to form a new parent population according to the crowding degree comparison operator.

[0089] Step 7, obtain the child population by selecting, crossing and mutating the parent population through the genetic algorithm.

[0090] Step 8, combine the parent population with the offspring population to form a new population with a size of , and select appropriate individuals to form the next parent

[0091] Similarly, repeat steps 4 to 8 until the iteration number reaches the pre-set maximum value, draw the corresponding Pareto front at this time, and the algorithm program ends.

[0092] The following describes how to initialize the population in step 3. The individuals in the population are row vectors composed of elements, which can be represented as . Among them is the number of available substreams in the network that can be used for data forwarding from the sender to the receiver, and each element is a non-negative integer corresponding to the number of split fixed-size containers that need to be transmitted for each available substream, indicating that the available substream is not assigned a data stream transmission task. In order to ensure that the receiver receives complete and non-repeated data, the total number of fixed-size containers transferred by all available substreams must be consistent with the number of fixed-size containers after the original data stream is split . In summary, the initialization population operation can be specifically described as randomly generating row vectors composed of integers with a total value of .

[0093] In optimization problems, the handling of constraints can be achieved through various ways, among which the commonly used ones are penalty function method and distinguishing feasible solution from infeasible solution method. The penalty function method needs to construct a penalty function according to the characteristics of the constraint, add the penalty function to the objective function, convert the constrained problem into an unconstrained problem, and then solve it by using the conventional method, but the penalty function method has the inherent defect that the penalty factor is difficult to set. The distinguishing feasible solution from infeasible solution method is a processing technique based on the comparison principle. According to whether it meets the constraint condition, the feasible solution and the infeasible solution are distinguished, and the solution that meets the constraint condition is the feasible solution, and the solution that does not meet the constraint condition is defined as the infeasible solution. It can handle most constrained optimization problems on the basis of small modification of the unconstrained optimization algorithm, so that the optimal solution search is closer to the feasible region, and then quickly closer to the Pareto front.

[0094] Based on this, the invention defines the violation value of each individual in step 4. The calculation method is , wherein is a row vector that satisfies The calculation principle shows that if an individual does not meet the constraints, the constraint violation value is the absolute value of the true constraint function; if the individual meets the constraints, the constraint violation value is 0. Normalizing the constraint violation value yields the constraint violation degree, which has a range of values. The feasible and infeasible solutions are distinguished by the degree of constraint violation. Individuals with a constraint violation degree of 0 correspond to feasible solutions, while individuals with a constraint violation degree of non-zero correspond to infeasible solutions.

[0095] In step 5, in the constrained NSGA-II multi-objective optimization that distinguishes between feasible and infeasible solutions, the principle for comparing the merits of solutions in fast non-dominated sorting is improved accordingly: (1) In any case, feasible solutions are unconditionally superior to infeasible solutions. (2) The comparison between feasible solutions still follows the rules of the original unconstrained optimization algorithm. (3) The comparison between infeasible solutions follows the principle of prioritizing solutions with lower constraint violation. The specific operation of fast non-dominated sorting is as follows: each solution is assigned two parameters. and ,in Indicates domination of the first The number of solutions. Indicates being the first The number of solutions dominated by each solution. First, calculate the number of solutions by iterating through each solution. and The value of is easily obtained. The solution lies in the first non-dominated layer, and all solutions in the first non-dominated layer can be represented by a set. Representation. For sets The solution in the middle, the solution that will be dominated by it. Subtract 1. At this point, the newly appearing The solution is the individuals on the second non-dominated layer. These newly emerging... The liberation of the set By repeating the above steps, all non-dominated layer sets can be found sequentially in rank order, and each set can be assigned a non-dominated rank.

[0096] Step 6 requires defining the crowding distance. For each objective function, find the two solutions adjacent to the current solution's function value and calculate the function difference between them. The sum of all calculated differences is the crowding distance of the current solution. Specifically, for individuals located on the boundary, since there are no adjacent solutions, the crowding distance cannot be calculated using the above definition and is set to infinity by default. From the above definition, it can be seen that a smaller crowding distance indicates a more crowded environment around the solution. To ensure the uniformity of solution distribution in the approximate Pareto solution set, a crowding comparison operator is designed based on two attributes: the non-dominant rank and the crowding distance for each individual in the population. This can be expressed using a partial order relation as follows:

[0097]

[0098] That is: prefer to select the solution with low rank value, if two solutions have the same rank value, the solution with large distance value is preferred to the solution with small distance value.

[0099] The multi-subflow parallel forwarding scheduling mechanism provided by the application is for a new type of intelligent computing center network service in the AI large model distributed training business background, and reducing the time delay of data communication and forwarding between parallel computing devices through the network is crucial for large model distributed training. The central controller in the embodiment provides a method for determining the optimal shunt ratio optimization, taking minimizing the AI large model training business data flow network forwarding time delay and maximizing the intelligent computing center network resource utilization as two optimization objectives, using a constrained NSGA-II multi-objective evolutionary optimization algorithm to determine the optimal number of subflows, and reasonably distributing the load for each subflow. It is the key step to ensure the efficient operation of the multi-subflow parallel forwarding scheduling mechanism and the new type of intelligent computing center network to meet the needs of large model distributed training business.

[0100] According to the results optimized by the central controller, the optimal shunt ratio of the elephant flow of the business request is determined, and the distribution plan of the sending end is designed and mapped to the physical network to complete the parallel transmission of the multi-subflow. For a business request, the total communication time delay of the elephant flow under the multi-subflow parallel scheduling mechanism provided by the application is composed of three parts: the shunt distribution time delay , the parallel transmission time delay , and the sorting and recombination time delay . Multi-subflow parallel transmission can fully utilize the available bandwidth of the network and effectively shorten the forwarding time delay of data on the physical link. Although additional time delay of sorting and recombination operation at the receiving end is introduced, the multi-objective optimization algorithm of the central controller can control the shunt situation, weigh the increase and decrease of time delay, and find the optimal solution to minimize the elephant flow completion time. In addition, multi-subflow can avoid the network resource fragmentation caused by single flow and alleviate the problem of elephant flow queuing at the sending end due to the lack of complete and sufficient remaining available bandwidth, effectively improving the utilization rate of network resources.

[0101] The above only describes the embodiments of the application, which are exemplary in nature and are intended to illustrate the technical solutions of the application, but the protection scope of the application is not limited thereto. Any person skilled in the art can make appropriate changes or modifications within the technical scope disclosed by the application. Such changes or modifications, without departing from the purpose of the application and the claims, should be covered within the protection scope of the application.

Claims

1. A multi-substream parallel forwarding scheduling method for novel intelligent computing center networks, characterized in that: The application discloses a method for scheduling multi-subflow parallel forwarding of a business data stream in distributed training of an AI large model in a new intelligent computing center. In the data stream cutting and distributing stage, the business data stream is cut first, and data packets are sequentially placed in a pre-set length fixed-length container for standby, and a continuous sequence number is sequentially allocated to each fixed-length container as a unique identifier; the sending end distributes the cut fixed-length container to the corresponding subflow in a round-by-round manner according to the configuration information issued by the central controller; In the parallel transmission stage, the mapping of the configuration information to the physical network is completed, the fixed-length container is taken as the smallest operation unit for path selection, different fixed-length containers select different subflow paths for forwarding, and parallel transmission is realized in the granularity of subflow; In the receiving end sorting and recombining stage, after each round of parallel transmission is completed, the receiving end sorts and recombines the disordered fixed-length containers in the cache and submits them to the upper layer; information is fed back to the central controller; finally, the corresponding cache space is emptied to wait for the next round of multi-subflow parallel transmission; A constrained multi-objective optimization algorithm is used to balance the increase and decrease of parallel transmission delay and sorting and recombination delay, to decide the best shunt ratio of the current business data stream, and to find the corresponding optimal forwarding path; The constrained multi-objective optimization algorithm specifically involves: generating random objects... One reason The sum of the integers is a constant. The row vectors are used as the initial population. Different sub-stream partitioning and forwarding path matching schemes are tried in the process of multiple selection, crossover and mutation. The results after multiple iterations are controlled to continuously approach the optimal balance between the parallel transmission delay of multiple sub-streams and the sorting and reorganization delay. Under the condition that the total communication forwarding delay of the service data stream is minimized and the network resource utilization is maximized, the best split ratio and the corresponding optimal forwarding path of the current service data stream are found. Finally, the optimal configuration of the number of fixed-length load containers on each virtual sub-stream path is determined. The processing method of the constraint is specifically as follows: Definition of violation value of each individual ; The calculation method is , wherein is a row vector, satisfying , represents the total number of fixed-length containers after the traffic data stream is divided, represents the number of fixed-length containers transmitted by each available substream, is the number of available substreams in the network for data forwarding from the sender to the receiver; if the individual does not meet the constraint condition, the constraint violation value is the absolute value of the real constraint function value; if the individual meets the constraint condition, the constraint violation value is 0; the constraint violation value is normalized to obtain the constraint violation degree, and the value range of the constraint violation degree is ; the multi-objective optimization evolutionary algorithm distinguishes between feasible solutions and infeasible solutions according to the constraint violation degree, and the individual with the constraint violation degree of 0 corresponds to the feasible solution, and the individual with the constraint violation degree not being 0 corresponds to the infeasible solution.

2. The method of claim 1, wherein, Before the current business data stream communication forwarding, the central controller integrates and issues configuration information and cache management information, which specifically includes: After receiving the business request, the sending end uploads the key information contained in the request to the central controller; The central controller obtains the physical network state in real time, determines the number of available subflows in the network that can be used for data forwarding from the sending end to the receiving end, and the link performance indicators corresponding to each subflow path, including the current available physical bandwidth and propagation delay; The obtained physical network state information is taken as input, and the optimization target is to minimize the total data stream communication forwarding delay and maximize the network resource utilization rate, to find the best shunt ratio of the current business data stream and the corresponding optimal forwarding path, to realize the optimal balance of parallel transmission delay and sorting and recombination delay, and to integrate the configuration information and cache management information to be issued by the central controller to the data forwarding plane for control and parallel forwarding scheduling.

3. The method of claim 1, wherein, The data stream cutting and distributing stage includes: The sending end cuts the business data stream for request communication forwarding, records the data packets in the pre-set length fixed-length container in sequence under the premise of ensuring the integrity of the data packets, and sequentially allocates a continuous sequence number to each fixed-length container as a unique identifier; After the cutting operation is completed, the sending end node determines the optimal load of each subflow according to the configuration information issued by the central controller, takes the maximum number of fixed-length containers that can be accommodated by the receiving end cache as the basic index, synchronizes the transmission completion time of multiple subflows in each round as much as possible, calculates the total number of fixed-length containers transmitted in each round, and accordingly distributes the cut fixed-length containers to the corresponding subflow for parallel transmission in rounds.

4. The method of claim 1, wherein, The parallel transmission stage includes: The path selection is performed by taking the cut fixed-length container as the smallest operation unit, all data packets belonging to the same fixed-length container are scheduled to the same sub-flow path for forwarding, and the data packets in the fixed-length container are guaranteed to be in order when reaching the receiving end; different fixed-length containers are mapped to the physical network according to the configuration information issued by the central controller, and different paths are selected for transmission, thereby realizing the parallel transmission of multiple sub-flows.

5. The method of claim 1, wherein, The receiving end sorting and reorganization stage includes: After receiving the buffer management information issued by the central controller, the receiving end opens an independent space for the current service data stream to classify and buffer the out-of-order fixed-length containers; the out-of-order fixed-length containers reaching the receiving end are buffered by the receiving end, if a fixed-length container is the first fixed-length container reaching the receiving end and needing to be buffered on the sub-flow, the receiving end newly creates a buffer queue in the corresponding independent space for temporarily storing the fixed-length container; otherwise, the fixed-length container is directly placed into the corresponding buffer queue for temporary storage; after the last fixed-length container in the same round reaches and is confirmed, the sorting and reorganization operation is uniformly performed; After each round of parallel transmission is completed, the receiving end sorts the out-of-order fixed-length containers in the buffer according to the sequence number as the key, submits to the upper layer after reorganization, and feeds back information to the central controller; finally, the corresponding buffer space is emptied to wait for the next round of multiple sub-flow parallel transmission; The central controller informs the sending end that the next round of parallel transmission can be started, until the entire service data stream is completely forwarded.

6. The method of claim 2, wherein, The key information in the service request includes: a sending end node , a receiving end node , an elephant flow length , and a longest communication delay .

7. The method of claim 3, wherein: The process of the sending end for cutting the service data stream for request communication forwarding includes: sending end node First, the central controller to transfer service request information, information need to include the data flow waiting for communication forwarding , Defined as a four-tuple , in which, and represent the sending end node and receiving end node of the data flow , indicates the length of the data flow, is the longest communication forwarding time that the data flow can accept; Sending node Data stream before distribution The raw data is segmented, and each data packet is sequentially placed into multiple fixed-length containers of the same length according to its position in the raw data stream. When the current fixed-length container is full or there is insufficient space to place a complete data packet, the process switches to placing the data packet into the next fixed-length container. Each fixed-length container is assigned a consecutive sequence number as a unique identifier. The length of the fixed-length container is pre-set to a certain value. The conditions are met. .

8. The method of Claim 3, wherein, The specific implementation of the cut fixed-length container being distributed to the corresponding sub-flow for preparing parallel transmission in rounds includes: The total number of fixed-length containers in each round of transmission is buffered at the receiving end As a reference index, as many fixed-length containers as possible are arranged for multi-substream parallel transmission according to the shunting ratio in the configuration information under the premise that the buffer can accommodate, and the total number of data packets in each round of transmission is calculated by the following formula , ； ； ； wherein represents the number of split fixed-size containers per available subflow transmission, is the number of available subflows in the network that can be used for data forwarding from the sender to the receiver, is a positive integer variable; The sending node, based on the configuration and deployment information issued by the central controller, and The optimal load condition of each sub-stream in each round is determined, and the segmented fixed-length containers are distributed to the corresponding sub-streams in rounds, in preparation for subsequent mapping to the physical network for parallel transmission of multiple sub-streams.

Images