Path planning method and device and related equipment

Abstract

The invention relates to the technical field of network communication, in particular to a path planning method and device and related equipment. The method comprises the steps that a set communication domain reported by each GPU and needed by a target training task is received, and the set communication domain reported by one GPU comprises the ID of the set communication domain, the Rank number and the communication relation between the GPU and other GPUs; based on the communication relationship between each GPU and other GPUs included in the set communication domain reported by each GPU, determining each training flow included in the target training task; based on the Rank number included in the set communication domain reported by each GPU, determining the training streams with the same Rank number corresponding to each training stream as the training streams of the same traffic type; and planning the training flows of the same flow type to different paths of networking based on the flow type of each training flow and the network topology of the networking for bearing the target training task.

Description

Technical Field

[0001] This application relates to the field of network communication technology, and in particular to a path planning method, apparatus and related equipment. Background Technology

[0002] As the scale of intelligent computing clusters continues to expand, the amount of data transmitted between servers within the cluster is growing exponentially. Each data stream has unique characteristics, including content purpose, data size, bandwidth requirements, and latency requirements. These differentiated needs often lead to mutual interference between data streams, causing local network congestion, which in turn results in performance degradation or even data loss. Although device network bandwidth and server network card bandwidth continue to increase, simply expanding bandwidth can no longer meet the needs of explosive business growth. Network congestion problems are becoming increasingly prominent in intelligent computing networks, especially in large-scale model training scenarios. Summary of the Invention

[0003] This application provides a path planning method, apparatus, and related equipment.

[0004] In a first aspect, this application provides a path planning method applied to a controller, wherein the controller manages a network for carrying training task traffic, and a GPU server for carrying training tasks is connected to the network; the method includes:

[0005] Receive the set communication domains reported by each GPU required for the target training task. The set communication domain reported by a GPU includes the ID of the set communication domain, the Rank number, and the communication relationship between the GPU and other GPUs. The Rank number is used to identify the number of GPUs in the GPU group to which the GPU belongs.

[0006] Based on the communication relationships between each GPU and other GPUs in the aggregate communication domain reported by each GPU, the training streams included in the target training task are determined.

[0007] Based on the Rank number included in the aggregate communication domain reported by each GPU, training streams with the same Rank number are identified as training streams of the same traffic type, wherein training streams of different traffic types have different Rank numbers.

[0008] Based on the traffic type of each training stream and the network topology of the network used to carry the target training task, training streams of the same traffic type are planned to different paths of the network.

[0009] Optionally, the network is a Spine-Leaf network architecture, and the GPU server connects to the network through Leaf devices.

[0010] Optionally, the method further includes:

[0011] Based on the traffic type of each training stream and the network topology of the network used to carry the target training task, training streams of different traffic types are planned to the same path / different paths of the network.

[0012] Optionally, the traffic type includes data parallel (DP) traffic and pipeline parallel (PP) traffic.

[0013] Optionally, the method further includes:

[0014] After planning any training flow to a path corresponding to the network, the weight values ​​of each link included in the network are maintained. If the training flow currently carried by a link belongs to the same traffic type as any training flow, the weight value of the link is incremented by 1. If the training flow currently carried by a link does not belong to the same traffic type as any training flow, the weight value of the link remains unchanged.

[0015] Based on the traffic type of each training stream, the network topology of the network used to carry the target training task, and the weight values ​​of each link included in the network, training streams of the same traffic type are planned to different paths of the network, or training streams of different traffic types are planned to the same path / different paths of the network.

[0016] Optionally, the communication relationship between a GPU and other GPUs includes: source IP, destination IP, source QP, and destination QP, where the source IP is the IP of the GPU and the destination IP is the IP of the other GPU.

[0017] Optionally, training streams with the same traffic type included in the target training task belong to time-overlapping streams; training streams with different traffic types belong to non-time-overlapping streams.

[0018] Secondly, this application provides a path planning device applied to a controller, wherein the controller manages a network for carrying training task traffic, and a GPU server for carrying training tasks is connected to the network; the device includes:

[0019] The receiving unit is used to receive the aggregate communication domains reported by each GPU required for the target training task. The aggregate communication domain reported by a GPU includes the ID of the aggregate communication domain, the Rank number, and the communication relationship between the GPU and other GPUs. The Rank number is used to identify the number of GPUs in the GPU group to which the GPU belongs.

[0020] The determining unit is used to determine each training stream included in the target training task based on the communication relationships between each GPU and other GPUs in the set communication domain reported by each GPU.

[0021] The determining unit is further configured to, based on the Rank number included in the aggregate communication domain reported by each GPU, determine the training streams with the same Rank number corresponding to each training stream as training streams of the same traffic type, wherein training streams of different traffic types correspond to different Rank numbers;

[0022] The planning unit is used to plan training flows of the same traffic type to different paths in the network based on the traffic type of each training flow and the network topology of the network used to carry the target training task.

[0023] Optionally, the network is a Spine-Leaf network architecture, and the GPU server connects to the network through Leaf devices.

[0024] Optionally, the planning unit is further configured to plan training flows of different traffic types to the same or different paths of the network based on the traffic type of each training flow and the network topology of the network used to carry the target training task.

[0025] Optionally, the traffic type includes data parallel (DP) traffic and pipeline parallel (PP) traffic.

[0026] Optionally, the device further includes a maintenance unit:

[0027] After the planning unit plans any training flow to a path corresponding to the network, the maintenance unit is used to maintain the weight value of each link included in the network. If the training flow currently carried by a link belongs to the same traffic type as any training flow, the weight value of the link is incremented by 1. If the training flow currently carried by a link does not belong to the same traffic type as any training flow, the weight value of the link remains unchanged.

[0028] The planning unit is further configured to, based on the traffic type of each training stream, the network topology of the network used to carry the target training task, and the weight values ​​of each link included in the network, plan training streams of the same traffic type to different paths in the network, or plan training streams of different traffic types to the same path / different paths in the network.

[0029] Optionally, the communication relationship between a GPU and other GPUs includes: source IP, destination IP, source QP, and destination QP, where the source IP is the IP of the GPU and the destination IP is the IP of the other GPU.

[0030] Optionally, training streams with the same traffic type included in the target training task belong to time-overlapping streams; training streams with different traffic types belong to non-time-overlapping streams.

[0031] Thirdly, embodiments of this application provide a path planning device, which includes:

[0032] Memory, used to store program instructions;

[0033] A processor is configured to invoke program instructions stored in the memory and execute the steps of the method as described in any one of the first aspects above, according to the obtained program instructions.

[0034] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the steps of the method as described in any of the first aspects above.

[0035] In summary, the path planning method provided in this application is applied to a controller, which manages a network used to carry training task traffic, and the GPU server used to carry the training task is connected to the network. The method includes: receiving a set of communication domains reported by each GPU required for the target training task, wherein a set of communication domains reported by a GPU includes the ID of the set of communication domains, the Rank number, and the communication relationship between the GPU and other GPUs, and the Rank number is used to identify the number of GPUs in the GPU group to which the GPU belongs; determining each training flow included in the target training task based on the communication relationship between each GPU and other GPUs included in the set of communication domains reported by each GPU; determining training flows with the same Rank number corresponding to each training flow as training flows of the same traffic type based on the Rank number included in the set of communication domains reported by each GPU, wherein training flows of different traffic types correspond to different Rank numbers; and planning training flows of the same traffic type to different paths of the network based on the traffic type of each training flow and the network topology of the network used to carry the target training task.

[0036] The path planning method provided in this application uses extended metadata (parameters used to identify the training stage to which the training stream belongs) carried by the collection communication domain to detect the time windows of each training stream included in the training task in real time. Training streams with overlapping time windows are identified as true concurrent streams and pre-planned to different physical paths, thus avoiding the planning of training streams with overlapping time onto the same physical path and reducing the risk of congestion. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings of the embodiments of this application.

[0038] Figure 1 A detailed flowchart of a path planning method provided in an embodiment of this application;

[0039] Figure 2 A schematic diagram of training flow planning provided in an embodiment of this application;

[0040] Figure 3 Another training flow planning diagram provided in this application embodiment;

[0041] Figure 4 This is a schematic diagram of the structure of a path planning device provided in an embodiment of this application;

[0042] Figure 5 This is a schematic diagram of the hardware architecture of a path planning device provided in an embodiment of this application. Detailed Implementation

[0043] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “the,” and “the” as used in this application and claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any and all possible combinations comprising one or more of the associated listed items.

[0044] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" may also be interpreted as "when," "when," or "in response to a determination."

[0045] Currently, the training efficiency of large AI models is closely related to the computing power of GPU clusters, and network performance is a key factor determining the effectiveness of computing power. Due to uneven distribution of training task load and frequent network congestion, effective bandwidth decreases and communication latency increases, severely restricting the overall efficiency of model training. Through in-depth analysis of the network traffic of large AI models, and using multi-dimensional indicators (including congestion count, latency, bandwidth utilization, cache queue, etc.) for performance evaluation, it was found that the communication data flow between nodes in the training task exhibits significant periodic characteristics.

[0046] Based on this finding, the network traffic of large AI models exhibits a high degree of predictability. AD-DC path navigation technology, by intelligently monitoring the start time of AI model training tasks, can accurately identify the communication needs of different training jobs and plan the optimal service path in advance. This proactive network traffic scheduling mechanism effectively achieves load balancing, significantly reduces the probability of network congestion, not only improves task processing efficiency but also greatly accelerates the training process of large-scale models.

[0047] Currently, the route navigation system performs unified route planning for all traffic based on the communication relationships (source IP + destination IP + source QP + destination QP) reported by the aggregated communication library, and performs load balancing at the network level.

[0048] However, it does not take into account the temporal separation of DP (Data Parallelism) and PP (Pipeline Parallelism) traffic during parallel training, resulting in a mismatch between path planning and actual traffic characteristics.

[0049] This application provides a dynamic path planning method based on time overlap detection to solve the problem of mismatch between static planning and actual traffic characteristics, which leads to unreasonable planning.

[0050] For example, see Figure 1 The diagram shown is a detailed flowchart of a path planning method provided in an embodiment of this application. This method is applied to a controller, which manages a network used to carry training task traffic. A GPU server used to carry training tasks connects to this network. In this embodiment, the network is described using a Spine-Leaf network architecture, with the GPU server connecting to the network via a Leaf device. Therefore, the path planning method in this embodiment may include the following steps:

[0051] Step 100: Receive the aggregate communication domains reported by each GPU required for the target training task.

[0052] In this embodiment of the application, the aggregate communication domain reported by a GPU includes the ID of the aggregate communication domain, the Rank number, and the communication relationship between the GPU and other GPUs. The Rank number is used to identify the number of GPUs in the GPU group to which the GPU belongs.

[0053] In practical applications, after defining a training task, the GPUs required for that task are determined; that is, which GPU servers will execute the training task are identified. Therefore, during initialization, the required GPUs for executing the training task are communicated to the controller via an initialization set communication domain. For example, 128 GPUs might be defined to execute training task 1.

[0054] Next, the training task is divided into multiple training stages, each with a different training flow. That is, the training flow is assigned to the corresponding GPUs for processing. For example, the initial GPUs are grouped, and the grouping rules are different in different training stages. At this time, each GPU will report the corresponding set communication domain to the controller based on the specific training flow. The set communication domain reported by a GPU includes the set communication domain ID, a parameter (e.g., Rank number) used to identify the number of GPUs in the GPU group to which the GPU belongs, and the communication relationship between the GPU and other GPUs. This communication relationship refers to the relationship of the training traffic being carried.

[0055] In this embodiment, the communication relationship between a GPU and other GPUs includes: source IP, destination IP, source QP, and destination QP, where the source IP is the IP of the GPU and the destination IP is the IP of the other GPU. A communication relationship refers to a training stream between the GPU and the destination GPU.

[0056] Step 110: Based on the communication relationships between each GPU and other GPUs in the aggregate communication domain reported by each GPU, determine the training streams included in the target training task.

[0057] As shown above, a communication relationship includes source IP, destination IP, source QP, and destination QP. Therefore, the training flow corresponding to each communication relationship can be determined based on each communication relationship.

[0058] Step 120: Based on the Rank number included in the aggregate communication domain reported by each GPU, the training streams with the same Rank number are identified as training streams of the same traffic type.

[0059] The training streams of different traffic types have different Rank numbers.

[0060] In this embodiment of the application, the traffic type includes data parallel DP traffic and pipeline parallel PP traffic.

[0061] In practical applications, taking a total of 128 GPUs as an example, for the DP traffic phase, the GPUs can be divided into 4 GPU groups; each GPU group has 32 Ranks (GPUs); for the PP traffic phase, the GPUs can be divided into 8 GPU groups; each GPU group has 16 Ranks (GPUs).

[0062] In this embodiment, since the rank number parameter is extended in the ensemble communication domain reported by the GPU, and the rank number corresponding to the training stream in different traffic stages is different, the traffic type can be identified by determining the training stream between the GPU and other GPUs based on the rank number carried in the ensemble communication domain and the communication relationship between the GPU and other GPUs.

[0063] Step 130: Based on the traffic type of each training stream and the network topology of the network used to carry the target training task, plan the training streams of the same traffic type to different paths of the network.

[0064] In this embodiment of the application, the training streams with the same traffic type included in the target training task belong to the time-overlapping streams; the training streams with different traffic types belong to the non-time-overlapping streams.

[0065] In practical applications, the traffic type of the training stream is related to the temporal overlap of the training stream. For example, training streams with the same DP traffic for the same training task are concurrent, that is, they overlap in time.

[0066] Different training phases for the same training task. For example, DP traffic and PP traffic do not overlap in time.

[0067] In this embodiment of the application, training streams with the same Rank number, i.e., belonging to the same training phase, are identified as training streams with overlapping time. For example, see [link to relevant documentation]. Figure 2 As shown, flow1 and flow2 belong to the same training task, and their Rank numbers are the same (time overlap). Therefore, during path planning, flow1 and flow2 are planned to different physical paths to avoid shared link congestion. For example, flow1 is planned to Path1, and flow2 is planned to Path2.

[0068] Furthermore, in this embodiment of the application, the above-mentioned path planning method may further include the following steps:

[0069] Based on the traffic type of each training stream and the network topology of the network used to carry the target training task, training streams of different traffic types are planned to the same path / different paths of the network.

[0070] For example, see Figure 3 As shown, flow3 and flow4 belong to the same training task, and their Rank numbers are different (their times do not overlap). Therefore, when planning the path, flow3 and flow4 can be planned onto the same physical path. For example, both flow3 and flow4 can be planned to Path1.

[0071] In this embodiment of the application, after any training flow is planned to a path corresponding to the network, the weight values ​​of each link included in the network are maintained. If the training flow currently carried by a link belongs to the same traffic type as any training flow, the weight value of the link is incremented by 1. If the training flow currently carried by a link does not belong to the same traffic type as any training flow, the weight value of the link remains unchanged.

[0072] Furthermore, when planning the path for the training stream, the same type of training stream can be planned to different paths in the network based on the traffic type of each training stream, the network topology of the network used to carry the target training task, and the weight values ​​of each link included in the network. Alternatively, different types of training streams can be planned to the same or different paths in the network.

[0073] For example, see Figure 4 The diagram shown is a schematic representation of a path planning device provided in an embodiment of this application. This device is applied to a controller, which manages a network used to carry training task traffic. A GPU server used to carry training tasks is connected to the network. The device includes:

[0074] The receiving unit 40 is used to receive the set communication domains reported by each GPU required for the target training task. The set communication domain reported by a GPU includes the ID of the set communication domain, the Rank number and the communication relationship between the GPU and other GPUs. The Rank number is used to identify the number of GPUs in the GPU group to which the GPU belongs.

[0075] The determining unit 41 is used to determine each training stream included in the target training task based on the communication relationship between each GPU and other GPUs included in the set communication domain reported by each GPU.

[0076] The determining unit 41 is further configured to, based on the Rank number included in the aggregate communication domain reported by each GPU, determine the training streams with the same Rank number corresponding to each training stream as training streams of the same traffic type, wherein training streams of different traffic types correspond to different Rank numbers;

[0077] Planning unit 42 is used to plan training flows of the same traffic type to different paths in the network based on the traffic type of each training flow and the network topology of the network used to carry the target training task.

[0078] Optionally, the network is a Spine-Leaf network architecture, and the GPU server connects to the network through Leaf devices.

[0079] Optionally, the planning unit 42 is further configured to plan training flows of different traffic types to the same or different paths of the network based on the traffic type of each training flow and the network topology of the network used to carry the target training task.

[0080] Optionally, the traffic type includes data parallel (DP) traffic and pipeline parallel (PP) traffic.

[0081] Optionally, the device further includes a maintenance unit:

[0082] After the planning unit 42 plans any training flow to a path corresponding to the network, the maintenance unit is used to maintain the weight value of each link included in the network. If the training flow currently carried by a link belongs to the same traffic type as any training flow, the weight value of the link is incremented by 1. If the training flow currently carried by a link does not belong to the same traffic type as any training flow, the weight value of the link remains unchanged.

[0083] The planning unit 42 is further configured to, based on the traffic type of each training stream, the network topology of the network used to carry the target training task, and the weight values ​​of each link included in the network, plan training streams of the same traffic type to different paths in the network, or plan training streams of different traffic types to the same path / different paths in the network.

[0084] Optionally, the communication relationship between a GPU and other GPUs includes: source IP, destination IP, source QP, and destination QP, where the source IP is the IP of the GPU and the destination IP is the IP of the other GPU.

[0085] Optionally, training streams with the same traffic type included in the target training task belong to time-overlapping streams; training streams with different traffic types belong to non-time-overlapping streams.

[0086] These units can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more digital signal processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when one of these units is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these units can be integrated together to form a system-on-a-chip (SOC).

[0087] Furthermore, regarding the path planning device provided in this application embodiment, from a hardware perspective, the hardware architecture schematic diagram of the path planning device can be found in [reference needed]. Figure 5 As shown, the path planning device may include: a memory 50 and a processor 51.

[0088] The memory 50 is used to store program instructions; the processor 51 calls the program instructions stored in the memory 50 and executes the above method embodiment according to the obtained program instructions. The specific implementation method and technical effect are similar, and will not be described again here.

[0089] Optionally, this application also provides a controller, including at least one processing element (or chip) for performing the above method embodiments.

[0090] Optionally, this application also provides a program product, such as a computer-readable storage medium storing computer-executable instructions for causing the computer to perform the above-described method embodiments.

[0091] Here, a machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, a machine-readable storage medium can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0092] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0093] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0094] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0095] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0096] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0097] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0098] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A path planning method, characterized in that, The method is applied to a controller that manages a network used to carry training task traffic, and a GPU server used to carry training tasks is connected to the network; the method includes: Receive the set communication domains reported by each GPU required for the target training task. The set communication domain reported by a GPU includes the ID of the set communication domain, the Rank number, and the communication relationship between the GPU and other GPUs. The Rank number is used to identify the number of GPUs in the GPU group to which the GPU belongs. Based on the communication relationships between each GPU and other GPUs in the aggregate communication domain reported by each GPU, the training streams included in the target training task are determined. Based on the Rank number included in the aggregate communication domain reported by each GPU, training streams with the same Rank number are identified as training streams of the same traffic type, wherein training streams of different traffic types have different Rank numbers. Based on the traffic type of each training stream and the network topology of the network used to carry the target training task, training streams of the same traffic type are planned to different paths of the network.

2. The method as described in claim 1, characterized in that, The network is a Spine-Leaf network architecture, and the GPU server connects to the network through Leaf devices.

3. The method as described in claim 1, characterized in that, The method further includes: Based on the traffic type of each training stream and the network topology of the network used to carry the target training task, training streams of different traffic types are planned to the same path / different paths of the network.

4. The method according to any one of claims 1-3, characterized in that, The traffic types include data parallel (DP) traffic and pipeline parallel (PP) traffic.

5. The method according to any one of claims 1-3, characterized in that, The method further includes: After planning any training flow to a path corresponding to the network, the weight values ​​of each link included in the network are maintained. If the training flow currently carried by a link belongs to the same traffic type as any training flow, the weight value of the link is incremented by 1. If the training flow currently carried by a link does not belong to the same traffic type as any training flow, the weight value of the link remains unchanged. Based on the traffic type of each training stream, the network topology of the network used to carry the target training task, and the weight values ​​of each link included in the network, training streams of the same traffic type are planned to different paths of the network, or training streams of different traffic types are planned to the same path / different paths of the network.

6. The method according to any one of claims 1-3, characterized in that, The communication relationship between a GPU and other GPUs includes: source IP, destination IP, source QP, and destination QP, where the source IP is the IP of the GPU and the destination IP is the IP of the other GPU.

7. The method as described in claim 4, characterized in that, The target training task includes training streams with the same traffic type that belong to time-overlapping streams; training streams with different traffic types belong to non-time-overlapping streams.

8. A path planning device, characterized in that, An application is made to a controller that manages a network used to carry training task traffic, and a GPU server used to carry training tasks is connected to the network; the device includes: The receiving unit is used to receive the aggregate communication domains reported by each GPU required for the target training task. The aggregate communication domain reported by a GPU includes the ID of the aggregate communication domain, the Rank number, and the communication relationship between the GPU and other GPUs. The Rank number is used to identify the number of GPUs in the GPU group to which the GPU belongs. The determining unit is used to determine each training stream included in the target training task based on the communication relationships between each GPU and other GPUs in the set communication domain reported by each GPU. The determining unit is further configured to, based on the Rank number included in the aggregate communication domain reported by each GPU, determine the training streams with the same Rank number corresponding to each training stream as training streams of the same traffic type, wherein training streams of different traffic types correspond to different Rank numbers; The planning unit is used to plan training flows of the same traffic type to different paths in the network based on the traffic type of each training flow and the network topology of the network used to carry the target training task.

9. A path planning device, characterized in that, The path planning device includes: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the steps of the method as described in any one of claims 1-7 according to the obtained program instructions.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing the computer to perform the steps of the method as described in any one of claims 1-7.

Images