Comprehensive optimization method, apparatus and device for aggregate communication, and medium

Abstract

Provided are a comprehensive optimization method, apparatus, device and medium for aggregate communication, relating to the technical field of communications, comprising: in response to an aggregate communication operation initiated by a process group of an application program, obtaining a communication network topology relationship among all processes in the process group, the communication network topological relation comprises that two processes belong to the same computing node and two processes belong to different computing nodes; determining all computing nodes corresponding to all processes in a process group according to the communication network topological relation; the computing nodes are used as basic units, a communication route map of the set communication operation among all the computing nodes is established, and the communication route map is composed of point-to-point communication among a plurality of computing nodes; according to the communication route map, the set communication operation is completed, and point-to-point communication between the computing nodes is completed through point-to-point communication of a plurality of process pairs between two corresponding computing nodes. According to the invention, the communication performance of aggregate communication is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a comprehensive optimization method, device, equipment and medium for collective communication. Background Art

[0002] As the scale of the problems handled by applications increases, the amount of computation required by applications increases rapidly. Developing parallel versions of these applications, taking advantage of the many processor cores on modern high-performance computers, can greatly accelerate the computational process. The development of parallel versions usually requires the parallel running and collaborative computing of programs on multiple processes, so it is necessary to use tools or programming specifications that support inter-process data communication (such as the message passing interface, or MPI). The communication modes between application processes can be divided into two categories: point-to-point communication and collective communication. Among them, collective communication usually requires all processes within a group of processes to communicate with each other, or the final result obtained through communication depends on all processes. Collective communication includes operations such as gather, scatter, allgather, bcast, reduce, and allreduce.

[0003] Since the network data transmission speed between computing nodes in high-performance computers is much slower than the computing speed and memory access speed of the processor, the communication overhead becomes the main factor affecting or even determining the parallel efficiency and parallel scalability of applications. Since global communication usually involves all processes, its overhead is often much larger than point-to-point communication, making global communication often the performance bottleneck of parallel programs. For collective communication, the main optimization ideas include: reducing the amount of communication data (such as involving the ring method), reducing the number of communications or the number of layers (such as the binary tree method), and increasing the amount of data in a point-to-point communication to increase the bandwidth (such as the binary tree method). The efficiency of a point-to-point communication can be quantified by the communication bandwidth (that is, the amount of data communicated per unit time). Affected by communication delay and network bandwidth capacity, the larger the amount of data in a point-to-point communication, the higher the communication bandwidth obtained. Since the ring method usually needs to divide the data of collective communication evenly according to a certain number of processes, the ring method will reduce the amount of data in a point-to-point communication, which is suitable for situations where the amount of collective communication data is large. Even if the amount of collective communication data is large, when the number of processes is large, the ring method will result in a small amount of data for a point-to-point communication and it is difficult to fully utilize the network bandwidth capacity. One solution is to decompose the number of processes into two layers, MxN, and perform the ring method on the two layers in turn. The binary tree method is the opposite, and it is suitable for situations where the amount of collective communication data is small and it is difficult to fully utilize the network bandwidth capacity.

[0004] The performance of a point-to-point communication between all processes involved in collective communication is related to the network topology, which is specifically manifested in the following aspects: (1) When two processes are in the same computing node, the communication between them can be achieved through memory copying, so there can be a high communication bandwidth capacity and a very low communication delay; (2) When two processes are between different computing nodes, the communication between them only needs to pass through the switch, so the communication bandwidth capacity is relatively low and the communication delay is relatively large. Therefore, in the collective communication completed by a group of processes in collaboration, multiple processes in the same computing node can be made to work together to improve communication performance. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a comprehensive optimization method, device, equipment and medium for collective communication to achieve the purpose of improving the communication performance of the collective communication mode.

[0006] The embodiment of the present application provides the following technical solution: a comprehensive optimization method for collective communication, the method comprising:

[0007] In response to a collective communication operation initiated by a process group of an application, obtaining a communication network topology relationship between all processes in the process group, wherein the communication network topology relationship includes two processes belonging to the same computing node and two processes belonging to different computing nodes;

[0008] Determine all computing nodes corresponding to all processes in the process group according to the communication network topology relationship;

[0009] Taking computing nodes as basic units, establishing a communication route map of the collective communication operation among all computing nodes, wherein the communication route map is composed of point-to-point communications among a number of computing nodes;

[0010] According to the communication route map, the collective communication operation is completed, wherein the point-to-point communication between each computing node is completed by the point-to-point communication of multiple process pairs between the corresponding two computing nodes.

[0011] According to an embodiment of the present application, in response to a collective communication operation initiated by a process group of an application, the collective communication operation includes any one of: data distributing from a main process to all processes, data of all processes converging to the main process, data of all processes converging to all processes, data of all processes being reduced to the main process, data of all processes being reduced to all processes, and data being broadcast from the main process to all processes.

[0012] According to one embodiment of the present application, taking computing nodes as basic units, a communication route map of the collective communication operation among all computing nodes is established, and the communication route map is a directed graph based on any one or more forms of direct communication, binary tree, and ring.

[0013] According to an embodiment of the present application, completing the collective communication operation according to the communication roadmap includes:

[0014] Determine a number of process pairs between two computing nodes corresponding to each directed edge in the communication route map, and according to the form of the collective communication operation, the number of process pairs between the two computing nodes collaborate to complete the point-to-point communication between the two computing nodes.

[0015] According to an embodiment of the present application, completing the collective communication operation according to the communication roadmap includes:

[0016] If the collective communication operation is in the form of reducing the data of all processes to all processes or the main process, the operation of reducing the data of all processes to all processes between multiple processes in each computing node is first performed, and then a communication roadmap between the computing nodes is established. Along the communication roadmap, multiple process pairs between any two computing nodes collaborate to complete the point-to-point communication between the two computing nodes and the operation of reducing the data of all processes to all processes or the main process.

[0017] According to an embodiment of the present application, completing the collective communication operation according to the communication roadmap includes:

[0018] If the collective communication operation is in the form of aggregating the data of all processes to all processes or a main process, a communication route map is established between computing nodes. Along the communication route map, multiple process pairs between any two computing nodes collaborate to complete the point-to-point communication between the two computing nodes. Then, within each computing node, efficient memory-based communication is used to complete the operation of aggregating the data of all processes to all processes or a main process between processes within the computing node.

[0019] According to an embodiment of the present application, determining a plurality of process pairs on two computing nodes corresponding to each directed edge in the communication route graph includes:

[0020] Determine the number of processes on the two computing nodes respectively. If the number of processes on the two computing nodes is the same, the number of process pairs between the two computing nodes is the number of processes. If the number of processes on the two computing nodes is different, the number of process pairs between the two computing nodes is the minimum number of processes between the two computing nodes or the common divisor of the number of processes between the two computing nodes.

[0021] According to an embodiment of the present application, determining a plurality of process pairs on two computing nodes corresponding to each directed edge in the communication route graph includes:

[0022] Determine the number of processes on the two computing nodes respectively, determine the amount of data for one communication corresponding to the directed edge, determine the optimal range of the number of communication process pairs based on the data amount, and determine the number of process pairs between the two computing nodes based on the number of processes on the two computing nodes and the optimal range of the number of communication process pairs.

[0023] The present application also provides a comprehensive optimization device for collective communication, the device comprising:

[0024] A topology relationship acquisition module, configured to, in response to a collective communication operation initiated by a process group of an application, acquire a communication network topology relationship between all processes in the process group, wherein the communication network topology relationship includes two processes belonging to the same computing node and two processes belonging to different computing nodes;

[0025] A computing node determination module, used to determine all computing nodes corresponding to all processes in the process group according to the communication network topology relationship;

[0026] A communication route map establishment module, used to establish a communication route map of the collective communication operation among all the computing nodes with computing nodes as basic units, wherein the communication route map is composed of point-to-point communications among a number of computing nodes;

[0027] The collective communication module is used to complete the collective communication operation according to the communication route map, wherein the point-to-point communication between each computing node is completed by the point-to-point communication of multiple process pairs between the corresponding two computing nodes.

[0028] The present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned comprehensive optimization method for collective communication when executing the computer program.

[0029] The present application also provides a computer-readable storage medium, which stores a computer program for executing the above-mentioned comprehensive optimization method for collective communication.

[0030] Compared with the prior art, the beneficial effects that can be achieved by at least one of the above-mentioned technical solutions adopted in the embodiments of this specification include at least the following: the embodiments of the present invention, for the collective communication mode, first obtain the communication network topology relationship between all processes in the process group, and then determine all computing nodes corresponding to all processes in the process group according to the communication network topology relationship; finally, with the computing node as the basic unit, establish a communication roadmap for the collective communication operation between all computing nodes, and complete the collective communication operation according to the communication roadmap, wherein the point-to-point communication between each computing node is completed by the point-to-point communication of multiple process pairs between the corresponding two computing nodes, thereby reducing communication delay, ensuring communication efficiency, and effectively improving the communication performance of the collective communication mode. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0032] Figure 1 It is a flowchart of a comprehensive optimization method for collective communication according to an embodiment of the present invention;

[0033] Figure 2 is a schematic diagram of a comprehensive optimization device for collective communication according to an embodiment of the present invention;

[0034] Figure 3 It is a structural schematic diagram of the computer device of the present invention. DETAILED DESCRIPTION

[0035] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0036] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0037] like Figure 1 As shown, an embodiment of the present invention provides a comprehensive optimization method for collective communication, the method comprising:

[0038] S101. In response to a collective communication operation initiated by a process group of an application, obtaining a communication network topology relationship between all processes in the process group, wherein the communication network topology relationship includes two processes belonging to the same computing node and two processes belonging to different computing nodes;

[0039] S102. Determine all computing nodes corresponding to all processes in the process group according to the communication network topology;

[0040] S103. Taking computing nodes as basic units, establishing a communication route map for the collective communication operation among all computing nodes, wherein the communication route map consists of point-to-point communications among several computing nodes;

[0041] S104. According to the communication route map, the collective communication operation is completed, wherein the point-to-point communication between each computing node is completed by the point-to-point communication of multiple process pairs between the corresponding two computing nodes.

[0042] In the embodiment of the present invention, it is found through testing that when communicating a piece of data between two computing nodes, the performance of using multiple pairs of processes for collaborative communication is significantly better than the communication performance of using only one pair of processes. For example, to send a piece of data with a length of 24MB from computing node A to B, when A uses only one process to send this piece of data to one process of B, the communication bandwidth is about 3GB / s; and when there are 24 pairs of processes between A and B to collaboratively complete the transmission of this 24MB of data (A and B provide 24 processes respectively, forming 24 process pairs between A and B), the communication bandwidth can reach 12GB / s. Therefore, in the collective communication completed by a group of processes in the embodiment of the present invention, multiple processes in the same computing node are enabled to work collaboratively, thereby improving the communication performance.

[0043] In one embodiment, in response to a collective communication operation initiated by a process group of an application, the collective communication operation includes any one of: data is scattered from a main process to all processes (scatter), data of all processes is gathered to the main process (gather), data of all processes is gathered to all processes (allgather), data of all processes is reduced to the main process (reduce), data of all processes is reduced to all processes (allreduce), and data is broadcast from the main process to all processes (bcast).

[0044] Common collective communications mainly include gather, scatter, allgather, bcast, reduce, allreduce and other operations, as shown in Table 1.

[0045] Table 1 Collection communication operation table

[0046]

[0047] In specific implementation, collective communication operations can be realized based on some optimization methods, such as binary tree form, ring form (ring form for communication), direct communication form (direct communication may appear in gather or scatter). For the ring method applicable to allgather, allreduce and bcast, two or even multiple layers of rings can be constructed to avoid the situation where the number of processes is large, resulting in small communication messages, low communication bandwidth and large communication delay.

[0048] In one embodiment, obtaining the communication network topology relationship between all processes in the process group includes: obtaining the computing node name (or number or network address) running each process through the corresponding application interface, comparing the computing node names (or numbers or network addresses) of two processes; if the computing node names (or numbers or network addresses) of the two processes are the same, then the communication network topology relationship between the two processes is that the two processes belong to the same computing node; if the computing node names (or numbers or network addresses) of the two processes are different, then the communication network topology relationship between the two processes is that the two processes belong to different computing nodes.

[0049] In specific implementation, on a high-performance computer, each computing node will have a unique node name (or number or network address), and the application can obtain the computing node name (or number or network address) running each process through the corresponding program interface. Therefore, for two processes, by comparing the corresponding computing node names (or numbers or network addresses), the communication network topology relationship between the two processes can be known. For example, when the computing node names (or numbers or network addresses) of two processes are the same, the two processes belong to the same computing node communication; when the computing node names (or numbers or network addresses) of two processes are different, the two processes belong to the communication between different computing nodes.

[0050] In one embodiment, all computing nodes corresponding to all processes in the process group are determined based on the communication network topology relationship, including: determining all computing nodes corresponding to the process group based on the computing node names (or numbers or network addresses) of all processes in the process group, and determining a number of processes contained in each computing node.

[0051] After obtaining the communication network topology, it is easy to determine all computing nodes corresponding to all processes in the process group. In specific implementation, all computing nodes corresponding to the process group can be determined according to the computing node names (or numbers or network addresses) of all processes, and the number of processes contained in each computing node can be determined.

[0052] In one embodiment, taking computing nodes as basic units, a communication route map of the collective communication operation among all computing nodes is established, and the communication route map is a directed graph based on any one or more forms of direct communication, binary tree, and ring.

[0053] In specific implementation, the communication route graph of the collective communication operation is a directed graph, mainly based on direct communication form, binary tree form, ring form and their combination. No matter what specific form the communication route graph is, it can be decomposed into multiple rounds of point-to-point communication. In the communication route graph between computing nodes, each point is a computing node, and each directed edge can be represented as a point-to-point communication from one computing node to another computing node.

[0054] In one embodiment, completing the collective communication operation according to the communication route map includes:

[0055] Determine a number of process pairs between two computing nodes corresponding to each directed edge in the communication route map, and according to the form of the collective communication operation, the number of process pairs between the two computing nodes collaborate to complete the point-to-point communication between the two computing nodes.

[0056] On modern general-purpose high-performance computers, each computing node has a large number of processor cores (dozens or even hundreds). Therefore, in collective communication, each computing node usually has multiple processes, and the number of processes on different computing nodes is often the same. Therefore, for point-to-point communication between two computing nodes, it can be completed by multiple process pairs formed by these two computing nodes. For example, when computing nodes A and B have k processes respectively, k process pairs can be formed between the two computing nodes, where each process pair consists of a unique process in A and B.

[0057] Different collective communication operations have different implementations, as follows:

[0058] (1) Gather based on binary tree (data from all processes are gathered to the main process). First, a communication binary tree is established between computing nodes. Then, in the process of gathering from low to high levels along the binary tree, the point-to-point communication between any two computing nodes is completed by multiple process pairs between the two computing nodes. After all data has been gathered to multiple processes of the computing node to which the main process (the process that needs to obtain the gather result) belongs, these processes use the efficient memory-based communication within the node to gather all data to the main process.

[0059] (2) Scatter based on binary tree (data is scattered from the main process to all processes). First, a communication binary tree is established between multiple processes in the computing node to which the main process belongs, and all data is scattered in the computing node to which the main process belongs; then a communication binary tree is established between computing nodes. In the process of scattering from high to low levels along the binary tree between nodes, the point-to-point communication between any two computing nodes is completed by multiple process pairs between the two computing nodes.

[0060] (3) Reduce based on binary tree (data of all processes are reduced to the main process). First, complete the allreduce (data of all processes are reduced to all processes) operation between multiple processes in each computing node (binary tree method or ring method can be used); then establish a communication binary tree between computing nodes. In the process of reducing from low to high levels along the binary tree, the point-to-point communication and reduce calculation between any two computing nodes are completed by multiple process pairs between the two computing nodes, where each process pair completes the communication and reduction of part of the data. For example, when a reduce operation is to be performed on N data and each node has k processes, each process pair is responsible for reducing approximately N / k data.

[0061] (4) Bcast based on binary tree (data is broadcast from the main process to all processes). First, the reduce operation between multiple processes in the computing node to which the main process belongs is completed; then a communication binary tree is established between the computing nodes. In the process of bcasting from high to low levels along the binary tree, the point-to-point communication between any two computing nodes is completed by multiple process pairs between the two computing nodes, where each process pair completes the communication of part of the data. For example, when a reduce operation is to be performed on N data and each node has k processes, each process pair is responsible for the communication of approximately N / k data.

[0062] (5) Allreduce (data of all processes are reduced to all processes). Similar to the reduce method based on the binary tree. First, the allreduce operation between multiple processes in each computing node is completed; then a communication route map between computing nodes is established. In the process of following the communication route map, the point-to-point communication and reduce calculation between any two computing nodes are completed by multiple process pairs between the two computing nodes, where each process pair completes the communication and reduce of part of the data.

[0063] (6) Allgather (data from all processes are gathered to all processes). Similar to the gather method based on the binary tree. When establishing a communication route map between computing nodes, point-to-point communication between any two computing nodes is completed through multiple process pairs between the two computing nodes along the communication route map; then, in each computing node, efficient memory-based communication is used to complete the allgather operation between processes in the computing node.

[0064] In one embodiment, determining a number of process pairs on two computing nodes corresponding to each directed edge in the communication route graph includes: determining the number of processes on the two computing nodes respectively; if the number of processes on the two computing nodes is the same, then the number of process pairs between the two computing nodes is the number of processes; if the number of processes on the two computing nodes is different, then the number of process pairs between the two computing nodes is the minimum number of processes between the two computing nodes or the common divisor of the number of processes between the two computing nodes.

[0065] Usually, in two computing nodes that are communicating, the number of processes of each computing node is the same. However, there are also cases where the number of processes is different. In this case, one solution is to determine the number of process pairs according to the minimum number of processes between the two computing nodes or the common divisor of the number of processes between the two computing nodes.

[0066] Therefore, in the specific implementation, the number of process pairs for communication between two computing nodes is the number of processes in each computing node. However, when the amount of data to be communicated between the two computing nodes is small, it is not necessary to use all the processes in each computing node, that is, a smaller number of process pairs can be used to achieve the best possible communication performance. Therefore, the number of process pairs can be determined in combination with the amount of data to be communicated. A specific implementation method is that the optimal range of the number of communication process pairs between the two computing nodes under various communication data amounts can be obtained through the setting of the system administrator or the automatic measurement of the system; then the optimal range of the number of communication process pairs is determined according to the communication data amount of the directed edge, and then the number of process pairs is finally determined in combination with the number of processes of the two computing nodes. For example, when the upper limit of the optimal range of the number of communication process pairs is less than or equal to the number of processes of the two computing nodes, the upper limit of the optimal range of the number of communication process pairs can be used as the number of process pairs; when the lower limit of the optimal range of the number of communication process pairs is greater than the number of processes of the two computing nodes, the number of process pairs is determined according to the number of processes of the two computing nodes.

[0067] like Figure 2 As shown, the present application also provides a comprehensive optimization device 200 for collective communication, the device comprising:

[0068] A topology relationship acquisition module 201 is used to obtain a communication network topology relationship between all processes in a process group in response to a collective communication operation initiated by the process group of the application, wherein the communication network topology relationship includes two processes belonging to the same computing node and two processes belonging to different computing nodes;

[0069] A computing node determination module 202, configured to determine all computing nodes corresponding to all processes in the process group according to the communication network topology relationship;

[0070] A communication route map establishing module 203 is used to establish a communication route map of the collective communication operation among all the computing nodes, taking the computing node as the basic unit, wherein the communication route map is composed of point-to-point communications among a number of computing nodes;

[0071] The collective communication module 204 is used to complete the collective communication operation according to the communication route map, wherein the point-to-point communication between each computing node is completed by the point-to-point communication of multiple process pairs between the corresponding two computing nodes.

[0072] When the device is implemented specifically, the form of the collective communication operation includes: data is distributed from the main process to all processes, data of all processes are aggregated to the main process, data of all processes are aggregated to all processes, data of all processes are reduced to the main process, data of all processes are reduced to all processes, and data is broadcast from the main process to all processes.

[0073] The topology relationship acquisition module 201 is used to obtain the computing node name (or number or network address) running each process through the corresponding application interface, and compare the computing node names (or numbers or network addresses) of two processes. If the computing node names (or numbers or network addresses) of the two processes are the same, then the communication network topology relationship between the two processes is that the two processes belong to the same computing node; if the computing node names (or numbers or network addresses) of the two processes are different, then the communication network topology relationship between the two processes is that the two processes belong to different computing nodes.

[0074] The computing node determination module 202 is used to determine all computing nodes corresponding to the process group according to the computing node names (or numbers or network addresses) of all processes in the process group, and to determine a number of processes contained in each computing node.

[0075] The communication route map establishing module 203 is used to establish a communication route map for the collective communication operation among all the computing nodes based on computing nodes as basic units. The communication route map is a directed graph based on any one or more forms of direct communication, binary tree, and ring.

[0076] The collective communication module 204 is used to determine a number of process pairs between two computing nodes corresponding to each directed edge in the communication route map, and according to the form of the collective communication operation, a number of process pairs between two computing nodes collaborate to complete the point-to-point communication between the two computing nodes.

[0077] In a specific implementation, a number of process pairs on two computing nodes corresponding to each directed edge in the communication route map are determined, including: respectively determining the number of processes on the two computing nodes; if the number of processes on the two computing nodes is the same, then the number of process pairs between the two computing nodes is the number of processes; if the number of processes on the two computing nodes is different, then the number of process pairs between the two computing nodes is the minimum number of processes between the two computing nodes or the common divisor of the number of processes between the two computing nodes.

[0078] In a specific implementation, determining a number of process pairs on two computing nodes corresponding to each directed edge in the communication route map also includes: respectively determining the number of processes on the two computing nodes, determining the amount of data in one communication corresponding to the directed edge, determining the optimal range of the number of communication process pairs based on the data amount, and determining the number of process pairs between the two computing nodes based on the number of processes on the two computing nodes and the optimal range of the number of communication process pairs.

[0079] In one embodiment, a computer device is provided, such as Figure 3 As shown, it includes a memory 301, a processor 302, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned comprehensive optimization method for collective communication is implemented.

[0080] Specifically, the computer device may be a computer terminal, a server or a similar computing device.

[0081] In this embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program for executing the above-mentioned integrated optimization method for collective communication.

[0082] Specifically, computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0083] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0084] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A comprehensive optimization method for collective communication, characterized in that: The method comprises: In response to a collective communication operation initiated by a process group of an application, obtaining a communication network topology relationship between all processes in the process group, wherein the communication network topology relationship includes two processes belonging to the same computing node and two processes belonging to different computing nodes; Determine all computing nodes corresponding to all processes in the process group according to the communication network topology relationship; Taking computing nodes as basic units, establishing a communication route map of the collective communication operation among all computing nodes, wherein the communication route map is composed of point-to-point communications among a number of computing nodes; According to the communication route map, the collective communication operation is completed, wherein the point-to-point communication between each computing node is completed by the point-to-point communication of multiple process pairs between the corresponding two computing nodes.

2. The comprehensive optimization method for collective communication according to claim 1, characterized in that: In response to a collective communication operation initiated by a process group of an application, the collective communication operation may take the form of: data being distributed from a main process to all processes, data from all processes being aggregated to the main process, data from all processes being aggregated to all processes, data from all processes being reduced to the main process, data from all processes being reduced to all processes, and data being broadcast from the main process to all processes.

3. The comprehensive optimization method for collective communication according to claim 1, characterized in that: Taking computing nodes as basic units, a communication route map of the collective communication operation among all computing nodes is established, and the communication route map is a directed graph based on any one or more forms of direct communication, binary tree, and ring.

4. The comprehensive optimization method for collective communication according to claim 3, characterized in that: According to the communication route map, completing the collective communication operation includes: Determine a number of process pairs between two computing nodes corresponding to each directed edge in the communication route map, and according to the form of the collective communication operation, the number of process pairs between the two computing nodes collaborate to complete the point-to-point communication between the two computing nodes.

5. The comprehensive optimization method for collective communication according to claim 4, characterized in that: According to the communication route map, completing the collective communication operation includes: If the collective communication operation is in the form of reducing the data of all processes to all processes or the main process, the operation of reducing the data of all processes to all processes between multiple processes in each computing node is first performed, and then a communication roadmap between the computing nodes is established. Along the communication roadmap, multiple process pairs between any two computing nodes collaborate to complete the point-to-point communication between the two computing nodes and the operation of reducing the data of all processes to all processes or the main process.

6. The comprehensive optimization method for collective communication according to claim 4, characterized in that: According to the communication route map, completing the collective communication operation includes: If the collective communication operation is in the form of aggregating the data of all processes to all processes or a main process, a communication route map is established between computing nodes. Along the communication route map, multiple process pairs between any two computing nodes collaborate to complete the point-to-point communication between the two computing nodes. Then, within each computing node, efficient memory-based communication is used to complete the operation of aggregating the data of all processes to all processes or a main process between processes within the computing node.

7. The comprehensive optimization method for collective communication according to claim 4, characterized in that: Determining a plurality of process pairs on two computing nodes corresponding to each directed edge in the communication route graph includes: Determine the number of processes on the two computing nodes respectively. If the number of processes on the two computing nodes is the same, the number of process pairs between the two computing nodes is the number of processes. If the number of processes on the two computing nodes is different, the number of process pairs between the two computing nodes is the minimum number of processes between the two computing nodes or the common divisor of the number of processes between the two computing nodes.

8. The comprehensive optimization method for collective communication according to claim 4, characterized in that: Determining a number of process pairs on two computing nodes corresponding to each directed edge in the communication route graph further includes: Determine the number of processes on the two computing nodes respectively, determine the amount of data for one communication corresponding to the directed edge, determine the optimal range of the number of communication process pairs based on the data amount, and determine the number of process pairs between the two computing nodes based on the number of processes on the two computing nodes and the optimal range of the number of communication process pairs.

9. A comprehensive optimization device for collective communication, characterized in that: The device comprises: A topology relationship acquisition module, configured to, in response to a collective communication operation initiated by a process group of an application, acquire a communication network topology relationship between all processes in the process group, wherein the communication network topology relationship includes two processes belonging to the same computing node and two processes belonging to different computing nodes; A computing node determination module, used to determine all computing nodes corresponding to all processes in the process group according to the communication network topology relationship; A communication route map establishment module, used to establish a communication route map of the collective communication operation among all the computing nodes with computing nodes as basic units, wherein the communication route map is composed of point-to-point communications among a number of computing nodes; The collective communication module is used to complete the collective communication operation according to the communication route map, wherein the point-to-point communication between each computing node is completed by the point-to-point communication of multiple process pairs between the corresponding two computing nodes.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the comprehensive optimization method for collective communication according to any one of claims 1 to 7 is implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the comprehensive optimization method for collective communication according to any one of claims 1 to 7.

Images