A collective communication method, computing node, and computing device cluster

Abstract

The embodiment of the application provides a collection communication method, a computing node and a computing device cluster, which can avoid the occurrence of the multiple-to-one situation when data is transmitted between the computing nodes, can reduce network congestion and improve collection communication efficiency. The method comprises the following steps: a first target node obtains a first data group from a second target matrix after transposing a first target matrix, determines a destination address of the first data group, sends the first data group and the destination address of the first data group to a second target node, receives a second data group and a destination address of the second data group from a third target node, writes the second data group into the destination address of the second data group, then updates the second target matrix, repeatedly executes the second step to the last step until the data of the first target node is all transposed, and the ppn data in the first data group and the second data group are not repeated in the ppn data in the previous loop process in the repeated execution process.

Description

Technical Field

[0001] This application relates to the field of high-performance computing, and in particular to a collective communication method, computing nodes, and computing device clusters. Background Technology

[0002] Set communication is a communication method in which all processes in a specific group participate. Global transpose (ALLtoALL) is a communication operation in set communication.

[0003] One type of aggregated communication works roughly as follows: A high-performance computing system includes N computing nodes, and each computing node includes ppn processes. The computing node sends the data of the i-th process to the i+k-th process and receives the data of the ik-th process according to the pairwise exchange function. When the pairwise exchange function is completed, the global transpose result is obtained.

[0004] However, when the above method transfers data between processes, it may cause multiple computing nodes to send data to a single computing node at the same time (i.e., a multi-to-one scenario). This situation can easily lead to network congestion and affect the efficiency of aggregated communication. Summary of the Invention

[0005] This application provides a collective communication method that can prevent multiple calls when transmitting data between computing nodes, thereby reducing network congestion and improving the efficiency of collective communication.

[0006] The first aspect provides a collective communication method in a high-performance computing system, comprising: a first target node transposing a first target matrix, obtaining a first data group from a second target matrix, determining the destination address of the first data group, sending the first data group and its destination address to a second target node, and the first target node receiving a second data group and its destination address from a third target node, writing the second data group to its destination address; when the second target matrix is ​​not the last matrix outside the first target matrix in the first target node, the first target node updates the second target matrix to the first matrix after the second target matrix and outside the first target matrix; when the second target matrix is ​​the last matrix outside the first target matrix in the first target node, the first target node updates the second target matrix to the first matrix outside the first target matrix in the first target node, and then repeats the second step to the last step, wherein the pppn data in the first data group and the second data group are not repeated with the pppn data in the previous loop process, until all the data of the first target node is transposed.

[0007] The high-performance computing system comprises N nodes, labeled with consecutive indices. Each node contains N square matrices, where each row of the matrix contains ppn data points from ppn processes, and each column contains ppn data points from a single process. N and ppn are both positive integers greater than 1. The first target node is any one of the N nodes, and its position in the first target matrix corresponds to its index. The second target matrix, different from the first target matrix, contains ppn data points. Any two data points in the first data group are located in different rows and columns of the second target matrix. The destination address of the first data group is used to transpose it. The difference between the index of the second target node and the index of the first target node is equal to the remainder obtained by dividing the difference between the index of the first target node and the index of the third target node by N. The second data group contains ppn data points, and its destination address is used to transpose it.

[0008] In this implementation, when transmitting data between nodes, the first target node sends data to one computing node and receives data from another computing node. This cyclical sending and receiving of data transposes all matrices except the first target matrix. The first target matrix can be transposed by the first target node itself without inter-node transmission. This global transposition process avoids situations where the first computing node simultaneously sends data to or receives data from multiple computing nodes, thus preventing "one-to-many" scenarios and reducing network congestion, thereby improving aggregate communication efficiency. It should be noted that each node in the high-performance computing system can synchronously execute this aggregate communication method to achieve global transposition.

[0009] In conjunction with the first aspect, in the first possible implementation of the first aspect, the first target node obtaining the first data group from the second target matrix includes: the first target node obtaining the i-th row of data from the (i+k)%ppn-th process in the second target matrix, using the i-th row of data from the (i+k)%ppn-th process as the i-th data of the first data group, and storing the i-th data of the first data group in the queue of the i-th process in the first target node; the first target node determining the destination address of the first data group includes: the first target node determining the receiving address of the i-th data in the first data group as the (i+k)%ppn-th row of the i-th process in the third target matrix of the second target node. i takes values ​​from 1, 2, ..., ppn, k is the quotient obtained by dividing the difference between the loop count and 1 by (N-1), and the position of the third target matrix in the second target node corresponds to the index of the first target matrix. When i is not equal to (i+k)%ppn, the i-th process can obtain data from other processes and then add it to the queue of its local process, thus solving the problem of each process obtaining data from other processes. The i-th data in the first data group is stored in the queue of the i-th process in the first target node, and the destination address of the i-th data in the first data group is the (i+k)%ppn row of the i-th process in the third target matrix of the second target node. This can be understood as the i-th process of the first target node and the i-th process of the second target node transmitting the i-th data in the first data group. That is, each process transmits one data to the process corresponding to it. This reduces the number of data links established and is convenient for implementation in large-scale parallel computing scenarios.

[0010] In another possible implementation, the data of the (i+1)th process in the first data group is located at row number r of the second target matrix. i+1 The data of the i-th process is located in the row number r of the second target matrix. i Satisfy the following formula: r i+1 =(r i +1)%ppn, where i iterates through 1, 2, ..., pppn. This provides another way to obtain the first data set.

[0011] In another possible implementation, the first target node obtaining the i-th row of data of the (i+k)%ppn-th process in the second target matrix includes: the first target node obtaining the local key of the (i+k)%ppn-th process and the virtual address of the i-th row of data of the (i+k)%ppn-th process through the remote direct memory access (RDMA) protocol; determining the physical address of the i-th row of data of the (i+k)%ppn-th process based on the local key and the virtual address of the i-th row of data of the (i+k)%ppn-th process; and obtaining the i-th row of data of the (i+k)%ppn-th process from the physical address of the i-th row of data of the (i+k)%ppn-th process. This provides a specific method for obtaining data from other processes.

[0012] In another possible implementation, the first target node obtains the first data group from the second target matrix by: retrieving the i-th row of data from the (i+k)%ppn-th process in the second target matrix from shared memory, and using the i-th row of data from the (i+k)%ppn-th process as the i-th data in the first data group. Here, i iterates through 1, 2, ..., ppn, and k is the quotient obtained by dividing the difference between the loop count and 1 by (N-1). This provides a method for obtaining the first data group from shared memory, improving the flexibility of the scheme implementation.

[0013] In another possible implementation, the first data group, its destination address, the second data group, and its destination address are all sent via the RDMA protocol. When transmitting data between nodes via the RDMA protocol, the first data group, its destination address, the second data group, and its destination address are not forwarded by the processor to the communication interface, but are transmitted directly through the communication interface. This reduces the processing latency of data in the computing nodes and improves the speed of aggregated communication.

[0014] In another possible implementation, transposing the first target matrix by the first target node involves repeatedly transposing the third data group of the first target matrix. The pppn data in the third data group are not repeated from the pppn data in the previous loop, until all data outside the main diagonal of the first target matrix has been transposed. In the third data group, any two data points are located in different rows and columns, and the third data group does not include data on the main diagonal of the first target matrix. This provides a specific scheme for transposing the first target matrix.

[0015] In another possible implementation, the data of the (i+1)th process in the third data set is located at row number r′ of the first target matrix. i+1The data of the i-th process is located at row number r of the first target matrix. i r′ satisfies the following formula: i+1 =(r i '+1)%ppn, where i iterates through 1, 2, ..., ppn. This provides a specific method for selecting the third data set.

[0016] The second aspect provides a computing node, which includes a communication module and a processing module. The processing module is used to transpose a first target matrix, and the communication module is used to obtain a first data group from a second target matrix, determine the destination address of the first data group, send the first data group to the second target node, and receive a second data group and the destination address of the second data group from a third target node, write the second data group to the destination address of the second data group, and when the second target matrix is ​​not the last matrix outside the first target matrix in the first target node, update the second target matrix to the first matrix after the second target matrix and outside the first target matrix; when the second target matrix is ​​the last matrix outside the first target matrix in the first target node, update the second target matrix to the first matrix outside the first target matrix in the first target node; repeat the second step to the last step, ensuring that the pppn data in the first data group and the second data group are not repeated with the pppn data in the previous loop process, until all the data in the first target node has been transposed.

[0017] In one possible implementation, the communication module is specifically used to acquire the i-th row of data of the (i+k)%ppn-th process in the second target array, use the i-th row of data of the (i+k)%ppn-th process as the i-th data of the first data group, store the i-th data of the first data group in the queue of the i-th process in the first target node, and determine the receiving address of the i-th data in the first data group as the (i+k)%ppn-th row of the i-th process in the third target array of the second target node.

[0018] In conjunction with the previous implementation, in another possible implementation, the communication module is specifically used to obtain the local key of the (i+k)%ppn process and the virtual address of the i-th row of data of the (i+k)%ppn process through the RDMA protocol; determine the physical address of the i-th row of data of the (i+k)%ppn process based on the local key of the (i+k)%ppn process and the virtual address of the i-th row of data of the (i+k)%ppn process; and obtain the i-th row of data of the (i+k)%ppn process from the physical address of the i-th row of data of the (i+k)%ppn process.

[0019] In another possible implementation, the data of the (i+1)th process in the first data group is located at row number r of the second target matrix. i+1The data of the i-th process is located in the row number r of the second target matrix. i Satisfy the following formula: r i+1 =(r i +1)%ppn.

[0020] In another possible implementation, the entire matrix of N nodes is located in shared memory. The communication module is specifically used to obtain the i-th row of data of the (i+k)%ppn process in the second target matrix from the shared memory, take the i-th row of data of the (i+k)%ppn process as the i-th data of the first data group, and determine the receiving address of the i-th data in the first data group as the (i+k)%ppn row of the i-th process in the third target matrix of the second target node.

[0021] In combination with the second aspect or any implementation of the second aspect, in another possible implementation, the first data group, the destination address of the first data group, the second data group, and the destination address of the second data group are all sent via the RDMA protocol.

[0022] In combination with the second aspect or any implementation of the second aspect, in another possible implementation, the communication module is an RDMA network card.

[0023] In another possible implementation, the processing module is specifically used to repeatedly transpose the third data group of the first target matrix. The third data group includes ppn data that are not repeated from the ppn data included in the third data group in the previous loop process, until all data outside the main diagonal of the first target matrix have been transposed.

[0024] In another possible implementation, the data of the (i+1)th process in the third data set is located at row number r′ of the first target matrix. i+1 The data of the i-th process is located at row number r of the first target matrix. i r′ satisfies the following formula: i+1 =(r i ′+1)%ppn, i takes values ​​from 1, 2, ..., ppn.

[0025] For the explanations of terms, the steps for each module, and the beneficial effects in the second part, please refer to the corresponding descriptions in the first part.

[0026] The third aspect provides a computing device including a processor, a memory, and a communication interface, wherein the processor is configured to execute instructions stored in the memory to cause the computing device to perform the methods of the first aspect or any implementation thereof.

[0027] The fourth aspect provides a computing device cluster, the computing device cluster including at least one computing device, each computing device including a processor, a memory and a communication interface, wherein the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method of the first aspect or any implementation thereof.

[0028] The fifth aspect provides a computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, perform the method of the first aspect or any implementation thereof.

[0029] The sixth aspect provides a computer program product containing instructions that, when executed by a cluster of computing devices, execute the method of the first aspect or any implementation thereof. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of a collection communication scenario in an embodiment of this application;

[0031] Figure 2 This is a schematic diagram of a multi-player game in the existing technology;

[0032] Figure 3 This is a flowchart of a collection communication embodiment in this application;

[0033] Figure 4 This is a schematic diagram of a collection communication method in an embodiment of this application;

[0034] Figure 5 This is another schematic diagram of the collection communication method in the embodiments of this application;

[0035] Figures 6A-6E This is another schematic diagram of the collection communication in the embodiments of this application;

[0036] Figure 7 This is a structural diagram of a computing node in an embodiment of this application;

[0037] Figure 8 This is a structural diagram of a computing device in an embodiment of this application;

[0038] Figure 9 This is a structural diagram of a computing device cluster in an embodiment of this application. Detailed Implementation

[0039] The ensemble communication method described in this application can be applied to high-performance computing systems comprising n computing nodes, where n is a positive integer. See also... Figure 1In one example, a high-performance computing system includes n computing nodes, each containing one process. The processes of computing node 1 include data 11, data 12, ..., data 1n; the processes of computing node 2 include data 21, data 22, ..., data 2n; the processes of computing node n include data n1, data n2, ..., data nn; and so on for other computing nodes. Each computing node in the high-performance computing system globally transposes the data of all its processes. After the global transpose, the processes of computing node 1 include data 11, data 21, ..., data n1; the processes of computing node 2 include data 12, data 22, ..., data n2; and the processes of computing node n include data 1n, data 2n, ..., data nn. It can be seen that the global transpose transposes the data of all processes. It should be noted that each computing node can have one or more processes. When each computing node includes ppn processes, each process includes n*PPN data items.

[0040] The following is a description of some of the terms used in this application:

[0041] The Message Passing Interface (MPI) is a communication protocol standard that can run on various parallel computing architectures. Users can write message passing programs using programming languages ​​such as C, C++, and Fortran. MPI supports point-to-point communication and aggregated communication. In high-performance computing (HPC) applications, MPI communication accounts for more than 30%, while aggregated communication accounts for the largest proportion of communication between all processes. Optimizing the communication latency of aggregated communication is crucial. HPC scenarios include molecular dynamics calculations and computational chemistry (first principles) calculations, among others, where the Fast Fourier Transform (FFT) is a core function.

[0042] Collective communication (COLL) is a specific set of collective communication behaviors defined on top of point-to-point communication. It can be divided into three communication modes: one-to-many, many-to-one, and many-to-many. There are many specific communication methods in COLL, such as broadcast (MPI_Bcast), reduction (MPI_Reduce), scattering (MPI_Scatter), gathering (MPI_Gather), global reduction (Allreduce), global gathering (Allgather), and global transpose (AlltoAll).

[0043] The global transpose in this application refers to an all-to-all personalized exchange, meaning that different messages are sent to different processes. This operation is frequently used in various parallel algorithms, such as Fast Fourier Transform, matrix transformation, sample sorting, and some database join operations.

[0044] See Figure 2 In existing aggregated communication methods, high-performance computing systems typically consist of four computing nodes, each with four processes. Node 1 includes processes P0, P1, P2, and P3; Node 2 includes processes P4, P5, P6, and P7; Node 3 includes processes P8, P9, P10, and P11; and Node 4 includes processes P12, P13, P14, and P15. During the fifth pairwise exchange, P11 of node 3 sends data to P0 of node 1, while P12, P13, and P14 of node 4 send data to processes P1, P2, and P3 of node 1, respectively. This results in a many-to-one communication scenario, which can lead to network congestion. Network congestion (NC) refers to a degraded quality of service when a network node or link carries more data than its processing capacity. The effects of network congestion include queuing delays, data loss or blocking of new connections, increased network load, and decreased network throughput.

[0045] To address this problem, this application provides a collective communication method in which each computing node sends data to one computing node and receives data from one computing node during each pair exchange, thus avoiding the situation of multiple nodes transmitting data simultaneously. See also... Figure 3 In one embodiment, the collection communication method of this application includes the following steps:

[0046] S301, The first target node is transposed into the first target matrix, and the position of the first target matrix corresponds to the sequence number of the first target node.

[0047] In this embodiment, the high-performance computing system includes N nodes, also referred to as computing nodes or computing devices. Each of the N nodes comprises N non-overlapping square matrices. Each row of the square matrix includes ppn data points from ppn processes. The columns of the square matrix correspond one-to-one with the processes, and each column includes ppn data points from a single process; that is, the order of the square matrix is ​​ppn. Both N and ppn are positive integers greater than 1, and their values ​​can be set according to actual conditions. The size of the data at each node can be set according to actual conditions and is not limited in this application.

[0048] The positions of the square matrix correspond one-to-one with the node indices. For example, the i′-th square matrix corresponds to the i′-th node. i′∈[1,N] or i∈[0,N-1]. The numbering of i′ can be set according to the actual situation, and this application does not impose any restrictions. It should be noted that the first target node is any one of the N nodes, and the position of the first target square matrix in the first target node corresponds to the index of the first target node. For example, if the first target node is the 2nd node, then the first target square matrix is ​​the 2nd square matrix of the 2nd node. And so on, the first target square matrix of each node can be determined. When the first target node receives the global transpose instruction, the first target node executes S301.

[0049] S302. The first target node obtains a first data group from the second target matrix. The first data group includes ppn data items. In the first data group, any two data items are located in different rows of the second target matrix and any two data items are located in different columns of the second target matrix.

[0050] The second target matrix is ​​a matrix within the first target node that differs from the first target matrix. The position of the second target matrix within the first target node corresponds to the node's index. For example, if the first target node is the second node, the first target matrix is ​​the second matrix within that second node; if the second target matrix is ​​the third matrix, the second target node's index is 3, meaning the second target node is the third node. Since the columns of the matrix correspond one-to-one with processes, any two data points from the first data group located in different columns of the second target matrix indicate that these two data points are located in different processes within the first target node.

[0051] In an optional embodiment, the data of the (i+1)th process in the first data group is located at row number r of the second target matrix. i+1 The data of the i-th process is located in the row number r of the second target matrix. i Satisfy the following formula: r i+1 =(r i +1)%ppn, where i takes values ​​from 1, 2, ..., ppn. This formula provides a feasible method for selecting the first data set from the second target matrix.

[0052] In another optional embodiment, S302 includes: the first target node obtains the i-th row of data of the (i+k)%ppn process in the second target matrix, uses the i-th row of data of the (i+k)%ppn process as the i-th data of the first data group, and stores the i-th data of the first data group in the queue of the i-th process in the first target node.

[0053] The value of i is taken from 1 to ppn, meaning that each process from the 1st to the ppnth process can obtain one piece of data. The data obtained by the 1st process to the ppnth process are considered as the first data group, and the data obtained by the ith process is the ith piece of data in the first data group. k is the quotient obtained by dividing the difference between the loop count and 1 by (N-1), where the loop count refers to the number of times S302 to S308 are executed in the loop process. For example, when N=3, k=0 in the first and second loop processes, and k=1 in the third and fourth loop processes. Other values ​​of k can be deduced similarly.

[0054] Optionally, the first target node obtaining the i-th row of data from the (i+k)%ppn-th process in the second target matrix includes: the first target node obtaining the local key of the (i+k)%ppn-th process and the virtual address of the i-th row of data from the (i+k)%ppn-th process through the RDMA protocol; determining the physical address of the i-th row of data from the (i+k)%ppn-th process based on the local key and the virtual address of the i-th row of data from the (i+k)%ppn-th process; and obtaining the i-th row of data from the (i+k)%ppn-th process from the physical address of the i-th row of data from the (i+k)%ppn-th process. Specifically, when i = (i+k)%ppn, the i-th process of the first target node can obtain the i-th row of data from the (i+k)%ppn-th process from the queue of the i-th process. When i≠(i+k)%ppn, the i-th process of the first target node can determine the physical address of the i-th row of data of the (i+k)%ppn process based on the local key of the (i+k)%ppn process and the virtual address of the i-th row of data of the (i+k)%ppn process, and read the data in the queue of other processes from the physical address.

[0055] S303, The first target node determines the destination address of the first data group.

[0056] Optionally, S303 includes: determining that the destination address of the i-th data in the first data group is the (i+k)%ppn row of the i-th process in the third target matrix of the second target node. The position of the third target matrix in the second target node corresponds to the sequence number of the first target matrix. For example, if the sequence number of the first target node is 1, the third target matrix is ​​the first matrix of the second target node; if the sequence number of the second target node is 2, the second target matrix is ​​the second matrix of the first target node.

[0057] When the first data in the first data group is the first line of data of the first process in the second matrix of the first node, its destination address is the first line of data of the first process in the first matrix of the second node. Similarly, when the second data in the first data group is the second line of data of the second process in the second matrix of the first node, its destination address is the second line of data of the second process in the first matrix of the second node. The other data addresses in the first data group can be deduced similarly. It should be understood that in this application, the data address of the data group is the send buffer address, and the destination address is the receive buffer address.

[0058] S304. The first target node sends the first data group and its destination address to the second target node. The destination address of the first data group is used to transpose the first data group. The second target node writes the first data group into the destination address of the first data group to transpose the first data group.

[0059] After the first target node obtains the first data group and its destination address, it sends the first data group and its destination address to the second target node via the RDMA protocol. Specifically, after retrieving the i-th packet from the queue of the i-th process of the first target node, it sends the i-th packet from the queue of the i-th process of the first data group to the queue of the i-th process of the second target node. Therefore, there is a one-to-one correspondence between the i-th process of the first target node and the i-th process of the second target node. From this, it can be deduced that the i-th processes of N nodes transmit data to each other, forming the i-th inter-node communication domain. When transmitting data between nodes, the data is transmitted within the same inter-node communication domain and will not be sent to other inter-node communication domains.

[0060] The processes between any two nodes are in a one-to-one correspondence, therefore, the number of data links established between any two nodes is ppn, and ppn*(N-1) data links can be established between N nodes. In existing ensemble communication methods, the data links between nodes are not in a one-to-one correspondence, so the number of data links established in the entire ensemble communication process far exceeds ppn*(N-1). Therefore, the method of this application can significantly reduce the number of links established, increase scalability, facilitate implementation in large-scale parallel computing scenarios, and reduce network congestion.

[0061] S305, The first target node receives the second data group and the destination address of the second data group from the third target node.

[0062] The difference between the index of the second target node and the index of the first target node is equal to the remainder obtained by taking the difference between the index of the first target node and the index of the third target node modulo N. Let the index of the second target node be S2, the index of the first target node be S1, and the index of the third target node be S3. They satisfy the following formula: (S2-S1)=(S1-S3)%N. For example, if the index of the second target node is 2, the index of the first target node is 1, and N is 5, then the index of the third target node S3 satisfies the following formula: (2-1)=(1-S3)%5. According to the formula, S3=5. Taking the remainder can be understood as a modulo operation; the remainder obtained by taking the remainder is a positive number.

[0063] The third target node obtains the second data group from the matrix corresponding to the index of the first target node. The second data group includes ppn data items, where any two data items are located in different rows and columns. For example, if ppn is 2, the index of the first target node is 1, and the matrix corresponding to the index of the first target node is the first matrix, the second data group selected by the third target node from the first matrix includes the data in the first row and first column, and the data in the second row and second column. The method by which the third target node determines the destination address of the second data group is similar to the method by which the first target node determines the destination address of the first data group.

[0064] S306. The first target node writes the second data group to the destination address of the second data group.

[0065] The destination address of the second data group is used to transpose the second data group. After executing S301 to S306, it is determined whether all data of the first target node has been transposed. If all data of the first target node has not been transposed, it is determined whether the second target matrix is ​​the last matrix outside the first target matrix in the first target node. If not, then execute S307; if yes, then execute S308. If all data of the first target node has been transposed, then the process ends.

[0066] S307. The first target node updates the second target square to the first square after the second target square and outside the first target square.

[0067] S308, The first target node updates the second target square to the first square outside the first target square in the first target node.

[0068] According to S307 and S308, one matrix can be cyclically selected from the N-1 matrices of the first target node as the second target matrix, thereby updating the first data group. S302 to S308 are executed repeatedly, ensuring that the ppn data in the first data group are not duplicates of the ppn data in the previous loop, and that the ppn data in the second data group are not duplicates of the ppn data in the previous loop.

[0069] In this embodiment, when transmitting data between nodes, the first target node sends data to one computing node and receives data from another computing node. The cyclic sending and receiving of data can transpose all the squares except the first target square. The first target square can be transposed by the first target node without the need for transmission between nodes. In this way, during the global transposition process, there will be no situation where the first computing node sends data to multiple computing nodes at the same time, nor will there be a situation where the first computing node receives data from multiple computing nodes at the same time. This avoids the situation of multiple parties attacking one other, thus reducing network congestion and improving the efficiency of aggregated communication.

[0070] It should be noted that, in another optional embodiment, S301 can be executed after S302 to S308. This allows for the transposition of other matrices besides the first target matrix first, and then the transposition of the first target matrix last, thus providing another implementation scheme for aggregated communication.

[0071] The method for transposing the first target matrix is ​​described below. In an optional embodiment, step S301 includes: the first target node repeatedly transposing the third data group, wherein the ppn data included in the third data group are not repeated with the ppn data in the third data group in the previous loop process, until all data outside the main diagonal of the first target matrix has been transposed. Specifically, any two data points in the third data group are located in different rows and columns, and the third data group does not include data on the main diagonal of the first target matrix.

[0072] In the first target matrix, the rows and columns of data on the main diagonal are equal, so there is no need to transpose the data on the main diagonal. Following the method of this embodiment, ppn data points of the first target matrix can be transposed each time, and repeating the process ppn-1 times will transpose all data points of the first target matrix.

[0073] Optionally, the data of the (i+1)th process in the third data set is located at row number r′ of the first target matrix. i+1 The data of the i-th process is located at row number r of the first target matrix. i r′ satisfies the following formula: i+1 =(r i'+1)%ppn. i takes values ​​from 1 to ppn, meaning each process from the 1st to the ppnth process can acquire one data point, and the data in the third data group is arranged in a hierarchical manner. This provides a method for selecting the third data group from the first target matrix. It should be understood that the method for selecting the third data group from the first target matrix is ​​not limited to the example above.

[0074] In another optional embodiment, S301 includes: the first target node repeatedly transposes the fourth data group, wherein the ppn data included in the fourth data group are not repeated with the ppn data of the fourth data group in the previous loop process, until all the data in the first target matrix has been transposed.

[0075] In this fourth data group, any two data points are located in different rows and columns. The fourth data group can be the data on the main diagonal of the first target matrix. In this embodiment, ppn data points of the first target matrix can be transposed each time; repeating this process ppn times will transpose all data points of the first target matrix.

[0076] In another optional embodiment, S302 includes: obtaining the i-th row of data of the (i+k)%ppn-th process in the second target matrix from shared memory, and using the i-th row of data of the (i+k)%ppn-th process as the i-th data of the first data group; S303 includes: determining that the destination address of the i-th data in the first data group is the (i+k)%ppn-th row of the i-th process in the third target matrix of the second target node, wherein the position of the third target matrix in the second target node corresponds to the sequence number of the first target matrix.

[0077] In this embodiment, each node of the high-performance computing system writes N square matrices into shared memory. Thus, all the square matrices of the N nodes reside in shared memory, and the N square matrices of the N nodes form a ppn*N matrix. Let i be a series of iterations 1, 2, ..., ppn, and k be the quotient obtained by dividing the difference between the iteration count and 1 by (N-1). Following the method of this embodiment, a first data group can be obtained from shared memory, and then the data group from shared memory can be sent and received via the RDMA protocol. This reduces the processing latency of the data group on the computing node and improves the speed of aggregated communication.

[0078] This application can transmit data between nodes via the RDMA protocol, such as a first data group, the destination address of the first data group, a second data group, or the destination address of the second data group. The method by which other computing nodes in a high-performance computing system send data groups via the RDMA protocol is similar to the method by which the first target node sends the first data group to the second target node via the RDMA protocol, and will not be described in detail here. It should be noted that this application can also transmit the above data between nodes via the Remote Direct Memory Access over Converged Ethernet (RoCE) protocol. Each computing node can send and receive data via the RDMA protocol or the RoCE protocol, which can reduce the data processing latency at the computing nodes and improve the speed of aggregated communication.

[0079] In one embodiment, the pseudocode of the collection communication method of this application is as follows:

[0080]

[0081]

[0082] In one embodiment, the MPI interface corresponding to the collection communication method of this application includes the following code:

[0083]

[0084] Each node in a high-performance computing system can synchronously execute the set communication method described in this application to achieve global transposition. A specific example is provided below to illustrate the set communication method of this application; please refer to [reference needed]. Figure 4 In one embodiment, each node includes two processes: Node 1 includes processes P0 and P1, Node 2 includes processes P2 and P3, and Node 3 includes processes P4 and P5. All processes of each node belong to the same intra-node communication domain, and processes with the same local sequence number in each node belong to the same inter-node communication domain. For example, P0, P2, and P4 all have a local sequence number of 0, and their inter-node communication domain sequence number can be recorded as 0; P1, P3, and P5 all have a local sequence number of 1, and their inter-node communication domain sequence number can be recorded as 1. Data is first sent in the inter-node communication domain, and then data is sent in the intra-node communication domain, thereby completing the global transpose. See also... Figure 5 The following describes the process of sending data in the inter-node communication domain and the intra-node communication domain:

[0085] The first pairwise exchange in the inter-node communication domain specifically includes: sending data from P0 in node 1 to P2 in node 2, sending data from P1 in node 1 to P3 in node 2, sending data from P2 in node 2 to P4 in node 3, sending data from P3 in node 2 to P5 in node 3, sending data from P4 in node 3 to P0 in node 1, and sending data from P5 in node 3 to P1 in node 1.

[0086] The second pairwise exchange in the inter-node communication domain specifically includes: sending data from P0 in node 1 to P4 in node 3, sending data from P1 in node 1 to P5 in node 3, sending data from P4 in node 3 to P2 in node 2, sending data from P5 in node 3 to P3 in node 2, sending data from P2 in node 2 to P0 in node 1, and sending data from P3 in node 2 to P1 in node 1.

[0087] The third pairwise exchange in the inter-node communication domain specifically includes: sending data from P0 in node 1 to P2 in node 2, sending data from P1 in node 1 to P3 in node 2, sending data from P2 in node 2 to P4 in node 3, sending data from P3 in node 2 to P5 in node 3, sending data from P4 in node 3 to P0 in node 1, and sending data from P5 in node 3 to P1 in node 1.

[0088] The fourth pairwise exchange in the inter-node communication domain specifically includes: sending data from P0 in node 1 to P4 in node 3, sending data from P1 in node 1 to P5 in node 3, sending data from P4 in node 3 to P2 in node 2, sending data from P5 in node 3 to P3 in node 2, sending data from P2 in node 2 to P0 in node 1, and sending data from P3 in node 2 to P1 in node 1.

[0089] A pairwise exchange within the intra-node communication domain specifically includes: data from P0 and P1 in transposed node 1, data from P2 and P3 in transposed node 2, and data from P4 and P5 in transposed node 3.

[0090] To facilitate understanding, the collection communication method of this application will be described in detail below with a specific example:

[0091] See Figures 6A to 6EIn one embodiment, ppn = 2 and N = 3. Node 1 includes two processes, P0 and P1; node 2 includes two processes, P2 and P3; and node 3 includes two processes, P4 and P5. The data for process P0 includes P00, P01, P02, P03, P04, and P05; the data for process P1 includes P10, P11, P12, P13, P14, and P15; the data for process P2 includes P20, P21, P22, P23, P24, and P25; the data for process P3 includes P30, P31, P32, P33, P34, and P35; the data for process P4 includes P40, P41, P42, P43, P44, and P45; and the data for process P5 includes P50, P51, P52, P53, P54, and P55.

[0092] Node 1 consists of three square matrices: the first square matrix includes P00, P01, P10, and P11; the second square matrix includes P02, P03, P12, and P13; and the third square matrix includes P04, P05, P14, and P15. Similarly, node 2 consists of three square matrices: the first square matrix includes P20, P21, P30, and P31; the second square matrix includes P22, P23, P32, and P33; and the third square matrix includes P24, P25, P34, and P35. Node 3 consists of three square matrices: the first square matrix includes P40, P41, P50, and P51; the second square matrix includes P42, P43, P52, and P53; and the third square matrix includes P44, P45, P54, and P55.

[0093] See Figure 6A During the first pairwise exchange in the inter-node communication domain, nodes 1, 2, and 3 synchronously send their first data sets. Node 1's second target array is the second array, and the first data set obtained from the second array includes P02 and P13. Node 1 sends P02, P13, and their destination addresses to node 2. Node 2 stores P02 in the first row of P2 and P13 in the second row of P3 based on their destination addresses. Node 2's second target array is the third array, and the first data set obtained from the third array includes P24 and P35. Node 2 sends P24, P35, and their destination addresses to node 3. Node 2 stores P24 in the third row of P4 and P35 in the fourth row of P5 based on their destination addresses. The second target array of node 3 is the first array. The first data group obtained from the first array includes P40 and P51. P40, P51, and their destination addresses are sent to node 1. Node 1 stores P40 in the 5th row of P0 and P51 in the 6th row of P1 according to their destination addresses. Since all data of each node has not been transposed, the second target array of each node is updated.

[0094] See Figure 6B During the second pairwise exchange in the inter-node communication domain, nodes 1, 2, and 3 synchronously send their first data sets. Node 1's second target array is the third array, and the first data sets obtained from the third array include P04 and P15. Node 1 sends P04, P15, and their destination addresses to node 3. Node 3 stores P04 in the first row of P4 and P15 in the second row of P5 based on their destination addresses. Node 2's second target array is the first array, and the first data sets obtained from the first array include P20 and P31. Node 2 sends P20, P31, and their destination addresses to node 1. Node 1 stores P20 in the third row of P0 and P31 in the fourth row of P1 based on their destination addresses. The second target matrix of node 3 is the second matrix. The first data group obtained from the second matrix includes P42 and P53. P42, P53, and their destination addresses are sent to node 2. Node 2 stores P42 in the 5th row of P2 and P53 in the 6th row of P3 according to their destination addresses. Since the transpose of all data of each node is not complete, the second target matrix of each node is updated.

[0095] See Figure 6C During the third pairwise exchange in the inter-node communication domain, nodes 1, 2, and 3 synchronously send their first data sets. Node 1's second target array is the second array, and the first data set obtained from the second array includes P03 and P12. Node 1 sends P03, P12, and their destination addresses to node 2. Node 2 stores P03 in the first row of P3 and P12 in the second row of P2 based on their destination addresses. Node 2's second target array is the third array, and the first data set obtained from the third array includes P25 and P34. Node 3 sends P25, P34, and their destination addresses to node 3. Node 4 stores P25 in the third row of P5 and P34 in the fourth row of P4 based on their destination addresses. The second target array of node 3 is the first array. The first data group obtained from the first array includes P41 and P50. P41, P50, and their destination addresses are sent to node 1. Node 1 stores P41 in the 5th row of P1 and P50 in the 6th row of P0 according to their destination addresses. Since the transpose of all data of each node is not complete, the second target array of each node is updated.

[0096] See Figure 6DDuring the fourth pairwise exchange in the inter-node communication domain, nodes 1, 2, and 3 synchronously send their first data sets. Node 1's second target array is the third array, and the first data sets obtained from the third array include P05 and P14. Node 1 sends P05, P14, and their destination addresses to node 3. Node 3 stores P05 in the first row of P5 and P14 in the second row of P4 based on their destination addresses. Node 2's second target array is the first array, and the first data sets obtained from the first array include P21 and P30. Node 2 sends P21, P30, and their destination addresses to node 1. Node 1 stores P21 in the third row of P1 and P30 in the fourth row of P0 based on their destination addresses. The second target array of node 3 is the second array. The first data group obtained from the second array includes P43 and P52. P43, P52 and their destination addresses are sent to node 2. Node 2 stores P43 in the 5th row of P3 and P52 in the 6th row of P2 according to their destination addresses.

[0097] See Figure 6E Nodes 1, 2, and 3 synchronously transpose the first target matrix of their respective nodes. Node 1's first target matrix is ​​the first matrix; it selects P01 and P10 from the first matrix as its third data set and transposes them. Node 2 determines its first target matrix as the second matrix, selects P32 and P23 from the second matrix as its third data set, and transposes them. Node 3 determines its first target matrix as the third matrix, selects P54 and P45 from the third matrix as its third data set, and transposes them. Therefore... Figure 6E The data processing procedure shown is a pairwise exchange within the communication domain of a node. After obtaining the main diagonal data of the first target matrix, the transpose matrix of the first target matrix can be obtained. At this point, all the data of each node has been transposed.

[0098] It should be understood that the address numbers can be set according to the actual situation. For example, the numbers for rows 1, 2, 3, 4, 5, and 6 can be 0, 1, 2, 3, 4, and 5, respectively. This application does not limit this.

[0099] The following section introduces the collective communication method using the RDMA protocol. The test results of this method and existing collective communication methods are shown in Table 1:

[0100]

[0101] Table 1

[0102] As shown in Table 1, this application can significantly reduce the latency of aggregated communication. The priority-based flow control (PFC) network flow control count represents the data flow links, demonstrating a significant reduction in data flow links, thus reducing network congestion and facilitating the scaling of high-performance computing systems.

[0103] See Figure 7 In one embodiment, the computing node 700 of this application includes a processing module 701 and a communication module 702. The processing module 701 is used to transpose a first target matrix, and the communication module 702 is used to obtain a first data group from a second target matrix, determine the destination address of the first data group, send the first data group and its destination address to a second target node, receive a second data group and its destination address from a third target node, write the second data group to its destination address, update the second target matrix to the first matrix after the second target matrix and outside the first target matrix when the second target matrix is ​​not the last matrix outside the first target matrix in the first target node, update the second target matrix to the first matrix outside the first target matrix in the first target node when the second target matrix is ​​the last matrix outside the first target matrix in the first target node, and repeat the second step to the last step, ensuring that the pppn data in the first data group and the second data group are not repeated with the pppn data in the previous loop process, until all the data in the first target node has been transposed.

[0104] In an optional embodiment, the data of the (i+1)th process in the first data group is located at row number r of the second target matrix. i+1 The data of the i-th process is located in the row number r of the second target matrix. i Satisfy the following formula: r i+1 =(r i +1)%ppn, i takes values ​​from 1, 2, ..., ppn.

[0105] In another optional embodiment, the communication module 702 is specifically used to obtain the i-th row of data of the (i+k)%ppn process in the second target array, take the i-th row of data of the (i+k)%ppn process as the i-th data of the first data group, and determine the receiving address of the i-th data in the first data group as the (i+k)%ppn row of the i-th process in the third target array of the second target node.

[0106] In another optional embodiment, the communication module 702 is specifically configured to obtain the local key of the (i+k)%ppn process and the virtual address of the i-th row of data of the (i+k)%ppn process via the RDMA protocol; determine the physical address of the i-th row of data of the (i+k)%ppn process based on the local key of the (i+k)%ppn process and the virtual address of the i-th row of data of the (i+k)%ppn process; and obtain the i-th row of data of the (i+k)%ppn process from the physical address of the i-th row of data of the (i+k)%ppn process.

[0107] In another optional embodiment, the communication module 702 is specifically used to obtain the i-th row of data of the (i+k)%ppn-th process in the second target array from the shared memory, use the i-th row of data of the (i+k)%ppn-th process as the i-th data of the first data group, store the i-th data of the first data group in the queue of the i-th process in the first target node, and determine the receiving address of the i-th data in the first data group as the (i+k)%ppn-th row of the i-th process in the third target array of the second target node.

[0108] In another alternative embodiment, the first data group, the destination address of the first data group, the second data group, and the destination address of the second data group are all sent via the RDMA protocol.

[0109] In another optional embodiment, the processing module 701 is specifically used to repeatedly transpose the third data group of the first target matrix, wherein the ppn data included in the third data group are not repeated with the ppn data included in the third data group in the previous loop process, until all data outside the main diagonal of the first target matrix has been transposed.

[0110] In another possible implementation, the data of the (i+1)th process in the third data set is located at row number r′ of the first target matrix. i+1 The data of the i-th process is located at row number r of the first target matrix. i r′ satisfies the following formula: i+1 =(r i ′+1)%ppn.

[0111] See Figure 8 In one embodiment, the computing device 800 of this application includes: a bus 802, a processor 804, a memory 806, and a communication interface 808. The processor 804, the memory 806, and the communication interface 808 communicate with each other via the bus 802. It should be understood that this application does not limit the number of processors and the number of memories in the computing device 800.

[0112] The 802 bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 8 The bus 804 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 804 may include a path for transmitting information between various components of the computing device 800 (e.g., memory 806, processor 804, communication interface 808).

[0113] The processor 804 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0114] The memory 806 may include volatile memory, such as random access memory (RAM). The processor 804 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD). In some embodiments, the memory 806 stores executable program code, which the processor 804 executes to implement the functions of the aforementioned processing module 701. In other embodiments, the memory 806 stores executable program code, which the processor 804 executes to implement the functions of the aforementioned processing module 701 and communication module 702, respectively.

[0115] The communication interface 808 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 800 and other devices or communication networks. In some embodiments, the communication interface 808 can implement the functions of the aforementioned communication module 702.

[0116] like Figure 9As shown, the computing device cluster includes at least one computing device 800. The memory 806 of one or more computing devices 800 in the computing device cluster may store the same instructions for executing the aggregate communication method. The computing device cluster can serve as the high-performance computing system of this application, and the computing devices can serve as computing nodes in the high-performance computing system.

[0117] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform a collective communication method.

[0118] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform a collective communication method.

[0119] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A collective communication method in a high-performance computing system, characterized in that, The high-performance computing system includes N nodes, which are labeled with consecutive numbers. Each node includes N square matrices, where each row of the square matrix includes ppn data points from ppn processes, and each column of the square matrix includes ppn data points from a single process. N and ppn are both positive integers greater than 1. The method includes: The first target node is a transpose of the first target matrix, where the first target node is any one of the N nodes, and the position of the first target matrix in the first target node corresponds to the index of the first target node. The first target node obtains the first data group from the second target matrix. The position of the second target matrix in the first target node corresponds to the sequence number of the second target node. The second target matrix is ​​different from the first target matrix. The first data group includes ppn data. In the first data group, any two data are located in different rows of the second target matrix and any two data are located in different columns of the second target matrix. The first target node determines the destination address of the first data group; The first target node sends the first data group and the destination address of the first data group to the second target node, whereby the destination address of the first data group is used to transpose the first data group; The first target node receives a second data group from the third target node and the destination address of the second data group. The difference between the sequence number of the second target node and the sequence number of the first target node is equal to the remainder obtained by taking the difference between the sequence number of the first target node and the sequence number of the third target node modulo N. The second data group includes ppn data. The first target node writes the second data group to the destination address of the second data group, and the destination address of the second data group is used to transpose the second data group; When the second target array is not the last array outside the first target array in the first target node, the first target node updates the second target array to the first array after the second target array and outside the first target array. When the second target matrix is ​​the last matrix outside the first target matrix in the first target node, the first target node updates the second target matrix to the first matrix outside the first target matrix in the first target node. The first target node repeats the second step to the last step. The ppn data in the first data group and the second data group are not repeated with the ppn data in the previous loop process, until all the data of the first target node has been transposed.

2. The method according to claim 1, characterized in that, The first target node obtains the first data group from the second target matrix by: the first target node obtains the i-th row of data of the (i+k)%ppn process in the second target matrix, takes the i-th row of data of the (i+k)%ppn process as the i-th data of the first data group, and stores the i-th data of the first data group in the queue of the i-th process in the first target node, where i takes the values ​​1, 2, ...,ppn, and k is equal to the quotient obtained by dividing the difference between the number of loops and 1 by (N-1); The first target node determines the destination address of the first data group by: the first target node determines that the destination address of the i-th data in the first data group is the (i+k)%ppn row of the i-th process in the third target matrix of the second target node, and the position of the third target matrix in the second target node corresponds to the sequence number of the first target matrix.

3. The method according to claim 2, characterized in that, The first target node obtains the i-th row of data from the (i+k)%ppn-th process in the second target matrix, including: The first target node obtains the local key of the (i+k)%ppn process and the virtual address of the i-th row of data of the (i+k)%ppn process through the Remote Direct Memory Access (RDMA) protocol; determines the physical address of the i-th row of data of the (i+k)%ppn process based on the local key of the (i+k)%ppn process and the virtual address of the i-th row of data of the (i+k)%ppn process; and obtains the i-th row of data of the (i+k)%ppn process from the physical address of the i-th row of data of the (i+k)%ppn process.

4. The method according to claim 1, characterized in that, The entire matrix of the N nodes is located in shared memory; The first target node obtains the first data group from the second target matrix by: the first target node obtains the i-th row of data of the (i+k)%ppn process in the second target matrix from the shared memory, and takes the i-th row of data of the (i+k)%ppn process as the i-th data of the first data group, where i takes all values ​​from 1, 2, ..., ppn, and k is the quotient obtained by dividing the difference between the number of loops and 1 by (N-1); The first target node determines the destination address of the first data group by: the first target node determines that the destination address of the i-th data in the first data group is the (i+k)%ppn row of the i-th process in the third target matrix of the second target node, and the position of the third target matrix in the second target node corresponds to the sequence number of the first target matrix.

5. The method according to any one of claims 1 to 4, characterized in that, The first data group, the destination address of the first data group, the second data group, and the destination address of the second data group are all sent via the RDMA protocol.

6. The method according to any one of claims 1 to 5, characterized in that, The transpose of the first target node into the first target matrix includes: The first target node repeatedly transposes the third data group of the first target matrix. The ppn data included in the third data group are not repeated with the ppn data included in the third data group in the previous loop process, until all data outside the main diagonal of the first target matrix is ​​transposed. In the third data group, any two data are located in different rows and any two data are located in different columns. The third data group does not include data on the main diagonal of the first target matrix.

7. The method according to claim 6, characterized in that, The data of the (i+1)th process in the third data group is located at row number r in the first target matrix. i+1 The data of the i-th process is located in the row number r of the first target matrix. i Satisfy the following formula: r i+1 =(r i +1)%ppn, where i takes values ​​from 1, 2, ..., ppn.

8. A computing node, characterized in that, The computing node is used as the first target node, and the computing node includes: The processing module is used to transpose the first target matrix, wherein the position of the first target matrix in the first target node corresponds to the sequence number of the first target node; A communication module is used to acquire a first data group from a second target array, wherein the position of the second target array in the first target node corresponds to the index of the second target node, the second target array is different from the first target array, the first data group includes ppn data items, and any two data items in the first data group are located in different rows and columns of the second target array; determine the destination address of the first data group; send the first data group and its destination address to the second target node, the destination address of the first data group being used to transpose the first data group; receive a second data group and its destination address from a third target node, the difference between the index of the second target node and the index of the first target node being equal to the difference between the index of the first target node and the index of the third target node. The remainder obtained by taking N modulo 1 indicates that the second data group includes ppn data items. The second data group is written to its destination address, which is used to transpose the second data group. When the second target matrix is ​​not the last matrix outside the first target matrix in the first target node, the second target matrix is ​​updated to be the first matrix after the second target matrix and outside the first target matrix. When the second target matrix is ​​the last matrix outside the first target matrix in the first target node, the second target matrix is ​​updated to be the first matrix outside the first target matrix in the first target node. The second step is repeated until the last step, ensuring that the ppn data items in the first and second data groups are not repeated with the ppn data items in the previous loop, until all data in the first target node has been transposed.

9. The computing node according to claim 8, characterized in that, The communication module is specifically used to acquire the i-th row of data of the (i+k)%ppn-th process in the second target matrix, where i takes values ​​from 1, 2, ..., ppn, and k is equal to the quotient obtained by dividing the difference between the number of cycles and 1 by (N-1); the i-th row of data of the (i+k)%ppn-th process is used as the i-th data of the first data group, and the receiving address of the i-th data in the first data group is determined to be the (i+k)%ppn-th row of the i-th process in the third target matrix of the second target node, where the position of the third target matrix in the second target node corresponds to the sequence number of the first target matrix.

10. The computing node according to claim 9, characterized in that, The communication module is specifically used to obtain the local key of the (i+k)%ppn process and the virtual address of the i-th row of data of the (i+k)%ppn process through the Remote Direct Memory Access (RDMA) protocol; determine the physical address of the i-th row of data of the (i+k)%ppn process based on the local key of the (i+k)%ppn process and the virtual address of the i-th row of data of the (i+k)%ppn process; and obtain the i-th row of data of the (i+k)%ppn process from the physical address of the i-th row of data of the (i+k)%ppn process.

11. The computing node according to claim 8, characterized in that, The entire array of the N nodes is located in shared memory. The communication module is specifically used to obtain the i-th row of data of the (i+k)%ppn process in the second target array from the shared memory, and take the i-th row of data of the (i+k)%ppn process as the i-th data of the first data group, where i takes values ​​from 1, 2, ..., ppn, and k is the quotient obtained by dividing the difference between the number of loops and 1 by (N-1); determine the receiving address of the i-th data in the first data group as the (i+k)%ppn row of the i-th process in the third target array of the second target node, where the position of the third target array in the second target node corresponds to the sequence number of the first target array.

12. The computing node according to any one of claims 8 to 11, characterized in that, The first data group, the destination address of the first data group, the second data group, and the destination address of the second data group are all sent via the RDMA protocol.

13. The computing node according to any one of claims 8 to 12, characterized in that, The processing module is specifically used to repeatedly transpose the third data group of the first target matrix. The third data group includes ppn data that are not repeated from the ppn data included in the third data group in the previous loop process, until all data outside the main diagonal of the first target matrix has been transposed. In the third data group, any two data are located in different rows and any two data are located in different columns. The third data group does not include data on the main diagonal of the first target matrix.

14. The computing node according to claim 13, characterized in that, The data of the (i+1)th process in the third data group is located at row number r in the first target matrix. i+1 The data of the i-th process is located in the row number r of the first target matrix. i Satisfy the following formula: r i+1 =(r i +1)%ppn, where i takes values ​​from 1, 2, ..., ppn.

15. A computing device, characterized in that, The computing device includes a processor, a memory, and a communication interface, wherein the processor is configured to execute instructions stored in the memory to cause the computing device to perform the method of any one of claims 1 to 7.

16. A computing device cluster, characterized in that, The computing device cluster includes at least one computing device, each computing device including a processor, a memory and a communication interface, wherein the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device to cause the computing device cluster to perform any one of methods 1 to 7.

17. A computer-readable storage medium, characterized in that, It includes computer program instructions that, when executed by a cluster of computing devices, perform the method of any one of claims 1 to 7.

18. A computer program product comprising instructions which, when executed by a cluster of computing devices, perform the method of any one of claims 1 to 7.

Images