Multi-process collaborative data collection method based on the new generation of Sunway supercomputer

Abstract

The invention relates to a multi-process collaborative data collection method based on a new generation of Shenwei supercomputer, and belongs to the field of electronic information technology. The method comprises: collecting all the data of six processes in the processor to a local root node in the processor through a small-scale tree communication inside the processor, so as to realize the first data packaging operation; establishing an offset array backup, so as to determine the size of the cache area to be applied for at the top level of message transmission, and exchanging the size information of the collected data with each other; storing the data collected after the first packaging operation in the entire processor in its own message buffer in sequence, taking it as needed, saving the buffer, and realizing the secondary data packaging operation, so as to reduce the number of message communications between processes and reduce the time consumption of inter-process communication.

Description

Technical Field

[0001] The present invention relates to the field of electronic information technology, and in particular to a multi-process collaborative data collection method based on a new generation of Sunway supercomputer. Background Art

[0002] At present, supercomputers play an indispensable supporting role in national security, economy and social progress. For a long time, my country has attached great importance to the independent research and development of domestic supercomputer chips. At present, domestic supercomputers have made significant progress, and many domestic high-performance processor chips have been successfully developed and put into use. The Shenwei series of multi-core processors represent the outstanding achievements of my country's independent research and development of high-performance processors.

[0003] The new generation of Sunway supercomputer "Sunway Blue Light II" is composed of 4 computing supernodes, each of which contains 128 computing nodes. Each computing node is equipped with two SW26010 pro processors, and the nodes are connected by a domestically-made, independently controllable high-speed interconnection network.

[0004] The SW26010pro processor independently developed by my country is a new generation of high-performance heterogeneous many-core processor. The SW26010pro processor is built with 6 core groups (Core-Group, CG), which are connected in a ring architecture through an on-chip network. Each core group contains a management processing element (MPE, also known as the main core) and a slave core array (Computing Processing Elements, CPEs). The slave core array consists of 64 slave cores (Computing Processing Elements, CPEs) arranged in an 8×8 mesh structure. The SW26010pro contains a total of 6 main cores and 384 slave cores. The hardware architecture of the SW26010pro processor is as follows Figure 1 Each node contains two SW26010pro processors. The internal diagram of the computing node is shown in Figure 2 shown.

[0005] Currently, there are relatively few process-level optimizations based on the Sunway series of supercomputers. Programs executed on Sunway supercomputers usually require hundreds or even thousands of processes. When a large number of processes communicate with each other, the load is unbalanced, communication bottlenecks are likely to occur, and it takes a lot of time and wastes resources. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes a multi-process collaborative data collection method based on a new generation of Sunway supercomputer;

[0007] The present invention proposes a method for twice packaging data based on the characteristics of the hardware architecture of the new generation of Sunway supercomputers. The data in the process is collected twice in a tree architecture and a special Sunway architecture respectively, and the data is packaged and transmitted uniformly, thereby optimizing inter-process communication.

[0008] The technical solution of the present invention is:

[0009] The multi-process collaborative data collection method based on the new generation of Sunway supercomputer includes:

[0010] Through small-scale tree-shaped communication inside the processor, the data of the six processes inside the processor are all collected to the local root node in the processor, realizing the first data packaging operation;

[0011] Any local root node obtains data information to be sent by other local root nodes from other local root nodes, and establishes an offset array backup to determine the size of the buffer area to be applied for at the top level of message transmission, and exchanges the size information of the collected data with each other;

[0012] Each local root node (including process No. 0) sends the contents of its cache to process No. 0. Process No. 0 applies for a large buffer block based on the serial number of the process receiving the data. The size of the large buffer block is the sum of the amount of data to be transmitted by each process in each started processor. It is used to accommodate the data collected after the first packing operation in the entire processor. The data collected after the first packing operation in the entire processor is stored in its own message buffer in sequence, and is accessed on demand, saving buffer space and realizing secondary data packaging operations to reduce the number of message communications between processes.

[0013] Preferably, according to the present invention, the first packaging operation refers to: collecting the data of the six processes on a processor into the first process of the processor (the first process is defined as the local root node); including:

[0014] Six processes pass data through a tree diagram, which is divided into three layers. First, all processes in the bottom processor transmit data to adjacent processes with smaller process numbers than their own. At this time, processes 0, 2, and 4 obtain the data. Then, process 2 continues to transmit data to process 0, and process 4 does not perform any operation. At this time, process 0 collects data from processes 0, 1, 2, and 3, and process 4 collects data from processes 4 and 5. Finally, process 4 sends data to process 0, which is the local root node, and the first packaging operation inside the processor is completed.

[0015] Preferably, according to the present invention, establishing an offset array backup means:

[0016] Each local root node applies for a piece of memory space and establishes an offset array backup stride to obtain the size of data transmitted by each process, so that the root process can eventually receive data of different sizes; the data is stored in the root process buffer in sequence according to the serial number in the source process. The data sent by the nth process to the root process is stored in the root process, and the nth part in the receiving buffer is received. The offset of the nth part in the buffer is stride[n].

[0017] Preferably, according to the present invention, any local root node obtains data information to be sent by other local root nodes from other local root nodes through RMA operation, ie, through MPI_GET(), and copies the data from the TARGET memory to the ORIGIN memory.

[0018] Further preferably, the data information to be sent by other local root nodes is obtained from other local root nodes through MPI_GET(); including:

[0019] Transfer data from the ORIGIN process starting address ORIGIN_ADDR to the TARGET process, the TARGET process is specified by the parameters WIN and TARGET_RANK; the number of transmitted data is ORIGIN_COUNT; ORIGIN_COUNT and TARGET_COUNT are the number of elements in the ORIGIN and TARGET buffers respectively, the type is specified by the parameter ORIGIN_DATATYPE, the data is written in the TARGET process buffer, the address TARGET_ADDR is obtained by the following formula: TARGET_ADDR = WINDOW_BASE+TARGET_DISP*DISP_UNIT, where WINDOW_BASE and DISP_UNIT are the base address (starting address) and the window displacement unit specified by the TARGET process when the window is initialized.

[0020] Preferably, according to the present invention, at the top level of message passing, that is, process No. 0 uses the function MPI_GATHERV() to determine the size of the buffer area to be applied for by establishing an offset array backup.

[0021] Further preferably, the size of the buffer area to be applied for is determined by establishing an offset array backup; including: the function MPI_GATHERV() receives information in the stride array, applies for a buffer on demand, and ensures that the buffer size is equal to the sum of data received by each local root node.

[0022] Preferably, according to the present invention, by calling the FORTRAN interface SDPO, the first packing operation of the tree structure communication of the data in the processor is transparently implemented, as well as the secondary packing operation of the data between the local root nodes based on the hardware architecture of the new generation Sunway supercomputer.

[0023] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of a multi-process collaborative data collection method based on a new generation of Sunway supercomputer are implemented.

[0024] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a multi-process collaborative data collection method based on a new generation of Sunway supercomputer.

[0025] The beneficial effects of the present invention are:

[0026] 1. When a parallel program communicates between a large number of processes, it is easy to cause load imbalance, communication bottlenecks, and waste resources. The present invention proposes a method to pack data twice, thereby reducing the time consumption of inter-process communication.

[0027] 2. The present invention avoids communication blocking and improves the efficiency of parallel programming by adjusting the inter-process communication mode while ensuring fast data transmission without data loss.

[0028] 3. The method and interface proposed in the present invention are applicable to the Shenwei supercomputer based on the new generation Shenwei multi-core processor, and have a certain degree of universality. After multiple case tests, the present invention has obvious acceleration effect and can well meet the needs of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is a schematic diagram of the SW26010 pro processor architecture;

[0030] Figure 2 This is a schematic diagram of the internal computing nodes of the new generation Sunway supercomputer;

[0031] Figure 3 A schematic diagram of a small-scale tree-shaped communication within a processor using the method of the present invention;

[0032] Figure 4 A schematic diagram of internal communication of a processor not using the method of the present invention;

[0033] Figure 5 A schematic diagram of storing arrays of different sizes in the buffer by offset;

[0034] Figure 6 This is a schematic diagram of the data secondary packaging operation. DETAILED DESCRIPTION

[0035] The present invention will be further defined below in conjunction with the accompanying drawings and embodiments, but is not limited thereto.

[0036] Example 1

[0037] The multi-process collaborative data collection method based on the new generation of Sunway supercomputer includes:

[0038] The processor collects all the data of the six processes in the processor to the local root node in the processor through a small-scale tree communication, thus realizing the first data packaging operation; this operation can effectively ensure the load balancing of the communication process and avoid communication blocking. Figure 3 As shown, the common method is Figure 4 shown.

[0039] Any local root node obtains data information to be sent by other local root nodes from other local root nodes, and establishes an offset array backup to determine the size of the buffer area to be applied for at the top level of message transmission, and exchanges the size information of the collected data with each other;

[0040] Each local root node (including process 0) sends the contents of its cache to process 0. Based on the serial number of the process receiving the data, process 0 applies for a large buffer block. The size of the large buffer block is the sum of the amount of data to be transmitted by each process in each processor started. It is used to accommodate the data collected after the first packaging operation in the entire processor. The data collected after the first packaging operation in the entire processor is stored in its own message buffer in sequence, and is used on demand to save buffers and implement secondary data packaging operations to reduce the number of message communications between processes. The specific process is as follows: Figure 6 shown.

[0041] Example 2

[0042] The difference between the multi-process collaborative data collection method based on the new generation Sunway supercomputer described in Example 1 is that:

[0043] Terminology explanation:

[0044] 1. MPI_GET(), copies data from TARGET memory to ORIGIN memory. MPI_GET(ORIGIN_ADDR,ORIGIN_COUNT, ORIGIN_DATATYPE, TARGET_RANK,

[0045] TARGET_DISP, TARGET_COUNT, TARGET_DATATYPE, WIN, IERROR)

[0046] <type>ORIGIN_ADDR(*)

[0047] INTEGER(KIND=MPI_ADDRESS_KIND)TARGET_DISP INTEGER ORIGIN_COUNT,

[0048] ORIGIN_DATATYPE,TARGET_RANK,TARGET_COUNT,TARGET_DATATYPE,WIN, IERROR The parameters have the following meanings: OUT ORIGIN_ADDR: the starting address of the ORIGIN buffer (optional);

[0049] IN ORIGIN_COUNT: the number of elements entering the ORIGIN buffer (non-negative integer);

[0050] IN ORIGIN_DATATYPE: data type (handle) of each element entering the ORIGIN buffer;

[0051] IN TARGET_RANK: TARGET's process number (non-negative integer);

[0052] IN TARGET_DISP: displacement from the start position of the window to the TARGET window (non-negative integer);

[0053] IN TARGET_COUNT: the number of elements entering the TARGET buffer (non-negative integer);

[0054] IN TARGET_DATATYPE: data type (handle) of each element entering the TARGET buffer;

[0055] IN WIN: The window object (handle) used for communication.

[0056] 2. MPI_GATHERV() can receive different amounts of data from different processes and store the received data anywhere in the root process message buffer.

[0057] MPI_GATHERV (SENDBUF, SENDCOUNT, SENDTYPE, RECVBUF, RECVCOUNTS, DISPLS,

[0058] RECVTYPE, ROOT, COMM, IERROR)

[0059] <type>SENDBUF(*), RECVBUF(*)

[0060] INTEGER SENDCOUNT, SENDTYPE, RECVCOUNTS (*), DISPLS (*) RECVTYPE, ROOT, COMM, IERROR

[0061] The parameters have the following meanings:

[0062] IN SENDBUF: the starting address of the send message buffer (variable);

[0063] IN SENDCOUNT: the number of data in the send message buffer (integer);

[0064] IN SENDTYPE: data type (handle) in the send message buffer;

[0065] OUT RECVBUF: starting address of the receive message buffer (variable, only for the root process);

[0066] IN RECVCOUNTS: integer array (length is the size of the group), whose value is received from each process

[0067] The number of data (only for the root process);

[0068] IN DISPLS: integer array, each entry represents the displacement relative to RECVBUF, this bit

[0069] The place where the input data received from the process is stored (only for the root process);

[0070] IN RECVTYPE: data type in the receive message buffer (only for root process) (handle);

[0071] IN ROOT: receiving process serial number (handle);

[0072] IN COMM: communicator (handle).

[0073] The first packaging operation is to collect the data of the six processes on a processor into the first process of the processor (the first process is defined as the local root node); including:

[0074] Six processes through Figure 3 The tree diagram shown in the figure transmits data, and the tree diagram is divided into three layers. First, all processes in the bottom processor transmit data to adjacent processes with smaller process numbers than their own. At this time, processes 0, 2, and 4 obtain data. Then, process 2 continues to transmit data to process 0, and process 4 does not perform any operation. At this time, process 0 collects data from processes 0, 1, 2, and 3, and process 4 collects data from processes 4 and 5. Finally, process 4 sends data to process 0, that is, the local root node, and the first packaging operation inside the processor is completed. The local root node obtains data from all other processes inside the processor through tree communication.

[0075] Create a backup of the offset array; Figure 5 As shown, it means:

[0076] Each local root node applies for a piece of memory space and establishes an offset array backup stride to obtain the size of data transmitted by each process, so that the root process can eventually receive data of different sizes; the data is stored in the root process buffer in sequence according to the serial number in the source process. The data sent by the nth process to the root process is stored in the root process, and the nth part in the receiving buffer is received. The offset of the nth part in the buffer is stride[n].

[0077] The new generation of Sunway supercomputers supports remote memory access by processes, i.e., process-level RMA operations, which allow a single process to specify all communication parameters so that each process can calculate the data it needs to access or update on other processes. Any local root node obtains data information to be sent by other local root nodes through RMA operations, i.e., through MPI_GET(), and copies the data from the TARGET memory to the ORIGIN memory.

[0078] Obtain data information to be sent by other local root nodes from other local root nodes through MPI_GET(); including:

[0079] Transfer data from the ORIGIN process starting address ORIGIN_ADDR to the TARGET process, the TARGET process is specified by the parameters WIN and TARGET_RANK; the number of transmitted data is ORIGIN_COUNT; ORIGIN_COUNT and TARGET_COUNT are the number of elements in the ORIGIN and TARGET buffers respectively, the type is specified by the parameter ORIGIN_DATATYPE, the data is written in the TARGET process buffer, the address TARGET_ADDR is obtained by the following formula: TARGET_ADDR = WINDOW_BASE+TARGET_DISP*DISP_UNIT, where WINDOW_BASE and DISP_UNIT are the base address (starting address) and the window displacement unit specified by the TARGET process when the window is initialized.

[0080] Due to the special architecture of the new generation of Sunway supercomputers, the cache area of ​​the new generation of Sunway supercomputers is very valuable. The method proposed in the present invention occupies as little buffer space as possible. At the top level of message passing, process 0 uses the function MPI_GATHERV() to determine the size of the cache area to be applied for by backing up the established offset array.

[0081] The size of the buffer area to be applied for is determined by backing up the established offset array; including: the function MPI_GATHERV() receives the information in the stride array, applies for the buffer on demand, and ensures that the buffer size is equal to the sum of the data received by each local root node.

[0082] The present invention designs a FORTRAN interface SDPO (Sunway Double Packing Optimization) for secondary packaging of data between processes. Programmers can transparently implement the first packaging operation of tree structure communication of data in the processor by calling the FORTRAN interface SDPO, as well as the secondary packaging operation of data between local root nodes based on the hardware architecture of the new generation of Sunway supercomputers. This interface improves the data communication speed between different processes of the Sunway series supercomputers when processing large-scale programs, and improves the operating efficiency of parallel applications. The configuration file and internal specific description of the interface are shown in Table 1, Table 2, and Table 3. Programmers can determine the number of processes and data types used by configuring the interface configuration file to handle different data transmission tasks.

[0083] Table 1 SDPO interface description table;

[0084]

[0085] Table 2 Interface SDPO configuration file table;

[0086]

[0087] Table 3 Specific content table of interface SDPO;

[0088]

[0089] Table 4 SDPO_Finalize interface internal specific content table; SDPO_Finalize interface internal specific content is shown in Table 4:

[0090]

[0091] Table 5 SDPO interface usage example (source program) table; SDPO interface usage example (source program) is shown in Table 5:

[0092]

[0093] Table 6 SDPO interface usage example (source program) table; SDPO interface usage example (source program) is shown in Table 6:

[0094]

[0095] The entire system of the Sunway Blueray II supercomputer has 512 nodes, each of which contains two sw26010pro processors, two Sunway message processing chips, 192GB of memory and 216 DDR4 memory particles. The peak performance of the machine can reach 3.13 PFlops, and the node computing performance can reach 6.12 TFlops.

[0096] The method of the present invention was tested on the latest generation of Sunway supercomputer, Sunway Blue Light II. In terms of software, the use cases tested the program running time when the number of processes was the optimal number of processes (6 or multiples of 6) required for small-scale tree communication, 6, 36, 180, 720, 1440, 1800, and the number of processes was the exponential power of 2, 8, 16, 32, 64, 128, 256, 512, 1024, 2048.

[0097] The global sum of the process numbers is calculated by using the common method and the secondary packaging method proposed by the present invention;

[0098] The number of processes in the processor and its multiples are used as the process number test results, as shown in Table 7 (time comparison between the normal version and the optimized version with a fixed number of processes each time):

[0099] Table 7 Time comparison and acceleration ratio of the common method and the method proposed by the present invention (fixed process, unit: microsecond);

[0100]

[0101] The test results of the number of processes are shown in Table 8 (all common versions and all optimized versions are tested at one time):

[0102] Table 8 Time comparison and acceleration ratio (2 exponential process, unit: microsecond) of the common method and the method proposed in the present invention;

[0103]

[0104] The conventional method and the secondary packing method proposed by the present invention are used to collect arrays of indefinite length in a REAl*8 floating point array of size 100*150.

[0105] The number of processes in the processor and its multiples are used as the process number test results, as shown in Table 9 (time comparison between the normal version and the optimized version with a fixed number of processes each time):

[0106] Table 9 Time comparison and acceleration ratio of the common method and the method proposed by the present invention (fixed process, unit: microsecond);

[0107]

[0108] The test results of the number of processes are shown in Table 10 (all common versions and all optimized versions are tested at one time):

[0109] Table 10 Time comparison and acceleration ratio (2 exponential process, unit: microsecond) of the common method and the method proposed in the present invention;

[0110]

[0111] The method proposed in the present invention focuses on packaging data twice, namely, the packaging method of the tree communication structure and the packaging method corresponding to the Shenwei architecture. By packaging twice, communication congestion is avoided and the data transmission speed is improved.

[0112] Example 3

[0113] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the multi-process collaborative data collection method based on the new generation Sunway supercomputer described in Example 1 or 2 are implemented.

[0114] Example 4

[0115] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the multi-process collaborative data collection method based on a new generation of Sunway supercomputer described in Example 1 or 2.< / type> < / type>

Claims

1. A multi-process collaborative data collection method based on the new generation of Sunway supercomputer, characterized in that: include: Through small-scale tree-shaped communication inside the processor, the data of the six processes inside the processor are all collected to the local root node in the processor, realizing the first data packaging operation; Any local root node obtains data information to be sent by other local root nodes from other local root nodes, and establishes an offset array backup, which is used to determine the size of the buffer area to be applied for at the top level of message transmission, and exchange the size information of the collected data with each other; Each local root node sends the contents of its cache to process No.

0. Process No. 0 applies for a large buffer block based on the serial number of the process receiving the data. The size of the large buffer block is the sum of the amount of data to be transmitted by each process in each started processor. It is used to accommodate the data collected after the first packing operation in the entire processor. The data collected after the first packing operation in the entire processor is stored in its own message buffer in sequence to realize the secondary packaging operation of the data.

2. The multi-process collaborative data collection method based on the new generation Sunway supercomputer according to claim 1 is characterized in that: The first packing operation means: collecting the data of the six processes on a processor into the first process of the processor; include: Six processes pass data through a tree diagram, which is divided into three layers. First, all processes in the bottom processor transmit data to adjacent processes with smaller process numbers than their own. At this time, processes 0, 2, and 4 obtain the data. Then, process 2 continues to transmit data to process 0, and process 4 does not perform any operation. At this time, process 0 collects data from processes 0, 1, 2, and 3, and process 4 collects data from processes 4 and 5. Finally, process 4 sends data to process 0, which is the local root node, and the first packaging operation inside the processor is completed.

3. The multi-process collaborative data collection method based on the new generation Sunway supercomputer according to claim 1 is characterized in that: Create an offset array backup; that is: Each local root node applies for a piece of memory space and establishes an offset array backup stride to obtain the size of data transmitted by each process, so that the root process can eventually receive data of different sizes; the data is stored in the root process buffer in sequence according to the serial number in the source process. The data sent by the nth process to the root process is stored in the root process, and the nth part in the receiving buffer is received. The offset of the nth part in the buffer is stride[n].

4. The multi-process collaborative data collection method based on the new generation Sunway supercomputer according to claim 1 is characterized in that: Any local root node obtains the data information to be sent by other local root nodes from other local root nodes through RMA operation, that is, through MPI_GET(), and copies the data from the TARGET memory to the ORIGIN memory.

5. The multi-process collaborative data collection method based on the new generation Sunway supercomputer according to claim 4 is characterized in that: Obtain data information to be sent by other local root nodes from other local root nodes through MPI_GET(); including: Transfer data from the ORIGIN process starting address ORIGIN_ADDR to the TARGET process, the TARGET process is specified by the parameters WIN and TARGET_RANK; the number of transmitted data is ORIGIN_COUNT; ORIGIN_COUNT and TARGET_COUNT are the number of elements in the ORIGIN and TARGET buffers respectively, the type is specified by the parameter ORIGIN_DATATYPE, the data is written in the TARGET process buffer, the address TARGET_ADDR is obtained by the following formula: TARGET_ADDR = WINDOW_BASE+TARGET_DISP*DISP_UNIT, where WINDOW_BASE and DISP_UNIT are the base address and the window displacement unit specified by the TARGET process when the window is initialized.

6. The multi-process collaborative data collection method based on the new generation Sunway supercomputer according to claim 1 is characterized in that: At the top level of message passing, process 0 uses the function MPI_GATHERV() to determine the size of the buffer to be requested by backing up the established offset array.

7. The multi-process collaborative data collection method based on the new generation Sunway supercomputer according to claim 6 is characterized in that: The size of the buffer area to be applied for is determined by backing up the established offset array; including: the function MPI_GATHERV() receives the information in the stride array, applies for the buffer on demand, and ensures that the buffer size is equal to the sum of the data received by each local root node.

8. The multi-process collaborative data collection method based on the new generation Sunway supercomputer according to any one of claims 1 to 7, characterized in that: By calling the FORTRAN interface SDPO, the first packing operation of the tree structure communication of data in the processor is transparently implemented, as well as the secondary packing operation of data between local root nodes based on the hardware architecture of the new generation Sunway supercomputer.

Images