Hierarchical aggregation method oriented to MPI aggregation I / O

Abstract

The invention discloses a hierarchical aggregation method oriented to MPI aggregation I / O. The method comprises the following steps: selecting a process on each computing node as an aggregator as a local aggregator, collecting and merging I / O requests of other processes on the computing node by the local aggregator, and merging the I / O requests into a larger continuous request; the method comprises the following steps of: dividing a group of computing nodes which are physically similar, namely good in communication quality, into a group according to a physical network communication topology, selecting a local aggregator in the group as an intra-group aggregator, collecting I / O (Input / Output) requests of all the local aggregators in the group by the intra-group aggregator, and combining the I / O requests into a larger continuous request; selecting a part of intra-group aggregators from all intra-group aggregators as global aggregators, collecting I / O requests on the intra-group aggregators by the global aggregators, and merging the I / O requests into larger continuous requests; and the global aggregator interacts with the storage through the parallel file system interface to complete a file read-write request.

Description

Technical Field

[0001] The present invention relates to the technical field of message passing interface, and in particular to a hierarchical aggregation method oriented to MPI aggregation I / O. Background Art

[0002] The Message Passing Interface (MPI) standard defines a set of programming interfaces for parallel shared file access, usually expressed as MPI-I / O. MPI-I / O utilizes the message passing mechanism of MPI and implements parallel data read and write access based on POSIX semantics. Scientific computing applications use MPI-I / O directly or indirectly through parallel I / O libraries to improve program I / O performance. In order to solve the problem of I / O performance degradation of parallel applications caused by large-scale discontinuous I / O requests, MPI Collective I / O is proposed. The core of collective I / O is the two-phase I / O strategy, which conceptually consists of a communication phase and an I / O phase. In the communication phase, a subset of MPI processes (defined as I / O aggregators) are selected to act as I / O agents for the rest of the processes. The files read and written by parallel applications are divided into non-overlapping areas between aggregators, called file domains. All processes send I / O requests to the I / O aggregator according to their assigned file domains. In the I / O phase, the I / O aggregator uses the POSIX interface to access the parallel file system to read and write files. In traditional MPI aggregated I / O, only the I / O aggregator is responsible for I / O interaction with the file system, and the other processes are only responsible for network communication with the I / O aggregator. The two-phase I / O process is as follows: Figure 1As shown in the figure, the circle represents the process, the dark color represents the I / O aggregator process, and the small square represents a computing node. MPI aggregate I / O essentially reduces I / O time at the expense of smaller network communication overhead. As the scale of HPC systems increases, the number of nodes, processors, and cores increases, and HPC enters the exascale era. The communication cost of MPI aggregate I / O, a method of redistributing I / O requests, becomes increasingly larger in large-scale parallel programs, gradually exceeding the performance improvement brought by aggregate I / O. In large-scale non-continuous I / O scenarios, the number of communications and costs of traditional MPI aggregate I / O increase significantly. Although some studies have reduced the number of network communications by using intra-node aggregation methods, they are limited to optimization within the node, only utilizing the locality of the node, and do not consider the relationship between nodes and the locality of the network. In large-scale non-continuous I / O scenarios, network competition between aggregators and I / O requests from a large number of aggregators to the parallel file system will affect the overall file reading and writing performance. Therefore, the traditional MPI aggregate I / O method and related optimization methods cannot be extended to large-scale parallel scenarios. Aggregators in traditional MPI aggregate I / O not only need to collect data from other processes through the network, but also access the file system to read and write data. The number and location of aggregators are not only related to the physical network communication architecture, but also closely related to the underlying parallel file system. Therefore, it is difficult for traditional MPI aggregate I / O methods to find an optimal number and location of aggregators to achieve optimal network communication performance and I / O performance.

[0003] The above information disclosed in this Background section is only for enhancement of understanding of the background of the invention and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0004] The present invention provides a hierarchical aggregation method for MPI aggregation I / O, which reduces network communication competition in the data aggregation process and improves data aggregation performance.

[0005] A hierarchical aggregation method for MPI aggregated I / O includes: Step 1: Intra-node aggregation: a process is selected as the local aggregator on each computing node in the supercomputing cluster. The local aggregator collects and merges the I / O requests of other processes on the computing node and merges the I / O requests into larger continuous requests. Step 2: Intra-group aggregation: A group of physically close computing nodes with good communication quality are divided into a group according to the physical network communication topology. A local aggregator is selected as the intra-group aggregator in the group. The intra-group aggregator collects the I / O requests of all local aggregators in the group and merges the I / O requests into larger continuous requests. Step 3, global aggregation, select some of the in-group aggregators from all in-group aggregators as global aggregators, the global aggregators collect I / O requests on the in-group aggregators and merge the I / O requests into larger continuous requests; Step 4: Asynchronous I / O pipeline optimization: The global aggregator applies for two aggregation buffers of the same size, replaces the original blocking I / O calls with non-blocking I / O calls, and alternately executes the communication operations and I / O operations in the two-stage I / O in the two buffers, overlapping the I / O stage of this round with the communication stage of the next round. Step 5: The global aggregator interacts with the storage through the parallel file system interface to complete the file read and write request.

[0006] In the hierarchical aggregation method for MPI aggregation I / O, in step 1, no network communication occurs between computing nodes in intra-node aggregation, and all communications are intra-node communications.

[0007] In the hierarchical aggregation method for MPI aggregated I / O, in step 2, network communication in intra-group aggregation only occurs between processes in the same group, and no network communication occurs between processes in different groups.

[0008] In the described hierarchical aggregation method for MPI aggregated I / O, in step 3, the number and position of global aggregators are determined according to the parallel file system stripe size, the number of stripes, and the number of OSTs.

[0009] In the hierarchical aggregation method for MPI aggregated I / O, files read and written by parallel applications are divided into non-overlapping areas between aggregators as file domains.

[0010] In the hierarchical aggregation method for MPI aggregated I / O, the global aggregator uses the POSIX interface to access the parallel file system to read and write files.

[0011] In the hierarchical aggregation method for MPI aggregated I / O, the MPI aggregated I / O utilizes the message passing mechanism of MPI to realize parallel data read and write access based on POSIX semantics.

[0012] In the hierarchical aggregation method for MPI aggregation I / O, the communication stage and the I / O stage constitute a two-stage I / O.

[0013] In the described hierarchical aggregation method for MPI aggregated I / O, the number and location of group aggregators are determined according to the physical network communication topology, thereby reducing the time overhead of network communication.

[0014] In addition, the present invention also discloses a hierarchical aggregation system for MPI aggregation I / O, which is characterized by comprising: The node aggregation unit is used to select a process as an aggregator on each computing node in the supercomputing cluster as a local aggregator. The local aggregator collects and merges I / O requests from other processes on the computing node and merges the I / O requests into larger continuous requests. The intra-group aggregation unit is used to divide a group of computing nodes that are physically close, i.e., have good communication quality with each other, into a group according to the physical network communication topology, select a local aggregator in the group as the intra-group aggregator, and the intra-group aggregator collects the I / O requests of all local aggregators in the group and merges the I / O requests into larger continuous requests; A global aggregation unit, which is used to select some of the aggregators in the group from all the aggregators in the group as global aggregators. The global aggregator collects I / O requests on the aggregators in the group and merges the I / O requests into larger continuous requests. Asynchronous I / O pipeline optimization unit, which is used to make the global aggregator apply for two aggregation buffers of the same size, replace the original blocking I / O calls with non-blocking I / O calls, and alternately perform communication operations and I / O operations in the two-stage I / O in the two buffers, overlapping the I / O stage of this round with the communication stage of the next round; The read / write unit is used by the global aggregator to interact with the storage through the parallel file system interface to complete file read / write requests.

[0015] In addition, the present invention also discloses a computer storage medium, wherein the storage medium includes computer instructions, and when the computer is run on the computer, the computer executes any of the methods described above.

[0016] In addition, the present invention also discloses an electronic device, wherein the electronic device comprises: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method described in any one of the above items is implemented.

[0017] Compared with the prior art, the present invention has the following advantages: The present invention optimizes the problem of high total communication cost in the two-stage I / O aggregation process, and effectively reduces the network communication between nodes by increasing intra-node aggregation. Traditional MPI aggregation I / O does not consider the communication topology relationship between computing nodes in the communication stage, resulting in network contention problems in the data aggregation process. Researchers calculate the communication cost based on the network communication topology, and then optimize the number and location of I / O aggregators in a targeted manner, reduce network communication competition in the data aggregation process, and improve data aggregation performance. The present invention adds an intra-node aggregation layer and an intra-group aggregation layer. In large-scale discontinuous I / O scenarios, it can effectively reduce the network communication overhead between large-scale I / O processes in the MPI aggregation I / O process, thereby improving the MPI aggregation I / O performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By reading the detailed description of the preferred specific embodiments below, various other advantages and benefits of the present invention will become clear to those of ordinary skill in the art. The drawings in the specification are only for the purpose of illustrating the preferred embodiments and are not considered to be limitations of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work. Moreover, the same reference numerals are used to represent the same components throughout the drawings.

[0019] In the attached picture: Figure 1 It is a schematic diagram of a two-stage I / O process of the prior art; Figure 2 It is a schematic diagram of the hierarchical aggregation process of MPI aggregation IO of this method.

[0020] The present invention is further explained below in conjunction with the accompanying drawings and embodiments. DETAILED DESCRIPTION

[0021] The following will refer to the attached Figure 1 to Figure 2 Specific embodiments of the present invention are described in more detail. Although specific embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0022] It should be noted that certain words are used in the specification and claims to refer to specific components. Those skilled in the art should understand that technicians may use different nouns to refer to the same component. This specification and claims do not use the difference in nouns as a way to distinguish components, but use the difference in the functions of the components as the criterion for distinction. As mentioned throughout the specification and claims, "including" or "comprising" is an open term, so it should be interpreted as "including but not limited to". The subsequent description of the specification is a preferred embodiment of the present invention, but the description is based on the general principles of the specification and is not intended to limit the scope of the present invention. The scope of protection of the present invention shall be determined by the attached claims.

[0023] To facilitate understanding of the embodiments of the present invention, further explanation will be given below by taking specific embodiments as examples in conjunction with the accompanying drawings, and each of the accompanying drawings does not constitute a limitation on the embodiments of the present invention.

[0024] like Figure 2 As shown in FIG. 1 , the hierarchical aggregation method for MPI aggregated I / O includes the following steps: Step 1: Aggregation within independent physical server computing nodes in the supercomputing cluster. A process is selected as the aggregator on each computing node in the supercomputing cluster as the local aggregator. The local aggregator collects and merges I / O requests from other processes on the computing node and merges the I / O requests into larger continuous requests. Step 2: Intra-group aggregation: A group of physically close computing nodes with good communication quality are divided into a group according to the physical network communication topology. A local aggregator is selected as the intra-group aggregator in the group. The intra-group aggregator collects the I / O requests of all local aggregators in the group and merges the I / O requests into larger continuous requests. Step 3, global aggregation, select some of the in-group aggregators from all in-group aggregators as global aggregators, the global aggregators collect I / O requests on the in-group aggregators and merge the I / O requests into larger continuous requests; Step 4: Asynchronous I / O pipeline optimization: The global aggregator applies for two aggregation buffers of the same size, replaces the original blocking I / O calls with non-blocking I / O calls, and alternately executes the communication operations and I / O operations in the two-stage I / O in the two buffers, overlapping the I / O stage of this round with the communication stage of the next round. Step 5: The global aggregator interacts with the storage through the parallel file system interface to complete the file read and write request.

[0025] In a preferred implementation of the hierarchical aggregation method for MPI aggregated I / O, in step 1, no network communication occurs between computing nodes in intra-node aggregation, and all communications are intra-node communications.

[0026] In a preferred implementation of the hierarchical aggregation method for MPI aggregated I / O, in step 2, network communication in intra-group aggregation only occurs between processes in the same group, and no network communication occurs between processes in different groups.

[0027] In a preferred implementation of the hierarchical aggregation method for MPI aggregated I / O, in step 2, the number and location of group aggregators are determined according to the physical network communication topology to reduce the time overhead of network communication.

[0028] In a preferred implementation of the hierarchical aggregation method for MPI aggregated I / O, in step 3, the number and position of global aggregators are determined based on the parallel file system stripe size, the number of stripes, and the number of OSTs.

[0029] In a preferred implementation of the hierarchical aggregation method for MPI aggregated I / O, files read and written by parallel applications are divided into non-overlapping areas between aggregators as file domains.

[0030] In a preferred implementation of the hierarchical aggregation method for MPI aggregated I / O, the global aggregator overlaps the communication and I / O stages in multiple rounds based on a pipeline optimization mechanism for asynchronous I / O.

[0031] In a preferred implementation of the hierarchical aggregation method for MPI aggregated I / O, the global aggregator uses a POSIX interface to access the parallel file system to read and write files.

[0032] In a preferred implementation of the hierarchical aggregation method for MPI aggregated I / O, the MPI aggregated I / O utilizes the message passing mechanism of MPI to implement parallel data read and write access based on POSIX semantics.

[0033] In a preferred implementation of the hierarchical aggregation method for MPI aggregated I / O, the communication stage and the I / O stage constitute a two-stage I / O.

[0034] In one embodiment, the hierarchical aggregation method for MPI aggregated I / O includes the following steps: Step 1: Select a process on each computing node as an aggregator, called the local aggregator. The local aggregator collects and merges I / O requests from other processes on the node, merging the requests into larger continuous requests. This step is called intra-node aggregation. In this step, there is no network communication between computing nodes, and all communication is intra-node communication.

[0035] Step 2: According to the physical network communication topology, a group of computing nodes that are physically close, that is, with good communication quality between each other, are divided into a group. A local aggregator is selected in the group as the intra-group aggregator. The intra-group aggregator collects the I / O requests of all local aggregators in the group and merges the requests into larger continuous requests, which is called intra-group aggregation. In this step, network communication only occurs between processes in the same group, and no network communication occurs between processes in different groups.

[0036] Step 3: From all the aggregators within the group, select some of the aggregators within the group as global aggregators. The global aggregator collects the I / O requests on the aggregators within the group and merges the requests into larger continuous requests, which is called global aggregation.

[0037] Step 4: Asynchronous I / O pipeline optimization: The global aggregator applies for two aggregation buffers of the same size, replaces the original blocking I / O calls with non-blocking I / O calls, and alternately executes the communication operations and I / O operations in the two-stage I / O in the two buffers, overlapping the I / O stage of this round with the communication stage of the next round. Step 5: The global aggregator interacts with the storage through the parallel file system interface to complete the final file read and write request.

[0038] In addition, the present invention also discloses a hierarchical aggregation system for MPI aggregation I / O, which is characterized by comprising: The node aggregation unit is used to select a process as an aggregator on each computing node in the supercomputing cluster as a local aggregator. The local aggregator collects and merges I / O requests from other processes on the computing node and merges the I / O requests into larger continuous requests. The intra-group aggregation unit is used to divide a group of computing nodes that are physically close, i.e., have good communication quality with each other, into a group according to the physical network communication topology, select a local aggregator in the group as the intra-group aggregator, and the intra-group aggregator collects the I / O requests of all local aggregators in the group and merges the I / O requests into larger continuous requests; A global aggregation unit, which is used to select some of the aggregators in the group from all the aggregators in the group as global aggregators. The global aggregator collects I / O requests on the aggregators in the group and merges the I / O requests into larger continuous requests. Asynchronous I / O pipeline optimization unit, which is used to make the global aggregator apply for two aggregation buffers of the same size, replace the original blocking I / O calls with non-blocking I / O calls, and alternately perform communication operations and I / O operations in the two-stage I / O in the two buffers, overlapping the I / O stage of this round with the communication stage of the next round; The read / write unit is used by the global aggregator to interact with the storage through the parallel file system interface to complete file read / write requests.

[0039] In addition, the present invention also discloses a computer storage medium, wherein the storage medium includes computer instructions, and when the computer is run on the computer, the computer executes any of the methods described above.

[0040] In addition, the present invention also discloses an electronic device, wherein the electronic device comprises: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method described in any one of the above items is implemented.

[0041] The hierarchical aggregation method of the present invention increases the intra-node aggregation and intra-group aggregation layers, which can effectively reduce the number of communications between nodes and groups. Since the communication overhead between nodes and groups is greater than the communication overhead within nodes and groups, the hierarchical aggregation method can significantly reduce the communication overhead in the MPI aggregation I / O process, effectively solve the communication contention problem during large-scale non-continuous I / O requests, and can expand the MPI aggregation I / O to a larger scale scenario. The hierarchical aggregation method decouples the network communication request and I / O request in the MPI aggregation I / O process by increasing the intra-node aggregation and intra-group aggregation layers. In the hierarchical aggregation method, the intra-group aggregator only collects data from other processes through the network and sends it to the global aggregator, and the appropriate number and position of the intra-group aggregators can be selected according to the network topology. The global aggregator is responsible for the actual file I / O operation, and the appropriate number and position of the global aggregators can be selected according to the parallel file system stripe size, the number of stripes, and the number of OSTs. Therefore, the hierarchical aggregation method can optimize network communication performance and I / O performance for a specific network system architecture and file system architecture, respectively, without affecting each other. At the same time, for multiple rounds of two-stage I / O, the pipeline optimization mechanism based on asynchronous I / O is used to overlap the communication and I / O stages in multiple rounds to reduce the impact of the time overhead of the communication stage on the overall aggregate I / O.

[0042] Although the embodiments of the present invention are described above in conjunction with the accompanying drawings, the present invention is not limited to the above specific embodiments and application fields, and the above specific embodiments are only illustrative and instructive, rather than restrictive. A person of ordinary skill in the art can also make many forms under the guidance of this specification and without departing from the scope of protection of the claims of the present invention, all of which belong to the protection of the present invention.

Claims

1. A hierarchical aggregation method for MPI aggregation I / O, characterized in that: The steps include: Step 1: Intra-node aggregation: a process is selected as the local aggregator on each computing node in the supercomputing cluster. The local aggregator collects and merges the I / O requests of other processes on the computing node and merges the I / O requests into larger continuous requests. Step 2: Intra-group aggregation: A group of physically close computing nodes with good communication quality are divided into a group according to the physical network communication topology. A local aggregator is selected as the intra-group aggregator in the group. The intra-group aggregator collects the I / O requests of all local aggregators in the group and merges the I / O requests into larger continuous requests. Step 3, global aggregation, select some of the in-group aggregators from all in-group aggregators as global aggregators, the global aggregators collect I / O requests on the in-group aggregators and merge the I / O requests into larger continuous requests; Step 4: Asynchronous I / O pipeline optimization: The global aggregator applies for two aggregation buffers of the same size, replaces the original blocking I / O calls with non-blocking I / O calls, and alternately executes the communication operations and I / O operations in the two-stage I / O in the two buffers, overlapping the I / O stage of this round with the communication stage of the next round. Step 5: The global aggregator interacts with the storage through the parallel file system interface to complete the file read and write request.

2. The hierarchical aggregation method for MPI aggregation I / O according to claim 1, characterized in that: Preferably, in step 1, no network communication occurs between computing nodes in the intra-node aggregation, and all communications are intra-node communications.

3. The hierarchical aggregation method for MPI aggregation I / O according to claim 1, characterized in that: Files read and written by parallel applications are divided into non-overlapping areas between aggregators as file domains.

4. The hierarchical aggregation method for MPI aggregation I / O according to claim 1, characterized in that: The global aggregator uses the POSIX interface to access the parallel file system for reading and writing files.

5. The hierarchical aggregation method for MPI aggregation I / O according to claim 1, characterized in that: MPI-oriented aggregate I / O uses MPI's message passing mechanism to implement parallel data read and write access based on POSIX semantics.

6. The hierarchical aggregation method for MPI aggregation I / O according to claim 1, characterized in that: The communication phase and the I / O phase constitute two-phase I / O.

7. The hierarchical aggregation method for MPI aggregation I / O according to claim 1, characterized in that: The number and location of group aggregators are determined based on the physical network communication topology to reduce the time overhead of network communication.

8. A hierarchical aggregation system for MPI aggregation I / O, characterized in that: include: The node aggregation unit is used to select a process as an aggregator on each computing node in the supercomputing cluster as a local aggregator. The local aggregator collects and merges I / O requests from other processes on the computing node and merges the I / O requests into larger continuous requests. The intra-group aggregation unit is used to divide a group of computing nodes that are physically close, i.e., have good communication quality with each other, into a group according to the physical network communication topology, select a local aggregator in the group as the intra-group aggregator, and the intra-group aggregator collects the I / O requests of all local aggregators in the group and merges the I / O requests into larger continuous requests; A global aggregation unit, which is used to select some of the aggregators in the group from all the aggregators in the group as global aggregators. The global aggregator collects I / O requests on the aggregators in the group and merges the I / O requests into larger continuous requests. Asynchronous I / O pipeline optimization unit, which is used to make the global aggregator apply for two aggregation buffers of the same size, replace the original blocking I / O calls with non-blocking I / O calls, and alternately perform communication operations and I / O operations in the two-stage I / O in the two buffers, overlapping the I / O stage of this round with the communication stage of the next round; The read / write unit is used by the global aggregator to interact with the storage through the parallel file system interface to complete file read / write requests.

9. A computer storage medium, wherein: The storage medium includes computer instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 7.

10. An electronic device, wherein: The electronic device comprises: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images