Multi-node MPI set communication performance modeling method and system

Abstract

The invention discloses a multi-node MPI set communication performance modeling method and system, and the method comprises the steps: firstly determining model parameters needing to be measured according to the information of a multi-core processor in a node and the hierarchical information of a platform network; then measuring intra-node communication parameters and inter-node communication parameters on different network levels on the platform; analyzing a set communication source code and simulating a communication algorithm behavior to obtain all communication characteristics of each round of communication of the algorithm; and finally, corresponding parameters are selected in combination with communication characteristics, and the communication time of each round is represented by a mathematical formula of the corresponding parameters. Compared with an existing model, supercomputing network hierarchical structure information and a point-to-point communication primitive type used by a communication algorithm bottom layer are introduced, and the application range of the model is expanded from a single-node shared memory model to a hybrid model of a multi-node shared memory and an interconnection network; the model can more accurately predict the communication time of the multi-node set communication operation.

Description

Background Art

[0002] As the most widely used communication middleware in high-performance computing, the Message Passing Interface (MPI) has become the de facto communication interface used by HPC applications. According to statistics by Chunduri et al., nearly 30% of the core time in modern production systems is spent on MPI, and collective communication accounts for 66% of the overall MPI execution time.

[0003] The purpose of establishing a collective communication performance model is to predict the performance of each communication operation under given communication characteristics and platform characteristics. It is a key technology for analyzing communication bottlenecks, reducing communication overhead, and achieving communication load balancing. Existing collective communication performance modeling methods mainly include machine learning-based modeling and statistical method-based modeling. Models based on machine learning methods such as KNN, XGBoost, and RandomForest can better handle the nonlinear relationship between communication time and communication characteristics, but the models require large data sets to train, which is costly and has limited extrapolation capabilities. Modeling based on statistical methods mainly predicts communication time by analyzing communication middle layer parameters. Typical communication performance models for communication middle layer modeling include τ-Lop and C-Lop. These models use communication middleware-related parameters to abstract hardware complexity, thereby avoiding dependence on hardware details. However, τ-Lop and C-Lop still have some shortcomings. They are difficult to predict the performance of cross-node communication operations in combination with the hierarchical structure of supercomputing networks.

[0004] As scientific computing problems become more complex and large-scale, the scale of scientific applications running on supercomputers is also expanding. The next generation of supercomputer systems will run applications on hundreds of thousands of nodes and tens of millions of cores. Therefore, how to make the communication performance model predict the cross-node communication overhead at a lower cost has become an urgent problem to be solved. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a multi-node MPI collective communication performance modeling method and system in view of the deficiencies in the above-mentioned prior art, so as to solve the technical problem that the communication performance model predicts the high cross-node communication overhead.

[0006] The present invention adopts the following technical solutions: A multi-node MPI collective communication performance modeling method comprises the following steps: Determine the model parameters required for measurement based on the multi-core processor information in the node and the platform network layer information; For all the model parameters measured, for shared memory communication within a single node, point-to-point communication parameters located on different NUMA domains were measured; for cross-node communication parameters, point-to-point communication parameters located on different network layers were measured; The communication behavior of the specific collective communication algorithm in the MPI source code is analyzed, and combined with the obtained model parameters, the communication time formula represented by each model parameter is obtained by simulating the execution process of the communication algorithm.

[0007] Preferably, the model parameters to be measured are determined based on the multi-core processor information in the node and the platform network layer information. For intra-node communication, the model uses Indicates a concurrent transmission on a shared memory channel The length is Point-to-point communication time of the message; Indicates the core location of the two processes bound, which are located in the same NUMA, across NUMA, or both; Pick or , Indicates that the underlying layer uses MPI_Send and MPI_Recv for communication. Indicates that the underlying layer uses MPI_Sendrecv for communication; For cross-node communication, the model uses Indicates that processes distributed on two nodes are transmitted concurrently through a network channel at a time The length is Point-to-point communication time of the message; Indicates the network level spanned by two nodes.

[0008] Preferably, for shared memory communication within a single node, by changing the message length, the number of concurrent transmissions, the process core binding method, and the point-to-point communication type, the time required for sending and receiving messages under different characteristics is measured, thereby obtaining parameter values ​​under different characteristics; For cross-node communication parameters, by changing the message length, the number of concurrent transmissions, the number of network layers spanned by the communication process, and the point-to-point communication type, the time required to send and receive messages under different characteristics is measured to obtain the parameter values ​​under different characteristics.

[0009] Preferably, for shared memory communication within a single node, the feature variation range is described as follows: message length ranges from 1 byte to 1M bytes; the number of concurrent transmissions ranges from 1 to n / 2, where n is the maximum number of cores in a single node; the process binding core mode ranges from , and ; Point-to-point communication primitives are used separately or Two types; For cross-node communication parameters, the characteristic variation range is described as follows: message length ranges from 1 byte to 1M bytes; the number of concurrent transmissions ranges from 1 to n (the maximum number of cores in a single node); the number of network layers spanned by the communication process ranges from 0 to k, where k is the number of platform network layers; the point-to-point communication primitives use or Two types.

[0010] Preferably, the communication behavior of the specific collective communication algorithm in the MPI source code is analyzed, and the obtained model parameters are combined to obtain the communication time formula represented by each model parameter by simulating the execution process of the communication algorithm, which is specifically: According to the code logic, the numbers of all source processes and destination processes participating in this round of communication are calculated, and the node numbers of the processes are determined by combining the mapping relationship, and then the channel type of each pair of communications is determined; If the communication is based on network channels, the network hierarchy information of the communication nodes needs to be further determined in combination with the platform node topology; Count the number of concurrent transmissions on the same channel in this round of communication; Determine the point-to-point communication type used in this round of communication based on the code; Combining the channel type, message length, number of concurrent transmissions, location information of each pair of communication processes and point-to-point communication type obtained in the first step, the time required for each pair of communication in this round is expressed by corresponding parameters; the maximum value of all communication times in this round is taken as the estimated time of this round of communication; The communication time of all rounds is accumulated, and the overall communication time of a collective communication operation is expressed by a mathematical formula of parameters.

[0011] Preferably, the point-to-point communication type used in this round of communication is determined according to the code. If it is MPI_Send and MPI_Recv primitives, the subsequent The type parameter represents the time cost; If it is the MPI_Sendrecv primitive, use type parameter; at the same time, the message length of this round of communication is determined according to the code logic.

[0012] Preferably, the communication time of all rounds is accumulated, and the overall communication time of a collective communication operation is expressed in a mathematical formula with parameters. For the binomial broadcast algorithm, the time consumption of the entire communication operation is It is expressed as:

[0013] in, is the number of processes, Indicates that the algorithm is in the channel Previous i Model parameters for round communication, is the message length, is the number of concurrent transmissions, is the network hierarchy information, Indicates that the underlying layer uses MPI_Send and MPI_Recv primitives for communication.

[0014] Preferably, the communication time of all rounds is accumulated, and the overall communication time of a collective communication operation is expressed in a mathematical formula with parameters. For the recursive doubling allgather algorithm, the time consumption of the entire communication operation is It is expressed as:

[0015] in, is the number of processes, Indicates that the algorithm is in the channel Previous i Model parameters for round communication, is the message length, is the number of concurrent transmissions, is the network hierarchy information, Indicates that the underlying layer uses the MPI_Sendrecv primitive for communication.

[0016] Preferably, the communication time of all rounds is accumulated, and the overall communication time of a collective communication operation is expressed in a mathematical formula with parameters. For the ring allgather algorithm, the time consumption of the entire communication operation is It is expressed as:

[0017] in, is the number of processes, Indicates that the algorithm is in the channel Previous i Model parameters for round communication, is the message length, is the number of concurrent transmissions, is the network hierarchy information, Indicates that the underlying layer uses the MPI_Sendrecv primitive for communication.

[0018] In a second aspect, an embodiment of the present invention provides a multi-node MPI collective communication performance modeling system, comprising: The parameter module determines the model parameters to be measured based on the multi-core processor information in the node and the platform network layer information; The measurement module measures all model parameters. For shared memory communication within a single node, the point-to-point communication parameters located in different NUMA domains are measured respectively. For cross-node communication parameters, the point-to-point communication parameters located in different network layers are measured respectively. The module is constructed, the communication behavior of the specific collective communication algorithm in the MPI source code is analyzed, and the communication time formula represented by each model parameter is obtained by simulating the execution process of the communication algorithm in combination with the obtained model parameters.

[0019] In a third aspect, a computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned multi-node MPI collective communication performance modeling method when executing the computer program.

[0020] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned multi-node MPI collective communication performance modeling method.

[0021] In a fifth aspect, a chip comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned multi-node MPI collective communication performance modeling method when executing the computer program.

[0022] In a sixth aspect, an embodiment of the present invention provides an electronic device, comprising a computer program, which, when executed by the electronic device, implements the steps of the above-mentioned multi-node MPI collective communication performance modeling method.

[0023] Compared with the prior art, the present invention has at least the following beneficial effects: A multi-node MPI collective communication performance modeling method first combines the multi-core processor information in the node and the platform network hierarchy information to determine the model parameters to be measured, then measures the model parameters under different feature combinations, and then analyzes the communication behavior of the collective communication algorithm. Finally, the overall communication time of the algorithm is expressed by the mathematical formula of the model parameters. By introducing the supercomputing network hierarchy information and the point-to-point communication primitive type used at the bottom of the communication algorithm, the model application scope is expanded from a single-node shared memory model to a hybrid model of multi-node shared memory and interconnected networks. The model can more accurately predict the communication time of multi-node collective communication operations; Furthermore, according to the multi-core processor information in the node and the platform network layer information, the maximum concurrency number of parameters to be measured and the maximum number of network layers for cross-node communication are determined. This step enables the model to adapt to different hardware platforms.

[0024] Furthermore, by changing the message length, number of concurrent transmissions, process core binding method, number of cross-node communication network layers and point-to-point communication type, and measuring the point-to-point communication parameters under different feature combinations, the point-to-point communication performance under different feature combinations can be reflected.

[0025] Furthermore, the message length ranges from 1 byte to 1M bytes, because the parameter value of a larger message length will show a linear change law, which can be derived from the parameter of 1M bytes; the point-to-point communication primitive uses or There are two types, which are the underlying communication primitives used by the collective communication algorithm; the maximum number of concurrent transmissions for single-node communication is n / 2, where n is the maximum number of cores in a single node, because one communication requires two cores; the process binding method within the node is desirable , and ; The number of concurrent transmissions of cross-node communication parameters ranges from 1 to n, where n is the maximum number of cores in a single node; the number of network layers spanned by the communication process ranges from 0 to k, where k is the number of platform network layers, because the communication performance between nodes at different network levels varies significantly.

[0026] Furthermore, the communication behavior of the specific collective communication algorithm in the MPI source code is analyzed. By simulating the execution process of the communication algorithm, the number of concurrent communications, message length, process core binding method, number of cross-node communication network layers and point-to-point communication type of each round of communication in the algorithm execution process are obtained, thereby obtaining the communication time formula represented by the model parameters. Furthermore, for MPI_Send and MPI_Recv primitives, The type parameter indicates the time cost; if it is the MPI_Sendrecv primitive, use Type parameters; through this classification, the performance of different communication primitives can be better distinguished.

[0027] It can be understood that the beneficial effects of the second aspect mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here.

[0028] In summary, by introducing the supercomputing network hierarchy information, the model of the present invention can more accurately predict the communication time of multi-node collective communication operations.

[0029] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2 Percent error graph of binomial broadcast algorithm for model prediction under multiple nodes; Figure 3 A schematic diagram of a computer device provided by an embodiment of the present invention; Figure 4 The present invention is a block diagram of an electronic device provided according to an embodiment. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0034] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0035] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects are in an "or" relationship.

[0036] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0037] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0038] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0039] The present invention provides a multi-node MPI collective communication performance modeling method, which measures the point-to-point communication parameters under different core binding modes within a single node and the point-to-point communication parameters at different network levels across nodes, and expresses the communication time of collective communication operations with a mathematical formula of parameters, thereby obtaining a collective communication performance model. The method combines the supercomputing network hierarchy information to expand the model application scope from a single-node shared memory model to a multi-node shared memory and network hybrid model, and the model can better predict the communication time of collective communication operations of large-scale parallel programs.

[0040] Example 1 See also Figure 1 The present invention provides a multi-node MPI collective communication performance modeling method, comprising the following steps: S1. Determine the model parameters to be measured based on the multi-core processor information in the node and the platform network layer information; the model parameters are used express; The modeling factors involved include: channel type , Message length , concurrent transmission number , Process binding method , network hierarchy information , underlying communication type .

[0041] For intra-node communication, the model uses Indicates a concurrent transmission on a shared memory channel The length is The point-to-point communication time of the message; where, Indicates the core location where the two processes are bound, which can be located in the same NUMA ( ), cross-NUMA ( ) and both situations exist at the same time ( ); Desirable or : Indicates that the underlying layer uses MPI_Send and MPI_Recv for communication. Indicates that the underlying layer uses the MPI_Sendrecv primitive for communication; For cross-node communication, the model uses Indicates that processes distributed on two nodes are transmitted concurrently through a network channel The length is The point-to-point communication time of the message; Indicates the network level spanned by two nodes. It has different value ranges depending on the specific number of network layers of the platform. For example The value ranges from 0 to 3, indicating communication within the same blade, across blades, across racks, and across cabinets, respectively.

[0042] S2, measure all parameters of the model on a given platform; For the relevant parameters of intra-node communication, by changing the message length, the number of concurrent transmissions, the process binding method and the point-to-point communication type, the time required to send and receive messages under different characteristics is measured, so as to obtain the parameter values ​​under different characteristics.

[0043] The feature variation range is described as follows: message length ranges from 1 byte to 1M bytes; concurrent transmission number ranges from 1 to n / 2, where n is the maximum number of cores in a single node; and the process binding method ranges from , and ; Point-to-point communication primitives are used separately or Two types.

[0044] For the relevant parameters of inter-node communication, by changing the message length, the number of concurrent transmissions, the number of network layers spanned by the communication process, and the point-to-point communication type, the time required to send and receive messages under different characteristics is measured, so as to obtain the parameter values ​​under different characteristics.

[0045] The feature variation range is described as follows: message length ranges from 1 byte to 1M bytes; the number of concurrent transmissions ranges from 1 to n (the maximum number of cores in a single node); the number of network layers spanned by the communication process ranges from 0 to k, where k is the number of platform network layers; the point-to-point communication primitives use or Two types.

[0046] S3. Analyze the communication behavior of the specific collective communication algorithm in the MPI source code, and obtain the communication time formula represented by each parameter by simulating the execution process of the communication algorithm.

[0047] S301, calculating all source process numbers and destination process numbers participating in this round of communication according to the code logic, determining the node number where the process is located in combination with the mapping relationship, and then determining the channel type of each pair of communication; S302: If the communication is based on a network channel, the network hierarchy information of the communication node needs to be further determined in combination with the platform node topology; S303, counting the number of concurrent transmissions on the same channel in this round of communication; S304. Determine the point-to-point communication type used in this round of communication according to the code: If it is MPI_Send and MPI_Recv primitives, then it will be used later The type parameter represents the time cost; If it is an MPI_Sendrecv primitive, use type parameter; at the same time, the message length of this round of communication is determined according to the code logic.

[0048] S305. Combining the previously obtained channel type, message length, number of concurrent transmissions, location information of each pair of communication processes, and point-to-point communication type, the time required for each pair of communication in this round can be represented by corresponding parameters; the maximum value of all communication times in this round is taken as the estimated time value of this round of communication; S306: Accumulate the communication time of all rounds, and the overall communication time of a collective communication operation can be expressed by a mathematical formula of parameters.

[0049] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0050] Example 2 The present invention provides a multi-node MPI collective communication performance modeling system, which can be used to implement the above-mentioned multi-node MPI collective communication performance modeling method. Specifically, the multi-node MPI collective communication performance modeling system includes a parameter module, a measurement module and a construction module.

[0051] Among them, the parameter module determines the model parameters required to be measured according to the multi-core processor information in the node and the platform network layer information; The measurement module measures all model parameters. For shared memory communication within a single node, the point-to-point communication parameters located in different NUMA domains are measured respectively. For cross-node communication parameters, the point-to-point communication parameters located in different network layers are measured respectively. The module is constructed, the communication behavior of the specific collective communication algorithm in the MPI source code is analyzed, and the communication time formula represented by each model parameter is obtained by simulating the execution process of the communication algorithm in combination with the obtained model parameters.

[0052] Example 3 The present invention provides a terminal device, which includes a processor and a memory, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, graphics processors (GPU), tensor processors (TPU), digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions to implement corresponding method processes or corresponding functions; the processor described in the embodiment of the present invention can be used for the operation of the multi-node MPI collective communication performance modeling method, including: According to the multi-core processor information in the node and the platform network layer information, the model parameters that need to be measured are determined; for all the model parameters measured, for shared memory communication within a single node, the point-to-point communication parameters located in different NUMA domains are measured respectively; for cross-node communication parameters, the point-to-point communication parameters located in different network layers are measured respectively; the communication behavior of the specific collective communication algorithm in the MPI source code is analyzed, and combined with the obtained model parameters, the communication time formula represented by each model parameter is obtained by simulating the execution process of the communication algorithm.

[0053] See also Figure 3 , the terminal device is a computer device, and the computer device 60 of this embodiment includes: a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When the computer program 63 is executed by the processor 61, the multi-node MPI collective communication performance modeling method in the embodiment is implemented. To avoid repetition, it is not described one by one here. Alternatively, when the computer program 63 is executed by the processor 61, the functions of each model / unit in the multi-node MPI collective communication performance modeling system in the embodiment are implemented. To avoid repetition, it is not described one by one here.

[0054] The computer device 60 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will appreciate that Figure 3 This is only an example of the computer device 60 and does not constitute a limitation of the computer device 60. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, buses, etc.

[0055] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, graphics processing units (GPU), tensor processing units (TPU), digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0056] The memory 62 may be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 may also be an external storage device of the computer device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device 60.

[0057] Furthermore, the memory 62 may include both an internal storage unit of the computer device 60 and an external storage device. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 may also be used to temporarily store data that has been output or is to be output.

[0058] See also Figure 4 The terminal device is an electronic device 600, which is in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0059] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 Follow the steps shown in .

[0060] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0061] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0062] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0063] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0064] Example 4 The present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a terminal device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the terminal device and extended storage media supported by the terminal device, and can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that more specific examples (non-exhaustive list) of the computer-readable storage medium here include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0065] Computer readable storage media also include data signals propagated in baseband or as part of a carrier wave, which carry readable program codes. Such propagated data signals can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than a readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0066] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0067] The processor may load and execute one or more instructions stored in a computer-readable storage medium to implement the corresponding steps of the multi-node MPI collective communication performance modeling method in the above embodiment; the processor may load and execute the following steps: According to the multi-core processor information in the node and the platform network layer information, the model parameters that need to be measured are determined; for all the model parameters measured, for shared memory communication within a single node, the point-to-point communication parameters located in different NUMA domains are measured respectively; for cross-node communication parameters, the point-to-point communication parameters located in different network layers are measured respectively; the communication behavior of the specific collective communication algorithm in the MPI source code is analyzed, and combined with the obtained model parameters, the communication time formula represented by each model parameter is obtained by simulating the execution process of the communication algorithm.

[0068] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0069] Example 5 Taking the Tianhe platform with a 4-layer network structure as an example, the following steps are included: S1. Determine the model parameters to be measured based on the multi-core processor information in the node and the platform network layer information; For intra-node communication, the model uses Indicates a concurrent transmission on a shared memory channel The length is The point-to-point communication time of the message. A single node of the Tianhe platform has 16 processor cores, divided into 8 NUMA domains, and each NUMA domain has 2 processor cores. Then the parameters that need to be measured include: the point-to-point communication parameters when the communication process is located in the same NUMA, across NUMA, and both situations exist at the same time, and the number of concurrent transmissions. Take 1 to 8; For cross-node communication, the model uses Indicates that processes distributed on two nodes are transmitted concurrently through a network channel The length is Since the Tianhe platform network has a four-layer structure, it is necessary to measure the communication parameters of the same blade, across blades, across racks, and across cabinets, and the number of concurrent transmissions. Take 1 to 16.

[0070] S2, measure all parameters of the model on a given platform; For intra-node communication, the communication processes are placed in the same NUMA domain, different NUMA domains, or both by controlling the process binding method. This can usually be controlled by the --cpu-bind option of the scheduling system. At the same time, the number of simultaneous communication pairs can be controlled by controlling the number of processes started. The number of processes can be a positive even number within 16. At the same time, the message length needs to be controlled from 1 byte to 1M bytes, and the point-to-point communication primitives can be used separately. or Two forms; For cross-node communication, the communication parameters between nodes at different network layers are measured. For the Tianhe platform, it is necessary to apply for two nodes on the same blade, across blades, across racks, and across cabinets, and then measure on these nodes. The number of processes started at the same time varies within 32 positive even numbers; at the same time, the message length is controlled from 1 byte to 1M bytes, and the point-to-point communication primitives are used respectively. or Two forms.

[0071] S3, analyzing the communication behavior of the specific collective communication algorithm in the MPI source code, and obtaining the communication time formula represented by each parameter by simulating the execution process of the communication algorithm; Taking the binomial broadcast algorithm as an example, the algorithm with P processes participating requires In each round of communication, the number of processes participating in the communication doubles, the number interval of the communication process doubles, and the length of the message sent remains unchanged. The communication primitives are MPI_Send and MPI_Recv. Therefore, the time consumption of the whole communication operation is expressed as:

[0072]

[0073] Among them, channel type c, number of concurrent transmissions and network hierarchy information Need further analysis.

[0074] First, all source process numbers and destination process numbers participating in this round of communication are calculated according to the code logic, and the node numbers of the processes are determined based on the mapping relationship, and then the channel type of each pair of communications is determined; if it is a communication based on a network channel, the network hierarchy information of the communication node must be further determined in combination with the platform node topology, and it is determined whether the two nodes where the communication process is located are located on the same blade, across blades but in the same rack, across racks but in the same cabinet, and across cabinets; then the number of concurrent transmissions on the same channel in this round of communication is counted.

[0075] For the recursive doubling allgather algorithm, the algorithm with P processes involved requires In each round of communication, the number of processes participating in the communication remains unchanged, but the length of the message sent doubles; and the underlying layer uses The communication primitive is MPI_Sendrecv. Therefore, the time consumption of the entire communication operation is expressed as:

[0076]

[0077] Similarly, channel type c, number of concurrent transmissions and network hierarchy information Need further analysis.

[0078] For the ring allgather algorithm, the algorithm with P processes participating needs Round communication, the number of processes participating in each round of communication remains unchanged, the length of the message sent remains unchanged, and the underlying type communication primitive. Therefore, the time consumption of the entire communication operation is expressed as:

[0079]

[0080] Similarly, channel type c, number of concurrent transmissions and network hierarchy information For intra-node communication, since the algorithm has both cross-NUMA domain communication and intra-NUMA domain communication, the characteristics of the algorithm are Should be taken For cross-node communication, the communication cost of each round of the algorithm needs to take the larger value of the intra-node communication time and the inter-node communication time.

[0081] See also Figure 2 ,The experiments were conducted using MPICH 4.0.2 on the Tianjin supercomputer MT-3000 platform with 2, 4, 8, 16 nodes and 32, 64, 128, 256 processes.,The experimental results show that the MAPE of the predicted multi-node binomial broadcast algorithm is 6.96%,,which shows that the modeling method of the present invention is effective.

[0082] In summary, the present invention provides a multi-node MPI collective communication performance modeling method and system, which takes into account the communication delay differences between nodes at different network levels in the model, and takes into account the communication performance differences caused by the point-to-point communication primitive types used in the underlying algorithm. The model can more accurately predict the collective communication performance across nodes.

[0083] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0084] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0085] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0086] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0087] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0088] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0089] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0090] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0091] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0092] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0093] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A multi-node MPI collective communication performance modeling method, characterized in that: The following steps are involved: Determine the model parameters required for measurement based on the multi-core processor information in the node and the platform network layer information; For all the model parameters measured, for shared memory communication within a single node, point-to-point communication parameters located on different NUMA domains were measured; for cross-node communication parameters, point-to-point communication parameters located on different network layers were measured; The communication behavior of the specific collective communication algorithm in the MPI source code is analyzed, and combined with the obtained model parameters, the communication time formula represented by each model parameter is obtained by simulating the execution process of the communication algorithm.

2. The multi-node MPI collective communication performance modeling method according to claim 1, characterized in that: According to the multi-core processor information in the node and the platform network layer information, the model parameters required for measurement are determined. For intra-node communication, the model uses Indicates a concurrent transmission on a shared memory channel The length is Point-to-point communication time of the message; Indicates the core location of the two processes bound, which are located in the same NUMA, across NUMA, or both; Pick or , Indicates that the underlying layer uses MPI_Send and MPI_Recv for communication. Indicates that the underlying layer uses MPI_Sendrecv for communication; For cross-node communication, the model uses Indicates that processes distributed on two nodes are transmitted concurrently through a network channel at a time The length is Point-to-point communication time of the message; Indicates the network level spanned by two nodes.

3. The multi-node MPI collective communication performance modeling method according to claim 1 is characterized in that: For shared memory communication within a single node, by changing the message length, number of concurrent transmissions, process core binding method, and point-to-point communication type, the time required to send and receive messages under different characteristics is measured to obtain parameter values ​​under different characteristics; For cross-node communication parameters, by changing the message length, the number of concurrent transmissions, the number of network layers spanned by the communication process, and the point-to-point communication type, the time required to send and receive messages under different characteristics is measured to obtain the parameter values ​​under different characteristics.

4. The multi-node MPI collective communication performance modeling method according to claim 3 is characterized in that: For shared memory communication within a single node, the feature variation range is described as follows: message length ranges from 1 byte to 1M bytes; the number of concurrent transmissions ranges from 1 to n / 2, where n is the maximum number of cores in a single node; and the process binding method ranges from 1 to n / 2. , and ; Point-to-point communication primitives are used separately or Two types; For cross-node communication parameters, the characteristic variation range is described as follows: message length ranges from 1 byte to 1M bytes; the number of concurrent transmissions ranges from 1 to n (the maximum number of cores in a single node); the number of network layers spanned by the communication process ranges from 0 to k, where k is the number of platform network layers; the point-to-point communication primitives use or Two types.

5. The multi-node MPI collective communication performance modeling method according to claim 1, characterized in that: The communication behavior of the specific collective communication algorithm in the MPI source code is analyzed, and the communication time formula represented by each model parameter is obtained by simulating the execution process of the communication algorithm in combination with the obtained model parameters. Specifically, it is: According to the code logic, the numbers of all source processes and destination processes participating in this round of communication are calculated, and the node numbers of the processes are determined by combining the mapping relationship, and then the channel type of each pair of communications is determined; If the communication is based on network channels, the network hierarchy information of the communication nodes needs to be further determined in combination with the platform node topology; Count the number of concurrent transmissions on the same channel in this round of communication; Determine the point-to-point communication type used in this round of communication based on the code; Combining the obtained channel type, message length, number of concurrent transmissions, location information of each pair of communication processes and point-to-point communication type, the time required for each pair of communication in this round is expressed by corresponding parameters; the maximum value of all communication times in this round is taken as the estimated time of this round of communication; The communication time of all rounds is accumulated, and the overall communication time of a collective communication operation is expressed by a mathematical formula of parameters.

6. The multi-node MPI collective communication performance modeling method according to claim 5, characterized in that: According to the code, the point-to-point communication type used in this round of communication is determined. If it is MPI_Send and MPI_Recv primitives, it will be used later. The type parameter represents the time cost; If it is the MPI_Sendrecv primitive, use type parameter; at the same time, the message length of this round of communication is determined according to the code logic.

7. The multi-node MPI collective communication performance modeling method according to claim 5, characterized in that: The communication time of all rounds is accumulated, and the overall communication time of a collective communication operation is expressed in a mathematical formula with parameters. For the binomial broadcast algorithm, the time consumption of the entire communication operation is It is expressed as: in, is the number of processes, Indicates that the algorithm is in the channel Previous i Model parameters for round communication, is the message length, is the number of concurrent transmissions, is the network hierarchy information, Indicates that the underlying layer uses MPI_Send and MPI_Recv primitives for communication.

8. The multi-node MPI collective communication performance modeling method according to claim 5, characterized in that: The communication time of all rounds is accumulated, and the overall communication time of a collective communication operation is expressed in a mathematical formula with parameters. For the recursive doubling allgather algorithm, the time taken for the entire communication operation is It is expressed as: in, is the number of processes, Indicates that the algorithm is in the channel Previous i Model parameters for round communication, is the message length, is the number of concurrent transmissions, is the network hierarchy information, Indicates that the underlying layer uses the MPI_Sendrecv primitive for communication.

9. The multi-node MPI collective communication performance modeling method according to claim 5, characterized in that: The communication time of all rounds is accumulated, and the overall communication time of a collective communication operation is expressed in a mathematical formula with parameters. For the ringallgather algorithm, the time consumption of the entire communication operation is It is expressed as: in, is the number of processes, Indicates that the algorithm is in the channel Previous i Model parameters for round communication, is the message length, is the number of concurrent transmissions, is the network hierarchy information, Indicates that the underlying layer uses the MPI_Sendrecv primitive for communication.

10. A multi-node MPI collective communication performance modeling system, characterized in that: include: The parameter module determines the model parameters to be measured based on the multi-core processor information in the node and the platform network layer information; The measurement module measures all model parameters. For shared memory communication within a single node, the point-to-point communication parameters located in different NUMA domains are measured respectively. For cross-node communication parameters, the point-to-point communication parameters located in different network layers are measured respectively. The module is constructed, the communication behavior of the specific collective communication algorithm in the MPI source code is analyzed, and the communication time formula represented by each model parameter is obtained by simulating the execution process of the communication algorithm in combination with the obtained model parameters.

Images