Explainable DeePMD Suite Performance Systems for Emerging Supercomputers

Abstract

The present invention relates to the technical field of supercomputers, and in particular to an interpretable DeepPMD kit performance system for emerging supercomputers, comprising: a shared DP‑perf model: used to calculate the peak performance of each DeepPMD‑kit operator after analyzing the execution time of each DeepPMD‑kit operator; a distributed DP‑perf model: used to analyze the communication time across MPI processes and to achieve large-scale simulation; and a customized DeepPMD kit operator: used to predict the achievable performance of the DeepPMD kit without actual operation after obtaining the total time of the DeePMD kit performance model simulation. Through the above technical solution, the present invention has the advantages of improving the analysis accuracy of the resource utilization of the DeepPMD kit and being able to predict the achievable performance of the DeepPMD kit without actual operation.

Description

Technical Field

[0001] The present invention relates to a DeePMD performance model, and in particular to an explainable DeePMD suite performance system for emerging supercomputers. Background Art

[0002] In recent years, ab initio molecular dynamics (AIMD) has provided tools for microscopic understanding of various phenomena in many different disciplines, including but not limited to chemistry, biology, and materials science. However, due to the complexity of AIMD, the spatial and temporal scales are limited (only hundreds of atoms can be processed in picoseconds) methods. Although some linear methods have been implemented on top supercomputers for millions of atoms in less than hundreds of femtoseconds per day, they are limited to insulating systems with large band gaps.

[0003] Meanwhile, the empirical force field (EFF) method enables many practical interests, including heterogeneous catalysis, electrochemical cells, and biomedical reactions that require trillion-atom or even larger simulations. However, EFF cannot accurately describe quantum mechanical phenomena such as bond breaking, complex chemical processes, quantum dots, etc.

[0004] Neural network molecular dynamics (NNMD) has attracted great research interest due to its ability to balance ab initio accuracy and computational efficiency. By leveraging neural networks and extensive training data, high-dimensional nonlinear potential energy surfaces (PES) can be accurately fitted. It was developed in 2007 by the Behler-Parrinello scheme, which laid the foundation for the development of various NNMD packages such as NEquIP, PaiNN, and the Deep Potential (DP) scheme.

[0005] DeePMD-kit is a paradigmatic implementation of the DP scheme, achieving unprecedented simulations of up to 10 billion atoms with nanosecond / day performance on top supercomputers, making it the most popular package for large-scale molecular dynamics simulations in the NNMD community.

[0006] DeePMD-kit is a abinitio molecular dynamics framework based on the DP scheme. Figure 1 An overview of the computational process is given, which includes two components: the descriptor D and the fitted network N. D is used to extract the input environment matrix The symmetry preserving features are calculated. N learns the relationship between the local environment features obtained from D and the atomic energy Ei. Therefore, the PES of the whole system is expressed as the sum of the atomic energy contributions .

[0007] Figure 2 The calculation of D is shown, expressed as:

[0008] ,

[0009] in, is the embedding matrix, yes The constructed submatrix is ​​taken Before Column, environment matrix It is expressed as:

[0010] ,

[0011] in, is the relative position of each neighboring atom, is a gating function. matrix By input Derivation of the embedding network. The embedding network is a multi-layer residual fully connected model. In each layer, the output size doubles, and the output size of the model is M.

[0012] like Figure 3 As shown in Figure 1, the fitted network N is a fully connected network that predicts Ei. Each layer is a fully connected layer with Tanh activation. Combining D and N, DeePMD-kit can achieve representation learning from the environment matrix to the system PES. The force of atom i is calculated by the back-propagation gradient of the DP model and is expressed as ,in Calculated by back propagation. The virial coefficient of the system is expressed as , which is calculated by the ProdVirial operator.

[0013] The computation of the embedding network in the descriptor occupies more than 90% of the total computation time. In order to reduce the computational burden, a tabulation method is proposed to approximate the embedding network, as shown in Figure 4 As shown, it is expressed as ,in, , which is built from Tabulate and TabulateGrad operators to fit the weights of the embedding network after training using a fifth-order polynomial via Weierstrass approximation.

[0014] Current supercomputers (i.e. Tianhe-3F, New Sunway, Fugaku and Summit) are usually equipped with many-core chips and innovative computing element designs to improve computing potential. These designs provide new opportunities to enhance the temporal and spatial scales of the DeePMD suite.

[0015] However, the performance gap between DeePMD-kit and emerging supercomputers still exists. The performance model of DeePMD kit is used to accurately identify the performance model and guide researchers to make better hardware-software co-design.

[0016] Currently, many performance models have been designed and used to analyze the performance of various applications or algorithms. Typical works include performance models to guide automatic parameter tuning, application-level performance models for analyzing computing time and machine computing power, and communication-level performance models for analyzing communication bandwidth and network latency of supercomputers.

[0017] However, DeePMD-kit is a complex application that contains many irregular custom operators and complex MPI communications. Current performance models cannot accurately describe the performance utilization of these special operators and communication patterns in the DeePMD suite. In addition, access to a full supercomputer is very limited, and there is no effective way to accurately estimate the simulation latency and achieved peak performance of DeePMD-kit without performing real MD simulations on the full machine. In addition, current performance models have poor transferability and ignore the impact of the simulated physical system and computing platform on the performance of DeePMD-kit.

[0018] Although the DeePMD suite has been successfully optimized on supercomputers and demonstrated excellent performance, there is a significant performance gap between the peak performance and the performance achieved by the DeePMD suite. Therefore, it is important to model and analyze the performance of the DeePMD suite respecting different physical systems and target supercomputers. However, several challenges remain:

[0019] 1) No performance model can accurately analyze the resource utilization of the DeePMD kit. The DeePMD-kit contains many irregular custom operators and complex MPI communication patterns. Its computational graph varies depending on the simulated physical system. Existing performance models only focus on specific aspects of some applications and lack generality in analyzing the DeePMD-kit. The Roofline-based model can analyze the computational resource utilization of the official TensorFlow operators in the DeePMD kit in a single process after completing the simulation, but cannot analyze the MPI communication. The LogP-based model can characterize network bandwidth and latency, but cannot characterize general operators.

[0020] 2) Current performance models cannot predict the achievable performance of the DeePMD suite without actually running it. We need to analyze the computational limitations of the DeePMD suite on specific platforms, including achievable performance, solution time, and physical systems. MD simulations of large physical systems require the resources of the entire machine, which makes simulations expensive and time-consuming. Providing accurate predictions can benefit this process. The long time steps required to simulate large physical systems consume a lot of computing resources. Therefore, it is critical to analyze costs and determine the hardware / software configuration before launching these simulations. Summary of the invention

[0021] In order to solve the problems in the related technology, the present application provides an explainable DeePMD suite performance system for emerging supercomputers, which solves the problem that there is no performance model to accurately analyze the resource utilization of the DeePMD suite and the existing performance model cannot predict the performance that the DeePMD suite can achieve without actual operation.

[0022] The technical solution is as follows:

[0023] The DeePMD suite of interpretable performance systems for emerging supercomputers features, including:

[0024] Shared DP-perf model on nodes: used to analyze the execution time of each DeePMD-kit operator and calculate the peak performance of each DeePMD-kit operator based on the obtained execution time;

[0025] Distributed DP-perf model across nodes: used to analyze the communication time across MPI processes and implement large-scale simulation based on the obtained communication time;

[0026] Customized DeePMD suite operator: used to obtain the total time of the DeePMD suite performance model simulation and predict the achievable performance of the DeePMD suite without actual operation based on the total simulation time. The total simulation time is the sum of the calculation time of multiple operators.

[0027] Through the above technical scheme, through the setting of the interpretable performance model of DP-perf, which characterizes the computing and communication modes of the DeepPMD kit at the operator level according to the machine configuration and the physical MD system, DP-perf can obtain accurate fitting and prediction to analyze the performance utilization and communication of each DeepPMD-kit operator; at the same time, DP-perf can accurately predict the MD simulation time and the achievable peak performance. DP-perf can provide valuable insights for the design and application innovation of new E-class supercomputers and bridge the performance gap between multi-core supercomputers and ab initio molecular dynamics, which is helpful for analyzing simulation efficiency and simulation cost.

[0028] Preferably, the method of sharing the DP-perf model to analyze the execution time of the DeePMD suite is:

[0029] When a custom operator in DeePMD-kit is selected, the execution time of the custom operator is expressed as: ,in, represents the total arithmetic work of the operator derived from the physical properties and algorithm of the corresponding operator, and the parameter Indicates the number of computing nodes; Indicates the delay caused by memory allocation and kernel startup; is the peak performance achievable; , P represents the theoretical peak performance of the CPU computing node;

[0030] The execution time of the custom operator is updated to: ,in, Obtained through linear regression;

[0031] When the official operator in DeePMD-kit that involves data format conversion in memory and does not include any calculation is selected, the official operator is represented by the bandwidth equation, and the execution time of the official operator is expressed as: ,in, Represents the total memory access of the operator, which is obtained from the physical characteristics and algorithms of the corresponding operator; Indicates the delay caused by memory allocation and kernel startup; is the memory bandwidth achievable on a single node; , represents the peak memory bandwidth of the computing node;

[0032] The execution time of the official operator is updated to: ,in, Obtained through linear regression.

[0033] Preferably, each MPI process in DeePMD-kit includes a local area and a ghost area; the method for analyzing the execution time of the DeePMD suite by the distributed DP-perf model is:

[0034] When the number of ghost atoms in each process remains constant and is positively correlated with the ghost area, according to The model communication time is expressed as: , where l represents the sum of the delay and overlap of each process, Indicates the theoretical network bandwidth, Indicates parameters related to network bandwidth utilization. represents the total number of ghost atoms in each direction, Indicates the total number of ghost atoms in each MPI process.

[0035] Preferably, the customized DeePMD suite operator obtains the computation time and the MPI communication time by obtaining the output log of the supercomputer, and fits the parameters according to the training log on the supercomputer;

[0036] The output log is in LAMMPS format, and the computation time of each operator is calculated using the MPI wtime() code.

[0037] Preferably, the shared DP-perf model includes and ;

[0038] In the customized DeePMD suite operator, Used to reflect peak performance utilization, The lower the value of, the higher the peak performance;

[0039] In the official operator, Used to reflect memory bandwidth utilization, The lower the value, the higher the bandwidth utilization;

[0040] It is used to reflect the latency of memory operations and memory startup. The latency includes memory copying, data loading, and kernel startup latency.

[0041] Preferably, the distributed DP-perf model includes and l; It is used to reflect the network bandwidth utilization, and l is used to reflect the network delay.

[0042] Preferably, a customized DeePMD suite operator is created for each DeePMD suite operator and communication wear based on the shared DP-perf model and the distributed DP-perf model; the total time of the customized DeePMD suite operator simulation is expressed as:

[0043] ,

[0044] in, ;

[0045] ;

[0046] ;

[0047] ;

[0048] ;

[0049] ;

[0050] ;

[0051] ;

[0052] ;

[0053] ;

[0054] N represents the number of atoms in the physical system, represents the number of neighboring atoms of each atom, T represents the number of atomic types in the physical system; the parameter Indicates the memory bandwidth of the CPU.

[0055] In summary, the beneficial effects of the present invention are that through the setting of the interpretable performance model of DP-perf, the model characterizes the computing and communication modes of the DeepPMD kit at the operator level according to the machine configuration and the physical MD system, and DP-perf can obtain accurate fitting and prediction to analyze the performance utilization and communication of each DeepPMD-kit operator; at the same time, DP-perf can accurately predict the MD simulation time and the achievable peak performance. DP-perf can provide valuable insights for the design and application innovation of new E-class supercomputers and bridge the performance gap between multi-core supercomputers and ab initio molecular dynamics, which is helpful for analyzing simulation efficiency and simulation cost.

[0056] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0058] Figure 1 A schematic diagram of the calculation progress of DeePMD in the schematic diagram of DeePMD overview;

[0059] Figure 2 The descriptor network in the schematic diagram outlined for DeePMD will be and Map to Schematic diagram;

[0060] Figure 3 It is a schematic diagram of the fitting network framework in the schematic diagram of DeePMD overview;

[0061] Figure 4 A schematic diagram of the framework of the tabulation operator in the schematic diagram outlined for DeePMD;

[0062] Figure 5 Schematic diagram for performing abinitio molecular dynamics and analyzing the physical or chemical properties of the simulated area as outlined in DeePMD;

[0063] Figure 6Schematic diagrams of processors of four top supercomputers in the embodiments of the present invention, wherein (a) is a schematic diagram of a processor in each Tianhe-3F supercomputer, (b) is a schematic diagram of a processor in a New Sunway supercomputer, (c) is a schematic diagram of a processor in a Fugaku supercomputer, and (d) is a schematic diagram of a processor in a Summit supercomputer;

[0064] Figure 7 Schematic diagrams of a shared DP-perf model and a distributed DP-perf model in the present invention, wherein (a) represents a schematic diagram of a shared DP-perf model, and (b) represents a schematic diagram of a distributed DP-perf model;

[0065] Figure 8 It is a schematic diagram of the simulation time accuracy of the DP-perf model fitting of the copper and water system by the Tianhe-3F supercomputer in a specific embodiment of the present invention, wherein (a) represents the comparison of the overall MD time, (b) represents the comparison of the calculation time using the shared DP-perf, and (c) represents the comparison of the communication time using the distributed DP-perf;

[0066] Fig. 9 It is a schematic diagram of the simulation time accuracy of the New Sunway supercomputer in the DP-perf model for fitting the copper and water system in a specific embodiment of the present invention, wherein (a) represents the comparison of the overall MD time, (b) represents the comparison of the calculation time using the shared DP-perf, and (c) represents the comparison of the communication time using the distributed DP-perf;

[0067] Fig.10 It is a schematic diagram of the simulation time accuracy of the Fugaku supercomputer in the DP-perf model fitting of the copper and water system in a specific embodiment of the present invention, wherein (a) represents the comparison of the overall MD time, (b) represents the comparison of the calculation time using the shared DP-perf, and (c) represents the comparison of the communication time using the distributed DP-perf;

[0068] Fig.11 It is a schematic diagram of the simulation time accuracy of the Summit supercomputer in the DP-perf model fitting of the copper and water system in a specific embodiment of the present invention, wherein (a) represents the comparison of the overall MD time, (b) represents the comparison of the calculation time using the shared DP-perf, and (c) represents the comparison of the communication time using the distributed DP-perf;

[0069] Fig.12 It is a schematic diagram of predicting the achievable peak performance of a water system using DP-perf in a specific embodiment of the present invention;

[0070] Fig.13It is a schematic diagram of predicting the achievable peak performance of a copper system using DP-perf in a specific embodiment of the present invention; DETAILED DESCRIPTION

[0071] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0072] As attached Figure 1-13 As shown: In this specific embodiment, four top supercomputers are selected to evaluate DP-perf, namely Tianhe-3F, New Sunway, Fugaku and Summit, because we can access the published performance results or the open source DeePMD-kit code of these supercomputers. These platforms include homogeneous ARMv8 architecture, MPE-CPE heterogeneous architecture and CPU-GPU heterogeneous architecture, which can represent the main supercomputing systems in the world.

[0073] The Tianhe-3 prototype consists of FT-2000+ processors and MT-2000+ processors. The FT-2000+ queue of the Tianhe-3 prototype is called the Tianhe-3F supercomputer, which consists of 5,000 computing nodes with a theoretical performance of 2.9PFLOPS and an accuracy of 2.9PFLOPS. Each node in Tianhe-3F is equipped with an FT-2000+ processor, such as Figure 6 As shown in (a), the processor adopts the ARMv8 architecture, has a 128-bit SIMD unit, and has a theoretical double-precision peak performance of 588.8GFLOPS. Each FT-2000+ consists of eight NUMA structures called panels, sharing 128GB of memory and 204.8GB / s memory bandwidth. Each panel has an independent memory control unit (MCU) and eight cores. In each panel, each core has a 32KB L1 cache, and 4 cores share an independent 2MB L2 cache. At the same time, each computing node is interconnected with a bidirectional network with a bandwidth of 200Gbps.

[0074] The New Sunway supercomputer contains 107,520 computing nodes with a theoretical double-precision peak performance of 1.5 EFLOPS. The computing nodes are interconnected through a fat-tree network topology, and every 256 nodes in a rack form a supernode. Each node is equipped with a SW26010-pro processor, which contains 6 core groups (CGs), providing a theoretical double-precision peak performance of 14 TFLOPS, such as Figure 6 As shown in (b) in the figure. Each CG contains a management button element (MPE) and an 8x8 computational processing element (CPE) grid connected to a 16GB DDR4 main memory with a bandwidth of 51.2GB / s. Each CPE has a 256KB manually controlled scratchpad memory called LDM. MPE and CPE have 256-bit and 512-bit SIMD units, respectively. The SW26010-pro processor supports direct memory access (DMA) to transfer continuous data between LDM and main memory with a theoretical bandwidth of 307GB / s. In addition, through remote memory access (RMA), data transfer between CPEs in the same CG has a bandwidth of approximately 400GB / s. The programming model of New Sunway is the Sunway Accelerated Computing Architecture (SACA), which provides a basic compilation environment, basic libraries, and runtime.

[0075] The Fugaku supercomputer has a theoretical peak performance of 537 PFLOPS at double precision. It consists of 158,976 nodes, each equipped with an A64FX processor, such as Figure 6 As shown in (c), the processor consists of 52 cores, 4 of which are used for the operating system and IO and 48 for computing. The chip consists of 4 NUMA domains called core memory groups (CMGs), each CMG is connected to its 8GB local HBM2 memory with a bandwidth of 256GB / s. The A64FX processor supports 512-bit SVE operations and two SIMD pipelines, enabling each core to perform 32 double-precision floating-point calculations per cycle. At the same time, Fugaku is equipped with a 6D ring / mesh interconnect called the Tofu-D network with a theoretical bandwidth of 400Gbps.

[0076] The Summit supercomputer can achieve about 200 PFLOPS of double precision. At the same time, its mixed precision capability can reach 3 EFLOPS. The building block of Summit is the AC922 node, such as Figure 6As shown in (d), each node contains 2 POWER9 processors and 6 NVIDIA Tesla V100 accelerators. Each processor is also connected through dual NVLINK modules, each module can provide 25GB / s bandwidth in each direction. The connection between each node uses dual-port InfiniBand interconnection, adopts non-blocking fat tree topology switching structure, and each port provides 200Gbps bandwidth.

[0077] Before building the model, the total arithmetic workload of the DeePMD-kit operator needs to be determined through physical parameters (including MD parameters and DeePMD-kit model configuration). and memory access . They are influenced by the characteristics of the molecular system and the algorithm.

[0078] For the total arithmetic workload For calculation, we take the Tabulate operator in DeePMD-kit as an example; input , , R , number of atoms , the number of neighbors and characteristic size K,

[0079] Initially, it traverses It loops k times to process the neighbors of all atoms in the physical system and determine the corresponding table weights. It then loops k times to calculate the polynomial result v and performs matrix multiplication with the embedding matrix R to obtain . Therefore, the arithmetic work of the tabulation operator is O( ). For each inner loop, we can compute 14 floating point operations. Finally, we construct the arithmetic workload as: .

[0080] For memory access Calculation, we take the Pad operator in DeePMD-kit as an example; input Pad feature , the number of atoms N, the size before filling M and , which accesses memory along the selected dimension of the input tensor x by adding the memory size of the input and output tensors. The input size is , the output size is . Then the total memory access (in bytes) is expressed as: ; In DeePMD-kit, , then the total memory access is byte.

[0081] The DP-perf model is constructed by collecting hardware parameters (i.e., machine configuration) and constructing the specific parameter description of the DP-perf model. In hardware, parameter P represents the theoretical peak performance of a single node, and parameter Indicates the memory bandwidth of the CPU, parameter represents the memory bandwidth of HPC, parameter L represents the network delay, and parameter Indicates the number of computing nodes;

[0082] In molecules, the parameter N represents the number of atoms in the physical system, and the parameter represents the number of neighboring atoms of each atom, parameter T represents the number of atomic types in the physical system, and parameter represents the number of ghost atoms in each node;

[0083] In the model, parameter K represents the size of the descriptor, and parameter represents the input feature dimension of the i-th layer, parameter represents the output feature dimension of the i-th layer, and the parameter L represents the number of dense layers in the fitting network.

[0084] The Explainable DeePMD Suite Performance System for Emerging Supercomputers includes:

[0085] Shared DP-perf model on nodes: used to analyze the execution time of each DeePMD-kit operator and calculate the peak performance of each DeePMD-kit operator based on the obtained execution time;

[0086] Distributed DP-perf model across nodes: used to analyze the communication time across MPI processes and implement large-scale simulation based on the obtained communication time;

[0087] Customized DeePMD suite operator: used to obtain the total time of the DeePMD suite performance model simulation and predict the achievable performance of the DeePMD suite without actual operation based on the total simulation time. The total simulation time is the sum of the calculation time of multiple operators.

[0088] Among them, the method of sharing the DP-perf model to analyze the execution time of the DeePMD suite is:

[0089] When a custom operator in DeePMD-kit is selected, the execution time of the custom operator is expressed as:

[0090] ,

[0091] in, represents the total arithmetic work of an operator derived from the physical properties and algorithm of the corresponding operator; Indicates the delay caused by memory allocation and kernel startup; is the peak achievable performance; in the shared DP-perf model, , P represents the theoretical peak performance of the CPU computing node;

[0092] The execution time of the custom operator is updated to:

[0093] ,

[0094] in, Obtained through linear regression.

[0095] When the official operator in DeePMD-kit that involves data format conversion in memory and does not include any calculation is selected, the official operator is represented by the bandwidth equation, and the execution time of the official operator is expressed as:

[0096] ,

[0097] in, Represents the total memory access of the operator, which is obtained from the physical characteristics and algorithms of the corresponding operator; represents the delay caused by memory allocation and kernel startup; and is the achievable memory bandwidth on a single node; in the shared DP-perf model, , represents the peak memory bandwidth of the computing node;

[0098] The execution time of the official operator is updated to:

[0099] ,

[0100] in, Obtained through linear regression.

[0101] DeePMD-kit is used as a plug-in embedded in LAMMPS to achieve large-scale simulation. The MPI process of DeePMD-kit follows the communication mode of LAMMPS to update the atomic positions. Figure 7 As shown in (b) in the figure, each MPI process in DeePMD-kit consists of a local region and a ghost region (shaded); the local region is mainly used to calculate the interactions between atoms, while the ghost region is used to calculate atomic forces and perform data communication. LAMMPS communication is represented by executing MPISend requests in the ghost region in each direction of the box.

[0102] Figure 7(b) in Figure 2 shows the communication of ghost atoms in a 2D box. Taking process 0 as an example, processes 2 and 3 send their own atoms in their green regions to process 0 in the x-axis exchange phase. Then in the y-axis exchange phase, processes 1 and 4 send atoms in their red regions to rank 0, which include the ghost atoms they received in the x-phase. Therefore, process 0 gets all of its ghost atoms; the atoms in the blue solid corner regions are communicated twice before process 0 receives them. For a 3D box, the communication can be represented as MPI sends in six directions, and the amount of communication is the number of atoms in the ghost regions of the send directions.

[0103] The method of analyzing the execution time of the DeePMD suite using the distributed DP-perf model is as follows:

[0104] When the number of ghost atoms in each process remains constant and is positively correlated with the ghost area, according to The model communication time is expressed as:

[0105] ,

[0106] Where l represents the sum of the delay and overlap of each process, Indicates the theoretical network bandwidth, Indicates parameters related to network bandwidth utilization. represents the total number of ghost atoms in each direction, Indicates the total number of ghost atoms in each MPI process;

[0107] Based on the shared DP-perf model and the distributed DP-perf model, a customized DeePMD suite operator is created for each DeePMD suite operator and communication wear. The customized DeePMD suite operator is used to obtain the total time of the DeePMD suite performance model simulation. The total simulation time is the sum of the calculation time of multiple operators.

[0108] The customized DeePMD suite operator obtains the computation time and MPI communication time by obtaining the output log of the supercomputer and fitting the parameters according to the training log on the supercomputer; the output log is in LAMMPS format and the computation time of each operator is calculated by the MPI wtime() code.

[0109] Shared DP-perf models include and ; In the customized DeePMD suite operator, Used to reflect peak performance utilization, The lower the value of, the higher the peak performance; in the official operator, Used to reflect memory bandwidth utilization, The lower the value, the higher the bandwidth utilization; It is used to reflect the latency of memory operations and memory startup. The latency includes memory copying, data loading, and kernel startup latency.

[0110] The distributed DP-perf model includes and l; It is used to reflect the network bandwidth utilization, and l is used to reflect the network delay.

[0111] Customized DeepPMD suite operators are created for each DeePMD suite operator and communication wear based on the shared DP-perf model and the distributed DP-perf model; the total time of the customized DeePMD suite operator simulation is expressed as:

[0112] ,

[0113] in, ;

[0114] ;

[0115] ;

[0116] ;

[0117] ;

[0118] ;

[0119] ;

[0120] ;

[0121] ;

[0122] ;

[0123] N represents the number of atoms in the physical system, represents the number of neighboring atoms of each atom, T represents the number of atomic types in the physical system; the parameter Indicates the memory bandwidth of the CPU.

[0124] The detailed performance equation of the DP-perf model and the training coefficients on four top supercomputers are as follows:

[0125] In Tianhe-3F: , ; , ; , ; , ; , ; , ; , ; , ; , ; , ; ;

[0126] In New Sunway: , ; , ; , ; , ; , ; , ; , ; , ; , ; , ; ;

[0127] In Fugaku: , ; , ; , ; , ; , ; , ; , ; , ; , ; , ; ;

[0128] In Summit: , ; , ; , ; , ; , ; , ; , ; , ; , ; , ; .

[0129] In this specific embodiment, two systems are used as evaluation benchmarks, namely a water system and a copper system. We selected extended results from four supercomputers to train the DP-perf model.

[0130] Specifically, the raw data for Summit and Fugaku are derived from published papers, and Tianhe-3F and NewSunway are derived from our real runs, based on our training datasets built by collecting weak and strong scaling logs. For strong scaling simulations, we use 32.5 / 14.9 / 8.3 / 41.5 million atoms of water system and 5.9 / 2.5 / 2.2 / 13.5 million atoms of copper system to saturate the memory of 50 nodes, and gradually increase to 4,600 nodes on Tianhe-3F, New Sunway, Fugaku, and Summit. For weak scaling simulations, we use 17.1 / 7.6 / 97.3 / 243.8 million atoms of water system and 7.1 / 3.4 / 67.6 / 21.25 million atoms of copper system to saturate 256 / 256 / 621 / 575 nodes of the total machine memory, then gradually increase the system size and the number of nodes of the whole machine, and measure the simulation peak performance achieved on these four platforms.

[0131] Due to the limitation of accessible resources, we only measured the weak scaling results of 2,048 / 90,000 / 9,936 nodes on Tianhe-3F, New Sunway, and Fugaku, respectively. After collecting the evaluation results, we built datasets of 26 / 31 / 21 / 32 samples on Tianhe-3F, NewSunway, Fugaku, and Summit, respectively. We randomly divided the datasets of Tianhe-3F, NewSunway, and Fugaku into training datasets and test datasets with a ratio of 4:1.

[0132] To evaluate the fitting accuracy on known datasets, we used the training datasets on the Tianhe-3F, New Sunway, Fugaku, and Summit supercomputers to fit the execution time. Figure 8 The detailed experimental results are shown in Figure 2, where the x-axis represents the actual execution time and the y-axis represents the fitting time of the DP-perf model. Figure 8 Draw the y=x reference line in .

[0133] like Figure 8 (a) Fig. 9 (a) Fig.10 (a) and Fig.11 As shown in (a), the fitted MD time points show a uniform distribution around the y=x line, indicating a linear correlation with the actual execution. Through Pearson correlation analysis, the Pearson coefficient of Tianhe-3F reached 0.9941, Fugaku reached 0.9925, New Sunway reached 0.999, and Summit reached 0.966, showing a significant linear correlation. Then, we evaluated the mean absolute percentage age error (MAPE) between the fitted MD time and the actual MD time.

[0134] On Tianhe-3F, DP-perf has an average MAPE of 5.7%. On New Sunway, DP-perf has an average MAPE of 8.1%. On Fugaku, DP-perf has an average MAPE of 14.3%. On Summit, DP-perf has an average MAPE of 13.5%.

[0135] We analyze the accuracy of shared DP-perf and distributed DP-perf models, as Figure 8 (b) Fig. 9 (b) Fig.10 (b) and Fig.11 (b) and Figure 8 (c) Fig. 9 (c) Fig.10 (c) and Fig.11 Both models show high accuracy, with the shared DP-perf model achieving a Pearson coefficient of 0.997 on Tianhe-3F, 0.999 on New Sunway, 0.943 on Fugaku, and 0.937 on Summit.

[0136] The distributed DP-perf model achieves a Pearson coefficient of 0.987 on Tianhe-3F, 0.991 on New Sunway, 0.829 on Tianhe-3F, and 0.833 on Summit. Figures 8 to 11 These results in show that the fitted parameters of the DP-perf model can accurately represent the peak performance utilization, memory bandwidth utilization, and network bandwidth utilization with low error.

[0137] The fitted mappings for each operator on the four top supercomputers were obtained by training the coefficients on the four top supercomputers:

[0138] In Tianhe-3F: ; ; ; ; ; ; ; ; ; ;

[0139] At New Sunway: ; ; ; ; ; ; ; ; ; ;

[0140] In Fugaku: ; ; ; ; ; ; ; ; ; ;

[0141] At Summit: ; ; ; ; ; ; ; ; ; .

[0142] According to the results, each operator achieved a low error rate of less than 20%.

[0143] We further perform prediction evaluation to analyze the accuracy of overall MD time prediction. We use the training dataset to train the DP-perf model and use the test dataset to verify its accuracy. To avoid the randomness and fitting uncertainty of DP-perf, we repeat the above steps 5 times and calculate the average metrics.

[0144] In Tianhe-3F: the Pearson coefficient is 0.990, and the MAPE is 13.5%. MAPE refers to the mean absolute percentage age error between the fitted MD time and the actual MD time. is 0.955, refers to the coefficient of determination;

[0145] In New Sunway: Pearson coefficient is 0.986, MAPE is 11.6%, is 0.952;

[0146] In Fugaku: Pearson coefficient is 0.997, MAPE is 8.9%, is 0.969;

[0147] In Summit: Pearson coefficient is 0.978, MAPE is 13.5%, is 0.830;

[0148] We can observe that the MD simulation times predicted by DP-perf maintain a very low average MAPE and show a strong linear correlation with the actual test results, which indicates that DP-perf can predict the achievable peak performance of MD simulations and DeepMD-kit with high accuracy.

[0149] The trained DP-perf model can accurately predict the execution time of DeepPMD-kit. Based on the predicted time and the DeePMD-kit workload, we can calculate the corresponding peak performance achievable by this MD simulation for a given physical system. Based on the previous weak scaling logs, we measured 120 / 290 / 160 / 3900 million atoms of water system and 0.80 / 1.30 / 120 / 3.40 billion atoms of copper system on 2,048 / 90,000 / 9,936 / 4,600 nodes on Tianhe-3F, New Sunway, Fugaku, and Summit, respectively. For the rest of the results scaled to the whole machine, we use DP-perf to further predict the achievable peak performance. Figure 12 to Figure 13Predictions of weak scaling results on these platforms are shown, where the solid lines represent actual results from our previous evaluation and the dashed lines represent predicted results based on DP-perf. Note that previous work has performed weak scaling evaluations on the entire Summit machine, but we do not need accurate predictions for DP-perf. Based on the results, we predict that the optimized DeePMD suite can achieve 938T / 124.8P / 60.3P / 46.3PFLOPS for 2.4 / 3.8 / 3.9 billion atom water simulations on Tianhe-3F, New Sunway, Fugaku, and Summit full machines. Meanwhile, the optimized DeePMD suite can achieve 780T / 119.1P / 49.3P / 43.7PFLOPS for 1.5 / 1.3 / 1.74 / 3.4 billion atom copper systems on the full machines of these supercomputers.

[0150] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that the present invention did not invent. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present invention are indicated by the appended claims.

[0151] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. The explainable DeePMD suite performance system for emerging supercomputers is characterized by: include: Shared DP-perf model on nodes: used to analyze the execution time of each DeePMD-kit operator and calculate the peak performance of each DeePMD-kit operator based on the obtained execution time; Distributed DP-perf model across nodes: used to analyze the communication time across MPI processes and implement large-scale simulation based on the obtained communication time; Customized DeePMD suite operator: used to obtain the total simulation time of the DeePMD suite performance model and predict the achievable performance of the DeePMD suite without actual operation based on the total simulation time. The total simulation time is the sum of the calculation time of multiple operators. The method of sharing the DP-perf model to analyze the execution time of the DeePMD suite is: When a custom operator in DeePMD-kit is selected, the execution time of the custom operator is expressed as: ,in, represents the total arithmetic work of the operator derived from the physical properties and algorithm of the corresponding operator, and the parameter Indicates the number of computing nodes; Indicates the delay caused by memory allocation and kernel startup; is the peak performance achievable; , P represents the theoretical peak performance of the CPU computing node; The execution time of the custom operator is updated to: ,in, Obtained through linear regression; When the official operator in DeePMD-kit that involves data format conversion in memory and does not include any calculation is selected, the official operator is represented by the bandwidth equation, and the execution time of the official operator is expressed as: ,in, Represents the total memory access of the operator, which is obtained from the physical characteristics and algorithms of the corresponding operator; Indicates the delay caused by memory allocation and kernel startup; is the memory bandwidth achievable on a single node; , represents the peak memory bandwidth of the computing node; The execution time of the official operator is updated to: ,in, Obtained through linear regression; Each MPI process in DeePMD-kit includes a local area and a ghost area. The distributed DP-perf model analyzes the execution time of the DeePMD suite as follows: When the number of ghost atoms in each process remains constant and is positively correlated with the ghost area, according to The model communication time is expressed as: , where l represents the sum of the delay and overlap of each process, Indicates the theoretical network bandwidth, Indicates parameters related to network bandwidth utilization. represents the total number of ghost atoms in each direction, Indicates the total number of ghost atoms in each MPI process; Customized DeepPMD Suite Operators are created for each DeePMD Suite Operator and Communication Wear based on the shared DP-perf model and distributed DP-perf model; The total time of the customized DeePMD suite operator simulation is expressed as: ， in, ; ； ； ； ； ； ； ； ； ； N represents the number of atoms in the physical system, represents the number of neighboring atoms of each atom, T represents the number of atomic types in the physical system; the parameter Indicates the memory bandwidth of the CPU.

2. The explainable DeePMD suite performance system for emerging supercomputers according to claim 1, characterized in that: The customized DeePMD suite operator obtains the computation time and MPI communication time by obtaining the output log of the supercomputer, and fits the parameters according to the training log on the supercomputer; The output log is in LAMMPS format, and the computation time of each operator is calculated using the MPI wtime() code.

Images