An efficient method for dividing physical process tasks in atmospheric models

Abstract

The present invention discloses an efficient atmospheric model physical process task division method, comprising the following steps: S1, obtaining the supernode number of each process in the atmospheric model; S2, grouping each process in the atmospheric model according to the supernode number information obtained in S1, and dividing the processes with the same supernode number into the same group; S3, first-level task division, using the same grid division method as the power framework, ensuring that the atmospheric model power framework and the physical process in the same group of processes have the same grid; S4, second-level task division, adjusting the grid division between processes in the group in the atmospheric model physical process, and redistributing the grid to the processes in the group. The present invention can not only make the atmospheric model physical process calculation tasks relatively balanced, but also reduce the communication overhead when the atmospheric model is coupled with the power framework, and can significantly improve the overall performance of the atmospheric model during large-scale parallelization.

Description

Technical Field

[0001] The invention relates to a high-efficiency atmospheric model physical process task division method, belonging to the technical field of climate and meteorological forecasting. Background Art

[0002] Climate and meteorological prediction is one of the most complex applied problems in the world today, involving many different scientific fields. Since the 1950s, numerical simulation methods have made great progress and played an important role in the field of climate and meteorological prediction. From extreme weather disasters to daily life planning, numerical simulation prediction provides a strong scientific basis for various practical problems.

[0003] As one of the important components of climate models, atmospheric models are an important method used by scientists and scholars to study and understand the laws of the atmosphere. They are also one of the most important scientific applications that promote the development of the world's most powerful supercomputers.

[0004] The atmospheric model mainly consists of two parts: the dynamic framework and the physical process. The dynamic framework solves the atmospheric motion equations, using grid points as the basic calculation unit, accompanied by neighbor communication and global reduction operations. The physical process mainly calculates grid vertical columns, mainly calculating rainfall, clouds, long-wave and short-wave radiation, etc., and there is no need for communication between grid vertical columns.

[0005] In order to adapt to the different calculation characteristics of the atmospheric model dynamic framework and physical process, different methods of dividing the atmospheric grid are required before the atmospheric model dynamic framework and physical process are calculated. In the atmospheric model calculation process, the dynamic framework and physical process calculations are performed alternately at each time step. Before the dynamic framework and physical process calculations, there needs to be a coupling information exchange process between the dynamic framework and the physical process. If the dynamic framework and physical process in the atmospheric model divide the atmospheric grid differently, that is, the dynamic framework process number and the physical process process number belonging to the same atmospheric grid are different, then when the dynamic framework and the physical process are coupled, the two processes need to communicate to obtain the physical quantity information of the grid.

[0006] With the increase of grid accuracy and the expansion of parallel scale, the calculation of physical processes in atmospheric models and the communication when coupling between physical processes and dynamical frameworks become more complicated. Balancing the calculation time of physical processes in atmospheric models and the communication time when coupling physical processes with dynamical frameworks is one of the key points to improve the overall performance of atmospheric models in earth system models. On the one hand, with the increase of grid accuracy, the calculation of physical processes in atmospheric models becomes more complicated, and the communication volume increases during coupling; on the other hand, in heterogeneous architectures, the parallel scale expands, the computing tasks of a single processor are reduced, but the communication form becomes more complicated. Faced with complex computing and communication problems, reasonable grid division of physical processes in atmospheric models has always been one of the key and difficult points of research.

[0007] The network structure of the Sunway TaihuLight supercomputer is divided into three levels. The top layer is the central switching network, the middle layer is the supernode network, and the bottom layer is the resource sharing network. The network diameter is 7, and the total bidirectional bandwidth of the entire network reaches 56TB / s. The supernode has a fully connected architecture, and the computing nodes are connected through the InfiniBand FDR network. The network between supernodes forms a two-layer fat tree structure. Supernodes are connected to each other through 64 network cables. Compared with the full connection within the supernode, the bandwidth is cut by 1:4.

[0008] On the Sunway TaihuLight supercomputer platform, the existing method of dividing the physical process calculation tasks of the atmospheric model can easily lead to unbalanced distribution of the physical process calculation tasks or complex communication when the physical process and the dynamic framework are coupled. It is difficult to balance the communication requirements when the physical process calculation of the atmospheric model and the physical process and the dynamic framework are coupled, which will cause a performance loss of 10% to 30% to the atmospheric model. Summary of the invention

[0009] The purpose of the present invention is to provide an efficient method for dividing the physical process tasks of an atmospheric model, which can not only make the physical process calculation tasks of the atmospheric model relatively balanced, but also reduce the communication overhead when the atmospheric model is coupled with the dynamic framework, and can significantly improve the overall performance of the atmospheric model when it is large-scale parallel.

[0010] To achieve the above object, the technical solution adopted by the present invention is: to provide a method for dividing the tasks of the physical process of an efficient atmospheric model, wherein the atmospheric model includes two processes, a power framework and a physical process, and the task division of the power framework and the physical process are independent of each other, and the task division of the power framework is performed first, and after the task division of the power framework is completed, each grid is mapped to a process;

[0011] The following steps are involved:

[0012] S1. Obtain the supernode number of each process in the atmospheric model, that is, obtain the mapping relationship between the process number and the supernode number;

[0013] S2, grouping the processes in the atmospheric model according to the supernode number information obtained in S1, and grouping the processes with the same supernode number into the same group;

[0014] S3, using the same grid division method as the dynamic framework, divide the first-level tasks of the physical process according to the super nodes, so that the dynamic framework and the physical process in the same group of processes have the same grid;

[0015] S4. Adjust the grid division between processes in the group in the physical process of the atmospheric model, redistribute the grid to the processes in the group, and allocate grids to each process in all supernodes. Specifically:

[0016] Count the number of grids N allocated to each supernode k k , and the number of processes P in supernode k k ;

[0017] Arrange the grids belonging to supernode k in ascending order according to the grid numbers, and arrange the processes belonging to supernode k in ascending order according to the process numbers;

[0018] The ii-th (0<=ii <N k ) The grid is mapped to the jjth process in supernode k, 0<=jj <P k , where % is the remainder operation, i.e. jj is ii divided by P k The remainder of .

[0019] Due to the application of the above technical solution, the present invention has the following advantages compared with the prior art:

[0020] Based on the Sunway TaihuLight supercomputer platform, the present invention proposes an efficient method for dividing the physical process tasks of the atmospheric model, which fully utilizes the communication network structure of the Sunway TaihuLight supercomputer system. It can not only make the physical process calculation tasks of the atmospheric model relatively balanced, but also reduce the communication overhead when the atmospheric model is coupled with the dynamic framework, and can significantly improve the overall performance of the atmospheric model when it is large-scale parallel. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Attached Figure 1 The present invention is a flowchart of a method for dividing tasks of physical processes of an efficient atmospheric model. DETAILED DESCRIPTION

[0022] Embodiment: The present invention provides a method for dividing tasks of physical processes in an efficient atmospheric model, wherein the atmospheric model includes two processes, a power framework and a physical process, wherein the task division of the power framework and the physical process are independent of each other, and the task division of the power framework is performed first, and after the task division of the power framework is completed, each grid is mapped to a process;

[0023] This patent studies the physical process task division method based on the dynamic framework task division results, including the following steps:

[0024] S1. Obtain the supernode number of each process in the atmospheric model, that is, obtain the mapping relationship between the process number and the supernode number;

[0025] S2, grouping the processes in the atmospheric model according to the supernode number information obtained in S1, and grouping the processes with the same supernode number into the same group;

[0026] S3. Adopt the same grid division method as the dynamic framework, and perform the first-level task division of the physical process according to the supernodes, so that the dynamic framework and the physical process have the same grid in the same group of processes;

[0027] After the dynamic framework task division, in the dynamic framework, there is a mapping relationship between the grid number and the process number: Suppose there are a total of N grids and P processes. After the dynamic framework task division, the grid numbered i (0 <= i < N) is mapped to the process numbered j (0 <= j < P); The dynamic framework task division result can ensure that any grid number will be mapped to a process number, and a grid will only be mapped to one process;

[0028] Obtain the mapping relationship between the process number and the supernode number. Suppose the supernode number where the process numbered j is located is k. The dynamic framework and the physical process use the same grid and the same process. Then, in the dynamic framework, the grid numbered i is mapped to the process numbered j, while in the physical process, the grid numbered i is mapped to the supernode numbered k, so as to ensure that the first-level task division of the physical process achieves the following effect: Any grid number will be mapped to a supernode number, and the same grid will only be mapped to one supernode number, realizing the first-level task division of the physical process by supernodes, and ensuring that the atmospheric model dynamic framework and the physical process have the same grid in the same group of processes;

[0029] S4. Adjust the grid division among the processes within the group in the atmospheric model physical process, reallocate the grids to the processes within the group, and allocate grids to each process in all supernodes. Specifically:

[0030] Count the number of grids N allocated in each supernode k k , and the number of processes P in the supernode k k ;

[0031] Arrange the grids belonging to the supernode k in a column in ascending order of the grid number, and arrange the processes belonging to the supernode k in a column in ascending order of the process number;

[0032] Map the ii-th (0 <= ii < N k ) grid in the supernode k to the jj-th process in the supernode k, 0 <= jj < P k , where % is the modulo operation, that is, jj is the remainder of ii divided by P k .

[0033] The further explanation of the above embodiment is as follows:

[0034] The present invention makes full use of the following features of the communication network structure of the Sunway TaihuLight supercomputer system: in the Sunway TaihuLight supercomputer system, a fully connected network is used within a supernode, and a 1:4 trimmed communication network is used between supernodes; using this feature, the first-level task division limits the communication requirements when the atmospheric model physical process is coupled with the power framework to the supernode, which can greatly improve the communication performance when the physical process and the power framework are coupled; and the second-level task division ensures the balance of computing tasks within the supernode when calculating the atmospheric model physical process, which can greatly improve the performance of the atmospheric model physical process; therefore, the use of an efficient atmospheric model physical process task division method can significantly improve the overall performance of the atmospheric model when it is large-scale parallel.

[0035] When the above-mentioned efficient atmospheric model physical process task division method is adopted, it can not only make the atmospheric model physical process calculation tasks relatively balanced, but also reduce the communication overhead when the atmospheric model is coupled with the dynamic framework, and can significantly improve the overall performance of the atmospheric model when it is large-scale parallel.

[0036] In order to facilitate a better understanding of the present invention, the terms used in this article are briefly explained below:

[0037] Earth System Model: English name CESM, a high-performance scientific and engineering computing application used for climate and meteorological research and prediction.

[0038] Atmospheric model: English name CAM, a component model in the Earth system model, usually the most time-consuming part of the Earth system model application software, mainly studies atmospheric movement and natural phenomena such as clouds and precipitation.

[0039] Dynamical framework: One of the components of the atmospheric model, mainly used to solve the atmospheric motion equations.

[0040] Physical process: One of the components of the atmospheric model, mainly used to solve phenomena such as rain, clouds and shortwave radiation.

[0041] Sunway TaihuLight: my country's first independently developed supercomputer platform.

[0042] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. An efficient atmospheric model physical process task division method, the atmospheric model includes two processes: a power framework and a physical process. The task division of the power framework and the physical process are independent of each other, and the task division of the power framework is performed first. After the task division of the power framework is completed, each grid is mapped to a process; It is characterized in that The following steps are involved: S1. Obtain the supernode number of each process in the atmospheric model, that is, obtain the mapping relationship between the process number and the supernode number; S2, grouping the processes in the atmospheric model according to the supernode number information obtained in S1, and grouping the processes with the same supernode number into the same group; S3, using the same grid division method as the dynamic framework, divide the first-level tasks of the physical process according to the super nodes, so that the dynamic framework and the physical process in the same group of processes have the same grid; S4. Adjust the grid division between processes in the group in the physical process of the atmospheric model, redistribute the grid to the processes in the group, and allocate grids to each process in all supernodes. Specifically: Count the number of grids N allocated to each supernode k k , and the number of processes P in supernode k k ; Arrange the grids belonging to supernode k in ascending order according to the grid numbers, and arrange the processes belonging to supernode k in ascending order according to the process numbers; Map the ii-th grid in supernode k to the jj-th process in supernode k, 0<=ii <N k , 0<=jj <P k ,jj=ii%P k , where % is the remainder operation, i.e. jj is ii divided by P k The remainder of .

Images