Grid data distributed storage service system, method, device, equipment and medium

Abstract

The invention provides a grid data distributed storage service method, device and equipment and a medium, and the method comprises the steps: based on a parallel subdivision principle, storing grid data in a memory of each process in a dispersed manner, and constructing distributed storage of a grid in a group of processes; according to the source parallel subdivision in the existing distributed storage on the grid, redistributing the grid data, and constructing the distributed storage of the target parallel subdivision corresponding to the source parallel subdivision on the grid; constructing a distributed storage service model according to the distributed storage of the grid in a group of processes and the distributed storage of the target parallel subdivision; according to the distributed storage service model, executing a service function; the operation memory of the grid data in one process can be effectively reduced, the computing processing capacity of a computer is improved, and the performance bottleneck of storing large-scale grids in development and application of numerical programs such as a coupling mode is solved.

Description

technical field [0001] The present invention relates to the technical field of computer virtual storage technology, in particular to a grid distributed storage service system, method, device, equipment and medium. Background technique [0002] Numerical programs for simulation calculations usually need to discretely represent the simulated object as a grid with a certain dimension and resolution and the data on it. For example, the certain dimension can be one-dimensional, two-dimensional, three-dimensional, four-dimensional, or even More dimensions, the data includes the grid data representing information such as the coordinates of each grid point, and the data used to represent the properties of the simulated object on each grid point; then perform numerical integration to solve the corresponding mathematical model on the grid. Numerical programs usually have a large amount of calculation, and it is necessary to use MPI (message passing interface, message passing interface...

AI Technical Summary

Problems solved by technology

Although this global storage method greatly simplifies the program realization of functions such as online calculation of interpolation weights in the coupler, acquisition of information on required grid points, and output of grid information where coupling variables are located; it seriously increases memory requirements, especially When the model resolution is very high (a large number of grid points), it will cause the entire coupled model to run in parallel

For example, when the model resolution reaches 3 kilometers around the world, there will be more than 50 million grid points, and the memory usage for storing the entire horizontal grid information in one process can exceed 4GB, and there are usually dozens of processor cores but only a few memory points. The capacity usually does not exceed 64GB, which makes a computing node of the current high-performance computer can only run at most a dozen processes; when trying to run more processes, the computer system will forcibly interrupt the operation of the mode due to insufficient memory capacity, so it cannot Make full use of the computing power of high-performance computers

Images