Data arrangement method matching high performance computer structure

Abstract

The invention discloses a data arrangement method matching a high performance computer structure. The method comprises the following steps that data identity information marking is completed accordingto input data user identity information; an authority structure grid and the physical quantity on the authority structure grid are determined according to data attribute recognition results, and theauthority structure grid is modeled to be a grid slice hierarchical structure formed by multiple grid layers; the grid layers are divided into multiple grid areas which are not mutually overlapped, aunique logical number is given to each grid area, each grid area is mapped to the unique server compute node, and the grid areas are distributed and stored in the memory space of corresponding serves;data arrangement locating and encryption processing are completed. Accordingly, data locality of storage and memory access of the structure grid and the physical quantity can be greatly improved, floating-point performance optimization developed by application software for the high performance computer complex architecture is supported, and the computing efficiency is greatly improved.

Description

technical field [0001] The invention relates to the field of data processing, in particular to a data arrangement method matching the structure of a high-performance computer. Background technique [0002] At present, the peak performance of high-performance computers has exceeded petaflops per second, and the architecture is becoming more and more complex, showing the typical characteristics of "six-level nested parallelism" and "heterogeneous many-core acceleration". Inter-distributed memory (DM)—distributed shared memory (DSM) among multiple CPUs in a node—symmetrical multiprocessing shared memory (SMP) among multi-cores in a CPU—multi-level cache (Cache) in a CPU core—multifunctional components in a CPU core Instruction-level parallelism (ILP) - long vector acceleration in the CPU core (SIMD)", the latter refers to "heterogeneous many-core acceleration", that is, each node is equipped with a graphics acceleration microprocessor GPU or a many-core coprocessor MIC, in Red...

AI Technical Summary

Problems solved by technology

However, this multi-dimensional array data management method and the corresponding two-level nested parallel computing are difficult to adapt to the complex architecture of "six-level nested parallelism" and "heterogeneous many-core acceleration". The positioning of the location can easily lead to confusion in data storage, and at the same time, it is easy to cause conflicts and duplication between data, which will undoubtedly increase the workload of data sorting, and also cause errors in the system when querying data, and all There is no traceability of the data