High-energy physics IO intensive computable storage system and service method

Abstract

The invention discloses a high-energy physical IO (input / output) intensive computable storage system and a service method. The method comprises the following steps of: 1), analyzing and filtering a file access request sent by a client by utilizing a metadata analysis module, forwarding the modified file access request to a metadata service process so that the file access request can correctly obtain file information, and transmitting a physical address of a file and file size information back to the client; using the client for redirecting and sending a file access request to a corresponding storage service process according to returned information; 2), enabling a storage service analysis module to call a corresponding method in a storage calculation function module to process data according to a file access request transmitted after redirection from a client; and 3), enabling a storage calculation function module to generate a file after processing the data and send the file to a storage service analysis module; and then enabling the storage service analysis module to call the metadata registration module to register the file back to a distributed storage system.

Description

technical field [0001] The invention belongs to the fields of mass data storage and computable storage, and relates to a high-energy physical IO-intensive computable storage system and service method. Background technique [0002] With the development of experimental equipment and detector technology, the energy resolution of detectors is getting higher and higher, and the amount of experimental data is also increasing sharply from TB level to PB level or even EB level, which poses great challenges to computing power and network transmission capacity. The huge requirements have promoted technological innovation and development in the fields of high-performance computing, mass storage systems, and network transmission. On the other hand, the development of technologies such as chip design and manufacturing, storage and network have also allowed high-energy physics to develop towards higher energy and higher detector resolution. [0003] The current high-energy physics comput...

AI Technical Summary

Problems solved by technology

[0010] In view of the high pressure of network transmission in the process of IO-intensive data analysis and processing in the field of high-energy physics, the performance and stability of the storage system and computing system are limited by the network bandwidth, and the CPU computing capacity of the storage device is excessive, resulting in waste of computing resources, etc., the present invention The purpose is to provide a computing storage system and service method for high-energy physical IO-intensive tasks

Images