Distributed file management system based on seismic data processing

Abstract

The invention discloses a distributed file management system based on seismic data processing, which includes data read-write interfaces, a file management server and communication modules, wherein the communication modules are provided with interfaces for information receiving and transmission, which are utilized by the data read-write interfaces and the file management server. The invention has the benefits as follows: the distributed file system principle is adopted to automatically slice and manage files, so as to realize the high-speed paralleling I / O and provide simple and efficient file management; users only simply call the corresponding interface to realize operation for data, and files can be sliced and managed completely and automatically through programs without manual intervention; the problem of competition conflict that a plurality of computation nodes access the same file on a unified magnetic disc array under the cluster environment can be solved; and the method for slicing the files fundamentally solves the problem of sharp performance reduction of I / O caused by the file locking protection of a file system.

Description

technical field [0001] The invention relates to a distributed file management system based on seismic data processing. Background technique [0002] Distributed File System (Distributed File System) means that the physical storage resources managed by the file system are not necessarily directly connected to the local node, but are connected to the node through a computer network. [0003] With the continuous acceleration of human informatization, the traditional storage structure has been unable to support the explosive growth of data volume. The storage and processing of big data has become a research hotspot in computer science today. When it comes to the processing of big data, the I / O performance of the traditional file system has become a bottleneck restricting the entire system, and the problems of fault tolerance and scalability are also more severe. The emergence of the distributed file system is solving this problem, and the I / O performance, fault tolerance, and ...

AI Technical Summary

Problems solved by technology

Although this method solves the conflict of reading and writing files, it introduces many problems at the same time: first, each file needs to be artificially divided into pieces, and seismic data has its own format, so it is necessary to ensure that each piece contains a complete The smallest processing unit (track or set) cannot be segmented by simple file segmentation; secondly, due to the large number of files after segmentation, it will bring great confusion to the management of the file itself, and also bring difficulties to the writing of applications. difficulty

In order to solve these problems, a lot of manual operations are required, which not only increases the burden on file managers and developers, but also reduces the reliability of the system itself.

Images