Distributed File Management System Based on Seismic Data Processing

Abstract

The invention discloses a distributed file management system based on seismic data processing, which includes a data read-write interface, a file management server and a communication module. The communication module provides an interface for sending and receiving messages for use by the data read-write interface and the file management server. The positive effects of the present invention are: adopting the principle of the distributed file system to automatically complete file fragmentation and management, so as to realize high-speed parallel I / O and provide simple and efficient file management. Users only need to simply call the corresponding interface to complete various operations on the data, and the file fragmentation and management are completely automatically completed by the program without manual intervention. It solves the problem of competition conflicts when multiple computing nodes access the same file on the same disk array in a cluster environment; fundamentally avoids the I / O caused by the file system's lock protection for files by making the file fragmented A problem with a significant drop in performance.

Description

technical field [0001] The invention relates to a distributed file management system based on seismic data processing. Background technique [0002] Distributed File System (Distributed File System) means that the physical storage resources managed by the file system are not necessarily directly connected to the local node, but are connected to the node through a computer network. [0003] With the continuous acceleration of human informatization, the traditional storage structure has been unable to support the explosive growth of data volume. The storage and processing of big data has become a research hotspot in computer science today. When it comes to the processing of big data, the I / O performance of the traditional file system has become a bottleneck restricting the entire system, and the problems of fault tolerance and scalability are also more severe. The emergence of the distributed file system is solving this problem, and the I / O performance, fault tolerance, and ...

AI Technical Summary

Problems solved by technology

Although this method solves the conflict of reading and writing files, it introduces many problems at the same time: first, each file needs to be artificially divided into pieces, and seismic data has its own format, so it is necessary to ensure that each piece contains a complete The smallest processing unit (track or set) cannot be segmented by simple file segmentation; secondly, due to the large number of files after segmentation, it will bring great confusion to the management of the file itself, and also bring difficulties to the writing of applications. difficulty

In order to solve these problems, a lot of manual operations are required, which not only increases the burden on file managers and developers, but also reduces the reliability of the system itself.

Images