Distributed transaction processing method on basis of shared storage pool

Abstract

The invention relates to a distributed transaction processing method on the basis of a shared storage pool. The distributed transaction processing method includes mapping all directory paths of a distributed file system to metadata servers on the basis of Hash; assigning a coordinator and a plurality of participators in a metadata server set with requirements on participating in distributed transaction; enabling the metadata servers to assess distributed transaction operation which need to be completed by the metadata servers, reporting 'termination' to the coordinator if the metadata servers cannot complete the distributed transaction operation, and reporting 'commitment' to the coordinator and locking resources if the metadata servers can complete the distributed transaction operation; enabling the coordinator to write the 'commitment' or the 'termination' into a global log of the shared storage pool according to report results of the participators and informing the participators; enabling the participators to execute the distributed transaction operation or cancel the locked resources; enabling the certain participator or the coordinator, which first checks a state of the distributed transaction, in the shared storage pool to set the distributed transaction into a 'termination' state when the coordinator or the participators fails. The distributed transaction processing method has the advantage that the transaction of metadata service in the distributed file system can be processed quickly in an extensible environment.

Description

technical field [0001] The invention relates to the technical field of distributed file system classification, in particular to a distributed transaction processing method based on a shared storage pool. Background technique [0002] Distributed File System (Distributed File System) means that the physical storage resources managed by the file system are not necessarily directly connected to the local node, but are connected to the node through a computer network. The design of the distributed file system is based on the client / server model. [0003] In recent years, with the emergence of more and more Web2.0 Internet services, data is gradually formed and stored centered on users. Therefore, the scale of data and the number of files have increased dramatically. Existing distributed file systems are either oriented to traditional application scenarios with high I / O bandwidth or are limited by the complexity of technical implementation, so they mostly use the structure of a ...

AI Technical Summary

Problems solved by technology

Due to the bottleneck of network, computing and memory resources, the structure of a single metadata server cannot well adapt to the growth of the storage system scale. At the same time, a single metadata server will also become a single point of failure (SPOF) of the system

[0004] In order to overcome these scalability problems, some recent studies have adopted multiple metadata servers to provide stronger metadata services, such as Ceph, PVFS, HDFS2, etc., but these systems either use complex and slow two-phase commit (Two-phase Phase Commit) to implement distributed metadata operations (such as Ceph), or disallow distributed transactions from metadata division (such as PVFS and HDFS2)

The slow two-phase commit method will affect the metadata access performance of the distributed file system, and the method that does not allow distributed transactions in terms of metadata division will reduce the flexibility, adaptability and scalability of metadata services, making Efficiency of metadata services is limited

In addition, some studies have adopted an improved three-phase commit method. Although it eliminates the above-mentioned waiting problem, it has more protocol communications, which reduces the performance of the system under normal execution conditions.

Images