A lightweight approach to autonomous block management for data-intensive file systems

A data-intensive, file system technology, applied in the direction of electrical digital data processing, data processing input/output process, instruments, etc., can solve the problem that the management method is not suitable for data-intensive file system management, increase address and other metadata maintenance cost, reduce the scalability of the master node, etc., to achieve the effect of improving recoverability and scalability, reducing storage space overhead, and improving scalability
CN106293537BActive Publication Date: 2019-11-12SHANGHAI MARITIME UNIVERSITY
4 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN Β· China
Patent Type
Patents(China)
Current Assignee / Owner
SHANGHAI MARITIME UNIVERSITY
Publication Date
2019-11-12

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses an autonomous block management method for a lightweight data-intensive file system. The method is characterized in that the mapping of a data block to a data storage node, quick search of the data block in the data storage node, fast recovery of the data block when the data storage node is in failure, fast redistribution of the data block in the process of newly adding the data storage node and the like are realized by means of ISD (Intersected Shifted Declustering), so that a main node is only in charge of the storage and maintenance of file namespace in the data-intensive file system, but the storage and maintenance of mapping relationship information between the data block and the data storage node, the replacement of the data block when the data storage node is in failure, the redistribution of the data block during the process of adding a new data storage node and the like are all autonomously completed by the data storage node. The method disclosed by the invention has the advantages that memory space of the main node in the data-intensive file system is saved, the processing capability of the main node is improved, and the data block management efficiency of the data-intensive file system under a big-data environment can be dramatically improved.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to computer security technology, in particular to a light-weight autonomous block management method of a data-intensive file system. Background technique

[0002] Data-intensive file system DiFS, such as Google File System GFS, Hadoop Distributed File System HDFS, etc., has become the main file system for big data storage management. The current data-intensive file system DiFS adopts a master-slave architecture. The master node (metadata server) manages all metadata, and the slave node (data storage node) is only responsible for data storage. In order to maintain high availability, these storage systems usually divide data files into blocks of fixed size, each data block usually has 3 copies, and distribute them to different data storage nodes of the cluster. The master node must record the addresses of hundreds or thousands of data storage nodes, as well as record the mapping information from the data blocks of all data files to...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More