Unlock instant, AI-driven research and patent intelligence for your innovation.

Method and device for constructing distributed file storage framework

A technology of distributed files and construction methods, which is applied in the construction field of distributed file storage architecture, can solve problems such as duplicate files, and achieve the effect of avoiding redundant duplicate files

Active Publication Date: 2012-08-08
CHENGDU HUAWEI TECH
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the embodiments of the present invention is to provide a method and device for constructing a distributed file storage framework, which are used to solve the problem of duplicate files in the file storage system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for constructing distributed file storage framework
  • Method and device for constructing distributed file storage framework
  • Method and device for constructing distributed file storage framework

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0017] This embodiment provides a method for constructing a distributed file storage framework. Refer below figure 1 , detailing the method, which includes:

[0018] Step S101: select a file;

[0019] Step S102: performing Hash calculation on the file content of the file to obtain the file identifier of the file;

[0020] Wherein, the HASH value of the file is obtained by performing Hash calculation on the file content to obtain the file identifier of the file, for example, calculated by methods such as MD5, SHA-1, SHA-2, and SHA-256;

[0021] Step S103: Construct a distributed file storage framework for files according to the file identifier.

[0022] Such as figure 2 As shown, the distributed file storage architecture includes a super block layer (super block), an index node matching layer (IMAP, inode map), an index node layer (inode) and a data block layer (datablock). Among them such as image 3 As shown in , the inode matching layer is used for matching between th...

Embodiment 2

[0029] This embodiment further describes in detail the construction method of the distributed file storage framework according to Embodiment 1 through writing operations. see Figure 5 , the method includes:

[0030] Step S501: The client calculates the HASH value based on the content of the file to be written to the storage node, and transmits the HASH value of the file to the metadata server. The HASH value of the file can be passed through MD5, SHA-1, SHA-2, SHA- 256 and other methods to calculate;

[0031] Step S502: the metadata server obtains the HASH value of the file;

[0032] Step S503: The metadata server constructs an index node matching layer (IMAPTree) according to the HASH value of the file, and simultaneously constructs a super block layer, an index node layer, and a data block layer; wherein, the index node matching layer includes a plurality of IMAP tables, using a B+ tree To build, the key value of the IMAP table is the file HASH value, and the value is th...

Embodiment 3

[0038] This embodiment provides a device for constructing a distributed file storage framework. Such as Figure 6 As shown, the device 600 includes:

[0039] The file identifier obtaining module 601 is used for performing Hash calculation on the file content of the file to obtain the file identifier of the file; the file identifier is the HASH value of the file, which is obtained through MD5, SHA-1, SHA-2, SHA-256 and other calculations.

[0040] The framework construction module 602 is configured to construct a distributed file storage framework according to the file identification. Such as figure 2 As shown, the distributed file storage architecture includes a super block layer (super block), an index node matching layer (IMAP, inode map), an index node layer (inode) and a data block layer (data block). Among them such as image 3 As shown in , the inode matching layer is used for matching between the file identifier and the inode of the file, and the inode layer is use...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a method and a device for a distributed file storage framework. The method comprises the following steps: selecting a file; carrying out Hash calculation on the content of the file; acquiring a file identification of the file; and constructing the distributed file storage framework of the file according to the file identification. The device comprises a file identification acquisition module used for carrying out Hash calculation on the content of the file and acquiring the file identification of the file and a framework construction module used for constructing the distributed file storage framework according to the file identification. By the matching of the file identification and an index node, the invention avoids repeated saving and deletion of the same file, thereby not only ensuring the reliability of data, but also saving a substantial amount of storage space.

Description

technical field [0001] The invention relates to an information processing method, in particular to a method and device for constructing a distributed file storage framework. Background technique [0002] With the rapid development of direct attachment storage (DAS, Direct Attachment Storage), network storage (NAS, Network Attachment Storage), and storage area network (SAN, Storage Area Network) in the storage field, new technologies are constantly emerging. In recent years, distributed file systems have been vigorously developed and started to be used in storage solutions. [0003] In the process of the present invention, the inventors found that the management of the existing distributed file system is based on the file name, but due to the different operating habits of users, many different names will gradually appear in the distributed file system. However, "duplicate files" with the same content occupy a large amount of storage space and are not conducive to file manage...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
Inventor 程菊生孔晗袁远
Owner CHENGDU HUAWEI TECH