Rapid data restoration method for distributed file system (DFS)

A distributed file and data technology, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve problems such as high overhead, limited usability, reliability, and difficulty in online expansion

Active Publication Date: 2013-03-13
SUGON INFORMATION IND
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] As the most traditional storage method, the direct connection system has the advantages of low latency, exclusiveness, and complete control, but it has the following disadvantages: 1) The scalability is limited, and it is difficult to expand online; 2) It occupies the host CPU , memory and other system resource overhead; 3) limited availability and reliability
If the information is not recorded during the normal operation of the system, when a fault occurs, all index nodes (inodes) in the system need to be scanned. In a distributed file system, this is intolerable; and if the file is created This information is recorded on the critical path. In order to prevent the power failure from losing some objects, synchronous recording is required. When a file involves multiple disks, this is intolerable, even if all the disks involved in the file are first recorded into the temporary file, and then process the temporary file asynchronously, a synchronous operation that is still expensive compared to other memory operations on the critical path

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Rapid data restoration method for distributed file system (DFS)
  • Rapid data restoration method for distributed file system (DFS)
  • Rapid data restoration method for distributed file system (DFS)

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0032] The specific implementation manner of the present invention will be described below with reference to the explanatory diagrams.

[0033] figure 1 It is a schematic diagram of the system structure of the parallel storage system, which mainly includes modules such as metadata, data, and client. Among them, the metadata server (mds) adopts a multi-data server architecture. The mds servers are used in groups, and the servers in the same group are copies of each other. module. The implementation system includes multiple data servers (ds), and file data storage provides fragmentation. In order to improve data reliability, a copy mechanism is introduced, and different copies of file objects are placed on different disks.

[0034] obj2disk is stored on the mds server, using local double-writing. In order to achieve simplicity, avoid communication overhead and synchronization between copies, obj2disk on the mds server only saves the files created by this mds, and does not broa...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a rapid data restoration method for a parallel file system, comprising the following steps: introducing the concept of a disc object file which is used for recording objects stored in each disc of a data server in the normal running process of the system, and storing the disc object files in a distributed mode for a multivariate data server to reduce communication and improving the concurrency in data restoration; performing asynchronous writing on the disc object file to minimize the influence on the critical path of a parallel file system; and brushing back the disc object file to the timing which is arranged on an inode and brushed back so as to make full use of the reliability mechanism of metadata.

Description

technical field [0001] The invention relates to data restoration of a distributed parallel file system, in particular to a method for fast data restoration of a distributed file system. Background technique [0002] With the development of storage system architecture, there are mainly the following types of disk storage systems: direct-attached systems; storage area networks (SANs); network-attached storage (NAS) and distributed cluster storage systems. [0003] As the most traditional storage method, the direct connection system has the advantages of low latency, exclusiveness, and complete control, but it has the following disadvantages: 1) The scalability is limited, and it is difficult to expand online; 2) It occupies the host CPU , memory and other system resource overhead; 3) availability and reliability are limited. With the rapid growth of the amount of data that needs to be stored, its shortcomings have become more and more prominent, and it is difficult to meet th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F17/30
Inventor 马照云苗艳超王勇杨浩付根希
Owner SUGON INFORMATION IND
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap