Method and device for performing file layout extraction on parallel file system

A file layout and file system technology, applied in transmission systems, electrical components, etc., can solve problems such as consistency maintenance, and achieve the effect of simplifying management, facilitating performance optimization, and reducing storage

Active Publication Date: 2011-08-24
TSINGHUA UNIV
View PDF2 Cites 39 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Common cluster file systems, such as GFS (Global File System), GPFS, etc., provide file management in a symmetrical manner, that is, data and metadata are mixed and placed, and each server role is equal. Disadvantages: Need to use complex cluster distributed lock management protocol for data consistency maintenance

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for performing file layout extraction on parallel file system
  • Method and device for performing file layout extraction on parallel file system
  • Method and device for performing file layout extraction on parallel file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0068] Reference figure 1 , Shows a flow chart of a method for extracting file layout from a parallel file system according to the present invention, and the method specifically includes:

[0069] Step S101, the application layer sends a read and write request to the file system through the VFS;

[0070] It should be noted that the method described in this embodiment is a method for extracting file layout from an asymmetric file system in a shared block storage device environment. Specifically, the asymmetric parallel file system of the shared block storage device consists of three parts: a client, a metadata server, and a shared storage device; the client runs on a server in the cluster and is used to receive file system operation requests submitted by user programs , Obtain the metadata information required in the submitted request through interaction with the metadata server, and then directly access the shared block storage device according to the obtained metadata information ...

Embodiment 2

[0181] In this embodiment, the method of the present invention will be further introduced in combination with practical applications. The required hardware structure is as follows: figure 2 As shown, it is composed of client, metadata server, shared block storage device (optical disk array, iSCSI disk array, etc.), SAN network, and high-speed Ethernet. Clients, metadata servers and shared block storage devices can be interconnected by SAN network, such as optical disk arrays, etc.; they can also be interconnected by high-speed Ethernet, such as iSCSI disk arrays. The interaction between the client and the metadata server is interconnected by high-speed Ethernet, and the software structure is as image 3 As shown, it consists of two parts: metadata server and client. The metadata server is divided into a metadata management module, a metadata file system, a block device driver module, and a communication module; the client is divided into a parallel file system client module, a ...

Embodiment 3

[0296] Reference Image 6 , Shows a structure diagram of an apparatus for extracting file layout from a parallel file system according to the present invention, and the apparatus specifically includes:

[0297] The first processing module 601 is used to complete the application layer sending read and write requests to the file system through the VFS;

[0298] The determination module 602 is used to complete the client side checking whether the file layout of the read and write request exists and is valid:

[0299] The second processing module 603 is configured to use a cached and effective file layout when the determination module determines yes;

[0300] The third processing module 604 is used to request the file layout to be obtained from a metadata server (MetaData Server, MDS) when the determination module determines that it is no:

[0301] The fourth processing module 605 is used for parsing the parameters after completing the MDS receiving the file layout obtaining request, and o...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a method and a device for performing file layout extraction on a parallel file system. The management and storage of data and meta data are separated so as to avoid the interference of data access to meta data access performance, simplify the management of meta data consistency and bring convenience to the performance optimization of the meta data. A meta data server adopts special meta data storage equipment, and establishes a special meta data file system. A parallel file system directory tree, file meta data and the like are stored in the meta data file system in form of file. The meta data file system organizes files in form of B+ tree, and rapid file reading performance is achieved.

Description

Technical field [0001] The present invention relates to the technical field of network file management, in particular to a method and device for extracting file layout from a parallel file system. Background technique [0002] In the field of high-performance computing, the demand for concurrent read and write operations by multiple servers has become increasingly apparent. Shared block storage devices represented by Storage Area Network (SAN) provide concurrent access capabilities of storage devices, that is, multiple front-end server nodes can directly access block devices through channels such as Fibre Channel and iSCSI. Therefore, a parallel file system is required to manage shared block storage devices to provide concurrent file access capabilities of the front-end server. [0003] Traditional network file systems, such as NFS, Samba, etc., provide access to multiple servers by exporting local file systems. In this way, data transmission still needs to be accessed through th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L29/08
Inventor 舒继武陆游游易乐天
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap