Data sharing method for Lustre storage system

A technology for data sharing and storage systems, applied in electrical digital data processing, special data processing applications, instruments, etc., can solve problems such as the inability of Lustre storage systems to provide, improve safety and reliability, save time, and make small changes Effect

Inactive Publication Date: 2015-12-09
INST OF HIGH ENERGY PHYSICS CHINESE ACAD OF SCI
View PDF3 Cites 10 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Aiming at the problem that the Luster storage system cannot provide wide area network access in the prior art, the present invention provides an access interface between Luster and dCache, the dCache system can realize data transmission of the wide area network, and realize remote remote sharing of Luster data through dCache, thereby Users of the WAN can access the local Luster file system through the dCache system, and the massive data stored on the local Luster system can be shared with the dCache system through the present invention and provided to the WAN users.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data sharing method for Lustre storage system
  • Data sharing method for Lustre storage system
  • Data sharing method for Lustre storage system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0044] Luster file: / lustre / director / file

[0045] dCache directory: / dcache (this is a virtual directory, which is the directory structure information generated by the dcache system through the nfs protocol, which can visually view and access data)

[0046] dCache cache directory: / storage / pool (the disk directory where dCache stores real files)

[0047] 1. Create a lustre directory in the dcache system: mkdir / dcache / lustre.

[0048] 2. Set the tag (directory identifier) ​​information of the directory under the / dcache / lustre directory, and the tag information specifies the cache pool of dcache. The cache pool specifies the program (dcache-lustre interface program) that calls the external storage.

[0049] 3. Create a directory under the / dcache / lustre directory: mkdir / dcache / lustre / director.

[0050] 4. Read the lustre file information. The file information read by the linux command stat includes the file modification time ctime, access time atime, modification time mtim...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data sharing method for a Lustre storage system. The method comprises: 1) adding a dCache system to a server where the lustre system is, and according to the difference between the dCache system and the lustre system, extending a meta-database of the dCache system; 2) reading metadata of the lustre system by the server, converting the metadata to a metadata structure identifiable for the dCache system, and then storing the metadata structure in a corresponding extension field of the meta-database of the dCache system; and 3) when receiving a user data transmission request in a wide area network by the dCache system, according to the request, querying matching metadata from the meta-database of the dCache system, then reading the data from the lustre system and caching the data to the dCache system, and returning a query result to the user. According to the method, the data security and reliability are improved and remote sharing of other local storage systems is realized.

Description

technical field [0001] The invention belongs to the field of network transmission of computing systems, and in particular relates to a data sharing method of a Luster storage system, which can be used for a transmission system and a transmission method of large batches of data in high-energy physics experiments. Background technique [0002] Luster is an open source project initiated by Dr. Peter Braam in 1999. Its purpose is to design a next-generation high-performance, high-scalability, and high-availability object-based storage cluster file system. Luster also adopts the technology of separating metadata and storage data, which can fully separate computing and storage resources, so that client computers can focus on user and application requests; storage servers and metadata servers focus on reading, transmitting and writing data . Luster uses Sandia's open Portals network transmission protocol and supports multiple networks, such as GigE, QSWElan, Myrinet, InfiniBand, T...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06G06F17/30
Inventor 闫晓飞郑伟
Owner INST OF HIGH ENERGY PHYSICS CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap