File quick reading and writing method under distributed environment

A distributed environment and distributed file technology, applied in the field of network communication, can solve the problems of large memory occupation of metadata nodes and low file writing efficiency, and achieve the effect of reducing memory usage, saving memory and increasing the number of files

Inactive Publication Date: 2013-05-08
HUAZHONG UNIV OF SCI & TECH
View PDF5 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] Aiming at the defects of the prior art, the purpose of the present invention is to provide a method for fast writing of files in a distributed environment, aiming at solving the problems in the existing methods that the metadata nodes occupy a large amount of memory and the writing efficiency of a large number of files is low

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • File quick reading and writing method under distributed environment
  • File quick reading and writing method under distributed environment
  • File quick reading and writing method under distributed environment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0059] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

[0060] At first the technical terms in the present invention are explained and defined:

[0061] Metadata node: save the metadata of the distributed file system (the namespace of the file system, the mapping of file name -> data block, and the mapping of data block -> data node).

[0062] Data node: stores the actual file data (generally stored in the form of data blocks). It accepts block operation commands from metadata nodes through heartbeat.

[0063] Client node: connect to metadata nodes for file information query, connect metadata nodes and data nodes for actual file transfer.

[0064] Th...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a file quick reading and writing method under a distributed environment. The method comprises that a client node sends a read file request to a metadata node, the client node judges whether the client node is kept to be connected with a data node which is connected with a last read file of the client node in a distributed file system, if not, the metadata node inquires whether the file exists in a data region of the file according to information in an index region, if not, the metadata node inquires a data node which stores the file according to primary index information, connection between the client node and the data node is built, the data node inquires a data block where the file is located according to secondary index information, the file is obtained through the secondary index information, the file is transmitted to the client node, and the client node receives the data and keeps connection with the data node. The method can solve the problems that in the prior art, an internal storage occupied by the metadata node is large, and writing efficiency of large amount of files is low.

Description

technical field [0001] The invention belongs to the field of network communication, and more specifically relates to a method for fast reading and writing of files in a distributed environment. Background technique [0002] With the rapid development of science and technology and the Internet, storage systems need to store massive amounts of data, respond to high-concurrent user access, and provide highly reliable and highly available services. Traditional stand-alone systems can no longer meet these requirements, and distributed file systems can quickly Good to meet these needs. In practical applications (personal applications, web applications, scientific computing, etc.), massive file information will be generated. How to efficiently store and access massive files in a distributed environment is still a problem and challenge. [0003] The current mainstream distributed file systems include google GFS, HDFS, Lustre, Ceph, etc. The architecture and basic principles of the...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 郑然金海章勤姚传威冯晓文
Owner HUAZHONG UNIV OF SCI & TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap