Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Distributed file system and data access method thereof

A technology of distributed file and file access, applied in the field of distributed file system and its data access, can solve the problems of burden, heavy NameNode, taking a long time, etc., to achieve the effect of reducing delay, reducing pressure, and speeding up

Inactive Publication Date: 2014-03-19
BEIJING UNIV OF POSTS & TELECOMM
View PDF7 Cites 58 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In large-scale applications, there may be thousands of requests at the same time, and it will take a long time to process all these requests, which will inevitably cause a very heavy burden on the NameNode
At the same time, when a single NameNode cluster has a large number of task requests, it cannot respond to load balancing requests in a timely manner. Due to task allocation and other related work, it will also affect the performance of the distributed file system.
[0010] (2) Expansion performance issues
A single NameNode structure will inevitably cause the bottleneck of HDFS system expansion

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed file system and data access method thereof
  • Distributed file system and data access method thereof
  • Distributed file system and data access method thereof

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0042] The present invention optimizes the above-mentioned problems existing in HDFS, enhances the stability of the distributed file system, and greatly improves the performance of the system. In addition, the whole system has better scalability, which is more conducive to the distributed file system. Deployment and application of the file system.

[0043] figure 1 It is a structural diagram of the distributed file system of the present invention.

[0044] Such as figure 1 As shown, the system includes a core control node and multiple sub-clusters, each sub-cluster includes a sub-cluster control node and multiple data nodes, where:

[0045] The core control node is configured to receive a file access request including a file name from the client, parse the file...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention brings forward a distributed file system and a data access method thereof. The system comprises a core control node and multiple sub-clusters. Each sub-cluster comprises a sub-cluster control node and multiple data nodes. The core control node is used for receiving file access requests comprising file names from a client, parsing the file names to determine sub-clusters which files belong to, and transmitting the file access requests to the sub-cluster control nodes in the sub-clusters. The sub-cluster control nodes are used for scheduling the distribution of stored resources of the data nodes in the sub-clusters which the sub-cluster control nodes belong to, establishing indexes to record correlations between the file names of the files stored in the data nodes of the sub-clusters which the sub-cluster control nodes belong to and the data nodes of the files, based on the file access requests transmitted by the core control node and the correlations, determining data nodes related with the file access requests, and informing the client of the identification of the determined data nodes through the forwarding of the core control node.

Description

technical field [0001] The invention belongs to the technical field of distributed storage, in particular to a distributed file system and a data access method thereof. Background technique [0002] Distributed File System (Distributed File System) means that the physical storage resources managed by the file system are not necessarily directly connected to the local node, but are connected to the node through a computer network. The existing distributed file system mainly adopts the client / server mode, and the manager of the file system and the stored data are scattered on different nodes and interconnected through the network. The distributed file system can provide services for multiple clients, allow multiple storage servers to share a global namespace, and realize intelligent distribution of I / O and data to improve system performance. Currently widely used distributed file systems mainly include GFS and Hadoop Distributed File System (HDFS). [0003] GFS is a scalable...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): H04L29/08
Inventor 林昭文陈立南赵彬郑楠马严吴军黄小红赵钦
Owner BEIJING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com