Reliable and efficient distributed file system

A technology of distributed files and active/standby mode, applied in the input/output process of data processing, instruments, input/output to record carrier, etc., can solve problems such as stretching, large memory consumption, single point failure, etc., and achieve reduction The effect of load pressure, improving reading efficiency, and solving single point failure problems

Pending Publication Date: 2019-07-16
JIANGSU ZHONGYUN TECH
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

There are also some problems in the use of HDFS, such as a single Master node, all file requests go through it, which becomes a bottleneck and leads to a single point of failure; in addition, Namenode stores the metadata of the file system in memory, and each small file Both need to occupy about 150 bytes of space, a large number of small files consume too much memory, and existing hardware is stretched to handle hundreds of millions or more small files

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Reliable and efficient distributed file system
  • Reliable and efficient distributed file system
  • Reliable and efficient distributed file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0035] see Figure 1-2 , a reliable and efficient distributed file system, including a log module, a system control module and a storage module. The log module includes log records. They are connected through the master-standby mode process. The judging condition for master-standby mode process switching is whether the master control node responds within 1 second after receiving the detection data packet. The storage module includes disk and memory cache layer. The backup node is connected, and the system control module is connected to the memory cache layer and the disk respectively. The system control module is also connected to multiple storage nodes. The system control module also includes a data transmission module. The mode is connected with a cloud memory module, and the cloud memory module includes a cloud data service platform and a cloud memory block, and the cloud memory block includes a memory process, a data transmission process and a timing process.

[0036] The...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a reliable and efficient distributed file system, belongs to the data storage technology field, the reliable and efficient distributed file system includes a log module, a system control module and a storage module, the log module comprises a log record; the system control module comprises a master control node, a standby node and a master-standby mode process. The master control node and the standby node are connected through a master-standby mode process. the storage module comprises a disk and a memory cache layer; the log records are respectively connected with themain control node and the standby node; wherein the system control module is respectively connected with the memory cache layer and the disk, and is also connected with a plurality of storage nodes, so that the problem of single point failure can be well solved, and the reading efficiency of the system on small files can be greatly improved by utilizing the memory cache layer in the system.

Description

technical field [0001] The invention relates to the technical field of data storage, and more specifically, relates to a reliable and efficient distributed file system. Background technique [0002] With the widespread application of cloud computing and big data technology, the storage of massive data, that is, the distributed file system, has also attracted attention. Among them, the reliability of the storage system and the efficiency of file reading are hot issues of research. [0003] In the prior art, "Distributed File System and Data Backup Method of Distributed File System" was applied for and published by the State Intellectual Property Office of China on January 13, 2016, with the publication number 105242988B. This patent designs a distributed file system with off-site backup function, adopts Syncserver to realize off-site data synchronization, and adopts DNS intelligent analysis to realize user data switching. [0004] This solution adopts off-site backup, which ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/182G06F3/06
CPCG06F3/061G06F3/0656G06F3/067
Inventor 戴元顺
Owner JIANGSU ZHONGYUN TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap