Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Big data storage and access system and method based on distributed file system

A distributed file and big data storage technology, applied in transmission systems, electrical components, etc., can solve problems such as single point of management server, solve disk bottleneck and bandwidth bottleneck, solve single point problem of management node server, easy to dynamic extended effect

Active Publication Date: 2015-01-28
北京思特奇信息技术股份有限公司
View PDF5 Cites 34 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] The technical problem to be solved by the present invention is to provide a large data storage access system and method based on a distributed file system, which can solve the problem of single point management server in the process of large data storage access

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Big data storage and access system and method based on distributed file system
  • Big data storage and access system and method based on distributed file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0023] Embodiment 1. A large data storage and access system based on a distributed file system. Combine below figure 1 The system provided in this embodiment will be described in detail.

[0024] figure 1 Among them, a large data storage access system based on a distributed file system provided in this embodiment includes a distributed file system, wherein the distributed file system includes a metadata server 101, a storage server cluster 102, an access client 103, and a log server 104.

[0025] Wherein, the metadata server 101 is mainly used for managing metadata information in the distributed file system and managing and monitoring the storage server cluster 102 .

[0026] The storage server cluster 102 is mainly used to store the data contained in the metadata information in several data storage nodes of the storage server cluster according to predetermined rules.

[0027] Specifically, in this embodiment, the storage server cluster 102 has several management node serv...

Embodiment 2

[0038]Embodiment 2, a large data storage access method based on a distributed file system. Combine below figure 2 The method provided by this implementation is described in detail.

[0039] figure 2 , S201. Store the data contained in the metadata information in several storage nodes of the storage server cluster according to predetermined rules, and the metadata server manages and monitors the storage server cluster.

[0040] Specifically, in the prior art, the storage server cluster has only one management node server, and the data is stored in the storage server cluster. When the access client accesses the data in the storage server cluster, the data needs to flow out through the network card of the management node server. Therefore, there will be storage space and bandwidth bottlenecks of the management node server. In this embodiment, the storage server cluster has several management node servers, and each management node server manages several storage server nodes t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a big data storage and access system and method based on a distributed file system. The big data storage and access system based on the distributed file system comprises the distributed file system which comprises a metadata server, a storage server cluster, an access client and a log server, the metadata server manages and monitors metadata information in the distributed file system and the storage server cluster, the storage server cluster stores data contained in the metadata information in multiple data storage nodes of the storage server cluster, the log server backs up the metadata information in the metadata server and synchronizes operation logs of the metadata server in real time, and the access client is used for selecting standby metadata service processes in the metadata server or in the log server according to the connection state of the metadata server to store the server cluster.

Description

technical field [0001] The invention relates to the technical field of data storage access, in particular to a large data storage access system and method based on a distributed file system. Background technique [0002] When the Hadoop-based Namenode (that is, the management node) is used in a production environment, there is a single point of storage problem. The current solution to this problem is mainly to store the metadata information of the Namenode on NFS (Network File System, Network File System), and the hosts where the primary and backup processes of the Namenode are located are both NFS clients, which can access NFS respectively. Metadata information stored on. When the primary node process of Namenode exits, the metadata maintained by the primary process is stored on NFS, and the standby node process of Namenode can still access the metadata on NFS normally, so that the standby node process of Namenode completely takes over the primary node of Namenode The pro...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): H04L29/06H04L29/08
CPCH04L41/08H04L67/1097H04L67/51H04L67/561
Inventor 王少华
Owner 北京思特奇信息技术股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com