Unlock instant, AI-driven research and patent intelligence for your innovation.

A big data storage access system and method based on a distributed file system

A distributed file and big data storage technology, applied in the transmission system, electrical components, etc., can solve the problem of single point of management server, solve the problem of disk bottleneck and bandwidth bottleneck, solve the problem of single point of management node server, easy to dynamic extended effect

Active Publication Date: 2018-06-22
北京思特奇信息技术股份有限公司
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] The technical problem to be solved by the present invention is to provide a large data storage access system and method based on a distributed file system, which can solve the problem of single point management server in the process of large data storage access

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A big data storage access system and method based on a distributed file system
  • A big data storage access system and method based on a distributed file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0023] Embodiment 1. A large data storage and access system based on a distributed file system. Combine below figure 1 The system provided in this embodiment will be described in detail.

[0024] figure 1 Among them, a large data storage access system based on a distributed file system provided in this embodiment includes a distributed file system, wherein the distributed file system includes a metadata server 101, a storage server cluster 102, an access client 103, and a log server 104.

[0025] Wherein, the metadata server 101 is mainly used for managing metadata information in the distributed file system and managing and monitoring the storage server cluster 102 .

[0026] The storage server cluster 102 is mainly used to store the data contained in the metadata information in several data storage nodes of the storage server cluster according to predetermined rules.

[0027] Specifically, in this embodiment, the storage server cluster 102 has several management node serv...

Embodiment 2

[0038]Embodiment 2, a large data storage access method based on a distributed file system. Combine below figure 2 The method provided by this implementation is described in detail.

[0039] figure 2 , S201. Store the data contained in the metadata information in several storage nodes of the storage server cluster according to predetermined rules, and the metadata server manages and monitors the storage server cluster.

[0040] Specifically, in the prior art, the storage server cluster has only one management node server, and the data is stored in the storage server cluster. When the access client accesses the data in the storage server cluster, the data needs to flow out through the network card of the management node server. Therefore, there will be storage space and bandwidth bottlenecks of the management node server. In this embodiment, the storage server cluster has several management node servers, and each management node server manages several storage server nodes t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a large data storage access system and method based on a distributed file system, wherein the large data storage access system based on a distributed file system includes a distributed file system, and the distributed file system includes a metadata server, Storage server cluster, access client and log server; the metadata server manages and monitors the metadata information in the distributed file system and the storage server cluster; the storage server cluster stores the data contained in the metadata information Stored in several data storage nodes of the storage server cluster; the log server backs up the metadata information in the metadata server and synchronizes the operation log of the metadata server in real time; the access client is used to connection status, choose to access the storage server cluster through the standby metadata service process in the metadata server or log server.

Description

technical field [0001] The invention relates to the technical field of data storage access, in particular to a large data storage access system and method based on a distributed file system. Background technique [0002] When the Hadoop-based Namenode (that is, the management node) is used in a production environment, there is a single point of storage problem. The current solution to this problem is mainly to store the metadata information of the Namenode on NFS (Network File System, Network File System), and the hosts where the primary and backup processes of the Namenode are located are both NFS clients, which can access NFS respectively. Metadata information stored on. When the primary node process of Namenode exits, the metadata maintained by the primary process is stored on NFS, and the standby node process of Namenode can still access the metadata on NFS normally, so that the standby node process of Namenode completely takes over the primary node of Namenode The pro...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/06H04L29/08
CPCH04L41/08H04L67/1097H04L67/51H04L67/561
Inventor 王少华
Owner 北京思特奇信息技术股份有限公司