Unlock instant, AI-driven research and patent intelligence for your innovation.

A data operation method, server and storage system

A storage system and data operation technology, applied in the storage field, can solve problems such as incomplete support of Hadoop functions

Active Publication Date: 2019-09-03
HUAWEI TECH CO LTD
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, since the realization of some functions of Hadoop directly depends on HDFS, for example, HBase (Hadoop database, Hadoop database) backup and system query impala, etc., directly using the key-value storage system to replace HDFS in Hadoop will lead to incomplete support of Hadoop functions , therefore, there is no perfect solution using key-value storage system combined with HDFS in the prior art

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A data operation method, server and storage system
  • A data operation method, server and storage system
  • A data operation method, server and storage system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0036] In order to make it easier for those skilled in the art to understand the improvements made by the embodiments of the present invention to the prior art, the solutions in the prior art will be briefly introduced below.

[0037] figure 1 It is a schematic diagram of HDFS architecture. As shown in the figure, the HDFS architecture includes HDFS client, name node, data node 1, and data node 2, wherein, the client is connected to name node and data node 1 respectively, and data node 1 is connected to Data node 2 is connected. Among them, the name node runs the HDFS file system, and each data node runs a local file system.

[0038] based on figure 1 The shown HDFS architecture illustrates the data writing process: the HDFS client sends a write request message to the name node, the write request message includes file name, start address information and data length information, and the name node receives the write request message Finally, first determine whether the file ex...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

A data operation method, a server and a storage system relate to the storage field and can ensure that Hadoop uses key-value storage under the premise of complete functional support. The method comprises: the name node module receives the operation request message sent by the HDFS client, the operation request message is based on the ClientProtocol communication protocol between the name node and the HDFS client in the Hadoop platform; according to the target file included in the operation request message Determine the key key by the file name, and determine the location of the storage space of the value value according to the key, and the value is the data of the target file; obtain the target data in the storage space according to the starting address information and data length information in the operation request message Target block address information in the space; sending a response message for responding to the operation request message to the HDFS client, where the response message includes the target block address information.

Description

technical field [0001] The invention relates to the field of storage, in particular to a data operation method, a server and a storage system. Background technique [0002] In the prior art, a Hadoop-based platform is used for processing big data. Hadoop is an open source distributed computing platform, its core includes HDFS (Hadoop Distributed Files System, Hadoop Distributed File System). [0003] HDFS includes name nodes and data nodes. Name nodes are used for metadata management and processing, and data nodes are used to store data in the form of files. Namenodes and datanodes can be dedicated devices. It can also be software running on ordinary computers. Usually, a dedicated machine runs the name node software, and each other machine runs a data node software. Multiple data node software can also run on one machine. Each machine running the data node software has a local file system. HDFS is a logical file system built on multiple machine file systems, and its un...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): H04L29/08H04L29/06
CPCH04L67/1097
Inventor 刘科佑王锋
Owner HUAWEI TECH CO LTD