Method and device for processing files in distributed file system

A distributed file and file technology, applied in the field of data processing, can solve the problems of unable to support expansion, limit the expansion of distributed file system, and the bottleneck of distributed file system capacity, and achieve the effect of solving storage and expansion, and realizing expansion

Inactive Publication Date: 2012-10-03
HUAWEI SOFTWARE TECH
View PDF7 Cites 25 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Whether it is file storage or reading, it is necessary to exchange information with the metadata server, and the metadata server is a centralized management mode, and the capacity of the metadata that can be stored or read is limited, which leads to metadata The data server becomes the capacity bottleneck of the distributed file system and cannot support greater expansion
[0005] Therefore, the existing distributed file system relies on a centralized metadata server to determine the route of the data storage node, which limits the greater expansion of the distributed file system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for processing files in distributed file system
  • Method and device for processing files in distributed file system
  • Method and device for processing files in distributed file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0020] figure 1 The flowchart of the file processing method in the distributed file system provided by Embodiment 1 of the present invention, such as figure 1 As shown, the method includes:

[0021] Step 101 , the client in the distributed file system receives a file operation request from a user, and the file operation request carries at least file description information and an operation type.

[0022] The client in the distributed file system is different from the client in the traditional sense. The client in the distributed file system can be understood as the interface layer between the distributed file system and the user, and the interface layer is located in the basic management layer ( Including metadata server and data block server) and storage facility layer (storage devices such as local disks, etc.), for applications (such as user generated content (User Generated Content, referred to as: UGC), social network services (Social Networking Services, referred to as ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a method and a device for processing files in a distributed file system. The method includes: receiving a user's file operation request which at least includes file description information and an operation type; determining a meta main node used for storing metadata of the file from all nodes of a metadata server according to the file description information, and forming a distributed hash table network by all nodes of the metadata server; determining a block main node used for storing data block of the file from all nodes of a data block server according to the file description information, and forming a distributed hash table network by all nodes of the data block server; and finishing an operation between the meta main node and the block main node according to the operation type. The device comprises a receiving module, a routing determination module and a processing module. By the method and the device, expansion of the distributed file system and mass file content storage can be effectively achieved.

Description

technical field [0001] The present invention relates to data processing technology, in particular to a file processing method and device in a distributed file system (Distributed File System, DFS for short). Background technique [0002] The storage system of the distributed file system can effectively meet the storage of massive data content of Internet business. Taking Google's distributed file system (Google File System, referred to as: GFS) as an example, the GFS cluster consists of a metadata server (master node) and multiple block servers (chunk servers). There are periodic heartbeats between chunkservers to communicate with each other, and the heartbeat communicates information and status of the chunkservers. The metadata server is responsible for managing the metadata of the file system of all files. The metadata may include information such as namespace, access control information, mapping relationship between files and blocks, and the location of the current block...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 唐金根袁洪锦
Owner HUAWEI SOFTWARE TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap