Unlock instant, AI-driven research and patent intelligence for your innovation.

Method for carrying out data processing by distributed file system and distributed file system

A distributed file and file technology, applied in the field of data processing, can solve problems such as long time, and achieve the effect of shortening the recovery time of downtime

Active Publication Date: 2014-02-12
TENCENT TECH (SHENZHEN) CO LTD +1
View PDF3 Cites 21 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] However, since a single master node corresponds to multiple data nodes, when a downtime occurs, the master node needs to obtain information from each data node. This process is a one-to-many information collection process, which takes a long time

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method for carrying out data processing by distributed file system and distributed file system
  • Method for carrying out data processing by distributed file system and distributed file system
  • Method for carrying out data processing by distributed file system and distributed file system

Examples

Experimental program
Comparison scheme
Effect test

example

[0054] see Figure 5 , is an example of a flow chart of data uploading based on a distributed file system in the present invention, which includes the following steps:

[0055] Step 501, the master node receives a data upload request including a file path sent by a client.

[0056] Step 502, the master node determines the corresponding file ID for the file path, determines the meta server ID corresponding to the file ID, and feeds back the determined file ID and meta server ID to the client.

[0057] Data upload can be divided into new method or append method. The new method is to create a new file ID for the file path, and the append method is to add data under the original file ID corresponding to the file path. It is also possible to carry the upload method in the data upload request, specifically:

[0058] If the upload mode is to cover, then the master node determines the corresponding file ID for the file path, including: the master node creates a new file ID for the f...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method for carrying out data processing by a distributed file system and the distributed file system. The system comprises node servers and a plurality of meta-information servers; each meta-information server is used for sending a data block information acquisition request comprising a meta server ID (Identity) to each node server, receiving a file ID, a node address and data block attribute information from each node server, storing a second mapping relation between each file ID and the corresponding node address in a memory, and storing the data block attribute information in the corresponding node addresses in the memory after downtime restart; each node server is used for receiving the data block information acquisition request from each meta-information server, determining whether a corresponding file is stored according to the corresponding meta server ID and feeding back the file ID, the node address and the data block attribute information of the corresponding file to the corresponding meta-information server if the corresponding file is stored. According to the scheme adopted by the invention, recovery time after downtime restart can be shortened.

Description

technical field [0001] The invention relates to data processing technology, in particular to a data processing method of a distributed file system and the distributed file system. Background technique [0002] see figure 1 , is a schematic structural diagram of a distributed file system (DFS, Distributed File System) in the prior art, and the system includes a client, multiple data nodes, and a single master node. [0003] The master node stores the first mapping relationship between the file path and the file identifier (ID, IDentity) on the local hard disk, and stores the file attribute information corresponding to the file ID; and stores the second mapping between the file ID and the node address in the memory relationship, and store data block attribute information in the memory corresponding to the node address. The file path is the logical path of a file displayed to the user; actually, each file is divided into multiple data blocks, which are stored on multiple data...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/182
Inventor 李锐伍海君朱会灿邓大付邹永强董乘宇阙太富王磊杨绍鹏张书鑫赵大勇刘畅陈晓东张银锋
Owner TENCENT TECH (SHENZHEN) CO LTD
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com