Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Storage method and device of distributed file system

A technology of distributed files and storage nodes, applied in the field of data processing, can solve problems such as difficulty in meeting the reliability of file storage, and achieve the effects of reducing data migration rate, balancing capacity, and increasing cluster stability

Active Publication Date: 2013-06-12
BEIJING QIHOO TECH CO LTD
View PDF5 Cites 45 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] However, the above-mentioned block allocation strategy only considers the reliability of data and the effect of data copy saving network bandwidth, and it is difficult to meet the subsequent use of Raid technology to achieve file storage reliability.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Storage method and device of distributed file system
  • Storage method and device of distributed file system
  • Storage method and device of distributed file system

Examples

Experimental program
Comparison scheme
Effect test

example 1

[0085] If there are 4 storage nodes DataNodes in the cluster, they are A1, A2, B, and C, among them, A1 and A2 are on the same rack, B and C are independent racks, and the block of the same band already exists on A2 ;

[0086] Client A1 triggers a write request to allocate blocks, the first copy is allocated on the local storage node A1, and the second copy can give priority to A2, but A2 already has a block in the stripe, so it can be randomly selected in the whole cluster B, the last copy can randomly select C in the whole cluster.

example 2

[0088] If there are 10 DataNodes in the cluster: A, B, C, D, E, F, G, H, I, J, K, L; if a file includes two blocks, the stripe length is 2; the client machine is A

[0089] Select three copies of the first Block:

[0090] First Copy: Localization Strategies, Option A

[0091] The second copy: A has been selected in the stripe, and a machine other than A is randomly selected, assuming that B is selected;

[0092] The third copy: randomly selected, C was chosen;

[0093] Select three copies of the second Block:

[0094] First Copy: Localization Strategies, Option A

[0095] Second copy: (A, B, C) in the stripe has been selected, randomly select machines other than these machines, assuming D is selected

[0096] Third copy: random selection, B was chosen

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a storage method and a storage device of a distributed file system. The method comprises the steps of receiving file information sent by a client side; determining a target storage node of each data block, wherein each target storage node comprises a first storage node, a second storage node and a third storage node; each first storage node is a storage node at which the client side is located or a storage node selected at random from a cluster; each second storage node is a storage node which is not allocated to other data blocks on a stripe where the data block is located or a storage node selected at random from the cluster; and the third storage node is a storage node selected at random from the cluster; and writing the data blocks into the first storage nodes, the second storage nodes and the third storage nodes in order. With the adoption of the storage method and the storage device of the distributed file system, in the case of considering the follow-up use of a Raid technique for the realization of a demand on file storage reliability, the capacity of each node in the cluster can be enabled to be more balanced, and the cluster stability is improved.

Description

technical field [0001] The present invention relates to the technical field of data processing, in particular to a storage method for a distributed file system and a storage device for a distributed file system. Background technique [0002] With the rapid development of information systems, massive amounts of information need to be reliably stored and quickly accessed by a large number of users. Traditional storage solutions have become more and more difficult to adapt to the rapid development of information system business in recent years in terms of architecture, and have become a bottleneck and obstacle for business development. HDFS (Hadoop Distributed File System, Hadoop Distributed File System) distributes data access and storage among a large number of servers through an efficient distributed algorithm, and distributes access in clusters while reliably storing multiple backups It is a subversive development of the traditional storage architecture on top of each serv...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): H04L29/08G06F17/30
Inventor 郭东东赵彦荣赵健博洪亮
Owner BEIJING QIHOO TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com