Storage method and device of distributed file system
What is Al technical title?
Al technical title is built by PatSnap Al team. It summarizes the technical point description of the patent document.
A technology of distributed files and storage nodes, applied in the field of data processing, can solve problems such as difficulty in meeting the reliability of file storage, and achieve the effects of reducing data migration rate, balancing capacity, and increasing cluster stability
Active Publication Date: 2013-06-12
BEIJING QIHOO TECH CO LTD
View PDF5 Cites 45 Cited by
Summary
Abstract
Description
Claims
Application Information
AI Technical Summary
This helps you quickly interpret patents by identifying the three key elements:
Problems solved by technology
Method used
Benefits of technology
Problems solved by technology
[0008] However, the above-mentioned block allocation strategy only considers the reliability of data and the effect of data copy
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more
Image
Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
Click on the blue label to locate the original text in one second.
Reading with bidirectional positioning of images and text.
Smart Image
Examples
Experimental program
Comparison scheme
Effect test
example 1
[0085] If there are 4 storage nodes DataNodes in the cluster, they are A1, A2, B, and C, among them, A1 and A2 are on the same rack, B and C are independent racks, and the block of the same band already exists on A2 ;
[0086] Client A1 triggers a write request to allocate blocks, the first copy is allocated on the local storage node A1, and the second copy can give priority to A2, but A2 already has a block in the stripe, so it can be randomly selected in the whole cluster B, the last copy can randomly select C in the whole cluster.
example 2
[0088] If there are 10 DataNodes in the cluster: A, B, C, D, E, F, G, H, I, J, K, L; if a file includes two blocks, the stripe length is 2; the client machine is A
[0089] Select three copies of the first Block:
[0090] First Copy: Localization Strategies, Option A
[0091] The second copy: A has been selected in the stripe, and a machine other than A is randomly selected, assuming that B is selected;
[0092] The third copy: randomly selected, C was chosen;
[0093] Select three copies of the second Block:
[0094] First Copy: Localization Strategies, Option A
[0095] Second copy: (A, B, C) in the stripe has been selected, randomly select machines other than these machines, assuming D is selected
[0096] Third copy: random selection, B was chosen
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
PUM
Login to view more
Abstract
The invention discloses a storage method and a storage device of a distributed file system. The method comprises the steps of receiving file information sent by a client side; determining a target storage node of each data block, wherein each target storage node comprises a first storage node, a second storage node and a third storage node; each first storage node is a storage node at which the client side is located or a storage node selected at random from a cluster; each second storage node is a storage node which is not allocated to other data blocks on a stripe where the data block is located or a storage node selected at random from the cluster; and the third storage node is a storage node selected at random from the cluster; and writing the data blocks into the first storage nodes, the second storage nodes and the third storage nodes in order. With the adoption of the storage method and the storage device of the distributed file system, in the case of considering the follow-up use of a Raid technique for the realization of a demand on file storage reliability, the capacity of each node in the cluster can be enabled to be more balanced, and the cluster stability is improved.
Description
technical field [0001] The present invention relates to the technical field of data processing, in particular to a storage method for a distributed file system and a storage device for a distributed file system. Background technique [0002] With the rapid development of information systems, massive amounts of information need to be reliably stored and quickly accessed by a large number of users. Traditional storage solutions have become more and more difficult to adapt to the rapid development of information system business in recent years in terms of architecture, and have become a bottleneck and obstacle for business development. HDFS (Hadoop Distributed File System, Hadoop Distributed File System) distributes data access and storage among a large number of servers through an efficient distributed algorithm, and distributes access in clusters while reliably storing multiple backups It is a subversive development of the traditional storage architecture on top of each serv...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more
Application Information
Patent Timeline
Application Date:The date an application was filed.
Publication Date:The date a patent or application was officially published.
First Publication Date:The earliest publication date of a patent with the same application number.
Issue Date:Publication date of the patent grant document.
PCT Entry Date:The Entry date of PCT National Phase.
Estimated Expiry Date:The statutory expiry date of a patent right according to the Patent Law, and it is the longest term of protection that the patent right can achieve without the termination of the patent right due to other reasons(Term extension factor has been taken into account ).
Invalid Date:Actual expiry date is based on effective date or publication date of legal transaction data of invalid patent.
Login to view more 
Login to view more