Distributed file system and data access method

A technology of distributed files and file data, applied in the network field, can solve problems such as data redundancy, and achieve the effect of preventing data redundancy and improving utilization.

Active Publication Date: 2020-07-24
ALIBABA GRP HLDG LTD
View PDF9 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In this way, users can only obtain 1 / 3 of the actual storage capacity, and the data redundancy caused by multiple copies is not what users want

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed file system and data access method
  • Distributed file system and data access method
  • Distributed file system and data access method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0018] refer to figure 1 , shows a structural block diagram of a distributed file system according to Embodiment 1 of the present invention.

[0019] The distributed file system of this embodiment includes: a file data node part 102 , a metadata node part 104 and a configuration center 106 . Wherein, the file data node part 102 includes a file data node 1022 and a first storage disk 1024 ; the metadata node part includes a metadata node 1042 and a second storage disk 1044 .

[0020] in:

[0021] Both the file data node 1022 and the first storage disk 1024 include multiple, each file data node 1022 is mounted with at least one first storage disk 1024, and each first storage disk 1024 is mounted on at least two file data nodes 1022, File data of files in the distributed file system is stored in the first storage disk 1024 .

[0022] In the embodiments of the present invention, unless otherwise specified, "multiple" means two or more. In addition, in the embodiment of the pre...

Embodiment 2

[0031] This embodiment is based on the distributed file system in the foregoing embodiment 1, and the distributed file system is further improved. In addition, in this embodiment, the distributed file system in the embodiment of the present invention is described in the form of a storage disk using a cloud disk. However, those skilled in the art should understand that in practical applications, as described in Embodiment 1, other forms of storage disks are also applicable to the solution of this embodiment.

[0032] The distributed file system in this embodiment includes the file data node part 102, the metadata node part 104 and the configuration center 106 as described in the first embodiment, wherein the file data node part 102 includes a plurality of file data nodes 1022 and multiple A first cloud disk 1024. In addition, the distributed file system in this embodiment further includes a client 108 . The client 108 may be an independent device, or may be an application or ...

Embodiment 3

[0065] refer to Figure 4 , shows a flowchart of steps of a data access method according to Embodiment 3 of the present invention.

[0066] The data access method in this embodiment is based on the distributed file system described in Embodiment 1 or Embodiment 2, and the distributed file system includes the file data node, metadata node, and configuration center as described above.

[0067] Based on the distributed file system, the data access method of this embodiment includes the following steps:

[0068]Step S302: the client sends a file operation request to the metadata node, and receives the information of the file data node matching the file operation request returned by the metadata node.

[0069] Wherein, the information of the file data node includes the information of at least two file data nodes, and the at least two file data nodes are both nodes that mount the target storage disk requested by the file operation request, and the file data node The information of...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The embodiment of the invention provides a distributed file system and a data access method, and the distributed file system comprises a file data node, a first storage disk mounted on the file data node, a metadata node, a second storage disk mounted on the metadata node, and a configuration center. The number of the file data nodes and the number of the first storage disks are multiple, each file data node mounts at least one first storage disk, each first storage disk mounts at least two file data nodes, and file data is stored in the first storage disks; the configuration center is used for storing a mounting relationship between the first storage disk and the file data node; the metadata node is used for receiving a file operation request of a client, obtaining metadata of the file from mounted second storage disk or sending the information of the file data node matched with the file operation request obtained from the configuration center to the client, so that the client accesses the first storage disk to perform file operation requested by the file operation request through the first storage disk.

Description

technical field [0001] The embodiment of the present invention relates to the field of network technology, and in particular to a distributed file system and a data access method. Background technique [0002] A distributed file system means that the physical storage resources managed by the file system are not necessarily directly connected to the local node, but are connected to the local node through a computer network. Based on the distributed file system, more and more high-level applications or frameworks have emerged. For example, the Hadoop framework is currently widely used. [0003] The Hadoop framework is a distributed system infrastructure developed by the Apache Foundation. Through Hadoop, users can develop distributed programs without knowing the underlying details of the distribution. The Hadoop framework mainly includes HDFS (Hadoop Distributed File System) and MapReduce. Among them, HDFS is a distributed file system that provides storage for massive data, ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/182G06F16/13
CPCG06F16/182G06F16/13
Inventor 孟庆义梁俊飞
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap