Distributed metadata management consistency ensuring method, device and system and application

A metadata and consistency technology, applied in special data processing applications, electronic digital data processing, database updates, etc., to achieve performance and low latency

Active Publication Date: 2019-04-05
SUN YAT SEN UNIV +1
View PDF7 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In short, in the distributed metadata management scenario, it is difficult for existing technologies to take into account the consistency and high performance of metadata updates at the same time.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed metadata management consistency ensuring method, device and system and application
  • Distributed metadata management consistency ensuring method, device and system and application
  • Distributed metadata management consistency ensuring method, device and system and application

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0038] A traditional large-scale file system consists of three types of nodes: clients, metadata servers, and data servers. Among them, the client is the initiator of all read and write requests, the metadata server is responsible for storing file metadata and responding to metadata requests sent by clients, and the data server is responsible for storing file data and responding to data requests sent by clients. This embodiment introduces a new type of node on the above infrastructure of the traditional large-scale file system: a consistent node (ConsistentNode), which is used to perform metadata management consistency assurance. The specific architecture is as follows figure 1 shown. figure 1 A file system organization structure including 4 clients, 3 metadata servers, and 2 consistency nodes is listed in , where dirty means dirty data, and the data server is ignored because it does not involve the related technology of the present invention. In the above architecture, the f...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distributed metadata management consistency guaranteeing method, Device, system and application, According to the invention, consistent nodes are deployed in a metadata cluster; the client sends a metadata updating request to the consistency node, the consistency node returns a message that the metadata is updated successfully to the client and records the metadata updating request in sequence, old metadata is marked as rejected data, and the rejected data is marked as rejected data after asynchronous data synchronization with the metadata server; The client metadataserver sends a metadata reading operation, and if an object of the metadata reading operation is marked as being invalid, the read data of which the asynchronous data synchronization is not completedare returned through the consistency node; And otherwise, directly returning the read data through the metadata server where the metadata is located. According to the method, the consistency of distributed metadata management can be ensured, and the metadata access performance is improved as much as possible while the consistency of metadata updating is ensured.

Description

technical field [0001] The present invention relates to the field of file systems for large-scale data storage, in particular to a distributed metadata management consistency assurance method, device, system and application, which are used to introduce consistent nodes into metadata clusters of distributed file systems, This ensures metadata consistency with very low overhead. Background technique [0002] As the amount of data accumulated by human society continues to grow, the scale of storage systems is getting larger and larger, and the construction of scalable storage systems has gradually attracted the attention of academia and industry. As an important means to realize scalable storage, distributed file system has attracted extensive attention. Traditional distributed file systems such as GFS, Luster, GPFS, and PVFS all use a metadata server. This centralized metadata management method has become the main obstacle for distributed file systems to achieve performance e...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/23G06F16/182
CPCG06F16/164G06F16/178G06F16/162G06F16/125G06F16/1734
Inventor 卢宇彤陈志广肖侬刘芳
Owner SUN YAT SEN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap