Supercharge Your Innovation With Domain-Expert AI Agents!

A read-write method and system for metadata information of a distributed file system

A technology of distributed files and metadata information, applied in the field of distributed file systems, can solve problems such as increasing startup time, data loss, and affecting user experience, and achieve the effects of shortening time, improving user experience, and improving system efficiency

Pending Publication Date: 2021-09-03
普赛微科技(杭州)有限公司
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] However, the above-mentioned existing restart method has the following defects: the existing metadata image file is generally stored in the volatile memory when the system is running, if the system suddenly loses power, the data stored in the volatile memory will be lost. Lost, resulting in the inability to obtain the latest data in time after the system is restarted; secondly, the metadata image files and log files stored locally in the distributed file system are usually stored in a regular update method, which is not performed in real time. When the name node occurs In case of failures such as downtime or power failure, the locally stored metadata image file may still not contain the latest namespace metadata, and the log file needs to be updated at startup, which increases the startup time; if the log file is restarted If there are many operation records stored in the system, it takes a long time to update the metadata file, and the user needs to wait for a long time before re-using the system, which affects the user experience.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A read-write method and system for metadata information of a distributed file system
  • A read-write method and system for metadata information of a distributed file system
  • A read-write method and system for metadata information of a distributed file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0048] This embodiment provides a method for reading and writing metadata information of a distributed file system, wherein the distributed file system adopts a master-slave structure and includes multiple data nodes and at least one name node. The distributed file system stores data in the form of data blocks, that is, distributed files are divided into different data blocks and stored in different data nodes, and all data nodes are managed through name nodes. The name node maintains all metadata of the file system tree and the folders and files contained in the file tree through the image file, and the existing name node loads the image file into the local memory at runtime, while this The name node in the embodiment loads the image file into the non-volatile memory during operation, and when the name node needs to be restarted after a failure such as downtime or power failure occurs, the image file in the non-volatile memory saves the fault occurrence For the latest namespa...

Embodiment 2

[0067] This embodiment provides a distributed file system, which executes the method for reading and writing metadata information of the distributed file system described in Embodiment 1, wherein, as image 3 As shown, the system specifically includes the following modules:

[0068] Create a module for creating an image file, where the image file is used to store the metadata of the distributed file system at runtime;

[0069] An operation module, configured to obtain metadata information when the distributed file system is running, and store it in the image file in real time;

[0070] A non-volatile storage module for storing the image file;

[0071] A restart module, configured to load metadata information of the image file from the non-volatile storage module, and perform a restart operation of the distributed file system.

[0072] The distributed file system of this embodiment saves the image file in the non-volatile storage module in real time during operation, and the ...

Embodiment 3

[0074] This embodiment provides an electronic device, including a program, a memory for storing the program, and a processor for loading the program to execute the method for reading and writing metadata information of a distributed file system as described above. In addition, this embodiment also provides a storage medium, which stores a program, and when the program is executed by a processor, implements the method for reading and writing metadata information of the distributed file system as described above.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a read-write method and system for metadata information of a distributed file system, and the read-write method comprises the steps of creating a mirror image file used for storing the metadata information of the distributed file system, and storing the mirror image file in a nonvolatile memory; receiving the metadata information generated in the running process of the distributed file system, and writing the metadata information into the mirror image file in the nonvolatile memory in real time; and in response to a restart instruction initiated by the distributed file system, loading the metadata information of the mirror image file from the nonvolatile memory for restarting the distributed file system. According to the invention, the time required for restarting the system can be shortened, so that the system efficiency and the user experience are improved.

Description

technical field [0001] The invention relates to the technical field of distributed file systems, in particular to a method, system, device and storage medium for reading and writing metadata information of a distributed file system. Background technique [0002] Big data is one of the most important technical directions for the development of the information industry. With the generation of massive data, traditional centralized computing and storage systems are difficult to meet the requirements for big data storage and processing, and distributed storage and computing systems have been widely used. [0003] The existing distributed file system generally adopts a master-slave structure, including multiple data nodes and a name node, and the name node contains two important data structures, one is the metadata image file, and the system can The metadata mirror file is used to maintain all the metadata of the file system tree and the folders and files contained in the file tr...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F8/61G06F8/65G06F16/172G06F16/18G06F16/182
CPCG06F8/63G06F8/65G06F16/172G06F16/1815G06F16/182
Inventor 蒋信刘瑞盛喻涛
Owner 普赛微科技(杭州)有限公司
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com