Metadata access performance optimization method, system and device and storage medium

A technology of access performance and optimization method, which is applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve problems such as time-consuming and cumbersome steps, achieve access performance optimization, reduce access time-consuming, and improve access operation performance Effect

Inactive Publication Date: 2018-11-30
ZHENGZHOU YUNHAI INFORMATION TECH CO LTD
View PDF3 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The existing distributed file system continues to use the metadata storage mechanism of the traditional local file system, that is, first use the hard disk to store the metadata persistently, and when receiving a file data access request, it is also necessary to find the metadata in the hard disk The metadata corresponding to the file data can be operated on after it is placed in the memory, and the metadata in the memory is usually moved out of the memory after a period of time after the access is completed. The access process of each metadata Similar, the steps are cumbersome and time-consuming

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Metadata access performance optimization method, system and device and storage medium
  • Metadata access performance optimization method, system and device and storage medium
  • Metadata access performance optimization method, system and device and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0048] The following combination figure 1 , figure 1 It is a flowchart of a metadata access performance optimization method provided in the embodiment of the present application, which specifically includes the following steps:

[0049] S101: Obtain metadata currently being accessed;

[0050] This step aims to obtain the metadata currently being accessed in the distributed file system. Specifically, it is possible to determine which metadata is being accessed according to acquiring currently active processes from the memory in the distributed file system.

[0051] S102: pre-caching associated metadata associated with the metadata into memory;

[0052] On the basis of S101, this step aims to first find other metadata that is related to the currently accessed metadata (in this application, the term associated metadata will be used to refer to it), and then associate the metadata pre-cached in memory. For example, assuming that the metadata A in the distributed file system i...

Embodiment 2

[0060] The following combination figure 2 , figure 2 It is a flowchart of another metadata access performance optimization method provided by the embodiment of the present application on the basis of the first embodiment. This embodiment is based on the first embodiment and aims at the metadata access operation in the existing distributed file system. Another problem that affects performance gives a solution. The specific steps are as follows:

[0061] S201: Count the number of visits of each metadata stored in the memory in each period by period, and obtain each visit popularity value;

[0062] S202: Arranging the visit popularity values ​​from top to bottom and from large to small to generate a visit popularity table;

[0063] S203: When the remaining storage space of the memory is less than the preset storage space, remove from the memory the metadata corresponding to a preset number of access heat values ​​at the bottom of the access heat table.

[0064] This embodime...

Embodiment 3

[0067] The following combination image 3 , image 3 It is a flow chart of another metadata access performance optimization method provided by the embodiment of the present application on the basis of the first embodiment. On the basis of the first embodiment, this embodiment aims at the metadata access operation in the existing distributed file system. Another problem that affects performance gives a solution. The specific steps are as follows:

[0068] S301: Extract the target access path from the received access request;

[0069] Wherein, the target access path is composed of directories with a preset number of levels and metadata names located in the last level of directories, and each level of directories is included in the upper level directory and includes the lower level directory. by

[0070] S302: sequentially inquire whether each level of directories forming the target access path has been cached in memory according to the order of the number of directories;

[...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a metadata access performance optimization method. The method comprises the steps of obtaining metadata currently accessed; caching related metadata related to the metadata into a memory in advance; when an access request of the related metadata is received, searching the memory for target metadata corresponding to the access request, and directly accessing the target metadata in the memory. By caching the related metadata related to the metadata currently accessed into the memory in advance, when the access request of the related metadata is received, the process of searching a hard disk and building in the memory can be skipped, various operations can be performed in the memory directly, the access consumption can be remarkably shortened, and the access operationperformance can be improved. The invention further discloses a data access performance optimization system and device and a readable storage medium which have the advantages.

Description

technical field [0001] The present application relates to the technical field of metadata access, and in particular to a metadata access performance optimization method, system, device, and computer-readable storage medium. Background technique [0002] With the continuous increase of data generation, the storage space for storing these data also needs to be expanded accordingly. Traditional storage technologies such as single node and disk array have been difficult to meet the storage requirements of massive data. Cluster storage system with its natural scalability Distributed file system (also known as network file system) is one of the more commonly used ones. The file system can be strictly accessed according to the access list and capacity of each server and client. By combining data and metadata Distribute to each server through certain protocols and rules, and provide standard interface of the file system for the client. [0003] In a distributed file system, data ca...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
Inventor 毕银龙
Owner ZHENGZHOU YUNHAI INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap