Distributed storage-based dynamic metadata optimizing method and system

A distributed storage and optimization method technology, applied in the field of dynamic metadata optimization, can solve the problems of wasting network traffic and migration time, high cost, etc., and achieve the effect of reducing network traffic and migration time, improving migration efficiency, and reducing the number of migrations

Inactive Publication Date: 2017-09-08
ZHENGZHOU YUNHAI INFORMATION TECH CO LTD
View PDF2 Cites 32 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, when high-configuration, high-load nodes are migrated to low-configuration, low-load nodes, and the low-configuration mds nodes receive too much load, system performance will become a bottleneck, although the next round of balancing will reduce the low-configuration nodes Req

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed storage-based dynamic metadata optimizing method and system
  • Distributed storage-based dynamic metadata optimizing method and system
  • Distributed storage-based dynamic metadata optimizing method and system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0054] Please refer to Figure 1~4 , figure 1 It is a schematic flow diagram of the steps of a specific implementation of the distributed storage-based dynamic metadata optimization method provided by the embodiment of the present invention; figure 2 A schematic structural diagram of a specific implementation of the distributed storage-based dynamic metadata optimization system provided by the embodiment of the present invention; image 3 A schematic struct...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distributed storage-based dynamic metadata optimizing method and system. The method comprises the following steps: obtaining the load of mds (metadata server) nodes; obtaining the average load of mds clusters; calculating the processing power on the basis of the memory of the mds nodes and a CPU (Central Processing Unit) to obtain corresponding weighting coefficients; and judging whether the time when the load of the mds nodes is greater than the average load exceeds the threshold or not, and if yes, migrating data of the mds nodes of which the load is greater than the average load to the mds nodes of which the load is lower than the average load. The processing power is calculated on the basis of the memory of the mds nodes and the CPU to obtain the corresponding weighting coefficients, so that the data processing power of all the mds nodes is evaluated more accurately; regional division is carried out on the metadata server; when sub-trees are migrated, the load and the machine performance are comprehensively considered; the selection strategy of sub-tree migration targets is optimized on the basis of the data processing power and the current load of each mds node; and therefore, the network flow and the migration time are reduced, and the migration efficiency is increased.

Description

technical field [0001] The invention relates to the field of computer hardware, in particular to a method and system for optimizing dynamic metadata based on distributed storage. Background technique [0002] Distributed File System (Distributed File System) means that the physical storage resources managed by the file system are not necessarily directly connected to the local node, but are connected to the node through a computer network. The design of the distributed file system is based on the client / server model. A typical network may include multiple servers accessed by multiple users. In addition, the peer-to-peer feature allows some systems to perform the dual roles of client and server. For example, a user can "publish" a directory that other clients can access, and once accessed, the directory appears to the client as if it were using a local drive. Therefore, the distributed file system can be a storage platform for applications in high-performance computing, da...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F3/06G06F9/50
CPCG06F3/0607G06F3/0631G06F3/0655G06F3/067G06F9/5083
Inventor 赵晓鹏
Owner ZHENGZHOU YUNHAI INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap