Metadata cluster parallel distribution processing method and metadata cluster parallel distribution processing system

A technology for distributing processing and metadata, applied in the field of data processing, which can solve the problems of difficult system state control, high real-time, and small overall ratio, and achieve the effect of realizing stable operation in high-load state and improving real-time processing

Active Publication Date: 2015-12-02
INSPUR BEIJING ELECTRONICS INFORMATION IND
View PDF6 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The distributed metadata cluster system has good scalability and can use the characteristics of distributed systems to solve the problem of poor scalability of the file system. However, with the increase in the size of the metadata cluster, system state control becomes more difficult, especially in high-level In load scenarios, the delayed processing of metadata cluster control messages will cause shocks and even system paralysis of the entire distributed metadata cluster system. In addition, the characteristics of cluster control messages are that the overall proportion is small but the real-time requirements are high. Therefore, metadata cluster control Message priority processing is very necessary

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Metadata cluster parallel distribution processing method and metadata cluster parallel distribution processing system
  • Metadata cluster parallel distribution processing method and metadata cluster parallel distribution processing system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0021] In order to make the purpose, technical solution and advantages of the present invention more clear, the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined arbitrarily with each other.

[0022] The steps shown in the flowcharts of the figures may be performed in a computer system, such as a set of computer-executable instructions. Also, although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.

[0023] The network packets received by the metadata cluster can be divided into cluster control packets and file operation request packets. Among them, the number of cluster control packets is small and requires high real-time processing, which directly af...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a metadata cluster parallel distribution processing method and a metadata cluster parallel distribution processing system. The method comprises the following steps: receiving network packets in network connection with metadata clusters in parallel, and distributing the received network packets to priority processing queues corresponding to the metadata clusters according to the preset priority, wherein each priority processing queue includes processing sub queues different in priority; and up-sending the network packets according to the priority of the processing sub queues in the priority processing queues, and up-sending the network packets of the processing sub queues with the same priority in different priority processing queues in parallel. By adopting the method and the system of the invention, the real-time performance of metadata cluster control packet processing is improved, and stable operation of metadata clusters at high load is realized.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a metadata cluster parallel distribution processing method and system based on message priority queues. Background technique [0002] With the explosion of unstructured data, the distributed file system has entered a golden age of development. From high-performance computing to data centers, from data sharing to Internet applications, it has penetrated into all aspects of data applications. For most distributed file systems, or cluster file systems, or parallel file systems, metadata and data are usually separated, that is, control flow and data flow are separated, so as to obtain higher system scalability and I / O o Concurrency. Therefore, the metadata management model is very important, which directly affects the scalability, performance, reliability and stability of the system. [0003] The distributed metadata cluster system has good scalability and can use the chara...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): H04L29/08H04L12/851H04L12/865H04L47/6275
CPCH04L47/24H04L47/6275H04L67/1097
Inventor 李雪生张延良
Owner INSPUR BEIJING ELECTRONICS INFORMATION IND
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap