Real-time dynamic management method and system for distributed memory file system

An in-memory file system, real-time dynamic technology, applied in transmission systems, resource allocation, electrical components, etc., can solve problems such as inability to distribute data-intensive and computationally-intensive computations efficiently at the same time, and achieve optimized memory usage, The effect of improving efficiency

Inactive Publication Date: 2017-11-10
杭州知物数据科技有限公司
View PDF5 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] In order to solve the technical problem that the static configuration scheme cannot enable distributed data-intensive and computing-intensive calculations to efficiently run on computer clusters at the same time, the present invention proposes a dynamic management method and system for a distributed memory file system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Real-time dynamic management method and system for distributed memory file system
  • Real-time dynamic management method and system for distributed memory file system
  • Real-time dynamic management method and system for distributed memory file system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0068] Embodiment 1 is a real-time dynamic management system of a specific distributed memory file system for practical application, such as image 3 As shown, it includes a distributed memory monitoring agent module 02, a distributed information flow processor 04, and a storage controller 05. The distributed memory monitoring agent module 02 is connected to the distributed information flow processor 04, and the distributed information flow processor 04 Connect storage controller 05;

[0069] The memory monitoring agent module 02 is used to collect the memory usage information of each computing node in the computer cluster in real time, and send the collected information to the distributed messaging bus 01;

[0070] The distributed message transfer bus 01 transmits the collected memory usage of each computing node from the distributed memory monitoring agent module 02 to the distributed information flow processor 04, and optimizes the computing node of each computing node in t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention provides a real-time dynamic management method and system for a distributed memory file system applied to a computer cluster. The method is used for dynamically adjusting a memory usage amount of the distributed memory file system on the computer cluster. The method comprises the following steps of collecting a memory usage amount of each computing node on the computer cluster at a control time point in real time; predicting a memory usage amount of the memory file system at the next control time point of each computing node according to the collected memory usage amount; and performing judgment through the predicted memory usage amount, and sending out a control instruction after the judgment to control the distributed memory file system to send out recovery and allocation instructions. By adopting the dynamic control system and method, the memory use of the computer cluster can be optimized, so that the efficiency of data-intensive and compute-intensive computing on the computer cluster is improved.

Description

technical field [0001] The invention relates to the technical field of distributed memory management and distributed file system, in particular to a real-time dynamic management method and system of a distributed memory file system. Background technique [0002] Data-intensive computing is becoming an increasingly important task for computer clusters. In order to accelerate data-intensive computing, existing big data analysis systems, such as Apache Spark or Apache Flink, when performing data-intensive computing on computer clusters, a large part of the memory is used to store data, and the remaining memory resources Will be used for other computational work. In this way, memory storage will compete with computing tasks for memory space. Existing distributed memory file systems are all statically configured. When the distributed memory file system is deployed, the space of the distributed memory file system on each computing node in the computer cluster is fixed, which le...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50H04L29/08
CPCG06F9/5016G06F9/5072H04L67/1097
Inventor 罗峰宣鹏飞陈虎
Owner 杭州知物数据科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap