A storage resource management method based on storage medium types and weighting quotas

A technology for storing resources and storage media, which is applied in the directions of resource allocation, electrical digital data processing, and input/output process of data processing, etc., to ensure scalability and fault tolerance, reduce memory consumption, and achieve the effect of high robustness.

Inactive Publication Date: 2017-07-28
BEIHANG UNIV
View PDF3 Cites 15 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, in the existing storage resource management or allocation methods

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A storage resource management method based on storage medium types and weighting quotas
  • A storage resource management method based on storage medium types and weighting quotas
  • A storage resource management method based on storage medium types and weighting quotas

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0039] The following explains in detail the storage resource management method based on the storage medium and the weighted consistency hash proposed by the present invention with reference to the accompanying drawings.

[0040] The storage resource management method of the present invention should include the following steps: mounting of storage devices, mounting of user space file systems, running Hadoop applications, and distributing read and write requests of Hadoop applications. Its system structure is like figure 1 Shown. First, the storage device is mounted to different directories according to different storage media, and then file operation requests for these directories are encapsulated through the user space file system. Then start the user space file system through multiple processes to provide FUSE file service, and mount it to the intermediate data storage directory of the Hadoop system to improve the concurrent utilization of storage resources. Then by modifying t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a storage resource management method based on storage medium types and weighting quotas. The method comprises the steps of mounting of storage apparatuses and user space file systems and distribution of reading-writing requests, wherein weighting distribution characteristics are imported into a Weighting Jump algorithm adopted in distribution of reading-writing requests, so that high calculating speed, stateless scheduling and extremely low memory consumption are achieved. The method is implemented by employing a method similar to jump in probability proposed by John Lamping and Eric Veach of Google. The method can guarantee that the distribution of storage resources complies with set weights at the level of probability, minimize standard error and achieve lower time complexity and memory consumption than a resource management system Yarn of an original Hadoop cluster. The method solves the problem of low efficiency in search directory polling by a disk resource distribution algorithm achieved in the conventional resource management system Yarn, has extendibility and can minimize movement of original data in probability during extension.

Description

technical field [0001] The invention relates to a storage resource management method based on storage medium type and weighted quota, and belongs to the fields of big data storage and distributed computing. Background technique [0002] With the development of big data storage and hybrid storage media technology, storage resource management and allocation methods are becoming more and more important and facing more and more severe tests. The Hadoop big data processing system developed by the Apache open source community has become the most representative solution in the field of big data. Hadoop includes HDFS (Hadoop Distributed FileSystem) distributed file system and Yarn (Yet Another Resource Negotiator) resource management system and MapReduce data processing framework. Among them, the running MapReduce application’s access to data is divided into access to the HDFS distributed file system when reading input files in the map stage, access to the HDFS distributed file sys...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F3/06G06F9/50
CPCG06F3/0604G06F3/0608G06F3/0611G06F3/0614G06F3/0631G06F3/0635G06F3/0643G06F3/0649G06F3/0659G06F3/067G06F9/5083
Inventor 吴文峻冯梦琦
Owner BEIHANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap