Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A storage resource management method based on storage medium types and weighting quotas

A technology for storing resources and storage media, which is applied in the directions of resource allocation, electrical digital data processing, and input/output process of data processing, etc., to ensure scalability and fault tolerance, reduce memory consumption, and achieve the effect of high robustness.

Inactive Publication Date: 2017-07-28
BEIHANG UNIV
View PDF3 Cites 15 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, in the existing storage resource management or allocation methods, there is no method that can effectively satisfy all the above conditions

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A storage resource management method based on storage medium types and weighting quotas
  • A storage resource management method based on storage medium types and weighting quotas
  • A storage resource management method based on storage medium types and weighting quotas

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0039] The method for managing storage resources based on storage media and weighted consistent hashing proposed by the present invention will be explained in detail below in conjunction with the accompanying drawings.

[0040] The storage resource management method of the present invention should include the following steps: mounting of storage devices, mounting of user space file systems, running Hadoop applications, and distribution of read and write requests of Hadoop applications. Its system structure is as figure 1 shown. First, the storage device is mounted to different directories according to different storage media, and then the file operation requests for these directories are encapsulated through the user space file system. Then start the user space file system through multiple processes to provide FUSE file services, and mount it to the intermediate data storage directory of the Hadoop system to improve the concurrent utilization of storage resources. Then by mo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a storage resource management method based on storage medium types and weighting quotas. The method comprises the steps of mounting of storage apparatuses and user space file systems and distribution of reading-writing requests, wherein weighting distribution characteristics are imported into a Weighting Jump algorithm adopted in distribution of reading-writing requests, so that high calculating speed, stateless scheduling and extremely low memory consumption are achieved. The method is implemented by employing a method similar to jump in probability proposed by John Lamping and Eric Veach of Google. The method can guarantee that the distribution of storage resources complies with set weights at the level of probability, minimize standard error and achieve lower time complexity and memory consumption than a resource management system Yarn of an original Hadoop cluster. The method solves the problem of low efficiency in search directory polling by a disk resource distribution algorithm achieved in the conventional resource management system Yarn, has extendibility and can minimize movement of original data in probability during extension.

Description

technical field [0001] The invention relates to a storage resource management method based on storage medium type and weighted quota, and belongs to the fields of big data storage and distributed computing. Background technique [0002] With the development of big data storage and hybrid storage media technology, storage resource management and allocation methods are becoming more and more important and facing more and more severe tests. The Hadoop big data processing system developed by the Apache open source community has become the most representative solution in the field of big data. Hadoop includes HDFS (Hadoop Distributed FileSystem) distributed file system and Yarn (Yet Another Resource Negotiator) resource management system and MapReduce data processing framework. Among them, the running MapReduce application’s access to data is divided into access to the HDFS distributed file system when reading input files in the map stage, access to the HDFS distributed file sys...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F3/06G06F9/50
CPCG06F3/0604G06F3/0608G06F3/0611G06F3/0614G06F3/0631G06F3/0635G06F3/0643G06F3/0649G06F3/0659G06F3/067G06F9/5083
Inventor 吴文峻冯梦琦
Owner BEIHANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com