Supercharge Your Innovation With Domain-Expert AI Agents!

Resource management method and device, and computer storage medium

A resource management and resource technology, applied in the direction of computing, resource allocation, multi-programming devices, etc., can solve the problems of unsupported, unable to uniformly manage multiple sub-cluster queue resources, unable to meet the needs of tenants, etc., to improve utilization rate and throughput Efficiency and high efficiency

Pending Publication Date: 2021-11-30
CHINA MOBILE SUZHOU SOFTWARE TECH CO LTD +1
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] However, there are some defects and deficiencies in the existing YARN Federation. For example, YARN Federation does not support network authorization protocol (kerberos) authentication, and the bottom layer of YARN Federation cannot implement multi-tenant isolation directly used in production systems. In the usage scenario of YARN Federation, Unable to uniformly manage the queue resources of multiple sub-clusters; even when scheduling multi-tenant and multi-cluster resources, it is impossible to configure them according to the actual needs of tenants, so that the needs of tenants cannot be met

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Resource management method and device, and computer storage medium
  • Resource management method and device, and computer storage medium
  • Resource management method and device, and computer storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention.

[0061] In the actual production environment, most of them use a platform as a service (Platform as a Service, PAAS) to manage multi-tenancy. At the bottom of the PAAS platform, there is usually only one distributed Hadoop cluster. However, as the number of businesses and tenants grows, the upper-level tenants’ demand for the resources of the underlying Hadoop cluster is also increasing. At this time, it is necessary to expand the capacity of the underlying Hadoop cluster. However, the performance of the Resource Manager usually limits the size of the cluster. Generally, when the cluster reaches When the number of nodes is 5000, the scheduling of YARN will become the bottleneck of cluster expansion. The current open source community uses the YARN Federation mechanism to solve ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a resource management method and device and a computer storage medium. The method comprises the following steps: receiving a task submitted to a target queue; detecting the queue type of the target queue, wherein the queue type comprises an exclusive queue type and a shared queue type; according to the detected queue type, allocating the task to a cluster resource corresponding to the queue type; and executing the task based on allocated cluster resources. Therefore, by reasonably using the exclusive queue type and the shared queue type, the special requirements of tenants on clusters can be met by using a specified queue mode, and meanwhile, the requirements of load balance, data locality and the like among sub-clusters can be considered, so the utilization rate and throughput rate of the cluster resources can be integrally improved.

Description

technical field [0001] The present invention relates to the technical field of big data applications, in particular to a resource management method, device and computer storage medium. Background technique [0002] With the rapid development of computer and information technology, data has penetrated into every industry and business function field today, becoming an important generating factor. Effective management, further mining and application of massive data has become the key for enterprises to improve their core competitiveness and seize market opportunities. At this time, big data technology emerged as the times require. [0003] On the existing big data platform, with the continuous development and maturity of Hadoop, major companies choose hadoop as the bottom layer of the big data platform. Among them, for small and medium-sized companies, a single hadoop cluster can often meet the needs; but for large-scale business scenarios, due to the diversity and complexity ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F9/48G06F9/50
CPCG06F9/505G06F9/4881G06F9/5083
Inventor 孙大鹏申银杰
Owner CHINA MOBILE SUZHOU SOFTWARE TECH CO LTD
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com