Unlock instant, AI-driven research and patent intelligence for your innovation.

The invention discloses a mMethod and a system for achievingrealizing multi-cluster multi-tenant resource isolation based on a Yarn queue

A multi-tenant and cluster technology, applied in the field of big data technology cluster resource management, can solve problems such as unsolvable multi-cluster resource allocation, large cluster size, resource competition, etc., and achieve the effect of multi-cluster multi-tenant resource isolation

Pending Publication Date: 2019-04-12
杭州玳数科技有限公司
View PDF1 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Although Yarn can also support cpu resources, the problem of resource competition is very serious when the cluster scale is large and there are many tasks;
[0004] 2. Unable to solve the resource allocation problem of multi-cluster under multi-tenant

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • The invention discloses a mMethod and a system for achievingrealizing multi-cluster multi-tenant resource isolation based on a Yarn queue
  • The invention discloses a mMethod and a system for achievingrealizing multi-cluster multi-tenant resource isolation based on a Yarn queue
  • The invention discloses a mMethod and a system for achievingrealizing multi-cluster multi-tenant resource isolation based on a Yarn queue

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0031] figure 1 It shows a schematic structural diagram of a system for implementing multi-cluster multi-tenant resource isolation based on Yarn queues provided by an embodiment of the present invention, see figure 1 The system for implementing multi-cluster multi-tenant resource isolation based on the Yarn queue provided by the embodiment of the present invention includes:

[0032] The client 101 that sends the task is used to ob...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a method and system for achieving multi-cluster multi-tenant resource isolation based on a Yarn queue. The method comprises the steps that a client sending a task obtains cluster configuration information according to a tenant to which the task belongs; T; the client sending the task splices the task and the cluster configuration information to obtain split information, andsends the split information to an agent submitting the task; A; and the agent for submitting the task receives the splicing information, extracts the task type and cluster configuration information inthe task, instantiates a client of the computing engine, and submits the task to an associated cluster by using the client of the computing engine to be executed through a Yarn queue. Therefore, multi-cluster multi-tenant resource isolation can be realized.

Description

technical field [0001] The present invention relates to the technical field of big data technology cluster resource management, in particular to a method and system for realizing multi-cluster multi-tenant resource isolation based on Yarn queues. Background technique [0002] Existing open source technology solutions have the following deficiencies: [0003] 1. When the cluster uses Yarn as the resource management framework, it only manages memory resources by default. Although Yarn can also support cpu resources, the problem of resource competition is very serious when the cluster scale is large and there are many tasks; [0004] 2. The resource allocation problem of multi-cluster under multi-tenant cannot be solved. Contents of the invention [0005] The present invention aims to provide a method and system for implementing multi-cluster multi-tenant resource isolation based on Yarn queues, which overcomes one of the above problems or at least partially solves any of t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F9/50
CPCG06F9/5061G06F2209/504
Inventor 赵章万
Owner 杭州玳数科技有限公司