Distributed system, allocation method of resource container, resource manager and application controller

Abstract

Embodiments of the present invention provide a resource allocation method, device and system for allocating resources for applications. The method includes: the application controller receives a resource allocation request from the client, and the resource allocation request carries resource requirement information of an application running on the client, wherein the application controller is used to manage a resource pool, and the resource pool contains at least one The started container, the container in the resource pool is applied to and started by the application controller in advance from the resource manager; the application controller selects an idle resource container from the resource pool and assigns it to the client according to the resource requirement information of the application. According to this method, when the application controller receives the resource allocation request of the application, it can realize the timely allocation of resource container resources, avoiding the waiting time for resource container startup, and avoiding resource allocation by reusing resource containers in the resource pool. The resource consumption of multiple opening and closing of the container.

Description

technical field [0001] The embodiments of the present invention relate to the field of computers, and in particular, to a resource allocation method. Background technique [0002] Since the development of cloud computing, the scale of clusters has continued to expand, and the types of clusters have also continued to increase. Such as common distributed computing Hadoop clusters for parallel programming technical tasks. From the changing trend of the industry using distributed systems and the long-term development of the Hadoop framework, the job tracker (Job Tracker) / task tracker (Task Tracker) mechanism in the first version of MapReduce MRv1 needs large-scale adjustments to repair It has flaws in scalability, memory consumption, threading model, reliability and performance. [0003] Another resource coordinator (Yet Another Resource Negotiator, yarn) is a new Hadoop resource manager, which is a general resource management system that can provide unified resource managem...

AI Technical Summary

Problems solved by technology

But this method will introduce a new problem: the job delay is large

Therefore, the Yarn job will generate a long running delay, which is not conducive to running small jobs, and because it needs to apply for an AM for each job, more computing resources are required

Images