Cluster resource management method and device based on virtual queue, and electronic equipment

Abstract

The invention discloses a cluster resource management method and device based on virtual queues and electronic equipment, and relates to the technical field of distributed computing, and the method comprises the following steps: obtaining the current queue length and the current quota use condition of each virtual queue, and confirming a tenant demand quota based on an offline task set submitted by each virtual queue, and under the condition that the current resource quota indicated by the current queue length of the virtual queue is lower than the quota required by the tenant, initiating a quota demolition request, and after idle quota demolited by other virtual queues are obtained, extracting resource state information of the physical container cluster reported by each resource quota management unit, and selecting a target physical container cluster according to the resource state information of each physical container cluster, issuing the offline task to the target physical container cluster, and executing the offline task. The technical problem that the resource utilization rate is low due to the fact that unbalanced resource requirements in a multi-tenant environment cannot be processed by adopting a static quota management mechanism in related technologies is solved.

Description

Technical Field

[0001] The present invention relates to the field of distributed computing technology or other related fields, and in particular to a cluster resource management method and device based on a virtual queue, and an electronic device. Background Art

[0002] With the rapid development of cloud computing and big data technologies, enterprises have an increasing demand for offline colocation. Especially in a multi-cluster environment, tenant resource management for offline services (such as data analysis, model training, etc.) becomes particularly important because it is directly related to resource utilization, task execution efficiency, and overall business operating costs.

[0003] In the prior art, tenant quota management for offline services mainly relies on static configuration, that is, the resource quota of each tenant on a specific cluster is pre-set before task scheduling. Although this method is simple and easy to implement, there are several major problems in actual application: 1. Since the resource quota is static, when the offline task volume of some tenants decreases and the task volume of other tenants increases, the static quota cannot achieve effective resource redistribution, resulting in resource waste. 2. The static quota management method cannot respond to changes in tenants' resource requests in a timely manner, especially in a multi-cluster environment. The dynamic and uncertain nature of resource demand requires the resource allocation mechanism to adapt quickly, and static configuration is difficult to meet this requirement. 3. In a multi-tenant environment, the imbalance of resource demand among tenants is the norm. However, static quota management lacks a mechanism, which makes it impossible for tenants with surplus resources to "lend" their resources to tenants with scarce resources, reducing the overall resource utilization efficiency of the cluster. 4. Under static quota management, when selecting a physical cluster, the task scheduler can only rely on fixed quota information, but cannot consider the resource status and quota borrowing possibility across clusters. This may lead to non-optimal selection of task scheduling and affect the efficiency of task execution.

[0004] To solve the above problems, existing technical solutions usually rely on offline business tenant quota management based on a multi-cluster management platform, and implement static resource management by configuring offline task quotas for multiple tenants in advance. However, this method still has the limitations of being unable to dynamically manage resources and unable to implement resource lending between tenants, especially in a multi-cluster environment where resource requirements change frequently. These limitations are particularly prominent.

[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0006] The embodiments of the present invention provide a cluster resource management method and device based on a virtual queue, and an electronic device, so as to at least solve the technical problem that the static quota management mechanism used in the related art cannot handle the imbalance of resource demand in a multi-tenant environment, resulting in low resource utilization.

[0007] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a cluster resource management method based on a virtual queue is provided, which is applied to an offline task scheduler, wherein the offline task scheduler is connected to multiple virtual queues and multiple physical container clusters, and manages the corresponding physical container clusters through a resource quota management unit, each of the virtual queues represents a tenant, and the queue length indicates the resource quota of the tenant. The cluster resource management method based on the virtual queue includes: obtaining the current queue length and current quota usage of each virtual queue, and confirming the tenant demand quota based on the offline task set submitted by each virtual queue; when the current resource quota indicated by the current queue length of the virtual queue is lower than the tenant demand quota, initiating a quota borrowing request, wherein the quota borrowing request is used to request free quota from other virtual queues that allow quota borrowing; after obtaining the free quota borrowed by other virtual queues, extracting the resource status information of the physical container cluster reported by each resource quota management unit; according to the resource status information of each physical container cluster, selecting a target physical container cluster, sending the offline task to the target physical container cluster, and having the target physical container cluster execute the offline task.

[0008] Optionally, the step of initiating a quota borrowing request includes: querying the current quota usage of other virtual queues that are allowed to borrow; based on the current quota usage of the other virtual queues, analyzing the quota free amount and the borrowing threshold of each of the other virtual queues, and selecting the other virtual queue with the lowest quota free amount and lower than the borrowing threshold as the target virtual queue for quota borrowing; initiating a quota borrowing request to the tenant of the target virtual queue.

[0009] Optionally, after initiating a quota borrowing request, it also includes: recording detailed information of the quota borrowing, wherein the detailed information of the quota borrowing includes: the quota borrowing amount, the quota lending amount, the tenant information of both parties to the borrowing, and the borrowing time; after the quota borrowing request is responded to, updating the resource quota of the original virtual queue to meet the quota requirements of the offline task set submitted by the original virtual queue.

[0010] Optionally, when initiating a quota borrowing request, it also includes: maintaining the borrowing quota amount within a specified quota range within a set time period.

[0011] Optionally, before extracting the resource status information of the physical container cluster reported by each resource quota management unit, it also includes: receiving an HTTP reporting request from each of the resource quota management units, wherein the HTTP reporting request carries the resource status information of the physical container cluster, and the resource status information includes at least one of the following: CPU usage, memory usage, storage space usage data, and network bandwidth status; summarizing the received resource status information and classifying and storing it according to the cluster identifier and tenant identifier of the physical container cluster; sending the summarized resource status information to a resource analysis model, and the resource analysis model outputs a quota update policy for each virtual queue and anomaly troubleshooting data and resource adjustment policy for each physical container cluster.

[0012] Optionally, before selecting a target physical container cluster based on the resource status information of each of the physical container clusters, it also includes: the resource quota management unit collects the resource status information of each of the physical container clusters; the resource quota management unit evaluates the resource usage value of each physical container cluster based on the resource status information, and selects a physical container cluster whose resource usage value is lower than a preset usage threshold as a candidate physical container cluster; the resource quota management unit uploads the candidate physical container cluster to the offline task scheduler, wherein the offline task scheduler selects a cluster from the candidate physical container clusters when selecting to execute the next offline task, and obtains the target physical container cluster.

[0013] Optionally, the cluster resource management method based on virtual queues also includes: according to pre-set priority rules, prioritizing the tenants represented by each virtual queue to obtain a queue priority division result, wherein the queue priority division result is used to determine the order of virtual queues for quota borrowing; for tenants with the same priority in the queue priority division result, a waiting queue is established based on the order of submission time of each offline task, wherein when quota release is detected or quota is increased through a borrowing mechanism, resource quota is scheduled for each offline task in turn according to the waiting queue.

[0014] According to another aspect of an embodiment of the present invention, a cluster resource management device based on a virtual queue is also provided, which is applied to an offline task scheduler. The offline task scheduler connects to multiple virtual queues and multiple physical container clusters, and manages the corresponding physical container clusters through a resource quota management unit. Each of the virtual queues represents a tenant, and the queue length indicates the resource quota of the tenant. The cluster resource management device based on the virtual queue includes: a demand quota determination unit, which is used to obtain the current queue length and current quota usage of each virtual queue, and confirm the tenant demand quota based on the offline task set submitted by each virtual queue; a quota borrowing request unit, which is used to In the case where the current resource quota indicated by the current queue length of the virtual queue is lower than the tenant demand quota, a quota borrowing request is initiated, wherein the quota borrowing request is used to request free quota from other virtual queues that allow quota borrowing; a cluster resource status extraction unit is used to extract the resource status information of the physical container cluster reported by each resource quota management unit after obtaining the free quota borrowed from other virtual queues; a task issuing unit is used to select a target physical container cluster according to the resource status information of each physical container cluster, issue the offline task to the target physical container cluster, and have the target physical container cluster execute the offline task.

[0015] Optionally, the quota borrowing request unit includes: a quota query module, used to query the current quota usage of other virtual queues that are allowed to borrow; a quota analysis module, used to analyze the quota free amount and borrowing threshold of each other virtual queue based on the current quota usage of the other virtual queues, and select the other virtual queue with the lowest quota free amount and lower than the borrowing threshold as the target virtual queue for quota borrowing; a quota borrowing request initiating module, used to initiate a quota borrowing request to the tenant of the target virtual queue.

[0016] Optionally, the cluster resource management device based on virtual queues also includes: a quota borrowing and lending recording unit, which is used to record detailed information of quota borrowing after initiating a quota borrowing and lending request, wherein the detailed information of quota borrowing and lending includes: quota borrowing amount, quota lending amount, tenant information of both parties of the borrowing and lending, and borrowing time; a quota updating unit, which is used to update the resource quota of the original virtual queue after the quota borrowing and lending request is responded to, so as to meet the quota requirements of the offline task set submitted by the original virtual queue.

[0017] Optionally, the cluster resource management device based on virtual queues further includes: a quota maintaining unit, configured to maintain the borrowed quota amount within a specified quota range within a set time period when a quota borrowing request is initiated.

[0018] Optionally, the cluster resource management device based on virtual queues also includes: a reporting request receiving unit, used to receive an HTTP reporting request from each resource quota management unit before extracting the resource status information of the physical container cluster reported by each resource quota management unit, wherein the HTTP reporting request carries the resource status information of the physical container cluster, and the resource status information includes at least one of the following: CPU usage, memory usage, storage space usage data and network bandwidth status; a resource aggregation unit, used to aggregate the received resource status information and classify and store it according to the cluster identifier and tenant identifier of the physical container cluster; a resource adjustment unit, used to send the aggregated resource status information to the resource analysis model, and the resource analysis model outputs a quota update policy for each virtual queue and anomaly troubleshooting data and resource adjustment policy for each physical container cluster.

[0019] Optionally, the cluster resource management device based on virtual queues also includes: a resource status collection unit, used by the resource quota management unit to collect resource status information of each of the physical container clusters before selecting a target physical container cluster based on the resource status information of each of the physical container clusters; a resource usage evaluation unit, used by the resource quota management unit to evaluate the resource usage value of each physical container cluster based on the resource status information, and select a physical container cluster whose resource usage value is lower than a preset usage threshold as a candidate physical container cluster; a candidate cluster uploading unit, used by the resource quota management unit to upload the candidate physical container cluster to the offline task scheduler, wherein the offline task scheduler selects a cluster from the candidate physical container clusters when selecting to execute the next offline task to obtain the target physical container cluster.

[0020] Optionally, the cluster resource management device based on virtual queues also includes: a tenant priority division unit, used to prioritize the tenants represented by each virtual queue according to a preset priority rule, and obtain a queue priority division result, wherein the queue priority division result is used to determine the order of virtual queues for quota borrowing; a waiting queue establishment unit, used to establish a waiting queue for tenants with the same priority in the queue priority division result based on the order of submission time of each offline task, wherein when quota release is detected or quota is increased through a borrowing mechanism, resource quota is scheduled for each offline task in turn according to the waiting queue.

[0021] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned virtual queue-based cluster resource management methods.

[0022] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned virtual queue-based cluster resource management methods.

[0023] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the steps of any one of the above-mentioned virtual queue-based cluster resource management methods are implemented.

[0024] In the present disclosure, the current queue length and current quota usage of each virtual queue are obtained, and the tenant demand quota is confirmed based on the offline task set submitted by each virtual queue; when the current resource quota indicated by the current queue length of the virtual queue is lower than the tenant demand quota, a quota borrowing request is initiated, wherein the quota borrowing request is used to request free quota from other virtual queues that allow quota borrowing; after obtaining the free quota borrowed from other virtual queues, the resource status information of the physical container cluster reported by each resource quota management unit is extracted; according to the resource status information of each physical container cluster, a target physical container cluster is selected, and the offline task is sent to the target physical container cluster, and the target physical container cluster executes the offline task.

[0025] Based on the above-mentioned public content, quota management of multiple offline business tenants across clusters is realized based on virtual queues, which can dynamically adjust resource quotas according to the real-time needs of tenants and allow resource borrowing between tenants, that is, temporary transfer of quotas between tenants with surplus resources and tenants with tight resources, thereby improving the utilization of the entire system resources and reducing the situation of idle and insufficient resources, thereby solving the technical problem that the static quota management mechanism used in related technologies cannot handle the uneven resource demand in a multi-tenant environment, resulting in low resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0027] Figure 1 is a flow chart of an optional cluster resource management method based on a virtual queue according to an embodiment of the present invention;

[0028] Figure 2 is a schematic diagram of an optional cross-cluster multi-offline service tenant quota management system based on a virtual queue according to an embodiment of the present invention;

[0029] Figure 3 is a schematic diagram of an optional cluster resource management device based on a virtual queue according to an embodiment of the present invention;

[0030] Figure 4 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] To facilitate those skilled in the art to understand the present invention, some terms or nouns involved in the embodiments of the present invention are explained below:

[0034] Tenant, in enterprise-level big data and cloud computing environments, different departments, teams or business lines can be regarded as different tenants. Each tenant has its specific resource requirements, such as CPU, memory, etc., ensuring that each business unit can use resources according to its quota while avoiding resource contention and conflict.

[0035] Virtual Queue is an abstract data structure used to manage tenant offline tasks in the present invention. Each virtual queue corresponds to a specific tenant. The length of the queue represents the resource quota of the tenant in a multi-cluster environment. The task set in the virtual queue reflects the resource request of the tenant's offline business. Through the virtual queue, dynamic management of resource quota and cross-cluster task scheduling can be achieved. There is no direct correspondence between the virtual queue and the physical cluster, but they are associated through the offline task scheduler. The scheduler intelligently selects the appropriate physical cluster to execute the task based on the task request in the virtual queue and the resource status of the physical cluster.

[0036] K8s cluster, i.e. Kubernetes cluster, is a core component in cloud computing environment, which is used to automate the deployment, expansion and management of container applications. In the present invention, K8s cluster serves as a physical resource pool for actually executing offline tasks. Multiple K8s clusters together constitute a multi-cluster environment. The offline task scheduler dynamically selects a K8s cluster with sufficient resources to execute offline tasks based on resource requests in the virtual queue. Resource status information (such as CPU and memory usage) in the K8s cluster will be reported to the resource quota management unit to support dynamic resource management and task scheduling decisions.

[0037] It should be noted that the cluster resource management method based on virtual queues and the device thereof in the present disclosure can be used in the field of distributed computing technology. When implementing quota management of multiple offline business tenants across clusters based on virtual queues, it can also be used in any field outside the field of distributed computing technology. When implementing quota management of multiple offline business tenants across clusters based on virtual queues, the present disclosure does not limit the application field of the cluster resource management method based on virtual queues and the device thereof.

[0038] It should be noted that in the present disclosure, when collecting and analyzing customer information, corresponding operation entrances are provided for users to choose to agree or reject the automated decision results; if the user chooses to reject, the expert decision-making process is entered.

[0039] The following embodiments of the present invention can be applied to various systems / applications / devices for cluster resource management based on virtual queues. Compared with the existing offline business tenant quota management based on a multi-cluster management platform, which has problems such as inability to dynamically manage and inability to implement resource borrowing between tenants, the present invention implements a method for cross-cluster multi-offline business tenant quota management based on virtual queues, which overcomes the limitations of static configuration of resource quotas in the prior art, can dynamically adjust resource quotas according to the real-time needs of tenants, improves the flexibility and efficiency of resource allocation, and allows resource borrowing between tenants, that is, temporary transfer of quotas between tenants with surplus resources and tenants with tight resources, thereby improving the utilization rate of the entire system resources and reducing resource idleness and shortage.

[0040] Furthermore, the present invention can also ensure the fairness of resource allocation through the management mechanism of priority and waiting queue, and at the same time, reduce the task waiting time and improve the task execution efficiency through efficient task scheduling and resource selection logic. The present invention is not limited to a single cluster, and can realize cross-cluster resource management and task scheduling. For large data centers or cloud platforms with multiple physical container clusters, it can greatly improve the efficiency of resource management and task execution.

[0041] The present invention is described in detail below in conjunction with various embodiments.

[0042] Embodiment 1

[0043] According to an embodiment of the present invention, an embodiment of a cluster resource management method based on a virtual queue is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] It should be noted that the cluster resource management method based on virtual queues provided in the embodiment of the present invention is applicable to large-scale distributed computing environments, especially in cloud platforms and big data centers, where there are multiple physical container clusters and multiple tenants (departments, teams or business lines) that need to share and dynamically allocate resources. For example, applicable scenarios include: 1. Multi-tenant environment, when a cloud platform or data center serves multiple independent business units, each tenant may have different resource requirements and priorities. The technical solution of the present application can help to achieve dynamic quota adjustment of resources to ensure that the resource requirements of each tenant are met fairly and efficiently. 2. Applicable to multi-cluster mixed deployment scenarios: In a mixed deployment environment with multiple physical container clusters (for example, K8s clusters), resource utilization and task execution efficiency are key considerations. The technical solution of the present application can perform intelligent scheduling and quota management of resources across clusters through the mechanism of virtual queues, thereby improving the overall resource utilization and execution efficiency of offline tasks. 3. Applicable to dynamic resource demand scenarios: In scenarios where resource requirements change frequently, such as data analysis and the execution of deep learning tasks, the resource requirements of tasks may increase or decrease sharply in a short period of time. The technical solution of the present application can quickly respond to changes in resource demand and avoid resource waste or task waiting through virtual queues and resource borrowing mechanisms.

[0045] An embodiment of the present invention provides a cluster resource management method based on virtual queues, which is applied to an offline task scheduler. The offline task scheduler is connected to multiple virtual queues and multiple physical container clusters, and manages the corresponding physical container clusters through a resource quota management unit. Each virtual queue represents a tenant, and the queue length indicates the tenant's resource quota.

[0046] Figure 1 is a flow chart of an optional cluster resource management method based on a virtual queue according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0047] Step S101, obtaining the current queue length and current quota usage of each virtual queue, and confirming the tenant demand quota based on the offline task set submitted by each virtual queue.

[0048] It should be noted that step S101 is the basis of the entire cross-cluster resource management process, which involves real-time monitoring of the virtual queue status and accurate assessment of tenant resource requirements. The offline task scheduler obtains the current queue length and resource usage from each virtual queue regularly or when triggered by a specific event. The queue length reflects the tenant's current resource quota, while the resource usage describes the amount of resources currently being used or occupied by the tenant, including dimensions such as CPU and memory. These data are usually collected and reported by a resource quota management unit (defined as an agent in this embodiment) to ensure the real-time and accuracy of the information.

[0049] The offline task scheduler further analyzes the offline task sets submitted in each virtual queue to confirm the resource requirements of the tenant. Each offline task has its own specific resource requirements, such as the number of CPU cores, memory size, etc. By summarizing the requirements of these tasks, the scheduler can derive the overall demand quota of the current tenant, which is the basis for comparison with the current queue length.

[0050] After analyzing the current set of offline tasks, the offline task scheduler can accurately determine the tenant's resource requirements, including the sum of resources required by tasks to be executed and waiting tasks. If the demand quota exceeds the resource quota indicated by the current queue length, this indicates that the resources are insufficient and further processing is required.

[0051] Step S102: when the current resource quota indicated by the current queue length of the virtual queue is lower than the tenant demand quota, a quota borrowing request is initiated, wherein the quota borrowing request is used to request free quota from other virtual queues that allow quota borrowing.

[0052] When the offline task scheduler confirms the resource demand quota of the tenant in step S101, if it is found that the existing resource quota is insufficient to support these demands, that is, the resource quota indicated by the current queue length is lower than the demand quota, this will trigger a resource shortage warning.

[0053] In the case of insufficient resources, the offline task scheduler will query the quota usage of all virtual queues in the system and identify virtual queues that are allowed to borrow resources. The offline task scheduler will select the most appropriate resources from these queues for borrowing based on real-time resource requirements and borrowing policies. The quota borrowing request includes detailed information about the source tenant (the party with insufficient resources) and the target tenant (the party with surplus resources), as well as the requested amount of resources.

[0054] Optionally, the step of initiating a quota borrowing request includes: querying the current quota usage of other virtual queues that are allowed to borrow; based on the current quota usage of other virtual queues, analyzing the quota free amount and the borrowing threshold of each other virtual queue, and selecting other virtual queues with the lowest quota free amount and lower than the borrowing threshold as the target virtual queue for quota borrowing; initiating a quota borrowing request to the tenant of the target virtual queue.

[0055] Among them, when querying the current quota usage of other virtual queues that are allowed to borrow, the offline task scheduler will start the quota borrowing mechanism. First, it is necessary to identify which virtual queues in the system can be used as the target queues for quota borrowing. The offline task scheduler sends a request to the resource quota management unit agent to obtain the real-time quota usage of all virtual queues, including CPU usage, memory occupancy, etc. This query process is not limited to the current cluster, but spans multiple physical clusters to ensure that the most comprehensive resource status information is obtained. Then, the quota free amount and the borrowing threshold of each other virtual queue are analyzed. Specifically, the offline task scheduler will further analyze the quota free amount of each virtual queue based on the obtained quota usage, that is, the difference between the current quota and the actual resource usage. At the same time, the scheduler will also consider the borrowing threshold of each queue, which is a preset quota usage level. Only queues below this level are considered candidates for resource borrowing. The scheduler screens out queues suitable for quota borrowing by comparing the quota free amount of each queue with the borrowing threshold.

[0056] After obtaining the quota free amount and borrowing threshold of each virtual queue, the offline task scheduler gives priority to the queue with the lowest quota free amount as the borrowing target. This is because such a queue can still maintain a high resource utilization rate after meeting the urgent needs of other tenants, avoiding excessive idleness of resources. At the same time, in order to ensure that resource borrowing does not affect the normal operation of the target queue itself, the scheduler will only borrow when the quota free amount of the target queue is lower than the borrowing threshold, so as to maintain a safe resource usage margin.

[0057] Once the target virtual queue is determined, the offline task scheduler will initiate a quota loan request to the tenant of the target virtual queue. The quota loan includes the original tenant's resource demand details, the amount of resources to be borrowed, and the effective time of the loan request. The target tenant decides whether to agree to the loan request based on its own resource status and strategy. If it agrees, both parties will carry out the resource loan operation.

[0058] Optionally, after initiating a quota borrowing request, it also includes: recording detailed information of the quota borrowing, wherein the detailed information of the quota borrowing includes: the quota borrowing amount, the quota lending amount, the tenant information of both parties to the borrowing, and the borrowing time; after the quota borrowing request is responded to, updating the resource quota of the original virtual queue to meet the quota requirements of the offline task set submitted by the original virtual queue.

[0059] After each quota borrowing operation is completed, the offline task scheduler will record detailed borrowing information, including the amount of quota borrowed, the amount of quota loaned, the tenant information of both parties of the borrowing, and the borrowing time, etc. The recording of this data is very important for subsequent historical queries, resource borrowing, and resource balance adjustment between tenants.

[0060] When the offline task scheduler receives the borrowing response from the target tenant and confirms that the quota borrowing is successful, it will update the resource quota of the original virtual queue, that is, increase the resource quota of the original virtual queue to meet the quota requirements of the offline task set submitted by the original virtual queue, ensuring that the task can be executed smoothly.

[0061] In order to ensure the fairness and stability of resource borrowing, a specific time period will be set in the system, during which the total amount of quota borrowing will be maintained within a certain range. Optionally, when initiating a quota borrowing request, it also includes: keeping the borrowed quota amount within the specified quota range within the set time period. The system will not only record the detailed information of a single quota borrowing, but also continuously monitor the overall trend of quota borrowing to ensure that within a certain period of time, the quota borrowing amount of each tenant will not be too high or too low, thereby maintaining a dynamic balance of resource borrowing between tenants and avoiding long-term unfair resource allocation.

[0062] After the quota loan request is sent to the target tenant, it will be processed according to the target tenant's resource usage, preset loan rules and the overall quota balance principle. If the target tenant confirms the loan, the corresponding resources will be transferred from its quota to the tenant with insufficient resources. At the same time, the offline task scheduler will record the detailed information of the loan, including the borrowed amount, the loaned amount, the tenant information of both parties of the loan and the loan time, in order to maintain the resource balance between tenants and the subsequent resource loan mechanism.

[0063] Step S103: after obtaining the idle quotas borrowed from other virtual queues, extract the resource status information of the physical container cluster reported by each resource quota management unit.

[0064] After the offline task scheduler successfully obtains additional free quotas from other virtual queues through the quota borrowing mechanism, the next step is to allocate these quotas to the physical container clusters to run offline tasks. This process requires intelligent decision-making based on the real-time resource status information of each physical container cluster to achieve optimal resource utilization. Extracting resource status information: The resource quota management unit (usually referring to the resource quota management agent) will regularly collect and report the resource status information of each physical container cluster (such as K8s cluster), including but not limited to key indicators such as CPU usage, memory utilization, storage space, and network bandwidth. Optionally, when obtaining resource status information, it can be obtained by accessing the Kubernetes API server, which can accurately reflect the current resource load of each cluster. After initiating quota borrowing and successfully obtaining resources, the offline task scheduler will immediately extract the latest resource status information to prepare for selecting the target physical container cluster.

[0065] Optionally, before extracting the resource status information of the physical container cluster reported by each resource quota management unit, it also includes: receiving HTTP reporting requests from each resource quota management unit, wherein the HTTP reporting request carries the resource status information of the physical container cluster, and the resource status information includes at least one of the following: CPU usage, memory usage, storage space usage data, and network bandwidth status; summarizing the received resource status information and classifying and storing it according to the cluster identifier and tenant identifier of the physical container cluster; sending the summarized resource status information to the resource analysis model, and the resource analysis model outputs a quota update policy for each virtual queue and anomaly troubleshooting data and resource adjustment policy for each physical container cluster.

[0066] The resource quota management unit reports the resource status information of the physical container cluster to the offline task scheduler through HTTP requests periodically or when triggered by specific events. The resource status information includes but is not limited to CPU usage, memory usage, storage space usage data and network bandwidth status, which are key indicators for evaluating the availability and health of cluster resources. After receiving the resource status information, the offline task scheduler needs to classify and summarize the data. The summary process will be carried out based on the cluster ID of each physical container cluster and the tenant ID represented by the virtual queue, clearly showing the resource usage of each cluster and each tenant, and providing a basis for subsequent resource analysis and scheduling decisions.

[0067] After the resource status information is summarized, it will be sent to the resource analysis model, which is a machine learning-based algorithm or a preset logical judgment process. Its main function is to analyze resource usage trends, identify potential abnormalities, and propose quota update strategies and resource adjustment suggestions based on current demand and resource status. For virtual queues, it may recommend increasing or decreasing quotas to adapt to changes in task requirements; for physical container clusters, it may prompt abnormal data that needs to be checked, or recommend specific resource adjustment measures, such as increasing memory, optimizing network configuration, etc.

[0068] Optionally, the cluster resource management method based on virtual queues also includes: according to pre-set priority rules, prioritizing the tenants represented by each virtual queue to obtain a queue priority division result, wherein the queue priority division result is used to determine the order of virtual queues for quota borrowing; for tenants with the same priority in the queue priority division result, a waiting queue is established based on the order of submission time of each offline task, wherein when it is detected that a quota is released or a quota is increased through a borrowing mechanism, resource quotas are scheduled for each offline task in turn according to the waiting queue.

[0069] In a multi-tenant environment, different tenants or business lines may have different resource priority requirements. The offline task scheduler will prioritize each virtual queue according to the pre-set priority rules to form a queue priority list. This list will serve as the basis for quota borrowing decisions. The scheduler will give priority to the resource requirements of tenants with higher priority to ensure the resource supply of key businesses.

[0070] When multiple tenants have the same priority, the offline task scheduler will further establish a waiting queue based on the submission time of the offline tasks. This means that tenants with earlier task submission times will be given priority in obtaining resource quotas, which helps to handle resource requests fairly and avoid the situation where all tenant tasks cannot be executed in time due to insufficient resources.

[0071] When the system detects that a tenant has actively released part of the resource quota, or successfully obtained additional resources from other virtual queues through the resource borrowing mechanism, the offline task scheduler will allocate resource quotas to each offline task in turn according to the waiting queue established above, ensuring the dynamic allocation and fairness of resources and giving priority to the needs of the first-come-first-served tasks. This can not only improve the overall resource utilization and task processing efficiency, but also achieve dynamic balance and fairness in resource allocation among tenants, and is an important part of building a high-performance, highly available multi-tenant multi-cluster resource management platform.

[0072] Step S104 : selecting a target physical container cluster according to the resource status information of each physical container cluster, sending the offline task to the target physical container cluster, and having the target physical container cluster execute the offline task.

[0073] Based on the resource status information of each physical container cluster extracted, the offline task scheduler uses a specific scheduling algorithm to select the optimal physical container cluster as the target cluster for executing offline tasks. This selection process takes into account multiple factors, such as the resource load of the cluster, the resource utilization of the cluster, the priority and resource requirements of the task, etc., to ensure that the task can be assigned to a cluster with sufficient resources, low load and fast response.

[0074] After determining the target physical container cluster, the scheduler will extract the offline tasks to be executed from the original tenant's virtual queue and send the tasks to the target cluster through HTTP requests and other methods. After receiving the task request, the target cluster will allocate resources according to its resource situation and task scheduling strategy, create and run task containers, and complete the execution of offline tasks. The target physical container cluster uses its allocated resources (including resources borrowed from other virtual queues) to execute offline tasks according to the instructions of the scheduler. In a Kubernetes cluster, this involves creating or adjusting Pods (container groups), ensuring that each task has sufficient resources to run, and executing multiple offline tasks in parallel as much as possible to improve execution efficiency if cluster resources allow.

[0075] The technical solution of the embodiment of the present invention not only realizes the dynamic borrowing of resources between tenants, but also realizes the cross-cluster intelligent scheduling of resources, ensuring the efficient execution of offline tasks and the full utilization of resources.

[0076] Through the above steps, the current queue length and current quota usage of each virtual queue can be obtained, and the tenant demand quota can be confirmed based on the offline task set submitted by each virtual queue; when the current resource quota indicated by the current queue length of the virtual queue is lower than the tenant demand quota, a quota borrowing request is initiated, wherein the quota borrowing request is used to request free quota from other virtual queues that allow quota borrowing; after obtaining the free quota borrowed from other virtual queues, the resource status information of the physical container cluster reported by each resource quota management unit is extracted; according to the resource status information of each physical container cluster, the target physical container cluster is selected, and the offline task is sent to the target physical container cluster, and the target physical container cluster executes the offline task. In this embodiment, quota management of multiple offline business tenants across clusters is implemented based on virtual queues, and resource quotas can be dynamically adjusted according to the real-time needs of tenants. At the same time, resource borrowing between tenants is allowed, that is, temporary transfer of quotas between tenants with surplus resources and tenants with tight resources, thereby improving the utilization of the entire system resources and reducing resource idleness and shortage, thereby solving the technical problem that the static quota management mechanism used in related technologies cannot handle the uneven resource demand in a multi-tenant environment, resulting in low resource utilization.

[0077] Optionally, before selecting a target physical container cluster based on the resource status information of each physical container cluster, the method further includes: collecting the resource status information of each physical container cluster by a resource quota management unit; evaluating the resource usage level value of each physical container cluster based on the resource status information by the resource quota management unit, and selecting a physical container cluster whose resource usage level value is lower than a preset usage level threshold as a candidate physical container cluster; uploading the candidate physical container cluster to the offline task scheduler by the resource quota management unit, wherein the offline task scheduler selects a cluster from the candidate physical container clusters when selecting to execute the next offline task, and obtains the target physical container cluster.

[0078] The resource quota management unit is usually an agent deployed in each physical container cluster, responsible for real-time monitoring and collecting resource status information of the physical container cluster. This includes but is not limited to key indicators such as CPU usage, memory usage, storage space occupancy, and network bandwidth usage. After collecting the resource status information, the resource quota management unit will further evaluate this information to calculate the resource usage value of each physical container cluster. This evaluation process may involve a comprehensive analysis of multiple resource usage indicators. For example, by calculating the average CPU and memory usage, as well as the storage space and network bandwidth usage, a value that reflects the overall resource tension is obtained. The lower the resource usage value, the more abundant the cluster resources are, and it is more suitable as a target cluster for running additional offline tasks.

[0079] The resource quota management unit will screen out clusters with looser resource usage as candidate physical container clusters based on the assessed resource usage values ​​and the pre-set usage thresholds. This threshold is set based on the overall resource strategy and maintenance goals of the system. Clusters with resource usage below a certain percentage are usually selected to ensure that these clusters have sufficient resources to cope with additional offline task loads. Once the candidate physical container clusters are determined, the resource quota management unit will upload the information of these clusters, including cluster identification, resource usage, etc., to the offline task scheduler through HTTP requests or other communication mechanisms. The scheduler will make the next resource scheduling decision based on the received candidate cluster information, combined with the resource requirements across clusters and the quotas of virtual queues.

[0080] After receiving the candidate physical container cluster information, the offline task scheduler will execute the resource scheduling strategy and select one or more of these candidate clusters as the target physical container cluster to run the next offline task. The scheduler's decision may also be based on multiple factors such as task priority, resource requirements, cluster location, network latency, etc. to ensure that the task can be executed in the most suitable environment. Once the target cluster is selected, the scheduler will immediately issue a task execution instruction, and the target cluster will take over the execution of the offline task until the task is completed.

[0081] Through the above implementation plan, not only dynamic quota allocation of resources and resource borrowing between tenants are realized, but also the resource scheduling of the physical container cluster is further optimized, ensuring that offline tasks can run in the cluster with the most abundant resources and the highest execution efficiency, thereby improving the overall system resource utilization and the reliability of task execution.

[0082] Another optional specific implementation is described in detail below.

[0083] Figure 2 is a schematic diagram of an optional cross-cluster multi-offline service tenant quota management system based on a virtual queue according to an embodiment of the present invention, such as Figure 2 As shown, including:

[0084] Offline task parties, multiple virtual queues ( Figure 2 The figure shows queue 1, queue 2 and queue 3), task scheduler and multiple K8S clusters. Each K8S cluster corresponds to a resource quota management unit Agent. The following describes these modules separately.

[0085] 1. Virtual queue.

[0086] Each virtual queue represents a tenant, and the queue length dynamically reflects the tenant's quota, including CPU and memory dimensions.

[0087] There is no direct correspondence between virtual queues and physical cluster resources. The tasks to be processed in the virtual queues are uniformly submitted to the task scheduler, which selects a physical cluster with sufficient resources to run the queue tasks.

[0088] Tenants' resource requests and allocation mechanisms are intelligently processed by the task scheduler rather than limited to manual approval, which improves the efficiency and flexibility of resource allocation.

[0089] Tenant quotas (queue lengths) are pre-configured by the business end and can be adjusted dynamically through the task scheduler.

[0090] When assigning batch tasks, the task scheduler will score them based on the resource idleness of the physical cluster. The idler clusters have higher scores and are selected first to ensure balanced resource allocation.

[0091] When tenant resource requests exceed the quota, there is a priority division mechanism. For tenants with the same priority, they are sorted according to the task submission time to form a waiting queue.

[0092] When dynamically adjusting quotas, the amount of borrowed and lent resources is calculated to maintain a balance between the amount of borrowed and lent resources within a specified time period to ensure fairness and stability.

[0093] Tenants can set whether to enable resource borrowing and the borrowing threshold by themselves, realizing flexible sharing of resources.

[0094] Tenants in a virtual queue can transfer quotas when the queue is idle and borrow quotas when the queue is busy. There are restrictions on lending and borrowing quotas, and the overall quota remains unchanged.

[0095] When the offline tasks of a virtual queue are full, the task scheduler queries the resource usage of the queues that are allowed to be borrowed, selects the queue with the lowest idle resources and below the borrowing threshold for resource borrowing, and records the amount of borrowed resources to maintain resource balance among tenants.

[0096] 2. Task scheduler.

[0097] The task scheduler interacts with the virtual queue through the HTTP interface, accepts task delivery requests, and intelligently selects the optimal physical cluster to allocate batch tasks based on the K8S cluster resource usage status information provided by the resource quota management unit Agent.

[0098] In tenant quota management, the task scheduler plays a core role. It not only maintains the resource quota usage of the virtual queue, but also serves as a bridge between the upper-level virtual queue and the lower-level K8s cluster, dynamically adjusting the task allocation strategy according to the current resource status and tenant needs.

[0099] 3. Resource quota management unit Agent.

[0100] The Agent is responsible for real-time monitoring and statistics of resource usage of different tenants in each K8S cluster. It obtains specific values ​​by accessing the Kubernetes API server (kube-apiserver) and distinguishes resource usage of different tenants through namespace identifiers.

[0101] The agent reports the collected resource status information to the task scheduler instead of directly reporting it to the virtual queue, ensuring that the scheduler can make the best decision based on real-time resource data.

[0102] Compared with the offline business tenant quota management of the existing multi-cluster management platform, the implementation method of the present invention realizes dynamic resource management and cross-tenant resource sharing by introducing virtual queues and intelligent task scheduling mechanisms, improves resource utilization and task execution efficiency in a multi-cluster environment, and solves the problems of static resource allocation and resource isolation between tenants.

[0103] The following is a detailed description in conjunction with another embodiment.

[0104] Embodiment 2

[0105] A cluster resource management device based on a virtual queue provided in this embodiment includes multiple implementation units, each of which corresponds to each implementation step in the above-mentioned embodiment 1. Its specific implementation methods and beneficial effects can refer to the above-mentioned method embodiments and will not be repeated here.

[0106] A cluster resource management device based on a virtual queue is provided in an embodiment of the present invention and is applied to an offline task scheduler. The offline task scheduler is connected to multiple virtual queues and multiple physical container clusters, and manages the corresponding physical container clusters through a resource quota management unit. Each virtual queue represents a tenant, and the queue length indicates the tenant's resource quota.

[0107] Figure 3 is a schematic diagram of an optional cluster resource management device based on a virtual queue according to an embodiment of the present invention, such as Figure 3 As shown, the cluster resource management device based on virtual queues may include: a demand quota determining unit 31 , a quota borrowing request unit 32 , a cluster resource status extracting unit 33 , and a task issuing unit 34 .

[0108] The demand quota determining unit 31 is used to obtain the current queue length and current quota usage of each virtual queue, and confirm the tenant demand quota based on the offline task set submitted by each virtual queue.

[0109] The quota borrowing request unit 32 is used to initiate a quota borrowing request when the current resource quota indicated by the current queue length of the virtual queue is lower than the tenant demand quota, wherein the quota borrowing request is used to request free quota from other virtual queues that allow quota borrowing.

[0110] The cluster resource status extraction unit 33 is used to extract the resource status information of the physical container cluster reported by each resource quota management unit after obtaining the idle quota borrowed by other virtual queues.

[0111] The task issuing unit 34 is used to select a target physical container cluster according to the resource status information of each physical container cluster, and issue the offline task to the target physical container cluster, so that the target physical container cluster executes the offline task.

[0112] The above-mentioned cluster resource management device based on virtual queues can obtain the current queue length and current quota usage of each virtual queue through the demand quota determination unit 31, and confirm the tenant demand quota based on the offline task set submitted by each virtual queue, and initiate a quota borrowing request through the quota borrowing request unit 32 when the current resource quota indicated by the current queue length of the virtual queue is lower than the tenant demand quota, wherein the quota borrowing request is used to request idle quota from other virtual queues that allow quota borrowing, and after obtaining the idle quota borrowed from other virtual queues, the cluster resource status extraction unit 33 extracts the resource status information of the physical container cluster reported by each resource quota management unit, and selects the target physical container cluster according to the resource status information of each physical container cluster through the task issuing unit 34, and issues the offline task to the target physical container cluster, and the target physical container cluster executes the offline task. In this embodiment, quota management of multiple offline business tenants across clusters is implemented based on virtual queues, and resource quotas can be dynamically adjusted according to the real-time needs of tenants. At the same time, resource borrowing between tenants is allowed, that is, temporary transfer of quotas between tenants with surplus resources and tenants with tight resources, thereby improving the utilization of the entire system resources and reducing resource idleness and shortage, thereby solving the technical problem that the static quota management mechanism used in related technologies cannot handle the uneven resource demand in a multi-tenant environment, resulting in low resource utilization.

[0113] Optionally, the quota borrowing request unit includes: a quota query module, used to query the current quota usage of other virtual queues that are allowed to borrow; a quota analysis module, used to analyze the quota free amount and borrowing threshold of each other virtual queue based on the current quota usage of other virtual queues, and select other virtual queues with the lowest quota free amount and lower than the borrowing threshold as the target virtual queue for quota borrowing; a quota borrowing request initiation module, used to initiate a quota borrowing request to the tenant of the target virtual queue.

[0114] Optionally, the cluster resource management device based on virtual queues also includes: a quota borrowing and lending recording unit, which is used to record detailed information of quota borrowing after initiating a quota borrowing and lending request, wherein the detailed information of quota borrowing and lending includes: quota borrowing amount, quota lending amount, tenant information of both parties of the borrowing and lending, and borrowing time; a quota updating unit, which is used to update the resource quota of the original virtual queue after the quota borrowing and lending request is responded to, so as to meet the quota requirements of the offline task set submitted by the original virtual queue.

[0115] Optionally, the cluster resource management device based on virtual queues further includes: a quota maintaining unit, configured to maintain the borrowed quota amount within a specified quota range within a set time period when a quota borrowing request is initiated.

[0116] Optionally, the cluster resource management device based on virtual queues also includes: a reporting request receiving unit, used to receive an HTTP reporting request from each resource quota management unit before extracting the resource status information of the physical container cluster reported by each resource quota management unit, wherein the HTTP reporting request carries the resource status information of the physical container cluster, and the resource status information includes at least one of the following: CPU usage, memory usage, storage space usage data, and network bandwidth status; a resource aggregation unit, used to aggregate the received resource status information and classify and store it according to the cluster identifier and tenant identifier of the physical container cluster; a resource adjustment unit, used to send the aggregated resource status information to the resource analysis model, and the resource analysis model outputs a quota update policy for each virtual queue and anomaly troubleshooting data and resource adjustment policy for each physical container cluster.

[0117] Optionally, the cluster resource management device based on virtual queues also includes: a resource status collection unit, which is used by the resource quota management unit to collect resource status information of each physical container cluster before selecting a target physical container cluster based on the resource status information of each physical container cluster; a resource usage evaluation unit, which is used by the resource quota management unit to evaluate the resource usage value of each physical container cluster based on the resource status information, and select a physical container cluster whose resource usage value is lower than a preset usage threshold as a candidate physical container cluster; and a candidate cluster uploading unit, which is used by the resource quota management unit to upload the candidate physical container cluster to the offline task scheduler, wherein the offline task scheduler selects a cluster from the candidate physical container clusters when selecting to execute the next offline task to obtain the target physical container cluster.

[0118] Optionally, the cluster resource management device based on virtual queues also includes: a tenant priority division unit, which is used to prioritize the tenants represented by each virtual queue according to a preset priority rule to obtain a queue priority division result, wherein the queue priority division result is used to determine the order of virtual queues for quota borrowing; a waiting queue establishment unit, which is used to establish a waiting queue for tenants with the same priority in the queue priority division result based on the order of submission time of each offline task, wherein when it is detected that a quota is released or a quota is increased through a borrowing mechanism, resource quotas are scheduled for each offline task in turn according to the waiting queue.

[0119] The above-mentioned cluster resource management device based on virtual queues may also include a processor and a memory. The above-mentioned demand quota determination unit 31, quota borrowing request unit 32, cluster resource status extraction unit 33, task issuing unit 34, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0120] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the virtual queue-based cross-cluster multi-offline service tenant quota management can be achieved by adjusting kernel parameters.

[0121] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0122] Embodiment 3

[0123] An embodiment of the present application may provide an electronic device, Figure 4 is a structural block diagram of an electronic device according to an embodiment of the present application. Figure 4 As shown, the electronic device may include: one or more ( Figure 4 (only one is shown) processor 402, memory 404, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0124] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the cluster resource management method and device based on virtual queues in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned cluster resource management method based on virtual queues. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0125] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain the current queue length and current quota usage of each virtual queue, and confirm the tenant demand quota based on the offline task set submitted by each virtual queue; when the current resource quota indicated by the current queue length of the virtual queue is lower than the tenant demand quota, initiate a quota borrowing request, wherein the quota borrowing request is used to request free quota from other virtual queues that allow quota borrowing; after obtaining the free quota borrowed from other virtual queues, extract the resource status information of the physical container cluster reported by each resource quota management unit; according to the resource status information of each physical container cluster, select the target physical container cluster, send the offline task to the target physical container cluster, and let the target physical container cluster execute the offline task.

[0126] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: query the current quota usage of other virtual queues that are allowed to borrow; based on the current quota usage of other virtual queues, analyze the quota free amount and borrowing threshold of each other virtual queue, and select other virtual queues with the lowest quota free amount and lower than the borrowing threshold as the target virtual queue for quota borrowing; initiate a quota borrowing request to the tenant of the target virtual queue.

[0127] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: after initiating a quota borrowing request, record the detailed information of the quota borrowing, wherein the detailed information of the quota borrowing includes: the quota borrowing amount, the quota lending amount, the tenant information of the borrowing and lending parties, and the borrowing time; after the quota borrowing request is responded to, update the resource quota of the original virtual queue to meet the quota requirements of the offline task set submitted by the original virtual queue.

[0128] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: within a set time period, keep the borrowing quota amount within a specified quota range.

[0129] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: before extracting the resource status information of the physical container cluster reported by each resource quota management unit, receive an HTTP reporting request from each resource quota management unit, wherein the HTTP reporting request carries the resource status information of the physical container cluster, and the resource status information includes at least one of the following: CPU usage, memory usage, storage space usage data, and network bandwidth status; summarize the received resource status information and classify and store it according to the cluster identifier and tenant identifier of the physical container cluster; send the summarized resource status information to the resource analysis model, and the resource analysis model outputs the quota update strategy for each virtual queue and the abnormality troubleshooting data and resource adjustment strategy for each physical container cluster.

[0130] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: before selecting the target physical container cluster based on the resource status information of each physical container cluster, the resource quota management unit collects the resource status information of each physical container cluster; the resource quota management unit evaluates the resource usage value of each physical container cluster based on the resource status information, and selects the physical container cluster whose resource usage value is lower than the preset usage threshold as the candidate physical container cluster; the resource quota management unit uploads the candidate physical container cluster to the offline task scheduler, wherein the offline task scheduler selects a cluster from the candidate physical container clusters when selecting to execute the next offline task to obtain the target physical container cluster.

[0131] The processor can call the information and application programs stored in the memory through the transmission device to perform the following steps: according to the preset priority rules, the tenants represented by each virtual queue are prioritized to obtain the queue priority division results, wherein the queue priority division results are used to determine the order of the virtual queues for quota borrowing; for tenants with the same priority in the queue priority division results, a waiting queue is established based on the order of submission time of each offline task, wherein when it is detected that the quota is released or the quota is increased through the borrowing mechanism, the resource quota is scheduled for each offline task in turn according to the waiting queue.

[0132] By adopting the embodiment of the present application, a method for quota management of multiple offline business tenants across clusters is implemented based on virtual queues, which overcomes the limitations of static configuration of resource quotas in the prior art, can dynamically adjust resource quotas according to the real-time needs of tenants, and improve the flexibility and efficiency of resource allocation. At the same time, it allows resource borrowing between tenants, that is, temporary transfer of quotas between tenants with surplus resources and tenants with tight resources, thereby improving the utilization rate of the entire system resources and reducing resource idleness and shortage.

[0133] It can be understood by those skilled in the art that Figure 4 The structure shown is for illustration only, and the electronic device may also be a terminal device such as a smart phone, a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, etc. Figure 4 The structure of the electronic device is not limited. Figure 4 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 4 Different configurations are shown.

[0134] A person skilled in the art may understand that all or part of the steps in the various virtual queue-based cluster resource management methods of the above-mentioned embodiments may be completed by instructing the hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0135] Embodiment 4

[0136] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the cluster resource management method based on virtual queues provided in the first embodiment.

[0137] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the virtual queue-based cluster resource management methods in the above-mentioned embodiment 1.

[0138] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0139] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the cluster resource management method based on a virtual queue described in each embodiment of the present application.

[0140] The present application also provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the cluster resource management method based on the virtual queue described in each embodiment of the present application are implemented.

[0141] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0142] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0143] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0144] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0145] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0146] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0147] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A cluster resource management method based on virtual queues, characterized in that: Applied to an offline task scheduler, the offline task scheduler is connected to multiple virtual queues and multiple physical container clusters, and manages the corresponding physical container clusters through a resource quota management unit. Each virtual queue represents a tenant, and the queue length indicates the resource quota of the tenant. The cluster resource management method based on the virtual queue includes: Obtain the current queue length and current quota usage of each virtual queue, and confirm the tenant demand quota based on the offline task set submitted by each virtual queue; In the case where the current resource quota indicated by the current queue length of the virtual queue is lower than the tenant demand quota, initiating a quota borrowing request, wherein the quota borrowing request is used to request free quota from other virtual queues that allow quota borrowing; After obtaining the idle quotas borrowed from other virtual queues, extracting the resource status information of the physical container cluster reported by each resource quota management unit; A target physical container cluster is selected according to the resource status information of each of the physical container clusters, and the offline task is sent to the target physical container cluster, so that the target physical container cluster executes the offline task.

2. The cluster resource management method according to claim 1, characterized in that: The steps to initiate a quota borrowing request include: Query the current quota usage of other virtual queues that are allowed to borrow; Based on the current quota usage of the other virtual queues, analyze the quota free amount and the borrowing threshold of each of the other virtual queues, and select the other virtual queue with the lowest quota free amount and lower than the borrowing threshold as the target virtual queue for quota borrowing; A quota borrowing request is initiated to the tenant of the target virtual queue.

3. The cluster resource management method according to claim 2, characterized in that: After initiating a quota borrowing request, it also includes: Recording detailed information of quota lending, wherein the detailed information of quota lending includes: quota borrowing amount, quota lending amount, tenant information of both parties of the lending and borrowing, and lending time; After the quota borrowing request is responded to, the resource quota of the original virtual queue is updated to meet the quota requirement of the offline task set submitted by the original virtual queue.

4. The cluster resource management method according to claim 2, characterized in that: When initiating a quota borrowing request, it also includes: Within the set time period, keep the borrowing quota amount within the specified quota range.

5. The cluster resource management method according to claim 1, characterized in that: Before extracting the resource status information of the physical container cluster reported by each resource quota management unit, the method further includes: Receiving an HTTP reporting request from each of the resource quota management units, wherein the HTTP reporting request carries resource status information of the physical container cluster, and the resource status information includes at least one of the following: CPU usage, memory usage, storage space usage data, and network bandwidth status; The received resource status information is aggregated and classified and stored according to the cluster identifier and tenant identifier of the physical container cluster; The aggregated resource status information is sent to a resource analysis model, which outputs a quota update strategy for each virtual queue and abnormality troubleshooting data and a resource adjustment strategy for each physical container cluster.

6. The cluster resource management method according to claim 1, characterized in that: Before selecting a target physical container cluster according to the resource status information of each physical container cluster, the method further includes: The resource quota management unit collects resource status information of each of the physical container clusters; The resource quota management unit evaluates the resource usage level of each physical container cluster based on the resource status information, and selects a physical container cluster whose resource usage level is lower than a preset usage level threshold as a candidate physical container cluster; The resource quota management unit uploads the candidate physical container cluster to the offline task scheduler, wherein the offline task scheduler selects a cluster from the candidate physical container clusters to obtain the target physical container cluster when selecting to execute the next offline task.

7. The cluster resource management method according to claim 1, characterized in that: Also includes: According to a preset priority rule, the tenants represented by each virtual queue are prioritized to obtain a queue priority division result, wherein the queue priority division result is used to determine the order of the virtual queues for quota borrowing; For tenants with the same priority in the queue priority division result, a waiting queue is established based on the chronological order of submission time of each offline task, wherein when quota release is detected or quota is increased through the borrowing mechanism, resource quota is scheduled for each offline task in turn according to the waiting queue.

8. A cluster resource management device based on a virtual queue, characterized in that: Applied to an offline task scheduler, the offline task scheduler is connected to multiple virtual queues and multiple physical container clusters, and manages the corresponding physical container clusters through a resource quota management unit, each of the virtual queues represents a tenant, and the queue length indicates the resource quota of the tenant. The cluster resource management device based on the virtual queue includes: A demand quota determination unit, configured to obtain the current queue length and current quota usage of each virtual queue, and to confirm the tenant demand quota based on the offline task set submitted by each virtual queue; A quota borrowing request unit, configured to initiate a quota borrowing request when the current resource quota indicated by the current queue length of the virtual queue is lower than the tenant demand quota, wherein the quota borrowing request is used to request free quota from other virtual queues that allow quota borrowing; A cluster resource status extraction unit, configured to extract resource status information of the physical container cluster reported by each resource quota management unit after obtaining the idle quota borrowed by other virtual queues; The task issuing unit is used to select a target physical container cluster according to the resource status information of each physical container cluster, and issue the offline task to the target physical container cluster so that the target physical container cluster executes the offline task.

9. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the virtual queue-based cluster resource management method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the cluster resource management method based on virtual queues described in any one of claims 1 to 7 are implemented.

Images