Resource management method, device, equipment and storage medium

Abstract

The present application discloses a resource management method, apparatus, equipment and storage medium, which belong to the field of geophysical exploration technology. The resource management scheme provided by the embodiment of the present application can realize efficient resource management of jobs. Among them, the scheme can dynamically allocate resources to non-running jobs and elastic jobs, thereby ensuring resource utilization when resources are sufficient; non-running jobs refer to jobs that are not currently running but meet the running conditions; elastic jobs refer to jobs that are currently running but meet the resource addition conditions. In addition, the scheme only allocates resources to jobs when the disk IO occupied by the application to which the job belongs is not overloaded and the resources currently applied for by the job do not exceed the resource quota limit, thereby ensuring the service quality when resources are tight. In other words, the scheme can dynamically adjust the use of resources by jobs, thereby improving the resource utilization of the entire system and the operating efficiency of jobs.

Description

Technical Field

[0001] The present application relates to the field of geophysical exploration technology, and in particular to a resource management method, device, equipment and storage medium. Background Art

[0002] Seismic data processing is a key technology in the oil and gas exploration industry. Its basic process involves organizing algorithmic modules with specific functions into seismic operations (also referred to as operations for short). Seismic operations are then run to process and calculate seismic data collected in the field, thereby generating images of underground geological structures that can then be used to guide subsequent drilling and oil extraction work.

[0003] With the continuous advancement of exploration technology, the amount of collected seismic data is rapidly increasing. For example, the data volume of a single work area can reach hundreds of terabytes, or even petabytes. In other words, seismic data processing has entered the era of massive data. This massive data volume also forces conventional serial processing to be parallelized, and parallel computing must be scaled up to meet production schedules. At the same time, to solve complex geological problems, new, computationally intensive algorithms, such as viscoelastic full waveform inversion (FWI), are constantly being put into production.

[0004] However, this is countered by the reality of low equipment utilization and severe waste. For example, physically, the computing and storage resources of large data centers are often divided into multiple independent node groups and allocated to different projects, making resource sharing difficult between different projects. In addition, the actual resource utilization of conventional processing clusters has always been relatively low. For example, although prestack migration equipment has a high resource utilization rate during calculations, its actual utilization rate is very low if the waste of manual scheduling gaps and the cost of data migration are taken into account. Considering these issues, it is very important to effectively manage resources for seismic operations in actual production environments. Summary of the Invention

[0005] The present invention provides a resource management method, apparatus, device, and storage medium, which can effectively manage resources for seismic operations. The technical solution is as follows:

[0006] In one aspect, a resource management method is provided, the method comprising:

[0007] Set resource quota rules for different resource usage objects in the job in the form of key-value pairs across multiple resource dimensions to obtain a resource quota limit list.

[0008] The object keyword of the resource usage object is used as a key, and the resource quota of the resource usage object is used as a value; the types of the resource usage object include at least: user, department, and project; the multiple resource dimensions include at least: CPU (Central Processing Unit), physical node, and GPU (Graphic Processing Unit);

[0009] Obtaining a job release queue; wherein the job release queue includes currently schedulable jobs; the currently schedulable jobs include jobs that are not currently running but meet running conditions and jobs that are currently running but meet resource addition conditions;

[0010] Start a job release cycle, where the number of cycles is the minimum value between the number of jobs in the job release queue and the set number of successful single job releases;

[0011] During the loop, for the i-th job in the job release queue, in response to the disk I / O (Input / Output) occupied by the application to which the i-th job belongs not being overloaded and the resources currently requested for use by the i-th job not exceeding the resource quota limit, an available resource list is generated for the i-th job; wherein i is a positive integer;

[0012] In response to a physical node that meets the current resource requirements of the i-th job existing in the available resource list, allocating resources on the physical node to the i-th job;

[0013] In response to the i-th job being a resource addition job, a resource addition notification is sent to the i-th job, and resource usage of the i-th job is updated.

[0014] In a possible implementation, the method further includes:

[0015] Generate a resource quota usage list based on the resource usage of the currently running job; wherein the resource quota usage list records the actual usage quota corresponding to each resource quota rule;

[0016] Looping through the resource quota restriction list and the resource quota usage list;

[0017] For the i-th job, matching the keys in the resource quota restriction list and the resource quota usage list with the job keywords of the i-th job;

[0018] In response to the current resource demand of the i-th job being less than the remaining resource quota corresponding to the corresponding resource quota rule, it is determined that the resources currently requested for use by the i-th job do not exceed the resource quota limit.

[0019] In a possible implementation, generating an available resource list for the i-th job includes:

[0020] Obtain a resource information list; wherein the resource information list includes static resource information and dynamic resource information;

[0021] Eliminate physical nodes without available resources from the resource information list;

[0022] Eliminate the physical node that cannot be used by the i-th job from the resource information list;

[0023] According to the node sorting strategy, the remaining available physical nodes in the resource information list are sequentially adjusted to obtain the available resource list.

[0024] In a possible implementation, adjusting the order of the remaining available physical nodes in the resource information list according to the node sorting strategy to obtain the available resource list includes:

[0025] According to the number of jobs currently running on the remaining available physical nodes, the computing characteristics of the currently running jobs and the load conditions of the remaining available physical nodes, the remaining available physical nodes are sequentially adjusted to obtain the available resource list.

[0026] In a possible implementation, obtaining a job release queue includes:

[0027] Periodically obtaining a first job list queued for scheduling; wherein the first job list includes jobs that are not currently running but meet running conditions;

[0028] adjusting the order of the jobs in the first job list according to the queue waiting time of each job in the first job list;

[0029] Periodically obtaining a second job list to which resources can be added; wherein the second job list includes jobs that are currently running but meet resource addition conditions;

[0030] Adjusting the order of the jobs in the second job list according to the target resource ratio of each job in the second job list; wherein, for any job in the second job list, the target resource ratio of the job refers to the ratio of the number of resources currently requested by the job to the minimum number of resources required to start the job;

[0031] The sorted second job list is appended to the sorted first job list to obtain the job release queue.

[0032] In a possible implementation, the method further includes:

[0033] In response to the absence of a physical node in the available resource list that meets the current resource requirements of the i-th job, suspending resource allocation for the i-th job and detecting whether there is currently an over-allocated job; wherein the over-allocated job refers to a job whose resource usage exceeds a corresponding resource quota limit;

[0034] In response to the resource over-allocation job using the resources of the i-th job, adding the resource over-allocation job to a third job list for which resources need to be reclaimed;

[0035] adjusting the order of the jobs in the third job list according to the resource over-allocation ratio of each job in the third job list;

[0036] Sending a resource reclaim request to the first resource overallocation job in the third job list according to the resource gap of the i-th job; wherein the resource reclaim request is used to instruct the first resource overallocation job to exit the resource overallocation node;

[0037] The running priority of the i-th job is set to the highest.

[0038] In a possible implementation, the method further includes:

[0039] In response to the j-th job in the job release queue finishing its operation, a resource recovery operation is performed, and according to the resource usage of the j-th job, a remaining resource quota corresponding to a corresponding resource quota rule is updated.

[0040] In another aspect, a resource management device is provided, the device comprising:

[0041] The setting module is configured to set resource quota rules for different resource usage objects of a job in multiple resource dimensions in the form of key-value pairs to obtain a resource quota restriction list; wherein the object keyword of the resource usage object is used as the key and the resource quota amount of the resource usage object is used as the value; the type of the resource usage object includes at least: user, department and project; and the multiple resource dimensions include at least: CPU, physical node and GPU;

[0042] An acquisition module is configured to acquire a job release queue; wherein the job release queue includes currently schedulable jobs; the currently schedulable jobs include jobs that are not currently running but meet running conditions and jobs that are currently running but meet resource addition conditions;

[0043] The scheduling module is configured to initiate a job release cycle, where the number of cycles is the minimum of the number of jobs in the job release queue and a set number of successful single job releases; during the cycle, for an i-th job in the job release queue, in response to the disk I / O occupied by the application to which the i-th job belongs not being overloaded and the resources currently requested for use by the i-th job not exceeding a resource quota limit, generate an available resource list for the i-th job; wherein i is a positive integer;

[0044] The scheduling module is further configured to, in response to the presence of a physical node in the available resource list that meets the current resource requirements of the i-th job, allocate resources on the physical node to the i-th job; in response to the i-th job being a resource addition job, send a resource addition notification to the i-th job and update the resource usage of the i-th job.

[0045] In a possible implementation, the scheduling module is further configured to:

[0046] Generate a resource quota usage list based on the resource usage of the currently running job; wherein the resource quota usage list records the actual usage quota corresponding to each resource quota rule;

[0047] Looping through the resource quota restriction list and the resource quota usage list;

[0048] For the i-th job, matching the keys in the resource quota restriction list and the resource quota usage list with the job keywords of the i-th job;

[0049] In response to the current resource demand of the i-th job being less than the remaining resource quota corresponding to the corresponding resource quota rule, it is determined that the resources currently requested for use by the i-th job do not exceed the resource quota limit.

[0050] In a possible implementation, the scheduling module is configured to:

[0051] Obtain a resource information list; wherein the resource information list includes static resource information and dynamic resource information;

[0052] Eliminate physical nodes without available resources from the resource information list;

[0053] Eliminate the physical node that cannot be used by the i-th job from the resource information list;

[0054] According to the node sorting strategy, the remaining available physical nodes in the resource information list are sequentially adjusted to obtain the available resource list.

[0055] In a possible implementation, the scheduling module is configured to:

[0056] According to the number of jobs currently running on the remaining available physical nodes, the computing characteristics of the currently running jobs and the load conditions of the remaining available physical nodes, the remaining available physical nodes are sequentially adjusted to obtain the available resource list.

[0057] In a possible implementation, the acquisition module is configured to:

[0058] Periodically obtaining a first job list queued for scheduling; wherein the first job list includes jobs that are not currently running but meet running conditions;

[0059] adjusting the order of the jobs in the first job list according to the queue waiting time of each job in the first job list;

[0060] Periodically obtaining a second job list to which resources can be added; wherein the second job list includes jobs that are currently running but meet resource addition conditions;

[0061] Adjusting the order of the jobs in the second job list according to the target resource ratio of each job in the second job list; wherein, for any job in the second job list, the target resource ratio of the job refers to the ratio of the number of resources currently requested by the job to the minimum number of resources required to start the job;

[0062] The sorted second job list is appended to the sorted first job list to obtain the job release queue.

[0063] In a possible implementation, the scheduling module is further configured to:

[0064] In response to the absence of a physical node in the available resource list that meets the current resource requirements of the i-th job, suspending resource allocation for the i-th job and detecting whether there is currently an over-allocated job; wherein the over-allocated job refers to a job whose resource usage exceeds a corresponding resource quota limit;

[0065] In response to the resource over-allocation job using the resources of the i-th job, adding the resource over-allocation job to a third job list for which resources need to be reclaimed;

[0066] adjusting the order of the jobs in the third job list according to the resource over-allocation ratio of each job in the third job list;

[0067] Sending a resource reclaim request to the first resource overallocation job in the third job list according to the resource gap of the i-th job; wherein the resource reclaim request is used to instruct the first resource overallocation job to exit the resource overallocation node;

[0068] The running priority of the i-th job is set to the highest.

[0069] In a possible implementation, the apparatus further includes:

[0070] The recycling module is configured to execute a resource recycling operation in response to the completion of the j-th job in the job release queue, and update the remaining resource quota corresponding to the corresponding resource quota rule according to the resource usage of the j-th job.

[0071] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the above-mentioned resource management method.

[0072] On the other hand, a computer-readable storage medium is provided, wherein at least one program code is stored in the storage medium, and the at least one program code is loaded and executed by a processor to implement the above-mentioned resource management method.

[0073] On the other hand, a computer program product or computer program is provided, which includes computer program code, which is stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device performs the above-mentioned resource management method.

[0074] The resource management solution provided in the embodiments of the present application can achieve efficient resource management of jobs.

[0075] Specifically, when performing resource scheduling, this solution first obtains a job release queue, which includes currently schedulable jobs. These currently schedulable jobs include not only jobs that are not currently running but meet the running conditions, but also jobs that are currently running but meet the resource addition conditions, i.e., flexible jobs. A job release cycle is then initiated. During this cycle, for each currently schedulable job, if the disk I / O occupied by the application to which the job belongs is not overloaded and the resources currently requested by the job do not exceed the resource quota limit, a list of available resources is automatically generated for the job. Based on this list of available resources, the job is then distributed to the most suitable physical node for execution.

[0076] Because this solution dynamically allocates resources to inactive and flexible jobs, it ensures resource utilization when resources are sufficient. Furthermore, it allocates resources to jobs only when the disk I / O usage of the job's application is not overloaded and the job's currently requested resources do not exceed the resource quota, thus ensuring service quality during resource shortages. In other words, this solution dynamically adjusts job resource usage, improving overall system resource utilization and job efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0078] Figure 1 This is a schematic diagram of a possible implementation environment of the resource management solution provided in the embodiment of the present application;

[0079] Figure 2 This is a flow chart of a resource management method provided by an embodiment of the present application;

[0080] Figure 3 This is a schematic diagram of a resource allocation process provided by an embodiment of the present application;

[0081] Figure 4 is a flowchart of another resource management method provided by an embodiment of the present application;

[0082] Figure 5 This is a comparison chart of the job operation efficiency of the present method and static resource partitioning provided in the embodiment of the present application;

[0083] Figure 6 This is a schematic diagram of the structure of a resource management device provided in an embodiment of the present application;

[0084] Figure 7 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0085] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0086] In this application, the terms "first," "second," and the like are used to distinguish identical or similar items having substantially the same role and function. It should be understood that "first," "second," and "nth" do not have a logical or temporal dependency, nor do they limit the quantity or execution order. It should also be understood that although the following description uses the terms "first," "second," and the like to describe various elements, these elements should not be limited by these terms.

[0087] These terms are simply used to distinguish one element from another. For example, a first element can be referred to as a second element, and similarly, a second element can be referred to as a first element, without departing from the scope of various examples. Both the first element and the second element can be elements, and in some cases, can be separate and different elements.

[0088] Here, at least one refers to one or more than one. For example, at least one element can be one element, two elements, three elements, or any other integer greater than or equal to one. And multiple refers to two or more than two. For example, multiple elements can be two elements, three elements, or any other integer greater than or equal to two.

[0089] The term "and / or" used in this document indicates that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0090] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0091] The First-Input-First-Output (FIFO) algorithm is widely used in mainstream resource management and job scheduling systems. Specifically, the system allocates resources to each job based on the order in which they were submitted. Specifically, when allocating resources to a job, the system determines whether the available resources meet the job's resource requirements based on the current system resource usage. If so, the job is released and begins running after acquiring resources. If not, the job remains in a waiting state until resources become available.

[0092] However, this resource allocation algorithm may lead to the following situation: the probability of obtaining resources for jobs with low resource usage is much higher than that of obtaining resources for jobs with high resource usage. In addition, some jobs may not be able to obtain resources and thus cannot run because they require too many resources.

[0093] In response to the above situation, in actual production environments, to ensure fairness in resource usage among resource users (such as projects or users) and to prevent certain projects or users from being unable to obtain resources to run jobs, administrators will split cluster resources and then use static partitioning to allocate resources to projects or users. This static partitioning method ensures that every project or user can obtain resources to run jobs. However, this static partitioning method also introduces other problems:

[0094] On the one hand, if a project does not have seismic operations temporarily, the resources reserved for it cannot be used by other projects, resulting in a waste of resources; on the other hand, if a project temporarily needs more resources due to urgency, this requires system administrators to perform relevant configurations, greatly increasing the workload of system configuration and maintenance, and the real-time performance of such configuration modifications is poor.

[0095] Seismic jobs used to process seismic data in related technologies have a major characteristic: running jobs cannot have their resources preempted, otherwise the entire job will fail. This characteristic creates a conflict between job execution and cluster resource utilization.

[0096] Specifically, if job execution is prioritized, since each project or user has strict static minimum resource requirements, idle resources will not be allocated to other jobs to meet the job's execution requirements. This makes resource reuse difficult, resulting in serious resource waste and, to a certain extent, reducing the system's peak computing power. Alternatively, if cluster resource utilization is prioritized, resources will be freely allocated to global jobs, and the first job submitted will receive resources first. When a project or user that does not meet the minimum resource requirements submits a new job, and if there are no available resources at this time, the system typically adopts the following two scheduling strategies:

[0097] Scheduling strategy 1: Do not allocate available resources to the job, which will affect the operation of the project or user's job;

[0098] Scheduling strategy 2: Preempt the computing resources of other jobs. Since most jobs (high-performance computing) have long and unpredictable running times, job failures caused by preemption will lead to waste of computing and are unreasonable for the projects or users whose resources are preempted.

[0099] In response to these problems, an embodiment of the present application proposes a resource management solution that dynamically allocates and recycles resources for seismic operations. In terms of resource allocation, it is no longer simply emphasized that the system throughput is maximized. The operation of the project or user is also very important. The overall best does not mean individual satisfaction, and it may be that no one is satisfied. Moreover, if there are no restrictions on the available resources of the project or user, then the project or user will tend to over-consume resources, which may result in the worst overall energy efficiency. Therefore, the overall goal of this solution is to ensure service quality when resources are tight and to improve utilization when resources are sufficient, that is, to have the characteristics of static resource division and dynamic sharing at the same time.

[0100] In other words, the embodiments of this application provide a dynamic resource allocation and recovery strategy for seismic operations. This strategy dynamically adjusts resource usage based on the current workload and resource availability in the system, improving overall system resource utilization and the efficiency of the operations. Specifically, for seismic operations, efficient resource allocation and recovery are achieved, both improving cluster resource utilization and ensuring that all projects or users in the system have access to resources to run their operations.

[0101] Figure 1 This is a schematic diagram of a possible implementation environment of the resource management solution provided in the embodiment of the present application.

[0102] See also Figure 1 , the implementation environment includes: a job scheduler 11 and a physical node 12.

[0103] For example, the job scheduler 11 is used to dynamically allocate and recycle resources for executing seismic jobs based on the C++ language, and the jobs are run on the physical nodes 12. In addition, taking the distribution of resources by department as an example, the jobs of each department are formed Figure 1 In addition, the job scheduler 11 also specifies resource quotas for each department in multiple resource dimensions (CPU, node, GPU), and sets up an additional elastic job queue to dynamically allocate resources among departments and achieve the purpose of rationally utilizing idle resources.

[0104] Based on the above implementation environment, this resource management solution achieves at least the following aspects:

[0105] 1. The embodiment of the present application provides a multi-dimensional resource quota management method, which supports resource quota setting in multiple resource dimensions through the user, department, project and other extended attributes of the job.

[0106] 2. The embodiment of the present application provides a resource allocation method based on resource quotas, which supports dividing jobs into multiple queues, queuing the jobs in the queues according to a certain job queuing strategy, and then selecting the job with the highest priority and distributing it to the most suitable physical node among all idle resources for execution. In addition, the resources used when the job is running are included in the quota to which it belongs, which will be described in detail later. In addition, after the job is completed, the resources are released, and other jobs can continue to use the released resources. If the resources used by the job reach the resource quota limit, the scheduling of such jobs will be suspended until sufficient idle resources are available.

[0107] 3. In response to the problem that nodes cannot be dynamically scaled after a job is running, an embodiment of the present application provides a resource negotiation mechanism, which realizes the elastic modification of the resources used by the job by negotiating with the job. For jobs that support elastic scaling of resources (the resources used by such jobs can vary between a minimum number of resources and a maximum number of resources), resources can be allocated to such jobs according to the minimum resource requirements, and the resources used when the job is running can be included in the corresponding quota, and then the corresponding job can be added to the elastic job queue. In addition, if there are still idle resources available, it is possible to negotiate with such jobs to continue to add resources until the maximum number of resources is reached; and if resources are tight, it is also possible to negotiate with such jobs to recover the extra resources allocated to it. For example, the scheduling algorithm of the elastic job queue can be carried out according to the principles of "first come, first served" and "whoever occupies more, gives first", and this application does not limit this.

[0108] Figure 2 This is a flow chart of a resource management method provided by an embodiment of the present application. The execution subject of this method is a computer device, such as a job scheduler (also known as a job scheduling system, or simply a scheduling system). Figure 2 , the method comprising:

[0109] 201. The computer device sets resource quota rules in multiple resource dimensions for different resource usage objects of the job in the form of key-value pairs, and obtains a resource quota restriction list.

[0110] For a detailed description of this step, please refer to the subsequent step 401.

[0111] 202. The computer device obtains a job release queue; wherein the job release queue includes currently schedulable jobs; currently schedulable jobs include jobs that are not currently running but meet running conditions and jobs that are currently running but meet resource addition conditions.

[0112] This step is used to obtain the currently schedulable jobs.

[0113] In the embodiment of the present application, the scheduling system automatically collects information about jobs currently waiting for resources at regular intervals, allocates resources to jobs waiting to be run, and adds resources to jobs that can be scheduled. For example, obtaining currently schedulable jobs includes but is not limited to the following steps:

[0114] 2021. Obtain jobs that are waiting to be scheduled but have not yet started. Scan the global job queue to obtain jobs that are not currently running but meet the running conditions, and add these jobs to the list of schedulable jobs. This job list is also referred to as the first job list in this article.

[0115] 2022. Sort the jobs in the list of schedulable jobs according to the first job sorting strategy. For example, the default sorting strategy is the queue waiting time of the job, which is not limited in this application.

[0116] 2023. Obtain currently running jobs that can be supplemented with resources, and add such jobs to a list of scalable resource jobs. This list of jobs is also referred to as a second job list in this document.

[0117] 2024. Sort the jobs in the scalable resource job list according to the second job sorting strategy. For example, the default sorting strategy is the ratio of the number of additional resources of the job to the number of resources required for startup.

[0118] 2025. Append the sorted list of scalable resource jobs to the sorted list of schedulable jobs.

[0119] In other words, obtaining the job release queue includes but is not limited to the following methods:

[0120] Periodically obtaining a first job list queued for scheduling; wherein the first job list includes jobs that are not currently running but meet running conditions;

[0121] Adjusting the order of the jobs in the first job list according to the queue waiting time of each job in the first job list;

[0122] Periodically obtaining a second job list to which resources can be added; wherein the second job list includes jobs that are currently running but meet resource addition conditions;

[0123] Adjusting the order of the jobs in the second job list based on the target resource ratio of each job in the second job list; wherein, for any job in the second job list, the target resource ratio of the job refers to the ratio of the number of resources currently requested by the job to the minimum number of resources required to start the job;

[0124] The sorted second job list is appended to the sorted first job list to obtain a job release queue.

[0125] 203. The computer device starts a job release cycle, where the number of cycles is the minimum of the number of jobs in the job release queue and the set number of successful single job releases; during the cycle, for the i-th job in the job release queue, in response to the fact that the disk IO occupied by the application to which the i-th job belongs is not overloaded and the resources currently requested for use by the i-th job do not exceed the resource quota limit, the computer device generates an available resource list for the i-th job.

[0126] This step is used to allocate resources to the job.

[0127] In an embodiment of the present application, the scheduling system initiates a job release cycle to release jobs from the job release queue. Furthermore, to avoid releasing too many jobs at once, which could impact the next release cycle, the scheduling system sets a default limit for successful single-time job releases. The number of cycles is the minimum of the number of jobs in the job release queue and the number of successful single-time job releases.

[0128] In one possible implementation, generating an available resource list for the i-th job includes but is not limited to the following steps:

[0129] 2031. Obtain a resource information list; the resource information list includes static resource information and dynamic resource information. This step is used to obtain all resource information. Exemplarily, the resource information obtained primarily includes static resource information such as the number of CPUs and GPUs of the node, and the node group to which the node belongs; and dynamic resource information such as the currently available number of CPUs and GPUs, which is not limited in this application.

[0130] 2032. Remove the physical nodes without available resources from the resource information list.

[0131] 2033. Remove the physical nodes that cannot be used by the i-th job from the resource information list. In this step, the physical nodes that cannot be used by the i-th job can be removed from the resource information list based on the access restriction rule of the i-th job, which is not limited in this application.

[0132] 2034. According to the node sorting strategy, the order of the remaining available physical nodes in the resource information list is adjusted to obtain an available resource list.

[0133] In another possible implementation, the remaining available physical nodes in the resource information list are sequentially adjusted according to the node sorting strategy to obtain a list of available resources, including but not limited to the following methods: according to the number of jobs currently running on the remaining available physical nodes, the computing characteristics of the currently running jobs and the load conditions of the remaining available physical nodes, the remaining available physical nodes are sequentially adjusted to obtain a list of available resources. For example, the physical nodes with fewer jobs and lower loads are ranked higher in the available resource list by default. Among them, the computing characteristics of the running job can be whether the running job is a CPU-intensive job or a GPU-intensive job, which is not limited in this application.

[0134] For example, assuming that the i-th job is a CPU-intensive job, for two physical nodes A and B with the same conditions, assuming that a CPU-intensive job is running on physical node A, while no CPU-intensive job is running on physical node B, the i-th job tends to run on physical node B.

[0135] Among them, the above steps 2032-2034 correspond to Figure 3 "Obtaining schedulable physical nodes according to filtering conditions and sorting the schedulable physical nodes" in [1].

[0136] 204. In response to the presence of a physical node in the available resource list that meets the current resource requirements of the i-th job, allocate resources on the physical node to the i-th job.

[0137] This step is used to select resources from the available resource list and assign them to the i-th job.

[0138] 205. In response to the i-th job being a resource addition job, send a resource addition notification to the i-th job and update the resource usage of the i-th job.

[0139] If the i-th job is already running as a job with dynamically scalable resources, resources are allocated to the job and the additional resource information is notified to the job.

[0140] The resource management solution provided in the embodiments of the present application can achieve efficient resource management of jobs.

[0141] Specifically, when performing resource scheduling, this solution first obtains a job release queue, which includes currently schedulable jobs. These currently schedulable jobs include not only jobs that are not currently running but meet the running conditions, but also jobs that are currently running but meet the resource addition conditions, i.e., flexible jobs. A job release cycle is then initiated. During this cycle, for each currently schedulable job, if the disk I / O occupied by the application to which the job belongs is not overloaded and the resources currently requested by the job do not exceed the resource quota limit, a list of available resources is automatically generated for the job. Based on this list of available resources, the job is then distributed to the most suitable physical node for execution.

[0142] Because this solution can dynamically allocate resources to inactive and elastic jobs, it ensures resource utilization when resources are sufficient. In addition, this solution allocates resources to a job only when the disk I / O occupied by the application to which the job belongs is not overloaded and the resources currently requested by the job do not exceed the resource quota limit, thus ensuring service quality when resources are scarce.

[0143] The above is just a brief introduction to some technical details of the resource management solution provided by the embodiment of this application. Figure 4 The illustrated embodiment provides a detailed introduction to the resource management solution.

[0144] Figure 4 This is a flow chart of another resource management method provided by an embodiment of the present application. The execution subject of this method is a computer device, such as a job scheduler. Figure 4 , the method comprising:

[0145] 401. The computer device sets resource quota rules in multiple resource dimensions for different resource usage objects of the job in the form of key-value pairs, and obtains a resource quota restriction list.

[0146] This step is used to set resource quotas.

[0147] It should be noted that in the resource quota restriction list, the object keyword (also called quota keyword) of the resource usage object is used as the key, and the resource quota amount of the resource usage object is used as the value; among them, the types of resource usage objects include at least: user, department and project; and the above-mentioned multiple resource dimensions include at least: CPU, physical node and GPU.

[0148] In one possible implementation, resource quota rules are set for different resource users of a job in multiple resource dimensions, including but not limited to the following steps:

[0149] 4011. Determine the quota keyword.

[0150] A quota keyword can be a single keyword, for example, only affecting a user or project, such as User A and Project B. Alternatively, a combined keyword can be used to affect multiple keywords simultaneously. Keywords can be connected using operators such as and or or, such as User A and Project B.

[0151] 4012. Determine resource quotas.

[0152] Resource quotas can be set for CPUs, nodes, and GPUs. Similar to keyword settings, you can limit quotas to a specific resource or to multiple resources simultaneously. If you limit quotas for multiple resources, the restrictions are in an OR relationship.

[0153] 4013. Adjust resource quotas. Adjust effective resource quotas. The resource dimensions that can be adjusted include CPU, node, and GPU.

[0154] 4014. Cancel resource quota. Cancel the effective resource quota setting. After cancellation, the resource quota will be invalid immediately, and subsequent job runs will no longer be restricted by this resource quota.

[0155] 4015. Obtain resource quota. For example, the scheduling system provides an interface to support managers to query the resource quota information currently in effect in the system.

[0156] 402. The computer device obtains a job release queue. The job release queue includes currently schedulable jobs. The currently schedulable jobs include jobs that are not currently running but meet running conditions and jobs that are currently running but meet resource addition conditions.

[0157] This step can refer to the aforementioned step 201 and will not be repeated here.

[0158] 403. The computer device initiates a job release cycle, wherein the number of cycles is the minimum of the number of jobs in the job release queue and the set number of successful single job releases. During the cycle, for the i-th job in the job release queue, in response to the fact that the disk IO occupied by the application to which the i-th job belongs is not overloaded and the resources currently requested for use by the i-th job do not exceed the resource quota limit, the computer device generates an available resource list for the i-th job.

[0159] During the loop, the scheduling system obtains information about the currently scheduled jobs, such as the resource type (CPU, node, or GPU) and the required resource quantity. Additionally, the scheduling system obtains the current number of tasks per physical node, though this application does not specify this.

[0160] In one possible implementation, the embodiment of the present application determines whether the resources currently requested for use by the i-th job exceed the resource quota limit through the following steps:

[0161] 4031. Obtain a resource quota restriction list. The resource quota restriction list records resource quota rules for different resource usage objects of a job in multiple resource dimensions in the form of key-value pairs.

[0162] 4032. Generate a resource quota usage list based on the resource usage of the currently running job; wherein the resource quota usage list records the actual usage quota corresponding to each resource quota rule.

[0163] That is, this step is used to obtain the actual usage of the resource quota restricted by each resource quota rule.

[0164] 4033. Loop through the resource quota limit list and the resource quota usage list; for the i-th job, match the keys in the resource quota limit list and the resource quota usage list with the job keywords of the i-th job; in response to the current resource demand of the i-th job being less than the remaining resource quota corresponding to the corresponding resource quota rule, determine that the resources currently requested for use by the i-th job do not exceed the resource quota limit.

[0165] For example, the job keyword includes a quota keyword to indicate which user, project or department the corresponding job comes from. Therefore, by matching the quota keywords in the two lists with the job keywords and the resource application information of the job, it can be determined whether the job exceeds the resource quota limit.

[0166] It should be noted that the resource quota rule described above refers to the resource quota rule that matches the job keyword of the i-th job. For example, suppose this resource quota rule limits user A's resource quota to 100 CPUs. Currently, user A's other jobs have already used 50 CPUs, leaving a remaining resource quota of 50 CPUs. If the number of CPUs currently requested by the i-th job (from user A) is less than 50, then the resources currently requested by the i-th job are determined to be within the resource quota limit.

[0167] In addition, this step corresponds to Figure 3 Determining Whether Resource Quotas Are Exceeded.

[0168] 404. In response to the disk IO of the application to which the i-th job belongs being overloaded, the computer device stops allocating resources to the i-th job.

[0169] like Figure 3As shown, the scheduling system will determine whether the disk IO occupied by the application to which the job belongs is overloaded; if so, it will directly terminate resource allocation for this job.

[0170] 405 . In response to the fact that the disk IO occupied by the application to which the i-th job belongs is not overloaded, but the resources currently requested for use by the i-th job exceed the resource quota limit, the computer device stops allocating resources to the i-th job.

[0171] like Figure 3 As shown in the figure, although the disk IO occupied by the application to which the job belongs is not overloaded, the resources currently requested by the job exceed the resource quota limit, and resource allocation for this job is directly terminated.

[0172] 406. In response to the presence of a physical node in the available resource list that meets the current resource requirements of the i-th job, the computer device allocates resources on the physical node to the i-th job.

[0173] In the embodiment of the present application, when resources are allocated to the i-th job, the corresponding resource usage information and quota information are also updated, that is, the resource usage of the job and the actual usage quota corresponding to the corresponding resource quota rule are updated.

[0174] If the job is already running as a dynamically scalable resource job, resources are allocated to the job and the job is notified of the additional resource information. Furthermore, adding resources only updates the job's resource usage information but does not count towards the quota information. This means that the actual usage quota corresponding to the corresponding resource quota rule is not updated, and the resource allocation ends.

[0175] 407. In response to the absence of a physical node in the available resource list that meets the current resource demand of the i-th job, the computer device suspends resource allocation for the i-th job and determines whether to perform a resource recycling operation.

[0176] In one possible implementation, the embodiment of the present application determines whether to perform a resource recovery operation through the following steps:

[0177] 4071. Detect whether there is currently a resource overallocation job; wherein, a resource overallocation job refers to a job whose resource usage exceeds the corresponding resource quota limit; in response to the resource overallocation job using the resources of the i-th job, the resource overallocation job is added to the third job list for which resources need to be reclaimed.

[0178] 4072. Adjust the order of the jobs in the third job list according to the resource over-allocation ratio of each job in the third job list.

[0179] Illustratively, the greater the resource over-allocation ratio of a job, the higher the job is ranked in the third job list, which is not limited in this application.

[0180] 4073. Send a resource reclaim request to the first resource overallocation job in the third job list according to the resource gap of the i-th job; wherein the resource reclaim request is used to instruct the first resource overallocation job to exit the resource overallocation node.

[0181] In an embodiment of the present application, a job that receives a resource reclaim request will automatically exit the resource over-allocation node.

[0182] 4074. Set the running priority of the i-th job to the highest and wait for the next resource allocation.

[0183] In another possible implementation, the embodiment of the present application also includes a resource recovery process.

[0184] Accordingly, in response to the jth job in the job release queue finishing running, the resource recovery operation is performed, and the remaining resource quota corresponding to the corresponding resource quota rule is updated according to the resource usage of the jth job. At the same time, the resource usage information of the job is updated to 0.

[0185] Wherein, i and j are both positive integers. For example, the j-th job ends, including but not limited to the following situations: normal exit, abnormal termination or termination by management personnel, which is not limited in this application.

[0186] The resource management solution provided in the embodiments of the present application can realize dynamic allocation and recycling of resources.

[0187] Specifically, this solution supports resource quota management across multiple resource dimensions. Furthermore, this solution features both static resource division and dynamic sharing, except that the division here is not a static grouping of physical resources, but rather the setting of resource quotas. Setting resource quotas provides basic resource guarantees for jobs. As long as the resources requested by a job do not exceed the corresponding resource quota limit, idle resources will be allocated to the job first. If no idle resources are available, resources will be recovered from elastic jobs to meet the job's operational needs, thereby ensuring service quality. The emergence of elastic jobs enables jobs to temporarily borrow idle resources outside of resource quotas, enabling resource sharing and improving resource utilization.

[0188] In summary, this solution achieves efficient resource management, that is, it achieves the scheduling goal of ensuring service quality when resources are tight and improving utilization when resources are sufficient. This solution guarantees the operation requirements of the job and realizes cluster resource sharing. This solution runs stably when processing earthquake operations and provides a good user experience. Among them, Figure 5This is a comparison chart of the job operation efficiency between this method and conventional static resource partitioning.

[0189] In addition, in actual application, taking resource distribution by department as an example, the sum of the resource quotas of all departments can be less than or equal to the actual number of physical resources to provide absolute resource guarantees. In addition, resource over-allocation can also be set according to a certain ratio. Since resources are shared and the operation and use of jobs vary, in most cases, the smooth operation of each job can still be guaranteed.

[0190] Figure 6 This is a schematic diagram of the structure of a resource management device provided in an embodiment of the present application. Figure 6 , the device comprises:

[0191] The setting module 601 is configured to set resource quota rules for different resource usage objects of a job in multiple resource dimensions in the form of key-value pairs, thereby obtaining a resource quota restriction list; wherein the object keyword of the resource usage object is used as the key, and the resource quota amount of the resource usage object is used as the value; the resource usage object type includes at least: user, department, and project; and the multiple resource dimensions include at least: CPU, physical node, and GPU;

[0192] The acquisition module 602 is configured to acquire a job release queue; wherein the job release queue includes currently schedulable jobs; the currently schedulable jobs include jobs that are not currently running but meet the running conditions and jobs that are currently running but meet the resource addition conditions;

[0193] The scheduling module 603 is configured to initiate a job release cycle, where the number of cycles is the minimum of the number of jobs in the job release queue and the set number of successful single job releases. During the cycle, for the i-th job in the job release queue, in response to the disk I / O occupied by the application to which the i-th job belongs not being overloaded and the resources currently requested for use by the i-th job not exceeding the resource quota limit, generate an available resource list for the i-th job.

[0194] The scheduling module 603 is further configured to allocate resources on the physical node to the i-th job in response to the presence of a physical node in the available resource list that meets the current resource requirements of the i-th job; in response to the i-th job being a resource addition job, send a resource addition notification to the i-th job and update the resource usage of the i-th job.

[0195] The resource management solution provided by the embodiments of the present application can achieve efficient resource management for jobs. Specifically, when performing resource scheduling, the solution first obtains a job release queue, which includes currently schedulable jobs. Among them, currently schedulable jobs include not only jobs that are not currently running but meet the running conditions, but also jobs that are currently running but meet the resource addition conditions, i.e., flexible jobs. Then, a job release cycle is started. During the cycle, for each currently schedulable job, in response to the disk I / O occupied by the application to which the job belongs is not overloaded and the resources currently requested for use by the job do not exceed the resource quota limit, a list of available resources is automatically generated for the job, and based on the available resource list, the job is distributed to the most suitable physical node for execution. Because the solution can dynamically allocate resources for non-running jobs and flexible jobs, resource utilization is ensured when resources are sufficient. In addition, the solution only allocates resources to the job when the disk I / O occupied by the application to which the job belongs is not overloaded and the resources currently requested for use by the job do not exceed the resource quota limit, thus ensuring service quality when resources are scarce.

[0196] In a possible implementation, the scheduling module is further configured to:

[0197] Generate a resource quota usage list based on the resource usage of the currently running job; wherein the resource quota usage list records the actual usage quota corresponding to each resource quota rule;

[0198] Looping through the resource quota restriction list and the resource quota usage list;

[0199] For the i-th job, matching the keys in the resource quota restriction list and the resource quota usage list with the job keywords of the i-th job;

[0200] In response to the current resource demand of the i-th job being less than the remaining resource quota corresponding to the corresponding resource quota rule, it is determined that the resources currently requested for use by the i-th job do not exceed the resource quota limit.

[0201] In a possible implementation, the scheduling module is configured to:

[0202] Obtain a resource information list; wherein the resource information list includes static resource information and dynamic resource information;

[0203] Eliminate physical nodes without available resources from the resource information list;

[0204] Eliminate the physical node that cannot be used by the i-th job from the resource information list;

[0205] According to the node sorting strategy, the remaining available physical nodes in the resource information list are sequentially adjusted to obtain the available resource list.

[0206] In a possible implementation, the scheduling module is configured to:

[0207] According to the number of jobs currently running on the remaining available physical nodes, the computing characteristics of the currently running jobs and the load conditions of the remaining available physical nodes, the remaining available physical nodes are sequentially adjusted to obtain the available resource list.

[0208] In a possible implementation, the acquisition module is configured to:

[0209] Periodically obtaining a first job list queued for scheduling; wherein the first job list includes jobs that are not currently running but meet running conditions;

[0210] adjusting the order of the jobs in the first job list according to the queue waiting time of each job in the first job list;

[0211] Periodically obtaining a second job list to which resources can be added; wherein the second job list includes jobs that are currently running but meet resource addition conditions;

[0212] Adjusting the order of the jobs in the second job list according to the target resource ratio of each job in the second job list; wherein, for any job in the second job list, the target resource ratio of the job refers to the ratio of the number of resources currently requested by the job to the minimum number of resources required to start the job;

[0213] The sorted second job list is appended to the sorted first job list to obtain the job release queue.

[0214] In a possible implementation, the scheduling module is further configured to:

[0215] In response to the absence of a physical node in the available resource list that meets the current resource requirements of the i-th job, suspending resource allocation for the i-th job and detecting whether there is currently an over-allocated job; wherein the over-allocated job refers to a job whose resource usage exceeds a corresponding resource quota limit;

[0216] In response to the resource over-allocation job using the resources of the i-th job, adding the resource over-allocation job to a third job list for which resources need to be reclaimed;

[0217] adjusting the order of the jobs in the third job list according to the resource over-allocation ratio of each job in the third job list;

[0218] Sending a resource reclaim request to the first resource overallocation job in the third job list according to the resource gap of the i-th job; wherein the resource reclaim request is used to instruct the first resource overallocation job to exit the resource overallocation node;

[0219] The running priority of the i-th job is set to the highest.

[0220] In a possible implementation, the device further includes:

[0221] The recycling module is configured to execute a resource recycling operation in response to the completion of the j-th job in the job release queue, and update the remaining resource quota corresponding to the corresponding resource quota rule according to the resource usage of the j-th job.

[0222] All of the above optional technical solutions can be combined in any way to form optional embodiments of the present application, and will not be described in detail here.

[0223] It should be noted that the resource management device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate resource management. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the resource management device provided in the above embodiment and the resource management method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0224] Figure 7 7 is a structural diagram of a computer device 700 provided in an embodiment of the present application.

[0225] The computer 700 may be a server. The computer device 700 may vary greatly due to different configurations or performances, and may include one or more processors (Central Processing Units, CPUs) 701 and one or more memories 702, wherein the memories 702 store at least one program code, and the at least one program code is loaded and executed by the processor 701 to implement the resource management methods provided by the above-mentioned various method embodiments. Of course, the computer device 700 may also have components such as a wired or wireless network interface, a keyboard, and an input / output interface for input and output. The computer device 700 may also include other components for implementing device functions, which will not be described in detail here.

[0226] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including program code. The program code can be executed by a processor in a computer device to implement the resource management method in the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0227] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer program code, which is stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device performs the above-mentioned resource management method.

[0228] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0229] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A resource management method, characterized in that: The method comprises: Resource quota rules are set for different resource usage objects of a job in multiple resource dimensions in the form of key-value pairs to obtain a resource quota restriction list; wherein the object keyword of the resource usage object is used as the key and the resource quota amount of the resource usage object is used as the value; the types of the resource usage object include at least: user, department, and project; and the multiple resource dimensions include at least: central processing unit (CPU), physical node, and graphics processing unit (GPU); Obtaining a job release queue; wherein the job release queue includes currently schedulable jobs; the currently schedulable jobs include jobs that are not currently running but meet running conditions and jobs that are currently running but meet resource addition conditions; Start a job release cycle, where the number of cycles is the minimum value between the number of jobs in the job release queue and the set number of successful single job releases; In a loop, for the i-th job in the job release queue, in response to the disk input / output (IO) occupied by the application to which the i-th job belongs not being overloaded and the resources currently requested for use by the i-th job not exceeding the resource quota limit, an available resource list is generated for the i-th job; wherein i is a positive integer; In response to a physical node that meets the current resource requirements of the i-th job existing in the available resource list, allocating resources on the physical node to the i-th job; In response to the i-th job being a resource addition job, a resource addition notification is sent to the i-th job, and resource usage of the i-th job is updated.

2. The method according to claim 1, characterized in that The method further comprises: Generate a resource quota usage list based on the resource usage of the currently running job; wherein the resource quota usage list records the actual usage quota corresponding to each resource quota rule; Looping through the resource quota restriction list and the resource quota usage list; For the i-th job, matching the keys in the resource quota restriction list and the resource quota usage list with the job keywords of the i-th job; In response to the current resource demand of the i-th job being less than the remaining resource quota corresponding to the corresponding resource quota rule, it is determined that the resources currently requested for use by the i-th job do not exceed the resource quota limit.

3. The method according to claim 1, characterized in that Generating an available resource list for the i-th job includes: Obtain a resource information list; wherein the resource information list includes static resource information and dynamic resource information; Eliminate physical nodes without available resources from the resource information list; Eliminate the physical node that cannot be used by the i-th job from the resource information list; According to the node sorting strategy, the remaining available physical nodes in the resource information list are sequentially adjusted to obtain the available resource list.

4. The method according to claim 3, characterized in that The step of adjusting the order of the remaining available physical nodes in the resource information list according to the node sorting strategy to obtain the available resource list includes: According to the number of jobs currently running on the remaining available physical nodes, the computing characteristics of the currently running jobs and the load conditions of the remaining available physical nodes, the remaining available physical nodes are sequentially adjusted to obtain the available resource list.

5. The method according to claim 1, wherein The obtaining of the job release queue includes: Periodically obtaining a first job list queued for scheduling; wherein the first job list includes jobs that are not currently running but meet running conditions; adjusting the order of the jobs in the first job list according to the queue waiting time of each job in the first job list; Periodically obtaining a second job list to which resources can be added; wherein the second job list includes jobs that are currently running but meet resource addition conditions; Adjusting the order of the jobs in the second job list according to the target resource ratio of each job in the second job list; wherein, for any job in the second job list, the target resource ratio of the job refers to the ratio of the number of resources currently requested by the job to the minimum number of resources required to start the job; The sorted second job list is appended to the sorted first job list to obtain the job release queue.

6. The method according to claim 1, characterized in that The method further comprises: In response to the absence of a physical node in the available resource list that meets the current resource requirements of the i-th job, suspending resource allocation for the i-th job and detecting whether there is currently an over-allocated job; wherein the over-allocated job refers to a job whose resource usage exceeds a corresponding resource quota limit; In response to the resource over-allocation job using the resources of the i-th job, adding the resource over-allocation job to a third job list for which resources need to be reclaimed; adjusting the order of the jobs in the third job list according to the resource over-allocation ratio of each job in the third job list; Sending a resource reclaim request to the first resource overallocation job in the third job list according to the resource gap of the i-th job; wherein the resource reclaim request is used to instruct the first resource overallocation job to exit the resource overallocation node; The running priority of the i-th job is set to the highest.

7. The method according to claim 1, characterized in that The method further comprises: In response to the j-th job in the job release queue finishing its operation, a resource recovery operation is performed, and according to the resource usage of the j-th job, a remaining resource quota corresponding to a corresponding resource quota rule is updated.

8. A resource management device, characterized in that: The device comprises: The setting module is configured to set resource quota rules for different resource usage objects of a job in multiple resource dimensions in the form of key-value pairs to obtain a resource quota restriction list; wherein the object keyword of the resource usage object is used as the key and the resource quota amount of the resource usage object is used as the value; the type of the resource usage object includes at least: user, department and project; and the multiple resource dimensions include at least: CPU, physical node and GPU; An acquisition module is configured to acquire a job release queue; wherein the job release queue includes currently schedulable jobs; the currently schedulable jobs include jobs that are not currently running but meet running conditions and jobs that are currently running but meet resource addition conditions; The scheduling module is configured to initiate a job release cycle, where the number of cycles is the minimum of the number of jobs in the job release queue and a set number of successful single job releases; during the cycle, for an i-th job in the job release queue, in response to disk input / output (IO) occupied by an application to which the i-th job belongs not being overloaded and resources currently requested for use by the i-th job not exceeding a resource quota limit, generate an available resource list for the i-th job; wherein i is a positive integer; The scheduling module is further configured to, in response to the presence of a physical node in the available resource list that meets the current resource requirements of the i-th job, allocate resources on the physical node to the i-th job; in response to the i-th job being a resource addition job, send a resource addition notification to the i-th job and update the resource usage of the i-th job.

9. A computer device, characterized in that: The device includes a processor and a memory, wherein at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement the resource management method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that At least one program code is stored in the storage medium, and the at least one program code is loaded and executed by the processor to implement the resource management method according to any one of claims 1 to 7.

Images