Resource management method and device, equipment and storage medium
By setting up resource quota rules and dynamic allocation and recovery mechanisms, the problem of low resource utilization in earthquake data processing is solved, efficient resource management and service quality are ensured, and the resource utilization and operating efficiency of the entire system are improved.
Patent Information
- Application Number
- CN202310611619.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-26
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2043-05-26
Smart Images

Figure CN119025255A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of geophysical exploration technology, and in particular to a resource management method, device, equipment and storage medium. Background Art
[0002] Seismic data processing is an important technology in the oil and gas exploration industry. Its basic process is to organize algorithm modules with specific functions into seismic operations (also referred to as operations), and then process and calculate the seismic data collected in the field by running seismic operations to obtain images of underground geological structures, which are then used to guide subsequent drilling and oil extraction work.
[0003] With the continuous advancement of exploration technology, the amount of collected seismic data is growing rapidly. For example, the amount of data in a single work area can reach hundreds of TB or even PB. In other words, seismic data processing has entered the era of massive data, and massive data also forces conventional serial processing to be parallelized, and parallel computing must increase the scale of parallelism to meet production schedule requirements. At the same time, in order to solve complex geological problems, new algorithms that take up a large amount of computing, such as viscoelastic full waveform inversion (FWI), are constantly being put into production.
[0004] However, the reality that corresponds to this is low equipment utilization and serious waste. For example, physically, the computing and storage resources of large data centers are usually divided into multiple independent node groups and allocated to different projects, but it is difficult to share resources between different projects. In addition, the actual resource utilization of conventional processing clusters has always been relatively low. For example, although the resource utilization of prestack migration equipment is high during calculation, its actual utilization is very low if the waste of manual scheduling gaps and the cost of data migration are taken into account. Taking these issues into consideration, it is very important to manage resources efficiently for seismic operations in an actual production environment. Summary of the invention
[0005] The embodiment of the present application provides a resource management method, device, equipment and storage medium, which can perform efficient resource management for seismic operations. The technical solution is as follows:
[0006] In one aspect, a resource management method is provided, the method comprising:
[0007] In the form of key-value pairs, resource quota rules are set for different resource users of jobs in multiple resource dimensions to obtain a resource quota restriction list.
[0008] The object keyword of the resource usage object is used as a key, and the resource quota amount of the resource usage object is used as a value; the types of the resource usage object include at least: user, department and project; the multiple resource dimensions include at least: CPU (Central Processing Unit, central processing unit), physical node and GPU (Graphic Processing Unit, graphics processing unit);
[0009] Obtaining a job release queue; wherein the job release queue includes currently schedulable jobs; the currently schedulable jobs include jobs that are not currently running but meet running conditions and jobs that are currently running but meet resource addition conditions;
[0010] Start a job release cycle, where the number of cycles is the minimum value between the number of jobs in the job release queue and the set number of successful single job releases;
[0011] In the loop process, for the i-th job in the job release queue, in response to the disk IO (Input / Output) occupied by the application to which the i-th job belongs not being overloaded and the resources currently applied for by the i-th job not exceeding the resource quota limit, an available resource list is generated for the i-th job; wherein i is a positive integer;
[0012] In response to the presence of a physical node in the available resource list that meets the current resource demand of the i-th job, allocating resources on the physical node to the i-th job;
[0013] In response to the i-th job being a resource addition job, sending a resource addition notification to the i-th job, and updating the resource usage of the i-th job.
[0014] In a possible implementation, the method further includes:
[0015] Generate a resource quota usage list based on the resource usage of the currently running job; wherein the resource quota usage list records the actual usage quota corresponding to each resource quota rule;
[0016] Loop through the resource quota restriction list and the resource quota usage list;
[0017] For the i-th job, matching the keys in the resource quota restriction list and the resource quota usage list with the job keywords of the i-th job;
[0018] In response to the current resource demand of the i-th job being less than the remaining resource quota corresponding to the corresponding resource quota rule, it is determined that the resources currently applied for by the i-th job do not exceed the resource quota limit.
[0019] In a possible implementation, generating an available resource list for the i-th job includes:
[0020] Obtain a resource information list; wherein the resource information list includes static resource information and dynamic resource information;
[0021] Eliminate physical nodes without available resources from the resource information list;
[0022] Remove the physical node that cannot be used by the i-th job from the resource information list;
[0023] According to the node sorting strategy, the remaining available physical nodes in the resource information list are sequentially adjusted to obtain the available resource list.
[0024] In a possible implementation, adjusting the order of the remaining available physical nodes in the resource information list according to the node sorting strategy to obtain the available resource list includes:
[0025] According to the number of jobs currently running on the remaining available physical nodes, the computing characteristics of the jobs currently running and the load conditions of the remaining available physical nodes, the remaining available physical nodes are sequentially adjusted to obtain the available resource list.
[0026] In a possible implementation, obtaining the job release queue includes:
[0027] Periodically obtaining a first job list that is queued and waiting to be scheduled; wherein the first job list includes jobs that are not currently running but meet running conditions;
[0028] Adjusting the order of the jobs in the first job list according to the queuing waiting time of each job in the first job list;
[0029] Periodically obtaining a second job list to which resources can be added; wherein the second job list includes jobs that are currently running but meet resource addition conditions;
[0030] According to the target resource ratio of each job in the second job list, the jobs in the second job list are sequentially adjusted; wherein, for any job in the second job list, the target resource ratio of the job refers to the ratio of the number of resources currently applied for by the job to the minimum number of resources required to start the job;
[0031] The sorted second job list is appended to the sorted first job list to obtain the job release queue.
[0032] In a possible implementation, the method further includes:
[0033] In response to the absence of a physical node in the available resource list that satisfies the current resource demand of the i-th job, suspending resource allocation for the i-th job, and detecting whether there is currently an over-allocated resource job; wherein the over-allocated resource job refers to a job whose resource usage exceeds the corresponding resource quota limit;
[0034] In response to the resource over-allocation job using the resources of the i-th job, adding the resource over-allocation job to a third job list of jobs that need to reclaim resources;
[0035] According to the resource over-allocation ratio of each job in the third job list, the jobs in the third job list are sequentially adjusted;
[0036] According to the resource gap of the i-th job, a resource recovery request is sent to the first resource overallocation job in the third job list; wherein the resource recovery request is used to instruct the first resource overallocation job to exit the resource overallocation node;
[0037] The running priority of the i-th job is set to the highest.
[0038] In a possible implementation, the method further includes:
[0039] In response to the jth job in the job release queue being finished, a resource recovery operation is performed, and according to the resource usage of the jth job, a remaining resource quota corresponding to a corresponding resource quota rule is updated.
[0040] In another aspect, a resource management device is provided, the device comprising:
[0041] The setting module is configured to set resource quota rules in multiple resource dimensions for different resource usage objects for jobs in the form of key-value pairs to obtain a resource quota restriction list; wherein the object keyword of the resource usage object is used as the key, and the resource quota amount of the resource usage object is used as the value; the type of the resource usage object includes at least: user, department and project; the multiple resource dimensions include at least: CPU, physical node and GPU;
[0042] The acquisition module is configured to acquire a job release queue; wherein the job release queue includes currently schedulable jobs; the currently schedulable jobs include jobs that are not currently running but meet the running conditions and jobs that are currently running but meet the resource addition conditions;
[0043] The scheduling module is configured to start a job release cycle, where the number of cycles is the minimum value between the number of jobs in the job release queue and the set number of successful single job releases; during the cycle, for the i-th job in the job release queue, in response to the disk IO occupied by the application to which the i-th job belongs not being overloaded, and the resources currently applied for by the i-th job not exceeding the resource quota limit, an available resource list is generated for the i-th job; wherein i is a positive integer;
[0044] The scheduling module is further configured to, in response to the presence of a physical node in the available resource list that meets the current resource requirements of the i-th job, allocate resources on the physical node to the i-th job; in response to the i-th job being a resource addition job, send a resource addition notification to the i-th job and update the resource usage of the i-th job.
[0045] In a possible implementation, the scheduling module is further configured to:
[0046] Generate a resource quota usage list based on the resource usage of the currently running job; wherein the resource quota usage list records the actual usage quota corresponding to each resource quota rule;
[0047] Loop through the resource quota restriction list and the resource quota usage list;
[0048] For the i-th job, matching the keys in the resource quota restriction list and the resource quota usage list with the job keywords of the i-th job;
[0049] In response to the current resource demand of the i-th job being less than the remaining resource quota corresponding to the corresponding resource quota rule, it is determined that the resources currently applied for by the i-th job do not exceed the resource quota limit.
[0050] In a possible implementation, the scheduling module is configured to:
[0051] Obtain a resource information list; wherein the resource information list includes static resource information and dynamic resource information;
[0052] Eliminate physical nodes without available resources from the resource information list;
[0053] Remove the physical node that cannot be used by the i-th job from the resource information list;
[0054] According to the node sorting strategy, the remaining available physical nodes in the resource information list are sequentially adjusted to obtain the available resource list.
[0055] In a possible implementation, the scheduling module is configured to:
[0056] According to the number of jobs currently running on the remaining available physical nodes, the computing characteristics of the jobs currently running and the load conditions of the remaining available physical nodes, the remaining available physical nodes are sequentially adjusted to obtain the available resource list.
[0057] In a possible implementation, the acquisition module is configured to:
[0058] Periodically obtaining a first job list that is queued and waiting to be scheduled; wherein the first job list includes jobs that are not currently running but meet running conditions;
[0059] Adjusting the order of the jobs in the first job list according to the queuing waiting time of each job in the first job list;
[0060] Periodically obtaining a second job list to which resources can be added; wherein the second job list includes jobs that are currently running but meet resource addition conditions;
[0061] According to the target resource ratio of each job in the second job list, the jobs in the second job list are sequentially adjusted; wherein, for any job in the second job list, the target resource ratio of the job refers to the ratio of the number of resources currently applied for by the job to the minimum number of resources required to start the job;
[0062] The sorted second job list is appended to the sorted first job list to obtain the job release queue.
[0063] In a possible implementation, the scheduling module is further configured to:
[0064] In response to the absence of a physical node in the available resource list that satisfies the current resource demand of the i-th job, suspending resource allocation for the i-th job, and detecting whether there is currently an over-allocated resource job; wherein the over-allocated resource job refers to a job whose resource usage exceeds the corresponding resource quota limit;
[0065] In response to the resource over-allocation job using the resources of the i-th job, adding the resource over-allocation job to a third job list of jobs that need to reclaim resources;
[0066] According to the resource over-allocation ratio of each job in the third job list, the jobs in the third job list are sequentially adjusted;
[0067] According to the resource gap of the i-th job, a resource recovery request is sent to the first resource overallocation job in the third job list; wherein the resource recovery request is used to instruct the first resource overallocation job to exit the resource overallocation node;
[0068] The running priority of the i-th job is set to the highest.
[0069] In a possible implementation manner, the device further includes:
[0070] The recycling module is configured to execute a resource recycling operation in response to the completion of the running of the jth job in the job release queue, and update the remaining resource quota corresponding to the corresponding resource quota rule according to the resource usage of the jth job.
[0071] On the other hand, a computer device is provided, the device comprising a processor and a memory, the memory storing at least one program code, the at least one program code being loaded and executed by the processor to implement the above-mentioned resource management method.
[0072] On the other hand, a computer-readable storage medium is provided, wherein at least one program code is stored in the storage medium, and the at least one program code is loaded and executed by a processor to implement the above-mentioned resource management method.
[0073] On the other hand, a computer program product or a computer program is provided, which includes a computer program code, the computer program code is stored in a computer-readable storage medium, a processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device executes the above-mentioned resource management method.
[0074] The resource management solution provided in the embodiment of the present application can realize efficient resource management of jobs.
[0075] In detail, when performing resource scheduling, the solution first obtains the job release queue, and the job release queue includes the currently schedulable jobs. Among them, the currently schedulable jobs include not only jobs that are not currently running but meet the running conditions, but also jobs that are currently running but meet the resource addition conditions, that is, elastic jobs. After that, the job release cycle is started, and during the cycle, for each job that can be scheduled currently, in response to the disk IO occupied by the application to which the job belongs is not overloaded, and the resources currently applied for by the job do not exceed the resource quota limit, a list of available resources is automatically generated for the job, and based on the list of available resources, the job is distributed to the most suitable physical node for execution.
[0076] Since the solution can dynamically allocate resources to non-running jobs and elastic jobs, it ensures resource utilization when resources are sufficient; in addition, the solution allocates resources to jobs only when the disk IO occupied by the application to which the job belongs is not overloaded and the resources currently applied for by the job do not exceed the resource quota limit, thus ensuring the service quality when resources are tight. In other words, the solution can dynamically adjust the use of resources by jobs, improving the resource utilization of the entire system and the operating efficiency of jobs. BRIEF DESCRIPTION OF THE DRAWINGS
[0077] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0078] Figure 1 This is a schematic diagram of a possible implementation environment of the resource management solution provided in the embodiment of the present application;
[0079] Figure 2 is a flow chart of a resource management method provided by an embodiment of the present application;
[0080] Figure 3 It is a schematic diagram of a resource allocation process provided by an embodiment of the present application;
[0081] Figure 4 is a flow chart of another resource management method provided by an embodiment of the present application;
[0082] Figure 5 This is a comparison chart of the job operation efficiency of the method provided in the embodiment of the present application and the static resource division;
[0083] Figure 6 It is a structural diagram of a resource management device provided in an embodiment of the present application;
[0084] Figure 7 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0085] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.
[0086] In this application, the terms "first", "second", etc. are used to distinguish the same or similar items with substantially the same role and function. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there a limitation on the quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms.
[0087] These terms are only used to distinguish one element from another element. For example, without departing from the scope of various examples, a first element can be referred to as a second element, and similarly, a second element can also be referred to as a first element. Both the first element and the second element can be elements, and in some cases, can be separate and different elements.
[0088] Here, at least one means one or more than one, for example, at least one element can be one element, two elements, three elements, or any other integer greater than or equal to one. And multiple means two or more than two, for example, multiple elements can be two elements, three elements, or any other integer greater than or equal to two.
[0089] The "and / or" mentioned in this article indicates that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship.
[0090] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.
[0091] At present, the First Input First Output (FIFO) algorithm is widely used in mainstream resource management or job scheduling systems. That is, the system allocates resources to each job according to the order in which the jobs are submitted. In detail, when allocating resources to any job, it is determined whether the current remaining resources meet the resource requirements of the job based on the resource usage in the current system; if they meet, the job is released and starts running after obtaining resources; if not, the job will remain in a waiting state until the resources are met.
[0092] However, this resource allocation algorithm may lead to the following situation: the probability of jobs with low resource usage obtaining resources is much higher than the probability of jobs with high resource usage obtaining resources. Some jobs may even fail to obtain resources and thus fail to run because they require too many resources.
[0093] In response to the above situation, in the actual production environment, in order to ensure the fairness of resource usage by resource users (such as projects or users) and avoid the situation where some projects or users cannot obtain resources to run jobs, managers will split the cluster resources and then use static partitioning to allocate resources to projects or users. This static partitioning method ensures that every project or user can obtain resources to run jobs. However, the static partitioning method will also introduce other problems:
[0094] On the one hand, if a project does not have seismic operations temporarily, the resources reserved for it cannot be used by other projects, resulting in a waste of resources; on the other hand, if a project temporarily needs more resources due to urgency, this requires system administrators to perform relevant configurations, which greatly increases the workload of system configuration and maintenance, and the real-time performance of such configuration modifications is poor.
[0095] Among them, the seismic job used to process seismic data in the related art has a major feature: the running job cannot be preempted for resources, otherwise the entire job will fail. This feature will cause a contradiction between the job running and the resource utilization of the cluster.
[0096] In detail, if job running is given priority, since each project or user has a strict static minimum resource requirement, in order to meet the running requirements of the job, idle resources will not be allocated to other jobs, making it difficult to achieve resource reuse, resulting in serious resource waste and reducing the peak computing power of the system to a certain extent. In addition, if the resource utilization of the cluster is given priority, resources will be freely allocated to global jobs, and the jobs submitted first will get resources first. When a project or user that does not meet the minimum resource requirement submits a new job, if there are no available resources at this time, the system usually adopts the following two scheduling strategies:
[0097] Scheduling strategy 1: Do not allocate available resources to the job, so the operation of the project or user's job will be affected;
[0098] Scheduling strategy 2: Preempt the computing resources of other jobs. Since most jobs (high-performance computing) take a long time to run and the running time is unpredictable, the failure of running jobs caused by preemption will lead to a waste of computing and is unreasonable for the projects or users whose resources are preempted.
[0099] In response to these problems, the embodiments of the present application propose a resource management solution, which dynamically allocates and recycles resources for seismic operations. In terms of resource allocation, it is no longer simply emphasized that the system throughput is maximized. The operation of the project or user is also very important. The overall best does not mean individual satisfaction, and it may be that no one is satisfied. Moreover, if there are no restrictions on the available resources of the project or user, then the project or user will tend to over-consume resources, which may result in the worst overall energy efficiency. Therefore, the overall goal of this solution is to ensure service quality when resources are tight and to improve utilization when resources are sufficient, that is, to have the characteristics of static resource division and dynamic sharing at the same time.
[0100] In other words, the embodiment of the present application provides a dynamic resource allocation and recovery strategy for seismic operations, which can dynamically adjust the use of resources by the operation according to the current workload and the busyness of the resources in the system, thereby improving the resource utilization of the entire system and the operation efficiency of the operation. That is, for seismic operations, efficient resource allocation and recovery can be achieved, which, on the one hand, achieves the purpose of improving the utilization of cluster resources, and on the other hand, ensures that all projects or users in the system can obtain resources to run operations.
[0101] Figure 1 This is a schematic diagram of a possible implementation environment of the resource management solution provided in the embodiment of the present application.
[0102] See also Figure 1 , the implementation environment includes: a job scheduler 11 and a physical node 12.
[0103] For example, the job scheduler 11 is used to dynamically allocate and recycle resources for performing seismic jobs based on the C++ language, and the jobs are run on the physical node 12. In addition, taking the distribution of resources by department as an example, the jobs of each department are formed Figure 1 In addition, the job scheduler 11 will specify resource quotas for each department in multiple resource dimensions (CPU, node, GPU), and set up an additional elastic job queue to dynamically allocate resources among departments and achieve the purpose of rationally utilizing idle resources.
[0104] Based on the above implementation environment, this resource management solution achieves at least the following aspects:
[0105] 1. The embodiment of the present application provides a multi-dimensional resource quota management method, which supports resource quota setting in multiple resource dimensions through the user, department, project and other extended attributes of the job.
[0106] 2. The embodiment of the present application provides a resource allocation method based on resource quotas, which supports dividing jobs into multiple queues, queuing jobs in the queues according to a certain job queuing strategy, and then selecting the job with the highest priority and distributing it to the most suitable physical node among all idle resources for execution. In addition, the resources used when the job is running are included in the quota, which will be described in detail later. In addition, after the job runs, the resources are released, and other jobs can continue to use the released resources. If the resources used by the job reach the resource quota limit, the scheduling of this type of job is suspended until sufficient idle resources are available.
[0107] 3. In response to the problem that nodes cannot be dynamically scaled after the job is running, the embodiment of the present application provides a resource negotiation mechanism, which realizes the elastic modification of the resources used by the job through negotiation with the job. For jobs that support elastic scaling of resources (the resources used by such jobs can vary between a minimum number of resources and a maximum number of resources), resources can be allocated to such jobs according to the minimum resource requirements when allocating resources, and the resources used when the job is running are included in the corresponding quota, and then the corresponding job is added to the elastic job queue. In addition, if there are still idle resources available, it is possible to negotiate with such jobs to continue to add resources until the maximum number of resources; and if resources are tight, it is also possible to negotiate with such jobs to recover the extra resources allocated to it. Exemplarily, the scheduling algorithm of the elastic job queue can be carried out according to the principles of "first come, first served" and "whoever occupies more, gives first", which is not limited in this application.
[0108] Figure 2 is a flow chart of a resource management method provided by an embodiment of the present application. The execution subject of the method is a computer device, such as a job scheduler (also called a job scheduling system, or scheduling system for short). Figure 2 , the method comprising:
[0109] 201. The computer device sets resource quota rules in multiple resource dimensions for different resource usage objects for the job in the form of key-value pairs to obtain a resource quota restriction list.
[0110] For a detailed description of this step, please refer to the subsequent step 401.
[0111] 202. The computer device obtains a job release queue; wherein the job release queue includes currently schedulable jobs; currently schedulable jobs include jobs that are not currently running but meet running conditions and jobs that are currently running but meet resource addition conditions.
[0112] This step is used to obtain the currently schedulable jobs.
[0113] In the embodiment of the present application, the scheduling system automatically collects information about jobs currently waiting for resources at regular intervals, allocates resources to jobs waiting to be run, and adds resources to jobs that can add resources. For example, obtaining the currently schedulable jobs includes but is not limited to the following steps:
[0114] 2021. Obtain jobs that are waiting to be scheduled but have not started running. By scanning the global job queue, obtain jobs that are not currently running but meet the running conditions, and add such jobs to the list of schedulable jobs. This job list is also referred to as the first job list in this article.
[0115] 2022. Sort the jobs in the list of schedulable jobs according to the first job sorting strategy. For example, the default sorting strategy is the queuing waiting time of the job, which is not limited in this application.
[0116] 2023. Obtain the jobs that are currently running but can be supplemented with resources, and add such jobs to the list of scalable resource jobs. This job list is also referred to as the second job list in this article.
[0117] 2024. Sort the jobs in the scalable resource job list according to the second job sorting strategy. Exemplarily, the default sorting strategy is the ratio of the number of additional resources of the job to the number of resources required for startup.
[0118] 2025. Append the sorted list of scalable resource jobs to the sorted list of schedulable jobs.
[0119] In other words, obtaining the job release queue includes but is not limited to the following methods:
[0120] Periodically obtaining a first job list queued for scheduling; wherein the first job list includes jobs that are not currently running but meet running conditions;
[0121] Adjusting the order of the jobs in the first job list according to the queue waiting time of each job in the first job list;
[0122] Periodically obtaining a second job list to which resources can be added; wherein the second job list includes jobs that are currently running but meet resource addition conditions;
[0123] According to the target resource ratio of each job in the second job list, the jobs in the second job list are sequentially adjusted; wherein, for any job in the second job list, the target resource ratio of the job refers to the ratio of the number of resources currently applied for by the job to the minimum number of resources required to start the job;
[0124] The sorted second job list is appended to the sorted first job list to obtain a job release queue.
[0125] 203. The computer device starts a job release cycle, wherein the number of cycles is the minimum value between the number of jobs in the job release queue and the set number of successful single job releases; during the cycle, for the ith job in the job release queue, in response to the fact that the disk IO occupied by the application to which the ith job belongs is not overloaded and the resources currently applied for by the ith job do not exceed the resource quota limit, the computer device generates an available resource list for the ith job.
[0126] This step is used to allocate resources to the job.
[0127] In the embodiment of the present application, the scheduling system will start a job release cycle to release jobs in the job release queue. In addition, in order to avoid too many jobs being released at one time, which affects the next release cycle, the scheduling system sets a single release job success number by default. The number of cycles is the minimum value of the two values, the number of jobs in the job release queue and the number of single release job success numbers.
[0128] In a possible implementation, generating an available resource list for the i-th job includes but is not limited to the following steps:
[0129] 2031. Obtain a resource information list; wherein the resource information list includes static resource information and dynamic resource information. This step is used to obtain all resource information. Exemplarily, the resource information obtained mainly includes static resource information such as the number of CPUs and GPUs of the node, the node group to which the node belongs, and dynamic resource information such as the currently available number of CPUs and GPUs, which is not limited in this application.
[0130] 2032. Remove the physical nodes without available resources from the resource information list.
[0131] 2033. Remove the physical nodes that cannot be used by the ith job from the resource information list. For this step, the physical nodes that cannot be used by the ith job can be removed from the resource information list based on the access restriction rule of the ith job, which is not limited in this application.
[0132] 2034. According to the node sorting strategy, the order of the remaining available physical nodes in the resource information list is adjusted to obtain an available resource list.
[0133] In another possible implementation, according to the node sorting strategy, the remaining available physical nodes in the resource information list are adjusted in order to obtain a list of available resources, including but not limited to the following methods: according to the number of jobs currently running on the remaining available physical nodes, the computing characteristics of the currently running jobs and the load conditions of the remaining available physical nodes, the remaining available physical nodes are adjusted in order to obtain a list of available resources. For example, the physical nodes with fewer jobs and lower loads are ranked higher in the list of available resources by default. Among them, the computing characteristics of the running job can be whether the running job is a CPU-intensive job or a GPU-intensive job, which is not limited in this application.
[0134] For example, assuming that the i-th job is a CPU-intensive job, for two physical nodes A and B with the same conditions, assuming that a CPU-intensive job is running on physical node A, but no CPU-intensive job is running on physical node B, the i-th job tends to run on physical node B.
[0135] Among them, the above steps 2032-2034 correspond to Figure 3 "Obtaining schedulable physical nodes according to filtering conditions and sorting the schedulable physical nodes" in.
[0136] 204. In response to the presence of a physical node in the available resource list that meets the current resource requirement of the i-th job, allocate resources on the physical node to the i-th job.
[0137] This step is used to select resources from the available resource list to be assigned to the i-th job.
[0138] 205. In response to the i-th job being a resource addition job, sending a resource addition notification to the i-th job, and updating resource usage of the i-th job.
[0139] If the i-th job has been running as a job with dynamically scalable resources, resources are allocated to the job and the additional resource information is notified to the job.
[0140] The resource management solution provided in the embodiment of the present application can realize efficient resource management of jobs.
[0141] In detail, when performing resource scheduling, the solution first obtains the job release queue, and the job release queue includes the currently schedulable jobs. Among them, the currently schedulable jobs include not only jobs that are not currently running but meet the running conditions, but also jobs that are currently running but meet the resource addition conditions, that is, elastic jobs. After that, the job release cycle is started, and during the cycle, for each job that can be scheduled currently, in response to the disk IO occupied by the application to which the job belongs is not overloaded, and the resources currently applied for by the job do not exceed the resource quota limit, a list of available resources is automatically generated for the job, and based on the list of available resources, the job is distributed to the most suitable physical node for execution.
[0142] Since this solution can dynamically allocate resources to non-running jobs and elastic jobs, it ensures resource utilization when resources are sufficient. In addition, this solution allocates resources to jobs only when the disk IO occupied by the application to which the job belongs is not overloaded and the resources currently requested by the job do not exceed the resource quota limit, thus ensuring service quality when resources are tight.
[0143] The above is only a brief introduction to some technical details of the resource management solution provided in the embodiment of the present application. Figure 4 The illustrated embodiment provides a detailed introduction to the resource management solution.
[0144] Figure 4 is a flow chart of another resource management method provided by an embodiment of the present application. The execution subject of the method is a computer device, such as a job scheduler. Figure 4 , the method comprising:
[0145] 401. The computer device sets resource quota rules in multiple resource dimensions for different resource usage objects for the job in the form of key-value pairs, and obtains a resource quota restriction list.
[0146] This step is used to set resource quotas.
[0147] It should be noted that in the resource quota restriction list, the object keyword (also called quota keyword) of the resource usage object is used as the key, and the resource quota amount of the resource usage object is used as the value; among which, the types of resource usage objects include at least: user, department and project; and the above-mentioned multiple resource dimensions include at least: CPU, physical node and GPU.
[0148] In a possible implementation, resource quota rules are set in multiple resource dimensions for different resource users of jobs, including but not limited to the following steps:
[0149] 4011. Determine the quota keyword.
[0150] The quota keyword can be a single keyword, for example, it only applies to users or projects, such as user A and project B. In addition, you can also use a combination keyword to apply to multiple keywords at the same time. Keywords can be connected by operators such as and or or, such as user A and project B.
[0151] 4012. Determine resource quotas.
[0152] Resource quotas can be set on CPU, nodes, and GPUs. Just like keyword settings, you can limit only a specific resource or limit multiple resources at the same time. If you limit the quotas of multiple resources, the restrictions are in an OR relationship.
[0153] 4013. Adjust resource quotas. Adjust the effective resource quotas. The resource dimensions that can be adjusted are CPU, node, and GPU.
[0154] 4014. Cancel resource quota. Cancel the effective resource quota setting. After cancellation, the resource quota will be invalid directly, and subsequent job runs will no longer be restricted by this resource quota.
[0155] 4015. Obtain resource quota. Exemplarily, the scheduling system supports managers to query resource quota information in effect in the current system by providing an interface.
[0156] 402. The computer device obtains a job release queue; wherein the job release queue includes currently schedulable jobs; currently schedulable jobs include jobs that are not currently running but meet running conditions and jobs that are currently running but meet resource addition conditions.
[0157] This step may refer to the aforementioned step 201 and will not be described in detail here.
[0158] 403. The computer device starts a job release cycle, wherein the number of cycles is the minimum value between the number of jobs in the job release queue and the set number of successful single job releases; during the cycle, for the i-th job in the job release queue, in response to the fact that the disk IO occupied by the application to which the i-th job belongs is not overloaded and the resources currently applied for by the i-th job do not exceed the resource quota limit, the computer device generates an available resource list for the i-th job.
[0159] During the cycle, the scheduling system will obtain the job information of the currently schedulable jobs, such as the resource type (CPU, node or GPU) and the required resource quantity of the job. In addition, the scheduling system will also obtain the current number of tasks of each physical node, which is not limited in this application.
[0160] In a possible implementation, the embodiment of the present application determines whether the resources currently applied for by the i-th job exceed the resource quota limit through the following steps:
[0161] 4031. Obtain a resource quota restriction list; wherein the resource quota restriction list records resource quota rules for different resource usage objects of a job in multiple resource dimensions in the form of key-value pairs.
[0162] 4032. Generate a resource quota usage list according to the resource usage of the currently running job; wherein the resource quota usage list records the actual usage quota corresponding to each resource quota rule.
[0163] That is, this step is used to obtain the actual usage of the resource quota restricted by each resource quota rule.
[0164] 4033. Loop through the resource quota restriction list and the resource quota usage list; for the ith job, match the keys in the resource quota restriction list and the resource quota usage list with the job keywords of the ith job; in response to the current resource demand of the ith job being less than the remaining resource quota corresponding to the corresponding resource quota rule, determine that the resources currently requested for use by the ith job do not exceed the resource quota limit.
[0165] Exemplarily, the job keyword includes a quota keyword to indicate which user, project or department the corresponding job comes from. Therefore, by matching the quota keywords in the two lists with the job keywords and the resource application information of the job, it can be determined whether the job exceeds the resource quota limit.
[0166] It should be noted that the corresponding resource quota rule is the resource quota rule that matches the job keyword of the i-th job. For example, suppose that this resource quota rule limits the resource quota of user A to 100 CPUs, and currently other jobs of user A have used 50 CPUs, and the remaining resource quota is 50 CPUs; suppose that the number of CPUs currently applied for by the i-th job (from user A) is less than 50, then it is determined that the resources currently applied for by the i-th job do not exceed the resource quota limit.
[0167] In addition, this step corresponds to Figure 3 Determine whether resource quotas are exceeded.
[0168] 404. In response to the disk IO occupied by the application to which the i-th job belongs being overloaded, the computer device stops allocating resources to the i-th job.
[0169] like Figure 3As shown, the scheduling system will determine whether the disk IO occupied by the application to which the job belongs is overloaded; if so, the resource allocation for this job will be directly terminated.
[0170] 405. In response to the fact that the disk IO occupied by the application to which the i-th job belongs is not overloaded, but the resources currently applied for by the i-th job exceed the resource quota limit, the computer device stops allocating resources to the i-th job.
[0171] like Figure 3 As shown, although the disk IO occupied by the application to which the job belongs is not overloaded, the resources currently applied for by the job exceed the resource quota limit, and the resource allocation for this job is directly terminated.
[0172] 406. In response to the presence of a physical node in the available resource list that meets the current resource requirements of the i-th job, the computer device allocates resources on the physical node to the i-th job.
[0173] In the embodiment of the present application, when resources are allocated to the i-th job, the corresponding resource usage information and quota information are also updated, that is, the resource usage of the job and the actual usage quota corresponding to the corresponding resource quota rule are updated.
[0174] In addition, if the job has been running as a dynamically scalable resource job, resources are allocated to the job and the additional resource information is notified to the job. In addition, after adding resources, only the resource usage information of the job is updated, but it is not included in the quota information, that is, the actual usage quota corresponding to the corresponding resource quota rule is not updated, and this resource allocation ends.
[0175] 407. In response to the absence of a physical node in the available resource list that meets the current resource demand of the i-th job, the computer device stops allocating resources to the i-th job and determines whether to perform a resource recovery operation.
[0176] In a possible implementation, the embodiment of the present application determines whether to perform a resource recovery operation through the following steps:
[0177] 4071. Detect whether there is currently any resource overallocation job; wherein, the resource overallocation job refers to a job whose resource usage exceeds the corresponding resource quota limit; in response to the resource overallocation job using the resources of the i-th job, the resource overallocation job is added to the third job list that needs to reclaim resources.
[0178] 4072. According to the resource over-allocation ratio of each job in the third job list, adjust the order of the jobs in the third job list.
[0179] Illustratively, the greater the resource over-allocation ratio of a job, the higher the job is ranked in the third job list, which is not limited in the present application.
[0180] 4073. Send a resource recovery request to the first resource overallocation job in the third job list according to the resource gap of the i-th job; wherein the resource recovery request is used to instruct the first resource overallocation job to exit the resource overallocation node.
[0181] In an embodiment of the present application, a job that receives a resource recycling request will automatically exit the resource over-allocated node.
[0182] 4074. Set the running priority of the i-th job to the highest and wait for the next resource allocation.
[0183] In another possible implementation, the embodiment of the present application also includes a resource recovery process.
[0184] Accordingly, in response to the jth job in the job release queue being finished, a resource recovery operation is performed, and the remaining resource quota corresponding to the corresponding resource quota rule is updated according to the resource usage of the jth job. At the same time, the resource usage information of the job is updated to 0.
[0185] Wherein, i and j are both positive integers. Exemplarily, the end of the j-th job includes, but is not limited to, the following situations: normal exit, abnormal termination, or termination by an administrator, which is not limited in this application.
[0186] The resource management solution provided in the embodiment of the present application can realize dynamic allocation and recovery of resources.
[0187] In detail, the solution supports resource quota management in multiple resource dimensions. In addition, the solution has the characteristics of both static resource division and dynamic sharing, but the division here is not a static grouping of physical resources, but the setting of resource quotas. Among them, setting resource quotas provides basic resource guarantees for jobs. As long as the resources applied for by the job do not exceed the corresponding resource quota limit, idle resources will be allocated to the job first; if there are no idle resources available, resources will be recovered from elastic jobs to meet the operation requirements of the job, thereby ensuring service quality. Among them, the emergence of elastic jobs enables jobs to temporarily borrow idle resources outside of resource quotas, realize resource sharing, and improve resource utilization.
[0188] In summary, this solution achieves efficient resource management, that is, it achieves the scheduling goal of ensuring service quality when resources are tight and improving utilization when resources are sufficient. This solution guarantees the operation requirements of the job and realizes cluster resource sharing. This solution runs stably when processing earthquake jobs and provides good user experience. Figure 5This is a comparison chart of the job operation efficiency between this method and conventional static resource partitioning.
[0189] In addition, in actual application, taking resource distribution by department as an example, the sum of resource quotas of all departments can be less than or equal to the actual number of physical resources to provide absolute resource guarantee; in addition, resource over-allocation can also be set according to a certain ratio. Since resources are shared and the operation and use of jobs vary, in most cases, each job can still run smoothly.
[0190] Figure 6 Schematic diagram of a resource management device provided in an embodiment of the present application. Figure 6 , the device comprises:
[0191] The setting module 601 is configured to set resource quota rules in multiple resource dimensions for different resource usage objects for jobs in the form of key-value pairs to obtain a resource quota restriction list; wherein the object keyword of the resource usage object is used as the key, and the resource quota amount of the resource usage object is used as the value; the type of the resource usage object includes at least: user, department and project; the multiple resource dimensions include at least: CPU, physical node and GPU;
[0192] The acquisition module 602 is configured to acquire a job release queue; wherein the job release queue includes currently schedulable jobs; the currently schedulable jobs include jobs that are not currently running but meet the running conditions and jobs that are currently running but meet the resource addition conditions;
[0193] The scheduling module 603 is configured to start a job release cycle, where the number of cycles is the minimum value between the number of jobs in the job release queue and the set number of successful single job releases; during the cycle, for the i-th job in the job release queue, in response to the disk IO occupied by the application to which the i-th job belongs not being overloaded and the resources currently applied for by the i-th job not exceeding the resource quota limit, an available resource list is generated for the i-th job;
[0194] The scheduling module 603 is also configured to allocate resources on the physical node to the i-th job in response to the presence of a physical node in the available resource list that meets the current resource requirements of the i-th job; in response to the i-th job being a resource addition job, send a resource addition notification to the i-th job and update the resource usage of the i-th job.
[0195] The resource management scheme provided by the embodiment of the present application can realize efficient resource management of jobs. In detail, when performing resource scheduling, the scheme first obtains the job release queue, and the job release queue includes the currently schedulable jobs. Among them, the currently schedulable jobs include jobs that are currently running but meet the resource addition conditions, that is, flexible jobs, in addition to jobs that are not currently running but meet the running conditions. After that, the job release cycle is started, and during the cycle process, for each job that can be scheduled currently, in response to the disk IO occupied by the application to which the job belongs is not overloaded, and the resources currently applied for by the job do not exceed the resource quota limit, a list of available resources is automatically generated for the job, and the job is distributed to the most suitable physical node for operation based on the available resource list. Since the scheme can dynamically allocate resources for non-running jobs and flexible jobs, resource utilization is ensured when resources are sufficient; in addition, the scheme only allocates resources to the job when the disk IO occupied by the application to which the job belongs is not overloaded and the resources currently applied for by the job do not exceed the resource quota limit, thereby ensuring the service quality when resources are tight.
[0196] In a possible implementation, the scheduling module is further configured to:
[0197] Generate a resource quota usage list based on the resource usage of the currently running job; wherein the resource quota usage list records the actual usage quota corresponding to each resource quota rule;
[0198] Loop through the resource quota restriction list and the resource quota usage list;
[0199] For the i-th job, matching the keys in the resource quota restriction list and the resource quota usage list with the job keywords of the i-th job;
[0200] In response to the current resource demand of the i-th job being less than the remaining resource quota corresponding to the corresponding resource quota rule, it is determined that the resources currently applied for by the i-th job do not exceed the resource quota limit.
[0201] In a possible implementation, the scheduling module is configured to:
[0202] Obtain a resource information list; wherein the resource information list includes static resource information and dynamic resource information;
[0203] Eliminate physical nodes without available resources from the resource information list;
[0204] Remove the physical node that cannot be used by the i-th job from the resource information list;
[0205] According to the node sorting strategy, the remaining available physical nodes in the resource information list are sequentially adjusted to obtain the available resource list.
[0206] In a possible implementation, the scheduling module is configured to:
[0207] According to the number of jobs currently running on the remaining available physical nodes, the computing characteristics of the jobs currently running and the load conditions of the remaining available physical nodes, the remaining available physical nodes are sequentially adjusted to obtain the available resource list.
[0208] In a possible implementation, the acquisition module is configured to:
[0209] Periodically obtaining a first job list that is queued and waiting to be scheduled; wherein the first job list includes jobs that are not currently running but meet running conditions;
[0210] Adjusting the order of the jobs in the first job list according to the queuing waiting time of each job in the first job list;
[0211] Periodically obtaining a second job list to which resources can be added; wherein the second job list includes jobs that are currently running but meet resource addition conditions;
[0212] According to the target resource ratio of each job in the second job list, the jobs in the second job list are sequentially adjusted; wherein, for any job in the second job list, the target resource ratio of the job refers to the ratio of the number of resources currently applied for by the job to the minimum number of resources required to start the job;
[0213] The sorted second job list is appended to the sorted first job list to obtain the job release queue.
[0214] In a possible implementation, the scheduling module is further configured to:
[0215] In response to the absence of a physical node in the available resource list that satisfies the current resource demand of the i-th job, suspending resource allocation for the i-th job, and detecting whether there is currently an over-allocated resource job; wherein the over-allocated resource job refers to a job whose resource usage exceeds the corresponding resource quota limit;
[0216] In response to the resource over-allocation job using the resources of the i-th job, adding the resource over-allocation job to a third job list of jobs that need to reclaim resources;
[0217] According to the resource over-allocation ratio of each job in the third job list, the jobs in the third job list are sequentially adjusted;
[0218] According to the resource gap of the i-th job, a resource recovery request is sent to the first resource overallocation job in the third job list; wherein the resource recovery request is used to instruct the first resource overallocation job to exit the resource overallocation node;
[0219] The running priority of the i-th job is set to the highest.
[0220] In a possible implementation, the device further includes:
[0221] The recycling module is configured to execute a resource recycling operation in response to the completion of the running of the jth job in the job release queue, and update the remaining resource quota corresponding to the corresponding resource quota rule according to the resource usage of the jth job.
[0222] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.
[0223] It should be noted that: the resource management device provided in the above embodiment only uses the division of the above functional modules as an example when performing resource management. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the resource management device provided in the above embodiment and the resource management method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.
[0224] Figure 7 It is a structural diagram of a computer device 700 provided in an embodiment of the present application.
[0225] The computer 700 may be a server. The computer device 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 701 and one or more memories 702, wherein the memory 702 stores at least one program code, and the at least one program code is loaded and executed by the processor 701 to implement the resource management method provided by the above-mentioned various method embodiments. Of course, the computer device 700 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output, and the computer device 700 may also include other components for implementing device functions, which will not be described in detail here.
[0226] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including a program code, and the program code can be executed by a processor in a computer device to complete the resource management method in the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.
[0227] In an exemplary embodiment, a computer program product or a computer program is also provided, which includes a computer program code, and the computer program code is stored in a computer-readable storage medium. A processor of a computer device reads the computer program code from the computer-readable storage medium, and the processor executes the computer program code, so that the computer device executes the above-mentioned resource management method.
[0228] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.
[0229] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. A resource management method, characterized in that: The method comprises: In the form of key-value pairs, resource quota rules are set for different resource usage objects for jobs in multiple resource dimensions to obtain a resource quota restriction list; wherein the object keyword of the resource usage object is used as the key, and the resource quota amount of the resource usage object is used as the value; the type of the resource usage object includes at least: user, department and project; the multiple resource dimensions include at least: central processing unit CPU, physical node and graphics processing unit GPU; Obtaining a job release queue; wherein the job release queue includes currently schedulable jobs; the currently schedulable jobs include jobs that are not currently running but meet running conditions and jobs that are currently running but meet resource addition conditions; Start a job release cycle, where the number of cycles is the minimum value between the number of jobs in the job release queue and the set number of successful single job releases; In the loop process, for the i-th job in the job release queue, in response to the disk input and output IO occupied by the application to which the i-th job belongs not being overloaded and the resources currently applied for by the i-th job not exceeding the resource quota limit, an available resource list is generated for the i-th job; wherein i is a positive integer; In response to the presence of a physical node in the available resource list that meets the current resource demand of the i-th job, allocating resources on the physical node to the i-th job; In response to the i-th job being a resource addition job, sending a resource addition notification to the i-th job, and updating the resource usage of the i-th job.
2. The method according to claim 1, characterized in that The method further comprises: Generate a resource quota usage list based on the resource usage of the currently running job; wherein the resource quota usage list records the actual usage quota corresponding to each resource quota rule; Loop through the resource quota restriction list and the resource quota usage list; For the i-th job, matching the keys in the resource quota restriction list and the resource quota usage list with the job keywords of the i-th job; In response to the current resource demand of the i-th job being less than the remaining resource quota corresponding to the corresponding resource quota rule, it is determined that the resources currently applied for by the i-th job do not exceed the resource quota limit.
3. The method according to claim 1, characterized in that The generating of the available resource list for the i-th job comprises: Obtain a resource information list; wherein the resource information list includes static resource information and dynamic resource information; Eliminate physical nodes without available resources from the resource information list; Remove the physical node that cannot be used by the i-th job from the resource information list; According to the node sorting strategy, the remaining available physical nodes in the resource information list are sequentially adjusted to obtain the available resource list.
4. The method according to claim 3, characterized in that The step of adjusting the order of the remaining available physical nodes in the resource information list according to the node sorting strategy to obtain the available resource list includes: According to the number of jobs currently running on the remaining available physical nodes, the computing characteristics of the jobs currently running and the load conditions of the remaining available physical nodes, the remaining available physical nodes are sequentially adjusted to obtain the available resource list.
5. The method according to claim 1, characterized in that The obtaining of the job release queue comprises: Periodically obtaining a first job list that is queued and waiting to be scheduled; wherein the first job list includes jobs that are not currently running but meet running conditions; Adjusting the order of the jobs in the first job list according to the queuing waiting time of each job in the first job list; Periodically obtaining a second job list to which resources can be added; wherein the second job list includes jobs that are currently running but meet resource addition conditions; According to the target resource ratio of each job in the second job list, the jobs in the second job list are sequentially adjusted; wherein, for any job in the second job list, the target resource ratio of the job refers to the ratio of the number of resources currently applied for by the job to the minimum number of resources required to start the job; The sorted second job list is appended to the sorted first job list to obtain the job release queue.
6. The method according to claim 1, characterized in that The method further comprises: In response to the absence of a physical node in the available resource list that satisfies the current resource demand of the i-th job, suspending resource allocation for the i-th job, and detecting whether there is currently an over-allocated resource job; wherein the over-allocated resource job refers to a job whose resource usage exceeds the corresponding resource quota limit; In response to the resource over-allocation job using the resources of the i-th job, adding the resource over-allocation job to a third job list of jobs that need to reclaim resources; According to the resource over-allocation ratio of each job in the third job list, the jobs in the third job list are sequentially adjusted; According to the resource gap of the i-th job, a resource recovery request is sent to the first resource overallocation job in the third job list; wherein the resource recovery request is used to instruct the first resource overallocation job to exit the resource overallocation node; The running priority of the i-th job is set to the highest.
7. The method according to claim 1, characterized in that The method further comprises: In response to the jth job in the job release queue being finished, a resource recovery operation is performed, and according to the resource usage of the jth job, a remaining resource quota corresponding to a corresponding resource quota rule is updated.
8. A resource management device, characterized in that: The device comprises: The setting module is configured to set resource quota rules in multiple resource dimensions for different resource usage objects for jobs in the form of key-value pairs to obtain a resource quota restriction list; wherein the object keyword of the resource usage object is used as the key, and the resource quota amount of the resource usage object is used as the value; the type of the resource usage object includes at least: user, department and project; the multiple resource dimensions include at least: CPU, physical node and GPU; The acquisition module is configured to acquire a job release queue; wherein the job release queue includes currently schedulable jobs; the currently schedulable jobs include jobs that are not currently running but meet the running conditions and jobs that are currently running but meet the resource addition conditions; The scheduling module is configured to start a job release cycle, where the number of cycles is the minimum value between the number of jobs in the job release queue and the set number of successful single job releases; during the cycle, for the i-th job in the job release queue, in response to the disk input and output IO occupied by the application to which the i-th job belongs not being overloaded, and the resources currently applied for by the i-th job not exceeding the resource quota limit, an available resource list is generated for the i-th job; wherein i is a positive integer; The scheduling module is further configured to, in response to the presence of a physical node in the available resource list that meets the current resource requirements of the i-th job, allocate resources on the physical node to the i-th job; in response to the i-th job being a resource addition job, send a resource addition notification to the i-th job and update the resource usage of the i-th job.
9. A computer device, characterized in that: The device comprises a processor and a memory, wherein at least one program code is stored in the memory, and the at least one program code is loaded and executed by the processor to implement the resource management method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: At least one program code is stored in the storage medium, and the at least one program code is loaded and executed by the processor to implement the resource management method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Application resource scheduling device and method based on weight
CN108121601A
Queue resource management method and device, and storage medium
CN109474543A
GPU resource management method, device and system and readable storage medium
CN114816741A
Resource management server, resource management system, resource management method, and resource management program
JP2012168585A
Performing resynchronization jobs in a distributed storage system based on a parallelism policy
US20210004163A1
