A method for allocating computing resources and related apparatuses

Abstract

The application provides a computing resource allocation method and a related device thereof. The method comprises the following steps: determining an available resource quota of a resource pool; based on the computing power requirement of a computing task and the available resource quota, configuring a computing resource quota comprising a planning quota and a temporary quota; in response to a submission request of the computing task, creating a training container in the resource pool, wherein the training container comprises a planning memory pool corresponding to the planning quota and a temporary memory pool corresponding to the temporary quota; calling the planning memory pool to execute the computing task in the training container; when the resource usage in the planning memory pool exceeds a first threshold, calling the temporary memory pool to execute the computing task, wherein the temporary memory pool and the resource pool perform resource flow conversion, so that the idle resource amount in the temporary memory pool does not exceed a second threshold. The application can be used to improve the utilization rate of computing power resources, facilitate the reasonable allocation of resources, improve the flow efficiency of heterogeneous computing power resources, and thus achieve the effect of cost reduction.

Description

Technical Field

[0001] This application relates to the field of heterogeneous computing power resource management technology, and in particular to a computing resource allocation method and related apparatus. Background Technology

[0002] With the rapid development of large-scale artificial intelligence (AI) models, the demand for computing power has exploded, directly leading to a dramatic increase in the scale of heterogeneous computing power. Heterogeneous computing power, as a key infrastructure supporting the efficient operation of complex AI tasks, has significantly improved data processing capabilities and energy efficiency. However, the deployment and maintenance costs of heterogeneous computing power are considerably higher than those of traditional CPU devices. The traditional approach employs a high-low priority allocation strategy, allocating high-priority quotas within the business plan and low-priority quotas outside the plan. Resources within the high-priority quota are readily available and will not be preempted during operation, while tasks submitted with low-priority quotas may be preempted and disrupted. Although using low-priority tasks allows AI personnel to submit tasks exceeding the planned quota, the task execution is subject to preemption and disruption, raising the barrier for AI personnel to access idle computing power and negatively impacting their user experience. Therefore, how to improve resource utilization while ensuring the stability of computing task execution is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0003] This application provides a computing resource allocation method and related apparatus. By configuring additional temporary quotas for computing tasks and transferring these temporary quotas to the resource pool, the method improves resource utilization while ensuring the stability of computing task execution.

[0004] One aspect of this application provides a method for allocating computing resources, including:

[0005] Determine the available resource quota for the resource pool;

[0006] Based on the computing power requirements of the computing task and the available resource quota, a computing resource quota is configured for the computing task, which includes a planned quota and a temporary quota.

[0007] In response to the submission request of the computing task, a training container is created in the resource pool based on the computing resource quota. The training container includes a planned memory pool corresponding to the planned quota and a temporary memory pool corresponding to the temporary quota.

[0008] The planned memory pool is invoked to execute the computational task in the training container;

[0009] When the resource usage in the planned memory pool exceeds the first threshold, the temporary memory pool is invoked to execute the computing task. The temporary memory pool and the resource pool perform resource transfer so that the amount of idle resources in the temporary memory pool does not exceed the second threshold.

[0010] In one possible implementation, before configuring computing resource quotas for the computing task based on its computing power requirements and available resource quotas, the following method is also included:

[0011] Based on the total quota of the resource pool, at least two dimensions of unusable resources are divided, including high-priority quota and flexible quota.

[0012] Based on the computing power requirements and available resource quotas of the computing task, configure computing resource quotas for the computing task, including:

[0013] Based on the available resource quota, determine the remaining quota in each dimension;

[0014] Obtain computational tasks from user input;

[0015] Assess the computing tasks and determine the computing power requirements;

[0016] Based on computing power requirements, a planned quota is allocated from the high-priority quota, and a temporary quota is allocated from the flexible quota.

[0017] In one possible implementation, prior to responding to the submission request for the computation task, the following is also included:

[0018] The remaining credit limit in the dimension and / or the utilization rate of the high-priority credit limit are output to the user. The utilization rate of the high-priority credit limit is the ratio of the configured planned credit limit to the high-priority credit limit.

[0019] In one possible implementation, after configuring computing resource quotas for the computing task based on its computing power requirements and available resource quotas, the method further includes:

[0020] If the configured temporary limit exceeds the flexible limit, a fault warning will be sent.

[0021] In one possible implementation, the computational resource quota also includes an unplanned quota, and the training container includes an unplanned memory pool corresponding to the unplanned quota. The resources in the planned memory pool and the temporary memory pool are unoccupiable resources, while the resources in the unplanned memory pool are occupiable resources.

[0022] After invoking the temporary memory pool to execute the computation task when the resource usage in the planned memory pool exceeds the first threshold, the following steps are also included:

[0023] When the resource usage in the temporary memory pool exceeds the third threshold, the unplanned memory pool is invoked to execute the computation task.

[0024] In one possible implementation, the planned memory pool is invoked to perform computational tasks within the training container, including:

[0025] Based on the submitted request, determine the task type of the computation task;

[0026] If the submitted request indicates that the task type is a low-priority task, then the unplanned memory pool is invoked to execute the computation task.

[0027] If the submitted request indicates that the task type is a high-priority task, then the planned memory pool is invoked to execute the computation task.

[0028] A second aspect of this application provides a computing resource allocation apparatus, comprising:

[0029] The determination module is used to determine the available resource quota of the resource pool;

[0030] The configuration module is used to configure computing resource quotas for computing tasks based on their computing power requirements and available resource quotas. The computing resource quotas include planned quotas and temporary quotas.

[0031] The creation module is used to respond to the submission request of computing tasks and create training containers in the resource pool based on the computing resource quota. The training containers include a planned memory pool corresponding to the planned quota and a temporary memory pool corresponding to the temporary quota.

[0032] The execution module is used to call the planned memory pool to perform computation tasks in the training container. When the resource usage in the planned memory pool exceeds the first threshold, the temporary memory pool is called to perform computation tasks. The temporary memory pool and the resource pool transfer resources to ensure that the amount of idle resources in the temporary memory pool does not exceed the second threshold.

[0033] One possible implementation also includes:

[0034] The partitioning module is used to partition the total amount of the resource pool into at least two dimensions of unavailable resources, including high-priority quota and flexible quota.

[0035] The configuration module is specifically used to determine the remaining quota in each dimension based on the available resource quota; obtain the computing tasks input by the user; evaluate the computing tasks to determine the computing power requirements; and, based on the computing power requirements, allocate the planned quota from the high-priority quota and the temporary quota from the flexible quota.

[0036] One possible implementation also includes:

[0037] The output module is used to output the remaining quota in the dimension and / or the utilization rate of the high-priority quota to the user. The utilization rate of the high-priority quota is the ratio of the configured planned quota to the high-priority quota.

[0038] One possible implementation also includes:

[0039] The reporting module is used to send a fault warning if the configured temporary limit exceeds the flexible limit.

[0040] In one possible implementation, the computational resource quota also includes an unplanned quota, and the training container includes an unplanned memory pool corresponding to the unplanned quota. The resources in the planned memory pool and the temporary memory pool are unoccupiable resources, while the resources in the unplanned memory pool are occupiable resources.

[0041] The execution module is also used to invoke the unplanned memory pool to execute computational tasks when the resource usage in the temporary memory pool exceeds the third threshold.

[0042] In one possible implementation,

[0043] Execution module 604 is specifically used to determine the task type of the computation task based on the submission request; if the submission request indicates that the task type is a low-priority task, the unplanned memory pool is invoked to execute the computation task; if the submission request indicates that the task type is a high-priority task, the planned memory pool is invoked to execute the computation task.

[0044] A third aspect of this application provides a computer device, comprising:

[0045] Memory and processor;

[0046] The memory stores instructions, which, when executed on the processor, perform the methods described above.

[0047] The fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described above.

[0048] A fifth aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the above aspects.

[0049] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0050] This application provides a method and related apparatus for allocating computing resources. It configures planned and temporary quotas for computing tasks. The computing tasks preferentially use resources within the planned quota. When the planned quota is exhausted or about to be exhausted, a temporary quota can be used, and this temporary quota is transferred between the temporary quota and the resource pool to ensure that the amount of idle resources in the temporary quota does not exceed a certain threshold. Excess idle resources can then be returned to the resource pool for use by other computing tasks, thereby improving the utilization rate of heterogeneous computing resources. Furthermore, since both the planned and temporary quotas are configured for computing tasks, resources will not be preempted during the execution of the computing task, ensuring the stability of the computing task execution and improving the user experience. Attached Figure Description

[0051] Figure 1 This is a flowchart of the computing resource allocation method provided in the embodiments of this application;

[0052] Figure 2 This is a schematic diagram of resource flow for the computing resource allocation method provided in the embodiments of this application;

[0053] Figure 3 This is a flowchart of the computing resource allocation method provided in the embodiments of this application;

[0054] Figure 4 This is a schematic diagram of the architecture of the computing resource allocation method provided in the embodiments of this application;

[0055] Figure 5 This is a schematic diagram illustrating an application scenario of the computing resource allocation method provided in the embodiments of this application;

[0056] Figure 6 This is a schematic diagram of one embodiment of the computing resource allocation device in this application;

[0057] Figure 7 This is a logic flowchart of the computing resource allocation device provided in the embodiments of this application;

[0058] Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0059] Figure 9 This is a schematic diagram of a server structure provided in an embodiment of this application. Detailed Implementation

[0060] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0061] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0062] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0063] With the rapid development of large-scale AI models, the demand for computing power is exploding, directly leading to a dramatic increase in the scale of heterogeneous computing power. Heterogeneous computing power, as a key infrastructure supporting the efficient operation of complex AI tasks, has significantly improved data processing capabilities and energy efficiency. However, the deployment and maintenance costs of heterogeneous computing power are considerably higher than those of traditional CPU devices. Therefore, how to achieve efficient utilization of heterogeneous computing power is a technical problem that urgently needs to be solved by those skilled in the art.

[0064] When heterogeneous computing power serves the research and development work of AI personnel, the computing units are not running all the time. For example, when AI personnel are collecting training data, writing code, and analyzing the results of training models, the heterogeneous computing power is in an idle waiting state. These idle resources cannot be used, resulting in a waste of heterogeneous computing power.

[0065] Traditionally, heterogeneous computing resources are managed using a high-priority / low-priority allocation strategy. This involves dividing computing resources into high-priority and low-priority allocations. High-priority allocations are used by high-priority tasks, while low-priority allocations are used by low-priority tasks. Resources within the high-priority allocation are available for immediate use after allocation to a computing task, falling within the task's planned allocation and not subject to preemption during task execution. Low-priority allocations, however, are outside the planned allocation. When a computing task is executing on low-priority resources, these resources can be preempted by a higher-priority task. Typically, tasks like data collection and result analysis during model training can be defined as low-priority because they can be paused or stopped at any time, allowing the use of low-priority allocations. However, for higher-priority tasks, once the planned allocation is exhausted, low-priority allocations must be used, leading to potential preemption and disruption during task execution, resulting in a poor user experience.

[0066] To address the aforementioned issues, this application provides a computing resource allocation method and related apparatus. The method allocates planned and temporary quotas for computing tasks. Computing tasks preferentially utilize resources within the planned quota. When the planned quota is exhausted or about to be exhausted, a temporary quota can be used. This temporary quota is transferred between the temporary quota and the resource pool to ensure that the amount of idle resources in the temporary quota does not exceed a certain threshold. Excess idle resources can then be returned to the resource pool for use by other computing tasks, thereby improving the utilization rate of heterogeneous computing resources. Furthermore, since both the planned and temporary quotas are allocated for computing tasks, resources will not be preempted during the execution of the computing task, thus not degrading the user experience.

[0067] The following section provides a detailed description of the computing resource allocation method provided in the embodiments of this application. Please refer to [link / reference]. Figure 1 , Figure 1 A flowchart of the computing resource allocation method provided in this application embodiment includes:

[0068] 101. Determine the available resource quota for the resource pool;

[0069] Understandably, to allocate computing resources reasonably, it is first necessary to determine the amount of available resources in the resource pool, including specifying the number of resources or containers currently available for use in the resource pool.

[0070] A resource pool specifically refers to the collection of various hardware and software components involved in a cloud computing data center. The purpose of resource pooling is to improve resource utilization by intelligently managing physical resources through software, abstracting them into virtual resources, and dynamically allocating and optimizing their scheduling in a pooled manner. Heterogeneous computing resources are uniformly managed and scheduled through resource pools to meet the computing power needs of different applications.

[0071] 102. Based on the computing power requirements and available resource quotas of the computing task, configure computing resource quotas for the computing task. The computing resource quotas include planned quotas and temporary quotas.

[0072] A computational task refers to a task submitted by a user or system that requires computational processing, such as training a machine learning model. When a computational task needs to be executed, the processing power required for that task must be determined. The computational power requirement is assessed based on factors such as the complexity of the task, the amount of data, and the computational accuracy. This computational power requirement can be assessed based on the corresponding model, or it can be requested by the task publisher.

[0073] Computational resource quota refers to the limitation of computing resources allocated to a computing task. After determining the computing power requirements of a task, and considering the available resource quota in the resource pool, a corresponding computing resource quota can be configured for that task. The computing resource quota includes planned quota and temporary quota. The planned quota is equivalent to the resource budget allocated to a computing task in traditional methods; it represents the resource quantity that the computing task strongly depends on and can be used at any time during the execution of the task. Unlike traditional methods, this application also configures a temporary quota for the computing task. This temporary quota is an unplanned available resource quantity to ensure that the task can continue to execute using the temporary quota when the planned quota is insufficient, and it will not be occupied.

[0074] 103. In response to the submission request of the computing task, a training container is created in the resource pool based on the computing resource quota. The training container includes a planned memory pool corresponding to the planned quota and a temporary memory pool corresponding to the temporary quota.

[0075] It is understandable that when a user submits a computing task, it means that the task needs to be executed. After configuring computing resource quotas for the task, one or more training containers can be created based on the resources corresponding to those quotas. A training container refers to a lightweight, portable runtime environment used to isolate and execute computing tasks.

[0076] Within the training container, one or more memory pools can be created based on the memory requirements of the computation task. The resources in the memory pools come from the resource pool, and the size and number of these memory pools match the computational resource quota of the task, ensuring that the computation task has sufficient computational resources available during execution.

[0077] Since the computing resource quota includes a planned quota and a temporary quota, the training container includes a planned memory pool corresponding to the planned quota and a temporary memory pool corresponding to the temporary quota. Computational tasks can be performed using resources in either the planned memory pool or the temporary memory pool.

[0078] 104. Invoke the planned memory pool and execute the computation task in the training container;

[0079] Understandably, when a computational task within the training container requires memory resources, it requests a corresponding memory block from the memory pool. The memory pool dynamically allocates memory blocks to the task based on the task's request and the current memory usage. After obtaining the memory block, the task can use it to perform computational operations. In this embodiment, the planned memory pool is invoked first to execute computational tasks.

[0080] 105. When the resource usage in the planned memory pool exceeds the first threshold, the temporary memory pool is invoked to execute the computing task. The temporary memory pool and the resource pool transfer resources to ensure that the amount of idle resources in the temporary memory pool does not exceed the second threshold.

[0081] Understandably, when resources in the planned memory pool are exhausted, computational tasks can obtain additional resources from the temporary memory pool. Under normal circumstances, resources in the temporary memory pool will not be used unless the resources in the planned memory pool are fully consumed. This application supports configuring a first threshold, which can be equal to the resource limit of the planned memory pool (i.e., the first threshold is the planned quota), meaning that resources in the temporary memory pool can only be used when the resources in the planned memory pool are exhausted. However, in some cases, tasks may require more resources to handle sudden computational demands or process larger datasets. To meet this need, the first threshold can be lower than the resource limit of the planned memory pool (i.e., the planned quota). When the resource usage in the planned memory pool exceeds the first threshold, releasing or preparing to release resources in the temporary memory pool in advance helps reduce resource switching latency and improve task execution efficiency.

[0082] Resource transfer between the temporary memory pool and the resource pool refers to the ability to move resources between the two pools, but the upper limit of resources in the temporary memory pool is the aforementioned temporary allowance. This resource transfer mechanism ensures efficient utilization of system resources and avoids waste. In this mechanism, the temporary memory pool acts as an intermediate buffer to store and manage temporarily unused memory resources. That is, when a computational task does not use resources in the temporary resource pool, or when there are many idle resources in the temporary resource pool, excess resources can be returned to the resource pool for use by other computational tasks. When there are few idle resources in the temporary resource pool, and the upper limit of resources in the temporary resource pool has not been reached, resources can be replenished from the resource pool.

[0083] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating the resource flow of the computing resource allocation method provided in the embodiments of this application.

[0084] like Figure 2 As shown, the resource flow between the temporary memory pool and the resource pool can be analogous to water level control. The capacity of the temporary memory pool is equivalent to a temporary allowance, and the water level is equivalent to a second threshold. When the amount of free resources in the temporary memory pool is lower than the water level, resources are added from the resource pool to the temporary memory pool to ensure the smooth execution of the current computing task. When the amount of free resources in the temporary memory pool is higher than the water level, excess resources are returned from the temporary memory pool to the resource pool, making them available for other computing tasks. This method enables the dynamic flow of computing resources between the resource pool and the temporary memory pool to meet constantly changing task requirements, achieving high resource efficiency and ensuring stable task execution.

[0085] The computing resource allocation method provided in this application provides an additional temporary quota for computing resources, which helps to solve the problem of unreasonable resource allocation caused by deviations in task resource planning. Specifically, when the planned quota allocated to computing resources is less than the actual required amount of resources, the resources in the temporary quota can be used to execute tasks, ensuring the stable execution of tasks. When the allocated temporary quota is large, the excess resources can be returned to the resource pool, solving the problem of idle resources being unusable during the execution of computing tasks and improving resource utilization. At the same time, resources will not be preempted during the execution of computing tasks, ensuring the stability of computing task execution and improving the user experience.

[0086] Please see Figure 3 and Figure 4 , Figure 3 This is a flowchart of the computing resource allocation method provided in the embodiments of this application. Figure 4This is a schematic diagram of the architecture of the computing resource allocation method provided in the embodiments of this application, wherein... Figure 3 include:

[0087] 301. Based on the total quota of the resource pool, divide the resources into at least two dimensions: high-priority quota and flexible quota. Resources other than the unoccupiable resources are available resources, and available resources correspond to low-priority quotas.

[0088] In this embodiment of the application, the total amount of the resource pool is first divided, including dividing the unoccupiable resources into at least two dimensions, namely the high-priority quota and the flexible quota.

[0089] In one possible implementation, the total amount of the resource pool can be divided into two categories: unavailable resources and available resources. Unavailable resources are used to execute high-priority tasks, such as model training tasks; available resources are used to execute low-priority tasks, corresponding to low-priority quotas, such as data collection tasks and result analysis tasks.

[0090] like Figure 5 As shown, Figure 5 This diagram illustrates an application scenario of the computing resource allocation method provided in this embodiment. The method provided in this embodiment can be applied to model training tasks at various product layers, including large language models, visual models, speech models, game models, medical models, and NLP models. The total allocation of the heterogeneous computing resource pool is divided into three dimensions: high-priority allocation, flexible allocation, and low-priority allocation. High-priority and flexible allocations are not preempted during task execution, while low-priority allocations can be preempted. Specifically, the high-priority quota refers to the total budget planned for the computing needs of a computing task. After being allocated to the computing task, it can be requested at any time during the task's operation based on the allocated resources, and it is the exclusive resource for that computing task. The flexible quota refers to the total amount of available resources outside the planning of the computing task. The design of this quota is to maintain a user-friendly experience while also solving the problem of wasted heterogeneous computing power due to idle resources during user operations such as R&D analysis and data processing. The low-priority quota refers to the total amount of resources other than the high-priority and flexible quotas. In other words, computing tasks can be submitted at any time when there are idle resources in the resource pool. However, when computing tasks use resources under the low-priority quota, they may be preempted.

[0091] The heterogeneous resource pool can create heterogeneous computing power containers corresponding to various dimensions, and tasks are executed within the computing power containers corresponding to the quota of each dimension.

[0092] 302. Determine the available resource quota for the resource pool.

[0093] It is understandable that step 302 and Figure 1 Step 101 in the corresponding embodiment is similar, and the relevant description is described above, so it will not be repeated here.

[0094] 303. Based on the available resource quota, determine the remaining quota in each dimension;

[0095] 304. Obtain the computation task input by the user;

[0096] 305. Evaluate the computing task and determine the computing power requirements;

[0097] 306. Based on computing power requirements, a planned quota is allocated from the high-priority quota, a temporary quota is allocated from the flexible quota, and an unplanned quota is also allocated from the low-priority quota.

[0098] It is understandable that steps 303 to 306 above are for... Figure 1 Further description of step 102 in the corresponding embodiment.

[0099] In this embodiment of the application, since the total resource pool quota has been divided into two dimensions of unusable quota, after determining the available resource quota, the remaining quota under these two dimensions can be determined separately. Furthermore, the remaining quota of the usable quota can also be determined. For example... Figure 4 As shown, for example: if the total amount of resources in the resource pool corresponds to A, B, and C respectively, and the amount of resources that have been used is A', B', and C' respectively, then the available resource quotas are A-A', B-B', and C-C' respectively.

[0100] On the one hand, available resource quotas can be directly provided to users, allowing them to evaluate their submitted tasks and determine their computing power requirements. On the other hand, after obtaining the computing tasks input by users, appropriate models can be used to evaluate the tasks and assist users in determining their computing power requirements.

[0101] Understandably, after allocating the total quota, based on computing power needs, a planned quota can be allocated from the high-priority quota, a temporary quota can be allocated from the flexible quota, and further, an unplanned quota can be allocated from the low-priority quota.

[0102] 307. Determine whether the configured temporary credit limit exceeds the flexible credit limit;

[0103] 308. If the configured temporary limit exceeds the flexible limit, a fault warning will be sent.

[0104] 309. If the configured temporary credit limit does not exceed the flexible credit limit, the remaining credit limit in each dimension and / or the utilization rate of the high-priority credit limit will be output to the user. The utilization rate of the high-priority credit limit is the ratio of the configured planned credit limit to the high-priority credit limit.

[0105] It is understandable that the three-dimensional quotas represent the upper limit of the available resources in those three dimensions. Since the resources in the flexible quota are resources reused by multiple tasks, it is important to avoid exceeding the allocated temporary quota. Otherwise, tasks may fail to obtain temporary resources, leading to task execution failure and affecting system stability. Therefore, when the configured temporary quota exceeds the flexible quota, a fault alert needs to be sent to relevant operations and maintenance personnel for manual intervention.

[0106] The specific flexible quota can be determined by the computing platform's operations and maintenance personnel based on the total peak usage (E) of cluster tasks and the total resource pool amount (F) of the cluster. The evaluation method is as follows:

[0107] Flexible credit limit = (FE) * w%;

[0108] The value of 'w' can be configured by the platform's operations and maintenance personnel based on their operational experience. The minimum computational requirement at the start of a computation task can be used as the initial value for the resource amount in the temporary quota.

[0109] If the configured temporary quota does not exceed the flexible quota, it means that the computing task can be executed smoothly. At this time, the remaining quota in each of the above dimensions and / or the utilization rate of the high-priority quota can also be output to the user so that the user can judge whether the execution requirements of the computing task are met based on the quota usage in each dimension.

[0110] 310. In response to the submission request of the computing task, a training container is created in the resource pool based on the computing resource quota. The training container includes a planned memory pool corresponding to the planned quota, a temporary memory pool corresponding to the temporary quota, and an unplanned memory pool corresponding to the unplanned quota. The resources in the planned memory pool and the temporary memory pool are unoccupiable resources, while the resources in the unplanned memory pool are occupiable resources.

[0111] It is understandable that step 310 and Figure 1 Similar to step 103 in the corresponding embodiment, when the computing resource quota includes an unplanned quota, the training container includes an unplanned memory pool corresponding to the unplanned quota, which is used to execute computing tasks that allow resources to be occupied.

[0112] 311. Based on the submitted request, determine the task type of the computation task;

[0113] 312. If the submitted request indicates that the task type is a low-priority task, then the unplanned memory pool is invoked to execute the computation task.

[0114] 313. If the submitted request indicates that the task type is a high-priority task, then the planned memory pool is invoked to execute the computation task;

[0115] It is understandable that steps 311 to 313 are... Figure 1 Supplementary explanation of step 104 in the corresponding embodiment.

[0116] Users can specify the type of computation task submitted, thereby determining the type of resources required to execute the task. The task type can be included in the submission request. When the computation task is a low-priority task, it means that resources can be occupied during its execution, and the task can be executed directly using the unplanned memory pool without occupying high-priority resources. When the computation task is a high-priority task, high-priority resources are required, meaning that the planned memory pool needs to be called for execution, and the planned quota will be deducted during execution. Figure 4 As shown.

[0117] 314. When the resource usage in the planned memory pool exceeds the first threshold, the temporary memory pool is invoked to execute the computing task. The temporary memory pool and the resource pool transfer resources to ensure that the amount of idle resources in the temporary memory pool does not exceed the second threshold.

[0118] It is understandable that step 314 and Figure 1 Step 105 in the corresponding embodiment is similar, and the relevant description is described above, so it will not be repeated here.

[0119] 315. When the resource usage in the temporary memory pool exceeds the third threshold, call the unplanned memory pool to execute the computation task.

[0120] Understandably, the third threshold can be equal to or less than the temporary limit, used to indicate that the resource usage in the temporary resource pool has reached or is about to reach its limit. When the resource usage in the temporary memory pool exceeds the third threshold, it means that the currently configured computing resources are insufficient to support the completion of the computing task. Figure 4 As shown, tasks executed in the temporary resource pool are temporary tasks, and temporary quotas are deducted during execution. When the temporary quota is insufficient, the unplanned memory pool can be invoked to execute computational tasks. Tasks executed in the unplanned memory pool are low-priority tasks. It should be noted that before invoking the unplanned memory pool to execute a computational task, the computational task can be evaluated. If the computational task does not support resource preemption, an expected interruption of task execution is directly reported; if the computational task supports resource preemption, the unplanned memory pool can be invoked for execution.

[0121] The computing resource allocation method provided in this application addresses the idle time issue that occurs in the AI ​​R&D process using heterogeneous computing power by allocating temporary quotas from the total resource pool and flexible quotas from these temporary quotas to computing tasks. Tasks running using the flexible quotas are not preempted or interrupted during operation, lowering the barrier to entry for AI R&D personnel and improving their user experience. This ensures the output of training computation results and avoids situations where time is spent performing heterogeneous computing computations without producing any final valuable results. Furthermore, it facilitates the rational allocation of resources, corrects the discrepancy between the budget planning of computing tasks and the actual usage demand, improves the efficiency of heterogeneous computing power resource flow, and thus achieves cost reduction, especially in large-scale heterogeneous computing power clusters where the cost reduction effect is significant.

[0122] The computing resource allocation device in this application will now be described in detail. Please refer to [link / reference]. Figure 6 . Figure 6 This is a schematic diagram of one embodiment of the computing resource allocation device 600 in this application. The computing resource allocation device 600 includes:

[0123] Module 601 is used to determine the available resource quota of the resource pool;

[0124] The configuration module 602 is used to configure computing resource quotas for computing tasks based on the computing power requirements and available resource quotas of the computing tasks. The computing resource quotas include planned quotas and temporary quotas.

[0125] Module 603 is created in response to a computing task submission request and creates a training container in the resource pool based on the computing resource quota. The training container includes a planned memory pool corresponding to the planned quota and a temporary memory pool corresponding to the temporary quota.

[0126] The execution module 604 is used to call the planned memory pool to execute computation tasks in the training container. When the resource usage in the planned memory pool exceeds the first threshold, the temporary memory pool is called to execute computation tasks. The temporary memory pool and the resource pool transfer resources so that the amount of idle resources in the temporary memory pool does not exceed the second threshold.

[0127] The computing resource allocation device provided in this application configures planned and temporary quotas for computing tasks. The computing tasks preferentially use the resources in the planned quota to execute the task. When the planned quota is used up or about to be used up, the temporary quota can be used, and the temporary quota is transferred with the resource pool to ensure that the amount of idle resources in the temporary quota does not exceed a certain threshold. Then, the excess idle resources can be returned to the resource pool for use by other computing tasks, thereby improving the utilization rate of heterogeneous computing resources. At the same time, since both the planned and temporary quotas are configured for computing tasks, resources will not be preempted during the execution of the computing task, and the user experience will not be reduced.

[0128] One possible implementation also includes:

[0129] The partitioning module 605 is used to partition the unoccupiable resources based on the total amount of the resource pool into at least two dimensions, including high-priority quota and flexible quota.

[0130] The configuration module 602 is specifically used to determine the remaining quota in each dimension based on the available resource quota; obtain the computing task input by the user; evaluate the computing task to determine the computing power requirement; and based on the computing power requirement, allocate the planned quota from the high-priority quota and the temporary quota from the flexible quota.

[0131] One possible implementation also includes:

[0132] The output module is used to output the remaining quota in the dimension and / or the utilization rate of the high-priority quota to the user. The utilization rate of the high-priority quota is the ratio of the configured planned quota to the high-priority quota.

[0133] One possible implementation also includes:

[0134] The reporting module is used to send a fault warning if the configured temporary limit exceeds the flexible limit.

[0135] In one possible implementation, the computational resource quota also includes an unplanned quota, and the training container includes an unplanned memory pool corresponding to the unplanned quota. The resources in the planned memory pool and the temporary memory pool are unoccupiable resources, while the resources in the unplanned memory pool are occupiable resources.

[0136] The execution module 604 is also used to invoke the unplanned memory pool to perform computation tasks when the resource usage in the temporary memory pool exceeds the third threshold.

[0137] In one possible implementation,

[0138] Execution module 604 is specifically used to determine the task type of the computation task based on the submission request; if the submission request indicates that the task type is a low-priority task, the unplanned memory pool is invoked to execute the computation task; if the submission request indicates that the task type is a high-priority task, the planned memory pool is invoked to execute the computation task.

[0139] It is understood that the aforementioned computing resource allocation device is related to this application. Figure 1 and Figure 3 The corresponding computing resource allocation method in the embodiment is described above and will not be repeated here.

[0140] Please see Figure 7 , Figure 7 This is a logic flowchart of a computing resource allocation device provided in an embodiment of this application.

[0141] The computing resource allocation device configures the flexible quota of the cluster based on the total quota of the resource pool, which is the upper limit of the temporarily allocated quota (701); it also allocates the planned quota (703), temporary quota (702), and unplanned quota (704) to each computing task according to the existing computing tasks and the available resource quota of the resource pool; after allocating the temporary quota to each computing task, it can be determined whether the accumulated value of the allocated temporary quota exceeds the upper limit (706) by reading the upper limit of the temporary quota (705); if it exceeds the upper limit, it is necessary to send an alarm to the operation and maintenance personnel of the heterogeneous computing power platform (707) so that the operation and maintenance personnel can manually intervene (708); if it does not exceed the upper limit of the temporary quota, a multi-dimensional quota model can be constructed based on the planned quota, temporary quota, and unplanned quota of each computing task (709); after constructing the multi-dimensional quota model, the corresponding quota of each task can be read through the quota model. The various types of quotas and the available resource quotas in the current resource pool (710) are used to first determine whether the planned quota meets the task execution requirements (711). If the high-priority quota meets the task execution requirements, the high-priority task execution calculation is started for the computing task (712). The computing task will not have the problem of resource occupation during the execution process. If the high-priority quota does not meet the task execution requirements, it is determined whether the temporary quota of the computing task is available (713). If yes, it means that the computing task has been allocated a temporary quota and the temporary quota has not been used up. The temporary task execution calculation can be started for the computing task (714). The computing task will also not have the problem of memory occupation during the execution process. If no, it means that the computing task may not have been allocated a temporary quota or the temporary quota has been used up. At this time, further judgment is needed to determine whether the computing task supports the forced occupation of computing resources (715). If it supports it, the low-priority task execution calculation can be started for the computing task (716). In addition, during the computation process of the computation task using the temporary quota, the amount of idle resources in the memory pool corresponding to the temporary quota is monitored in real time to determine whether the amount of idle resources is higher than the threshold (717). If it is higher than the threshold, the excess idle resources can be returned to the resource pool (718) to improve the utilization rate of idle resources.

[0142] This application also provides a computing device; please refer to [link / reference]. Figure 8 , Figure 8 This is a schematic diagram of a computing device provided in an embodiment of this application. The computing device can be used to execute computer programs or computer instructions stored in memory to perform... Figure 1 or Figure 3 The methods in the illustrated embodiments can be referred to the relevant descriptions in the above method embodiments.

[0143] The communication device includes a processor 1301. Optionally, the communication device may also include a memory 1302 and a transceiver 1303.

[0144] This application also provides a computer program product including instructions that, when run on a computer, cause the computer to perform the above-described actions. Figure 1 or Figure 3 The method in the illustrated embodiment.

[0145] This application also provides a computer-readable storage medium, including computer instructions that, when executed on a computer, cause the computer to perform the actions described above. Figure 1 or Figure 3 The method of the embodiment shown.

[0146] This application also provides a chip device, including a processor for connecting to a memory and calling a program stored in the memory, so that the processor executes the above-described... Figure 1 or Figure 3 The method of the embodiment shown.

[0147] The processor mentioned above can be a general-purpose central processing unit, a microprocessor, an application-specific integrated circuit (ASIC), or one or more devices used to control the above. Figure 1 or Figure 3 The illustrated embodiment is an integrated circuit for program execution of the method. The memory mentioned above may be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, such as random access memory (RAM).

[0148] Figure 9 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 300 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 322 (e.g., one or more processors) and memory 332, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 342 or data 344. The memory 332 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the server. Furthermore, the CPU 322 may be configured to communicate with the storage media 330 and execute the series of instruction operations stored in the storage media 330 on the server 300.

[0149] Server 300 may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0150] The steps performed by the server in the above embodiments can be based on this Figure 9 The server structure shown.

[0151] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0152] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0153] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0154] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0155] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0156] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for allocating computing resources, characterized in that, include: Determine the available resource quota for the resource pool; Based on the computing power requirements of the computing task and the available resource quota, a computing resource quota is configured for the computing task, which includes a planned quota and a temporary quota. In response to the submission request of the computing task, a training container is created in the resource pool based on the computing resource quota. The training container includes a planned memory pool corresponding to the planned quota and a temporary memory pool corresponding to the temporary quota. The planned memory pool is invoked to execute the computational task in the training container; When the resource usage in the planned memory pool exceeds the first threshold, the temporary memory pool is invoked to execute the computing task. The temporary memory pool and the resource pool perform resource transfer so that the amount of idle resources in the temporary memory pool does not exceed the second threshold.

2. The method according to claim 1, characterized in that, Before configuring computing resource quotas for the computing task based on the computing power requirements of the computing task and the available resource quotas, the method further includes: Based on the total amount of the resource pool, at least two dimensions of unusable resources are divided, including high-priority quota and flexible quota. The configuration of computing resource quotas for the computing task based on the computing power requirements of the computing task and the available resource quotas includes: Based on the available resource quota, determine the remaining quota in each dimension; Obtain computational tasks from user input; The computational task is evaluated to determine the computing power requirements; Based on the computing power requirements, the planned quota is allocated from the high-priority quota, and the temporary quota is allocated from the flexible quota.

3. The method according to claim 2, characterized in that, Prior to the response to the submission request for the computation task, the method further includes: The remaining quota in the dimension and / or the utilization rate of the high-preferred quota are output to the user, where the utilization rate of the high-preferred quota is the ratio of the configured planned quota to the high-preferred quota.

4. The method according to claim 2 or 3, characterized in that, After configuring computing resource quotas for the computing task based on the computing power requirements of the computing task and the available resource quotas, the method further includes: If the configured temporary limit exceeds the flexible limit, a fault warning is sent.

5. The method according to any one of claims 1 to 4, characterized in that, The computing resource quota also includes an unplanned quota. The training container includes an unplanned memory pool corresponding to the unplanned quota. The resources in the planned memory pool and the temporary memory pool are unoccupiable resources, while the resources in the unplanned memory pool are occupiable resources. After invoking the temporary memory pool to execute the computation task when the resource usage in the planned memory pool exceeds the first threshold, the method further includes: When the resource usage in the temporary memory pool exceeds the third threshold, the unplanned memory pool is invoked to execute the computation task.

6. The method according to claim 5, characterized in that, The step of invoking the planned memory pool and executing the computation task in the training container includes: Based on the submission request, determine the task type of the computing task; If the submission request indicates that the task type is a low-priority task, then the unplanned memory pool is invoked to execute the computation task; If the submission request indicates that the task type is a high-priority task, then the planned memory pool is invoked to execute the computation task.

7. A computing resource allocation device, characterized in that, include: The determination module is used to determine the available resource quota of the resource pool; A configuration module is used to configure computing resource quotas for the computing task based on the computing power requirements of the computing task and the available resource quotas. The computing resource quotas include planned quotas and temporary quotas. A creation module is used to respond to the submission request of the computing task and create a training container in the resource pool based on the computing resource quota. The training container includes a planned memory pool corresponding to the planned quota and a temporary memory pool corresponding to the temporary quota. An execution module is used to call the planned memory pool to execute the computation task in the training container; when the resource usage in the planned memory pool exceeds a first threshold, the temporary memory pool is called to execute the computation task, wherein the temporary memory pool and the resource pool perform resource transfer so that the amount of idle resources in the temporary memory pool does not exceed a second threshold.

8. A computer device, characterized in that, include: Memory and processor; The memory stores instructions that, when executed on the processor, implement the computing resource allocation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the computing resource allocation method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, The computer program is executed by a processor using the computing resource allocation method as described in any one of claims 1 to 6.

Images