GPU (Graphics Processing Unit) resource scheduling method and device and computer program product

Abstract

Embodiments of the invention disclose a GPU resource scheduling method and apparatus, and a computer program product. The method comprises the steps of obtaining GPU resource use information of each item in a plurality of historical preset time periods; on the basis of the GPU resource use information, GPU resource demand information of each project in a preset time period in the future is predicted, and the GPU resource demand information comprises resource types of needed GPU resources and the number of the GPU resources under the resource types; on the basis of the GPU resource demand information, a preset GPU resource cost control factor and the private GPU resource information, public GPU resource demand information of each project in a preset time period in the future is determined; and according to the public GPU resource demand information, carrying out capacity expansion or capacity reduction on the public GPU resources for each project in advance. According to the technical scheme, the utilization rate of GPU resources can be improved, and the resource cost is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method, device and computer program product for scheduling GPU resources. Background Art

[0002] AIGC (AI Generative Content) can use big models to generate content with certain creativity and quality. For example, by inputting keywords, descriptions or samples, AIGC can use big models to generate matching articles, images, audio, etc. Among them, the application of AIGC strongly relies on GPU (Graphic Process Unit) resources.

[0003] Currently, resource buyers (such as companies or organizations) often purchase a corresponding number of public GPU resources for each project based on the needs of each project, and then use these public GPU resources to perform the AICG tasks of the project. In related solutions, resource buyers often purchase the number of public GPU resources that meet the peak task volume demand of the project to meet the task requirements of the project.

[0004] However, the inventors found in the process of implementing this application that the same project has different task volumes in different time periods, that is, different GPU resources are required. This results in that in some time periods, the task volume of the project is small, and the public GPU resources corresponding to the project are idle, resulting in resource waste.

[0005] Therefore, how to reasonably schedule public GPU resources between various projects of resource buyers, thereby improving the utilization rate of the purchased public GPU resources and reducing cost losses has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] In order to solve the problems in the related art, the embodiments of the present disclosure provide a method, an apparatus and a computer program product for scheduling GPU resources.

[0007] In a first aspect, a method for scheduling GPU resources is provided in an embodiment of the present disclosure.

[0008] Specifically, the GPU resource scheduling method includes: Get the GPU resource usage information of each project in multiple preset time periods in history; Based on the GPU resource usage information, predict the GPU resource demand information of each project in the preset time period in the future, wherein the GPU resource demand information includes the resource type of the required GPU resources and the quantity of GPU resources under the resource type; Determine the public GPU resource demand information of each project in the preset time period in the future based on the GPU resource demand information, the preset GPU resource cost control factor and the private GPU resource information; the private GPU resource information includes the resource type and usage status of the GPU resources owned by the private GPU resource, and the number of GPU resources under the resource type; The public GPU resources are expanded or reduced in advance for each project according to the public GPU resource demand information.

[0009] In a second aspect, an embodiment of the present disclosure provides a GPU resource scheduling device, including: A historical information acquisition module is configured to acquire GPU resource usage information of each project in multiple preset time periods in history; A prediction module is configured to predict the GPU resource demand information of each project in the preset time period in the future based on the GPU resource usage information, wherein the GPU resource demand information includes the resource type of the required GPU resources and the quantity of the GPU resources under the resource type; A determination module is configured to determine the public GPU resource demand information of each project in the preset time period in the future based on the GPU resource demand information, a preset GPU resource cost control factor and private GPU resource information; the private GPU resource information includes the resource type and usage status of the GPU resources owned by the private GPU resource, and the number of GPU resources under the resource type; The first expansion / contraction module is configured to expand or reduce the GPU resources for each project in advance according to the public GPU resource demand information.

[0010] In a third aspect, a computer program product is provided in an embodiment of the present disclosure, comprising computer instructions, which, when executed by a processor, implement the method steps as described in any one of the first aspects.

[0011] According to the technical solution provided by the embodiment of the present disclosure, the GPU resource usage information of each project in multiple preset time periods in history can be obtained; based on the GPU resource usage information, the GPU resource demand information of each project in the preset time period in the future is predicted, and the GPU resource demand information includes the resource type of the required GPU resource and the number of GPU resources under the resource type; based on the GPU resource demand information, the preset GPU resource cost control factor and the private GPU resource information, the public GPU resource demand information of each project in the preset time period in the future is determined; the private GPU resource information includes the resource type and usage status of the owned GPU resources, and the number of GPU resources under the resource type; according to the public GPU resource demand information, the public GPU resources are expanded or reduced in advance for each project. In this way, the public GPU resources are automatically expanded or reduced for each project in advance based on the needs of each project, that is, the GPU resources are reasonably and dynamically allocated to each project of the resource buyer in each time period, the utilization rate of the GPU resources purchased by the resource buyer is guaranteed, and the utilization rate of the GPU resources is improved. Moreover, when scaling public GPU resources, this embodiment not only takes into account the future GPU resource demand information and private GPU resource information of each project within the preset time period, but also takes into account the GPU resource cost control factor. In this way, the corresponding public GPU resources can be purchased for scaling while meeting cost control requirements, thereby reducing resource costs.

[0012] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Other features, objectives and advantages of the present disclosure will become more apparent through the following detailed description of non-limiting embodiments in conjunction with the accompanying drawings. In the accompanying drawings:

[0014] Figure 1 A flowchart showing a GPU resource scheduling method according to an embodiment of the present disclosure is shown; Figure 2 A structural block diagram of a GPU resource scheduling device according to an embodiment of the present disclosure is shown; Figure 3 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown; Figure 4 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0015] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for the sake of clarity, parts not related to the description of the exemplary embodiments are omitted in the accompanying drawings.

[0016] In the present disclosure, it should be understood that terms such as "include" or "have" are intended to indicate the presence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in the present specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof exist or are added.

[0017] It should also be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0018] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0019] As mentioned above, resource buyers (such as companies or organizations) often purchase a corresponding number of public GPU resources for each project according to the needs of each project, and then use these public GPU resources to perform the AICG tasks of the project. In related schemes, resource buyers often purchase the number of public GPU resources that meet the peak task volume demand of the project to meet the task requirements of the project. However, in the process of implementing this application, the inventors found that the task volume of the same project in different time periods is different, that is, the required GPU resources are different. This results in that in some time periods, the task volume of the project is small, and the public GPU resources corresponding to the project are idle, resulting in a waste of resources. Therefore, how to reasonably schedule public GPU resources between the projects of resource buyers, thereby improving the utilization rate of the purchased public GPU resources and reducing cost losses has become a technical problem that needs to be solved urgently.

[0020] The present disclosure provides a scheduling method for GPU resources. The method can predict the GPU resource demand information of each project in the preset time period in the future, and then determine the public GPU resource demand information of each project in the preset time period in the future based on the GPU resource demand information, the preset GPU resource cost control factor and the private GPU resource information (such as the resource type and usage status of the private GPU, and the number of GPU resources under the resource type). In this way, the public GPU resources can be expanded or reduced in advance for each project according to the predicted public GPU resource demand information of each project in the preset time period in the future. In this way, GPU resources are reasonably allocated to each project of the resource buyer in each preset time period, and the utilization rate of the GPU resources purchased by the resource buyer is guaranteed, thereby improving the utilization rate of the GPU resources. Moreover, since the GPU resources of the required quantity can be allocated to each project according to the different requirements of each project in each preset time period, the resource buyer can purchase the public GPU resources on demand under the restriction of the GPU resource cost control factor and allocate them to each project in time periods according to the requirements of each project, thereby reducing the resource cost.

[0021] Figure 1 FIG. 1 is a flowchart of a GPU resource scheduling method according to an embodiment of the present disclosure. Figure 1 As shown, the GPU resource scheduling method includes the following steps S101-S104:

[0022] In step S101, the GPU resource usage information of each project in multiple preset time periods in history is obtained; In step S102, based on the GPU resource usage information, predict the GPU resource demand information of each project in the preset time period in the future, the GPU resource demand information including the resource type of the required GPU resources and the quantity of GPU resources under the resource type; In step S103, based on the GPU resource demand information, the preset GPU resource cost control factor and the private GPU resource information, the public GPU resource demand information of each project in the preset time period in the future is determined; the private GPU resource information includes the resource type and usage status of the GPU resources owned by itself, and the number of GPU resources under the resource type; In step S104, the capacity of public GPU resources is expanded or reduced in advance for each project according to the public GPU resource demand information.

[0023] In a possible implementation, the GPU resource scheduling method is applicable to a computer, computing device, server, server cluster, or other device that can execute GPU resource scheduling, hereinafter referred to as a resource scheduling server.

[0024] In a possible implementation, the GPU resources that can be scheduled by the resource scheduling server are all GPU resources registered in the resource scheduling server. These GPU resources can be provided by a private cloud platform, or by one or more public cloud platforms, or by a hybrid cloud platform that is a combination of private and public clouds, and there is no limitation here. Among them, the main difference between private clouds and public clouds is that the GPU resources provided by private clouds are maintained on the dedicated private network of their own enterprises or organizations and do not need to be purchased; while the GPU resources provided by public clouds need to be purchased from other GPU resource providers and maintained by other GPU resource providers.

[0025] In a possible implementation, the resource buyer refers to a company or organization that has insufficient or no private GPU resources and needs to purchase public GPU resources. Each project in the resource buyer may correspond to a project maintained by each department or technical team under the resource buyer. The operation of each project requires each task submitter (such as a client or server) under the project to submit tasks under the project. The resource scheduling server can schedule GPU resources to execute the tasks submitted by the task submitter. Tasks of different task types require GPUs of different resource types to execute.

[0026] In one possible implementation, when registering, the GPU resource will provide the resource scheduling server with resource information of the GPU resource, where the resource information includes the resource type of the GPU resource, where the resource type is used to indicate the type of service that the GPU resource can provide. For example, the resource type may be a resource that provides raw image services, a resource that provides reasoning services (such as image classification, object detection, face recognition, and other predictions), a resource that provides cutout services, a resource that provides raw video services, and so on.

[0027] In a possible implementation, the GPU resource usage information of each project in multiple preset time periods in history may include information such as resource type and resource quantity of GPU resources used by each project in multiple preset time periods of each day (such as multiple time periods from 7:00 to 8:00 in the morning, 8:00 to 9:00, 9:00 to 10:00 in the morning, etc.) on historical dates such as the past month or half a year. Of course, it can also be multiple preset time periods in the target holiday (such as multiple preset time periods on May 1, May 2, May 5, etc.). Based on the GPU resource usage information of each project in multiple preset time periods in history, the resource type of GPU resources required in the preset time period in the future and the quantity of GPU resources under the resource type can be predicted.

[0028] In a possible implementation, since GPU resources are limited and the cost of GPU resources is high, in order to control the cost budget of the resource buyer to purchase GPU resources, a GPU resource cost control factor can be preset, and the preset GPU resource cost control factor includes the following cost control strategy: by predicting the GPU resource demand information of each project in the preset time period in the future to evaluate the elastic demand of each project for GPU resources, according to the elastic demand, GPU resources are allocated to each project, when allocating GPU resources, private GPU resources provided by the private cloud are preferentially allocated to each project, and when private GPU resources are insufficient, the purchased public GPU resources are allocated to control the cost of purchasing public GPU resources. The GPU resource cost control factor can be a control factor for the purchase cost of public GPU resources. For example, the GPU resource cost control factor can be that the total cost of a month does not exceed the predetermined total cost, and the maximum cost per day does not exceed the predetermined maximum cost. Multiple projects of a resource buyer all require a large amount of GPU resources during the week, and only a small number of projects require a small amount of GPU resources on weekends. Based on the GPU resource cost control factor, more public GPU resources can be purchased during the week, but the daily cost will not exceed the predetermined maximum cost, and less public GPU resources can be purchased on weekends to ensure that the total cost of a month does not exceed the predetermined total cost. Alternatively, the GPU resource cost control factor can be that the daily minimum cost is not less than the predetermined minimum cost, and the maximum cost does not exceed the predetermined maximum cost. The resource purchaser can evaluate the elastic demand for GPU resources of each project based on the predicted GPU resource demand information of each project in the preset time period in the future, and allocate GPU resources to each project based on the elastic demand. When allocating GPU resources, private GPU resources provided by the private cloud are preferentially allocated to each project. When private GPU resources are insufficient, the purchased public GPU resources are allocated. In this way, the daily public GPU resource demand is evaluated, and public GPU resources are purchased according to the daily demand, ensuring that the minimum cost is not less than the predetermined minimum cost and the maximum cost does not exceed the predetermined maximum cost. Of course, the GPU resource cost control factor can also be other control strategies, which are not listed here one by one. Compared with the prior art of not controlling resource costs and purchasing according to peak demand, this can greatly save resource costs.

[0029] In a possible implementation, based on the predicted GPU resource demand information of each project in the preset time period in the future, when configuring GPU resources for each project in the preset time period in accordance with the preset GPU resource cost control factor, the GPU resources provided by the private cloud can be first configured for each project according to the resource type and usage status (including occupied status or idle status) of the private GPU resources, and the number of GPU resources under the resource type; if the private GPU resources are insufficient according to the predicted GPU resource demand information, the public GPU resource demand information of each project in the preset time period in the future is determined, and the public GPU resources are expanded or reduced in advance for each project according to the public GPU resource demand information.

[0030] For example, assuming that the private GPU resources of the resource buyer are 10 raw image GPU resources, and the resource buyer is currently running projects A and B. Projects A and B can be configured with 5 private raw image GPU resources respectively. Among them, project A requires 30 raw image GPU resources at the peak of task volume, and project B requires 40 raw image GPU resources at the peak of task volume. In order to meet the demand, the existing technology can purchase 25 raw image public GPU resources for project A and 35 raw image public GPU resources for project B, for a total of 60 raw image public GPU resources. In this way, it is guaranteed that project A has 30 raw image GPU resources in each time period and project B has 40 raw image GPU resources in each time period. However, during non-peak task volume periods, some GPU resources will be idle, resulting in resource waste. By using the solution provided by the present embodiment, the resource buyer can reduce the purchase cost and only purchase 40 public GPU resources of the raw image class according to demand. Assuming that the resource scheduling server predicts that in the future 8-9 o'clock period, the resource buyer's project A will have a large number of raw image tasks, and it is estimated that 30 raw image GPU resources will be needed. In the future 9-10 o'clock period, the raw image tasks of project A will be reduced, and it is estimated that 10 raw image GPU resources will be needed. Project B has a large number of raw image tasks, and it is estimated that 40 raw image GPUs will be needed. Then, if the resource scheduling server determines that 5 private GPU resources configured for project A in the 8-9 period are not enough, 25 public GPU resources of the raw image class can be expanded for resource buyer A in the 8-9 period at 7:50 in advance. Since the number of raw image tasks of Project A will decrease during the period from 9:00 to 10:00 in the future, it is estimated that 10 raw image GPU resources will be needed. The previously configured 5 private GPU resources and 25 public GPU resources of the raw image class exceed the project's needs. At this time, it is determined that the project needs 5 public GPU resources of the raw image class in addition to the 5 private GPU resources of the raw image class during the period from 8:00 to 9:00. Therefore, at 8:50, the capacity of Project A can be reduced to 5 public GPU resources of the raw image class during the period from 9:00 to 10:00. Project B is expected to need 40 raw image GPUs during the period from 9:00 to 10:00 in the future. Therefore, 5 private GPU resources can be configured for Project B during the period from 9:00 to 10:00 and 35 raw image GPU resources can be expanded. In this way, it can be ensured that the tasks of the two projects can be efficiently completed in each preset time period (including the task peak time period), and the purchased public GPU resources can be efficiently utilized in each preset time period, reducing cost losses.

[0031] This implementation method can obtain the GPU resource usage information of each project in multiple preset time periods in history; based on the GPU resource usage information, predict the GPU resource demand information of each project in the preset time period in the future, the GPU resource demand information includes the resource type of the required GPU resource and the number of GPU resources under the resource type; based on the GPU resource demand information, the preset GPU resource cost control factor and the private GPU resource information, determine the public GPU resource demand information of each project in the preset time period in the future; the private GPU resource information includes the resource type and usage status of the owned GPU resources, and the number of GPU resources under the resource type; according to the public GPU resource demand information, the public GPU resources are expanded or reduced in advance for each project, so that the public GPU resources are automatically expanded or reduced in advance for each project based on the needs of each project. In this way, the public GPU resources are automatically expanded or reduced for each project in advance based on the needs of each project, and GPU resources are reasonably and dynamically allocated to each project of the resource buyer in each time period, so as to ensure the utilization rate of the GPU resources purchased by the resource buyer and improve the utilization rate of the GPU resources. Moreover, when scaling public GPU resources, this embodiment not only takes into account the future GPU resource demand information and private GPU resource information of each project within the preset time period, but also takes into account the preset GPU resource cost control factor. In this way, the corresponding public GPU resources can be purchased for scaling while meeting cost control requirements, thereby reducing resource costs.

[0032] In a possible implementation, after GPU resources are expanded or reduced for each project in advance according to the public GPU resource demand information, the method further includes: Based on the usage rate of GPU resources of each project in the current time period, the public GPU resources are expanded or reduced for each project.

[0033] In this implementation, the Serverless method can be used to expand or shrink the public GPU resources for each project. The Serverless method is a fully automated elastic expansion and contraction method. When the task volume of a project is at a peak, the public GPU resources of the project are automatically expanded to carry more tasks for the project. When the task volume decreases, the public GPU resources of the project will also shrink at the same time to avoid resource waste. The serverless method is based on the utilization rate of public GPU resources in the current time period. If the utilization rate of public GPU resources of a project is low in the current period, it triggers a contraction. If the utilization rate is high, it triggers an expansion. The current time period here refers to a period of time before the current moment, such as a few minutes before the current moment.

[0034] In this implementation, unlike the above-mentioned prediction of the future GPU resource demand information of each project within the preset time period and the expansion and contraction in advance, the serverless method expands and contracts based on the utilization rate of GPU resources in the current time period. The triggering conditions of these two expansion and contraction methods are different. The GPU resource scheduling method provided in this implementation can use these two different methods to expand and contract. For example, after the public GPU resources are expanded for the project in the 8-9 time period at 7:50 in advance, the utilization rate of the public GPU resources of the project exceeds the threshold at 8:20, which can trigger the expansion of the serverless method.

[0035] It should be noted here that in actual applications, there are two ways for resource buyers to purchase GPU resources provided by public clouds. One way is that the resource buyer can share the public GPU resources provided by the public cloud with other resource buyers, and each resource buyer only needs to pay for the public GPU resources used to process the tasks of the resource buyer, and does not need to pay for the use of public GPU resources during idle periods, thus reducing costs. Of course, another way is for the resource buyer to purchase a fixed number of public GPU resources. At this time, the above-mentioned prediction method or serverless method can be used to scale the public GPU resources of each project of the resource buyer. However, the cost of the resource buyer is fixed at this time, but the utilization rate of GPU resources can be improved.

[0036] This implementation can be combined with the serverless method to perform elastic expansion and contraction, further improving the utilization of GPU resources and avoiding resource waste.

[0037] In a possible implementation manner, after the GPU resource capacity is expanded or reduced for each project in advance according to the public GPU resource demand information, the method further includes: When a preset time period in the future is reached, task information is obtained, wherein the task information includes a target task type, target task parameters, and a target project identifier of the project to which the task belongs; Allocate the task indicated by the task information to a queue of pending tasks; the tasks in the queue of pending tasks are pulled according to a preset pulling frequency; When the task is pulled from the pending task queue, based on the amount of data to be processed indicated by the target task parameters, one or more GPU resources whose resource type matches the target task type are searched from the GPU resources allocated to the project corresponding to the target project identifier to process the task.

[0038] In this implementation, when a preset time period is reached in the future, the task submitter of a certain project will submit some tasks. The resource scheduling server can obtain the task information of the task submitted by the task submitter to allocate the corresponding GPU resources to execute the task. The task information includes the target task type, target task parameters and the target project identifier of the project to which it belongs. For example, the target task type can be various types such as raw image task, model evaluation task, reasoning task (such as image classification, object detection, face recognition, etc.), cutout task, raw video task, etc. The task parameter refers to the set of parameters required for the GPU resource to execute the task, which may include parameters corresponding to the metadata describing the content of the task, parameters of the model information required to execute the task, etc. For example, when the task is a text-based image task, the task parameter may include parameters corresponding to the metadata describing the content of the task "generating an image of size YY using keyword XX" and parameters corresponding to the text-based image model required to execute the text-based image task, etc. The target project identifier of the project to which it belongs is used to identify the project to which the task belongs.

[0039] In this embodiment, after the resource scheduling server obtains the task information of the task submitted by the task submitter, it can assign the task to a queue of pending tasks. The resource scheduling server can pull the task of the project corresponding to the target project identifier from the queue of pending tasks according to the task processing capability of the GPU resources allocated to the project corresponding to the target project identifier. For example, if it is predicted in advance that the task volume of the project corresponding to the target project identifier in this time period is large, and more GPU resources are allocated to the project corresponding to the target project identifier, then the task processing capability of the GPU resources allocated to the project corresponding to the target project identifier is strong, and the resource scheduling server will pull the task of the project corresponding to the target project identifier from the queue of pending tasks faster according to a larger preset pulling frequency for processing. If it is predicted in advance that the task volume of the target project in this time period is small, and fewer GPU resources are allocated to the project corresponding to the target project identifier, then the task processing capability of the GPU resources allocated to the project corresponding to the target project identifier is weak, and the resource scheduling server will pull the task of the project corresponding to the target project identifier from the queue of pending tasks slower according to a smaller preset pulling frequency for processing.

[0040] In this implementation, when the resource scheduling server pulls the task from the pending task queue, it can search for idle GPU resources whose resource type matches the target task type from the GPU resources allocated to the project corresponding to the target project identifier to process the task. At the same time, it will also allocate a corresponding number of matching idle GPU resources to process the task according to the amount of data to be processed indicated by the target task parameters. For example, when the target task type is a picture generation task and the target task parameter of the picture generation task indicates that a large number of pictures need to be generated, a large number of idle GPU resources of the picture generation class can be searched for to process the task. When the target task type is a picture generation task and the target task parameter of the picture generation task indicates that a picture described by a keyword needs to be generated, an idle GPU resource of the picture generation class can be searched for to process the task.

[0041] This implementation can cache the tasks submitted by users in a queue of pending tasks by setting up a task queue, and then pull the tasks for orderly processing based on the task processing capability of the GPU resources of the project to which the task belongs. Regardless of whether the submission speed of the task submitter is very high or very low, the resource scheduling server can pull tasks at a uniform speed according to the task processing capability of the project, thereby achieving the effect of shaving peaks and filling valleys and smoothly processing tasks, avoiding problems such as system crashes caused by the inability of GPU resources to carry a sudden large number of tasks.

[0042] In a possible implementation, the to-be-processed task queue includes an online task queue and an offline task queue, the task information includes task mode indication information, and the task mode indication information is used to indicate whether the task is in an online mode or an offline mode; and allocating the task indicated by the task information to the to-be-processed task queue includes: When the task mode indication information indicates that the task is in online mode, assigning the task to an online task queue; When the task mode indication information indicates that the task is in offline mode, the task is allocated to an offline task queue.

[0043] In this embodiment, the task information also includes task mode indication information, which is used to indicate the task mode of the task. The task mode includes an online mode and an offline mode. The online mode task focuses on real-time processing of the task and requires real-time response and update results, while the offline task pays more attention to batch processing and analysis of the task, usually does not require real-time response, and can run in the background.

[0044] In this embodiment, when the task mode indication information indicates that the task is in online mode, the task is assigned to an online task queue; when the task mode indication information indicates that the task is in offline mode, the task is assigned to an offline task queue; wherein the tasks in the offline task queue can be pulled slowly, while the tasks in the online task queue need to be pulled and processed quickly.

[0045] This implementation can distinguish the task modes of various tasks so that online tasks that require real-time response can be quickly pulled and executed.

[0046] In a possible implementation, the GPU resources allocated to the project corresponding to the target project identifier include a GPU resource offline cluster and a GPU resource online cluster, the number of GPU resources in the GPU resource offline cluster is less than the number of GPU resources in the GPU resource online cluster, and when the task is pulled from the queue of pending tasks, searching for a GPU resource whose resource type matches the target task type from the GPU resources allocated to the project corresponding to the target project identifier to process the task, including: When the task is pulled from the offline task queue, a GPU resource whose resource type matches the target task type is searched from the offline GPU resource cluster allocated to the project corresponding to the target project identifier to process the task; When the GPU resources in the GPU resource offline cluster are in an idle state and the task is pulled from the online task queue, GPU resources whose resource types match the target task type are preferentially searched from the idle GPU resources in the GPU resource offline cluster allocated by the project corresponding to the target project identifier to process the task; when the idle GPU resources in the GPU resource offline cluster are exhausted and GPU resources still need to be called, GPU resources whose resource types match the target task type are searched from the GPU resource online cluster allocated by the project corresponding to the target project identifier to process the task; Or, when the GPU resources in the GPU resource offline cluster are in an idle state and the task is pulled from the online task queue, the GPU resources whose resource types match the target task type are preferentially searched from the GPU resource online cluster allocated by the project corresponding to the target project identifier to process the task; when the GPU resources in the GPU resource online cluster are exhausted and GPU resources still need to be called, the GPU resources whose resource types match the target task type and are in an idle state are searched from the GPU resource offline cluster allocated by the project corresponding to the target project identifier to process the task.

[0047] In this implementation, the resource scheduling server can divide the GPU resources allocated to the project corresponding to the target project identifier into a GPU resource offline cluster and a GPU resource online cluster. The GPU resources in the GPU resource offline cluster are used to execute tasks in the offline task queue, and the GPU resources in the GPU resource online cluster are used to execute tasks in the online task queue. When pulling the task of the project from the online task queue, the GPU resource whose resource type matches the target task type can be found from the GPU resource online cluster allocated to the project to process the task; when pulling the task of the project from the offline task queue, the GPU resource whose resource type matches the target task type can be found from the GPU resource offline cluster allocated to the project to process the task. Usually, the number of GPU resources in the GPU resource offline cluster is less than the number of GPU resources in the GPU online cluster, so that a small amount of GPU resources can be used to slowly execute offline tasks, and a large amount of GPU resources can be used to quickly respond to online tasks.

[0048] In this implementation, in order to further improve the utilization rate of GPU resources, when the GPU resources in the GPU resource offline cluster are in an idle state, if the task of the project is pulled from the online task queue, the GPU resources in the GPU resource offline cluster allocated by the project can be searched for GPU resources whose resource type matches the target task type to process the task, and the idle GPU resources in the GPU resource offline cluster execute the task in the online task queue. When the GPU resources in the GPU resource offline cluster that are in an idle state are called up and the GPU resources still need to be called up, the idle GPU resources whose resource type matches the target task type are searched for from the GPU resource online cluster allocated by the project to process the task. Alternatively, the idle GPU resources whose resource type matches the target task type can be searched for from the GPU resource online cluster allocated by the project to process the task; when the GPU resources in the GPU resource online cluster are called up and the GPU resources still need to be called up, the idle GPU resources whose resource type matches the target task type and is in an idle state can be searched for from the GPU resource offline cluster allocated by the project to process the task.

[0049] In a possible implementation, before determining the future public GPU resource demand information of each project within the preset time period based on the GPU resource demand information, the preset GPU resource cost control factor and the private GPU resource information, the method further includes: Receive a registration request for a target GPU resource, wherein the registration request carries resource information of the target GPU resource, wherein the resource information includes a resource address and a resource type; Recording the resource information of the target GPU resource into a database, and recording that the usage status of the target GPU resource is an idle status; The step of searching, based on the amount of data to be processed indicated by the target task parameter, one or more GPU resources whose resource types match the target task type from the GPU resources allocated to the project corresponding to the target project identifier to process the task includes: Searching a database, based on the amount of data to be processed indicated by the target task parameter, to search for a target GPU resource from GPU resources allocated to a project corresponding to the target project identifier, wherein the target GPU resource is in an idle state and a resource type matches a task type of the task; Based on the resource address of the target GPU resource, issuing the task to the target GPU resource; The usage status of the target GPU resource in the database is updated to an occupied status, and the task being executed by the target GPU resource is recorded.

[0050] In this implementation, when the GPU resource is registered, the host where the GPU resource is located will provide resource information to the resource scheduling server. The resource information includes information such as the resource ID, resource address, and resource type of the GPU resource.

[0051] In this embodiment, the resource address includes an IP address and a port number. The IP address is an identifier used to uniquely identify a host on the network. On the same host, services provided by different GPU resources are identified by different port numbers. The resource address of the IP address and port combination can accurately locate the GPU resource on the host in the network. Alternatively, the resource address includes a URI (Uniform Resource Identifier), which is used to identify the GPU resource in the network. The URI address includes the host name where the resource is stored and the name of the resource itself (the name of the resource itself is represented by a path). The URI address can also locate the GPU resource on the host in the network.

[0052] In this implementation, after the target GPU resource is registered in the resource scheduling server, the resource scheduling server can record the resource information of the target GPU resource in the database, and record the usage status of the target GPU resource as an idle state. The target GPU resource can be the private GPU resource of the resource buyer. After the GPU resource is registered, the resource scheduling server can determine the public GPU resource demand information of each project in the preset time period in the future based on the GPU resource demand information, the preset GPU resource cost control factor and the private GPU resource information. Of course, the target GPU resource can also be a public GPU resource. After the GPU resource is registered, the resource scheduling server can allocate the target GPU resource to a project when expanding the public GPU resource for a project according to the predicted public GPU resource demand information of each project in the preset time period in the future.

[0053] In this implementation, when the resource scheduling server searches for a GPU resource whose resource type matches the target task type in the GPU resources allocated from the project corresponding to the target project identifier to process the task, it can determine the resource type that can match the target task type according to the target task type of the task, and then search one or more idle GPU resources under the resource type from the database based on the amount of data to be processed indicated by the target task parameter, and the one or more GPU resources include the target GPU resource, and then the task can be issued to the target GPU resource based on the resource address of the target GPU resource, so that the target GPU resource performs the task, and at the same time, the resource scheduling server updates the usage status of the called target GPU resource in the database from the idle state to the occupied state, and records the task being executed by the called target GPU resource in the database (which can be represented by the task ID), or also records the time of executing the task, so as to query the execution status of the task later. When the GPU resource completes the task, the resource scheduling server can release the target GPU resource and update the usage status of the target GPU resource in the database from the occupied state to the idle state, so that it can be assigned to perform other tasks.

[0054] This implementation method manages these GPU resources in a unified manner through the resource information and usage status of the registered target GPU resources recorded in the database.

[0055] In a possible implementation, the database includes a KV storage database and / or a relational database.

[0056] In this embodiment, the KV storage database can read data quickly, but the KV storage database is not as good as the relational database when querying complex table relationships. For example, the KV storage database can be a Redis database, and the relational database can be a MySQL database.

[0057] In this implementation, the KV storage database can store the usage status, resource address and resource type of the GPU resources corresponding to each project of the resource buyer, and the ID of the task being executed by the GPU resources in the occupied state. The amount of stored data is small, which is convenient for quick query; the relational database can store the resource information of the GPU resources corresponding to each project of the resource buyer, the task information of the tasks being executed by the GPU resources in the occupied state, and also stores the task information and task execution time of the tasks executed by each GPU resource in the historical time period. It stores a large amount of data, which is convenient for subsequent task tracing.

[0058] In this embodiment, the database provided in this embodiment may be a KV storage database, or a relational database, or may be a KV storage database and a relational database.

[0059] In a possible implementation, before acquiring the task information, the method further includes: receiving a task request, wherein the task request carries the task information; Determine whether to discard the task request using a flow limiter, wherein the flow limiter is used to limit the number of tasks that can be received in the current time period; The obtaining of task information includes: When the flow limiter receives and passes the task request, task information in the task request is obtained.

[0060] In this implementation, when a task requester needs to complete a task, he can send a task request to the resource scheduling server. The task request is used to request the resource scheduling server to schedule the corresponding GPU resources to process the requested task. The task request carries task information such as the target task type, target task parameters, and the target project identifier of the project to which it belongs. For example, when a client of a project needs to perform a raw image task based on a keyword to generate an image, the user of the client can enter the keyword and click the "raw image" button. The client can send a task request to the resource scheduling server to request the resource scheduling server to schedule the corresponding raw image GPU resources to process the raw image task.

[0061] In this embodiment, after receiving the task request, the resource scheduling server can use a current limiter to determine whether to discard the task request. The current limiter is used to limit the number of tasks that can be currently received. The current limiter can be a funnel current limiter, a token bucket current limiter, a sliding window current limiter, and the like. Taking the funnel current limiter as an example, when the task request reaches the funnel current limiter, it is directly placed in the funnel. If the current number of task requests in the funnel reaches the upper limit (predetermined value), the funnel current limiter will discard the task request. At the same time, the funnel current limiter passes the task request at a fixed rate (i.e., releases the task request) until the funnel is empty. It should be noted here that the funnel current limiter can send a task discard notification to the task requester after discarding the task request, so that the task requester continues to send the task request at a suitable time later.

[0062] In this implementation, if the rate limiter determines not to discard the task request, it will receive and subsequently pass the task request. When the rate limiter passes the task request, the resource scheduling server can obtain the task information carried in the task request from the task request.

[0063] It should be noted here that the task request may also carry a callback address URI, which is used to feedback a task execution start notification to the requester of the task request when the GPU resource starts to execute the task requested by the task request.

[0064] Figure 2 The structure block diagram of the GPU resource scheduling device according to the embodiment of the present disclosure is shown. The device can be implemented as part or all of the electronic device through software, hardware or a combination of both. Figure 2 As shown, the GPU resource scheduling device includes:

[0065] The historical information acquisition module 201 is configured to acquire the GPU resource usage information of each project in multiple preset time periods in history; The prediction module 202 is configured to predict the GPU resource demand information of each project in the preset time period in the future based on the GPU resource usage information, wherein the GPU resource demand information includes the resource type of the required GPU resources and the quantity of the GPU resources under the resource type; The determination module 203 is configured to determine the public GPU resource demand information of each project in the preset time period in the future based on the GPU resource demand information, the preset GPU resource cost control factor and the private GPU resource information; the private GPU resource information includes the resource type and usage status of the GPU resources owned by itself, and the number of GPU resources under the resource type; The first expansion / contraction module 204 is configured to expand or reduce the public GPU resources for each project in advance according to the public GPU resource demand information.

[0066] In a possible implementation, the device further includes: The second expansion / contraction module is configured to expand or reduce the public GPU resources for each project based on the utilization rate of the GPU resources of each project in the current time period after expanding or reducing the GPU resources for each project in advance according to the public GPU resource demand information.

[0067] In a possible implementation, the device further includes: A task acquisition module is configured to acquire task information when a preset time period is reached after the GPU resource capacity is expanded or reduced for each project in advance according to the public GPU resource demand information, wherein the task information includes a target task type, target task parameters, and a target project identifier of the project to which the task belongs; A task allocation module is configured to allocate the task indicated by the task information to a queue of pending tasks; the tasks in the queue of pending tasks are pulled according to a preset pulling frequency; The task processing module is configured to, when pulling the task from the pending task queue, search for one or more GPU resources whose resource type matches the target task type from the GPU resources allocated to the project corresponding to the target project identifier based on the amount of data to be processed indicated by the target task parameters to process the task.

[0068] In a possible implementation, the to-be-processed task queue includes an online task queue and an offline task queue, the task information further includes task mode indication information, and the task mode indication information is used to indicate whether the task is in an online mode or an offline mode; the task allocation module indicated by the task information is configured as follows: When the task mode indication information indicates that the task is in online mode, assigning the task to an online task queue; When the task mode indication information indicates that the task is in offline mode, the task is allocated to an offline task queue.

[0069] In a possible implementation, the GPU resources allocated to the project corresponding to the target project identifier include a GPU resource offline cluster and a GPU resource online cluster, the number of GPU resources in the GPU resource offline cluster is less than the number of GPU resources in the GPU resource online cluster, and the task processing module is configured as follows: When the task is pulled from the offline task queue, a GPU resource whose resource type matches the target task type is searched from the offline GPU resource cluster allocated to the project corresponding to the target project identifier to process the task; When the GPU resources in the GPU resource offline cluster are in an idle state and the task is pulled from the online task queue, GPU resources whose resource types match the target task type are preferentially searched from the idle GPU resources in the GPU resource offline cluster allocated by the project corresponding to the target project identifier to process the task; when the idle GPU resources in the GPU resource offline cluster are exhausted and GPU resources still need to be called, GPU resources whose resource types match the target task type are searched from the GPU resource online cluster allocated by the project corresponding to the target project identifier to process the task; Or, when the GPU resources in the GPU resource offline cluster are in an idle state and the task is pulled from the online task queue, the GPU resources whose resource types match the target task type are preferentially searched from the GPU resource online cluster allocated by the project corresponding to the target project identifier to process the task; when the GPU resources in the GPU resource online cluster are exhausted and GPU resources still need to be called, the GPU resources whose resource types match the target task type and are in an idle state are searched from the GPU resource offline cluster allocated by the project corresponding to the target project identifier to process the task.

[0070] In a possible implementation, the device further includes: A registration module is configured to receive a registration request for a target GPU resource before determining the public GPU resource demand information of each project in the preset time period in the future based on the GPU resource demand information, the preset GPU resource cost control factor and the private GPU resource information, wherein the registration request carries resource information of the target GPU resource, and the resource information includes a resource address and a resource type; a recording module configured to record the resource information of the target GPU resource into a database, and record that the usage status of the target GPU resource is an idle status; The part of the task processing module that searches for a GPU resource whose resource type matches the target task type from the GPU resources allocated to the project corresponding to the target project identifier to process the task is configured as follows: Searching the database, based on the amount of data to be processed indicated by the target task parameter, from the GPU resources allocated to the project corresponding to the target project identifier, searching for the target GPU resource whose resource type matches the target task type; Based on the resource address of the target GPU resource, issuing the task to the target GPU resource; The usage status of the target GPU resource in the database is updated to an occupied status, and the task being executed by the target GPU resource is recorded.

[0071] In a possible implementation, the database includes a key-value pair KV storage database and / or a relational database; the resource address of the GPU resource includes an IP address and a port address or the resource address of the GPU resource includes a URI address.

[0072] In a possible implementation, the device further includes: A task request receiving module is configured to receive a task request before acquiring the task information, wherein the task request carries the task information; A current limiting module is configured to use a current limiter to determine whether to discard the task request, wherein the current limiter is used to limit the number of tasks that can be received in the current time period; The task acquisition module is configured as follows: When the flow limiter receives and passes the task request, task information in the task request is obtained.

[0073] The technical terms and technical features mentioned in the implementation manner of this device are the same as or similar to the technical terms and technical features mentioned in the implementation manner of the above method. For the explanation and description of the technical terms and technical features involved in this device, reference may be made to the explanation and description of the implementation manner of the above method, and no further details will be given here.

[0074] The present disclosure also discloses an electronic device, Figure 3 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0075] like Figure 3 As shown, the electronic device 300 includes a memory 301 and a processor 302, wherein the memory 301 is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor 302 to implement the method according to an embodiment of the present disclosure.

[0076] Figure 4 A schematic diagram showing the structure of a computer system suitable for implementing the method according to an embodiment of the present disclosure is shown.

[0077] like Figure 4As shown, the computer system 400 includes a processing unit 401, which can perform various processes in the above-mentioned embodiments according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage part 408 to a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the computer system 400 are also stored. The processing unit 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0078] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage section 408 as needed. Among them, the processing unit 401 can be implemented as a processing unit such as a CPU, a GPU, a TPU, an FPGA, an NPU, etc.

[0079] In particular, according to an embodiment of the present disclosure, the method described above can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes computer instructions, and the computer instructions are executed by a processor to implement the method steps described above. In such an embodiment, the computer program product can be downloaded and installed from a network through the communication part 409, and / or installed from a removable medium 411.

[0080] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0081] The units or modules involved in the embodiments described in the present disclosure may be implemented by software or programmable hardware. The units or modules described may also be set in a processor, and the names of these units or modules do not constitute limitations on the units or modules themselves in some cases.

[0082] As another aspect, the present disclosure further provides a computer-readable storage medium, which may be a computer-readable storage medium included in the electronic device or computer system in the above embodiment; or a computer-readable storage medium that exists independently and is not assembled into a device. The computer-readable storage medium stores one or more programs, and the programs are used by one or more processors to execute the method described in the present disclosure.

[0083] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

Claims

1. A GPU resource scheduling method, characterized in that: include: Get the GPU resource usage information of each project in multiple preset time periods in history; Based on the GPU resource usage information, predict the GPU resource demand information of each project in the preset time period in the future, wherein the GPU resource demand information includes the resource type of the required GPU resources and the quantity of GPU resources under the resource type; Determine the public GPU resource demand information of each project in the preset time period in the future based on the GPU resource demand information, the preset GPU resource cost control factor and the private GPU resource information; the private GPU resource information includes the resource type and usage status of the GPU resources owned by the private GPU resource, and the number of GPU resources under the resource type; The public GPU resources are expanded or reduced in advance for each project according to the public GPU resource demand information.

2. The method according to claim 1, characterized in that After expanding or reducing the GPU resources for each project in advance according to the public GPU resource demand information, the method further includes: Based on the usage rate of GPU resources of each project in the current time period, the public GPU resources are expanded or reduced for each project.

3. The method according to claim 1, characterized in that After the GPU resources are expanded or reduced for each project in advance according to the public GPU resource demand information, the method further includes: When a preset time period in the future is reached, task information is obtained, wherein the task information includes a target task type, target task parameters, and a target project identifier of the project to which the task belongs; Allocate the task indicated by the task information to a queue of pending tasks; the tasks in the queue of pending tasks are pulled according to a preset pulling frequency; When the task is pulled from the pending task queue, based on the amount of data to be processed indicated by the target task parameters, one or more GPU resources whose resource type matches the target task type are searched from the GPU resources allocated to the project corresponding to the target project identifier to process the task.

4. The method according to claim 3, characterized in that The to-be-processed task queue includes an online task queue and an offline task queue, the task information also includes task mode indication information, and the task mode indication information is used to indicate whether the task is in an online mode or an offline mode; the task indicated by the task information is assigned to the to-be-processed task queue, comprising: When the task mode indication information indicates that the task is in online mode, assigning the task to an online task queue; When the task mode indication information indicates that the task is in offline mode, the task is allocated to an offline task queue.

5. The method according to claim 4, characterized in that The GPU resources allocated to the project corresponding to the target project identifier include an offline GPU resource cluster and an online GPU resource cluster, the number of GPU resources in the offline GPU resource cluster is less than the number of GPU resources in the online GPU resource cluster, and when the task is pulled from the queue of pending tasks, based on the amount of pending data indicated by the target task parameter, one or more GPU resources whose resource types match the target task type are searched from the GPU resources allocated to the project corresponding to the target project identifier to process the task, including: When the task is pulled from the offline task queue, a GPU resource whose resource type matches the target task type is searched from the offline GPU resource cluster allocated to the project corresponding to the target project identifier to process the task; When the GPU resources in the GPU resource offline cluster are in an idle state and the task is pulled from the online task queue, the GPU resources in the GPU resource offline cluster allocated by the project corresponding to the target project identifier and in an idle state are preferentially searched for a GPU resource whose resource type matches the target task type to process the task; When all idle GPU resources in the offline GPU resource cluster are used up and GPU resources are still needed, searching for a GPU resource whose resource type matches the target task type in the online GPU resource cluster allocated to the project corresponding to the target project identifier to process the task; or, When the GPU resources in the GPU resource offline cluster are in an idle state and the task is pulled from the online task queue, the GPU resource online cluster allocated to the project corresponding to the target project identifier is preferentially searched for a GPU resource whose resource type matches the target task type to process the task; When all GPU resources in the GPU resource online cluster are used up and GPU resources still need to be called, a GPU resource in an idle state whose resource type matches the target task type is searched from the GPU resource offline cluster allocated to the project corresponding to the target project identifier to process the task.

6. The method according to claim 3, characterized in that Before determining the public GPU resource demand information of each project in the preset time period in the future based on the GPU resource demand information, the preset GPU resource cost control factor and the private GPU resource information, the method further includes: Receive a registration request for a target GPU resource, wherein the registration request carries resource information of the target GPU resource, wherein the resource information includes a resource address and a resource type; Recording the resource information of the target GPU resource into a database, and recording that the usage status of the target GPU resource is an idle status; The step of searching, based on the amount of data to be processed indicated by the target task parameter, one or more GPU resources whose resource types match the target task type from the GPU resources allocated to the project corresponding to the target project identifier to process the task includes: Searching the database, based on the amount of data to be processed indicated by the target task parameter, from the GPU resources allocated to the project corresponding to the target project identifier, searching for the target GPU resource whose resource type matches the target task type; Based on the resource address of the target GPU resource, issuing the task to the target GPU resource; The usage status of the target GPU resource in the database is updated to an occupied status, and the task being executed by the target GPU resource is recorded.

7. The method according to claim 6, characterized in that The database includes a key-value pair KV storage database and / or a relational database; the resource address includes an IP address and a port address or the resource address includes a uniform resource identifier URI address.

8. The method according to claim 3, characterized in that Before acquiring the task information, the method further includes: receiving a task request, wherein the task request carries the task information; Determine whether to discard the task request using a flow limiter, wherein the flow limiter is used to limit the number of tasks that can be received in the current time period; The obtaining of task information includes: When the flow limiter receives and passes the task request, task information in the task request is obtained.

9. A GPU resource scheduling device, characterized in that: include: A historical information acquisition module is configured to acquire GPU resource usage information of each project in multiple preset time periods in history; A prediction module is configured to predict the GPU resource demand information of each project in the preset time period in the future based on the GPU resource usage information, wherein the GPU resource demand information includes the resource type of the required GPU resources and the quantity of the GPU resources under the resource type; A determination module is configured to determine the public GPU resource demand information of each project in the preset time period in the future based on the GPU resource demand information, a preset GPU resource cost control factor and private GPU resource information; the private GPU resource information includes the resource type and usage status of the GPU resources owned by the private GPU resource, and the number of GPU resources under the resource type; The first expansion / contraction module is configured to expand or reduce the public GPU resources for each project in advance according to the public GPU resource demand information.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the method steps described in any one of claims 1 to 8 are implemented.

Images