Task processing method

Abstract

Embodiments of the present specification provide a task processing method, which comprises: determining current state information of an initial virtual node based on a received target task, wherein the current state information is determined based on a physical computing unit corresponding to the initial virtual node; determining a candidate virtual node corresponding to task type information of the target task from the initial virtual node; determining a target virtual node corresponding to the target task based on current state information of the candidate virtual node, and executing the target task through the target virtual node, thereby meeting the requirement of accurately determining a target virtual node for a target task.

Description

TECHNICAL FIELD

[0001] Embodiments of the present specification relate to the technical field of computer technology, and particularly relate to a task processing method. BACKGROUND

[0002] With the continuous development of computer technology, the proportion of implementing computing tasks based on computing devices is rapidly increasing. For example, in a heterogeneous computing scenario, the proportion of heterogeneous computing based on heterogeneous hardware (such as a graphics processing unit GPU) is rapidly increasing, and thus is widely applied in fields such as audio / video production, graphics / image processing, AI training, etc. With different computing service providers supporting the execution of heterogeneous computing tasks through computing instances, the demand for computing instances needs to be accurately determined based on different workload types of heterogeneous computing tasks. Therefore, there is an urgent need to provide a scheme capable of accurately allocating computing instances to meet the elastic expansion needs of heterogeneous computing tasks. SUMMARY

[0003] Therefore, the embodiments of the present specification provide a task processing method. One or more embodiments of the present specification also relate to a task processing apparatus, a computing device, a computer-readable storage medium, and a computer program to solve the technical defects in the prior art.

[0004] According to a first aspect of the embodiments of the present specification, a task processing method is provided, including:

[0005] determining current state information of an initial virtual node based on a received target task, wherein the current state information is determined based on a physical computing unit corresponding to the initial virtual node;

[0006] determining a candidate virtual node corresponding to task type information of the target task from the initial virtual node based on the task type information;

[0007] determining a target virtual node corresponding to the target task based on current state information of the candidate virtual node, and executing the target task through the target virtual node.

[0008] According to a second aspect of the embodiments of the present specification, a task processing apparatus is provided, including:

[0009] a receiving module configured to determine current state information of an initial virtual node based on a received target task, wherein the current state information is determined based on a physical computing unit corresponding to the initial virtual node;

[0010] a determining module configured to determine a candidate virtual node corresponding to task type information of the target task from the initial virtual node based on the task type information;

[0011] The execution module is configured to determine a target virtual node corresponding to the target task based on the current state information of the candidate virtual node, and execute the target task through the target virtual node.

[0012] According to a third aspect of the embodiments of the present specification, a computing device is provided, comprising:

[0013] a memory and a processor;

[0014] The memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, which implement the steps of the task processing method when executed by the processor.

[0015] According to a fourth aspect of the embodiments of the present specification, a computer readable storage medium is provided, which stores computer executable instructions, which implement the steps of the task processing method when executed by the processor.

[0016] According to a fifth aspect of the embodiments of the present specification, a computer program is provided, which causes a computer to execute the steps of the task processing method when the computer program is executed in the computer.

[0017] The task processing method provided by the present specification comprises determining the current state information of the initial virtual node based on the received target task, wherein the current state information is determined based on the physical computing unit corresponding to the initial virtual node; determining the candidate virtual node corresponding to the task type information of the target task from the initial virtual node based on the task type information of the target task; determining the target virtual node corresponding to the target task based on the current state information of the candidate virtual node, and executing the target task through the target virtual node.

[0018] Specifically, the method determines the target virtual node corresponding to the target task based on the current state information of the initial virtual node and the task type information of the target task when the target task is received, and executes the target task through the target virtual node, thereby meeting the requirement of accurately determining the target virtual node for the target task. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a Serverless platform scheduling framework diagram provided by the present specification;

[0020] Figure 2 is a schematic diagram of a Serverless scheduling algorithm based on request concurrency provided by the present specification;

[0021] Figure 3 is a flowchart of a task processing method provided by an embodiment of the present specification;

[0022] Figure 4 is a flow chart of task scheduling in a task processing method provided by one embodiment of the present specification;

[0023] Figure 5 is a schematic diagram of elastic scaling in a task processing method provided by one embodiment of the present specification;

[0024] Figure 6 is a process flow chart of a task processing method provided by one embodiment of the present specification;

[0025] Figure 7 is a structural schematic diagram of a task processing apparatus provided by one embodiment of the present specification;

[0026] Figure 8 is a structural block diagram of a computing device provided by one embodiment of the present specification. DETAILED DESCRIPTION

[0027] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present specification. However, the present specification can be practiced without the specific details, other than in the examples, set forth in this description. Those skilled in the art, in light of the description, can implement the present specification without limiting to the specific details disclosed in this description.

[0028] The terminology used in one or more embodiments of the present specification is for the purpose of describing particular embodiments only and is not intended to be limiting of one or more embodiments of the present specification. As used in one or more embodiments of the present specification and the accompanying claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in one or more embodiments of the present specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0029] It will be understood that, although the terms first, second, etc. can be employed in describing various information, such information should not be limited to these terms. These terms are only used to differentiate one piece of information from another. For example, without departing from the scope of one or more embodiments of the present specification, first can be termed second, and similarly, second can be termed first. Depending on the context, the word "if' as used herein can be interpreted to mean "when" or "in response to determining" or "in response to a determination."

[0030] First, the noun terms related to one or more embodiments of the present specification are explained.

[0031] GPU: Generally refers to a graphics processor. A graphics processing unit (GPU), also called a display core, visual processor, or display chip, is a microprocessor that is designed to handle the computations for graphics and images on personal computers, workstations, game consoles, and some mobile devices (such as tablets and smartphones).

[0032] VPU: VPU (Video Processing Unit) is a new video processing platform core engine with hard decoding function and the ability to reduce CPU (Central Processing Unit) load. In addition, VPU can reduce the consumption of server load and network bandwidth. Used to distinguish from GPU (Graph Process Unit). The graphics processing unit includes three main modules: video processing unit, external video module, and post-processing module.

[0033] TPU: A neural network training processor, mainly used for deep learning and AI (Artificial Intelligence) operations. TPU has programming like GPU and CPU, as well as a set of CISC instruction sets (Complex Instruction Set). As a machine learning processor, it not only supports a certain type of neural network, but also supports convolutional neural networks, LSTM (Long Short-Term Memory Artificial Neural Network), fully connected networks, and many others. TPU uses low-precision (8-bit) calculations to reduce the number of transistors used in each operation.

[0034] GPGPU: General-purpose computing on graphics processing units (GPGPU) is a type of graphics processing unit that uses graphics processing tasks to calculate general-purpose computing tasks that are normally handled by central processing units. These general-purpose computing tasks often have nothing to do with graphics processing. Due to the powerful parallel processing capability and programmable pipeline of modern graphics processors, stream processors can handle non-graphic data. Especially when facing single instruction stream multi-data stream (SIMD), and the amount of data processing operations is much larger than the need for data scheduling and transmission, the general-purpose graphics processor greatly surpasses the central processing unit application in performance.

[0035] Hardware encoder: A video encoding unit built into a graphics card.

[0036] Hardware decoder: A video decoding unit built into a graphics card.

[0037] SP (Streaming Processor, Stream Processing Units): Stream processor, which is a device that directly maps multimedia graphics data stream to the stream processor for processing, and has programmable and non-programmable types.

[0038] CUDACore: A kind of stream processor.

[0039] TensorCore: A special execution unit designed for performing tensor or matrix operations.

[0040] NVLINK: A bus and its communication protocol. NVLink uses a point-to-point structure and serial transmission, and is used for the connection between a central processing unit (CPU) and a graphics processing unit (GPU), and can also be used for the mutual connection between multiple graphics processing units.

[0041] GPU instance: A container type that can run GPU tasks.

[0042] Serverless platform: A serverless computing platform, or a microservice platform.

[0043] RR polling: RR polling scheduling, which means that each request signal will obtain a response in a polling response request.

[0044] With the continuous development of computer technology, on the one hand, the proportion of heterogeneous computing based on GPU is rapidly increasing, for example: in the fields of audio and video production, graphics image processing, AI training, AI inference, rendering scene, etc. GPU is widely used to achieve several times, even tens of thousands of times speedup compared with CPU; on the other hand, with the popularization of cloud computing and the continuous upward movement of the computing interface, more and more customers migrate from VM (virtual machine), container to Serverless elastic computing platform, so that customers can focus on their own computing tasks and shield many non-computing task details such as cluster management, observability, diagnosis, etc.

[0045] With the support of Serverless platform for GPU heterogeneous computing tasks, the need for elastic scaling based on different types of workload based on heterogeneous computing tasks naturally arises. That is, based on GPU hardware, heterogeneous computing tasks will use different hardware computing units built-in GPU according to different types of workload. For example: audio and video production will use hardware encoding unit / hardware decoding unit of GPU, high-precision AI training task will use TensorCore computing unit of GPU, low-precision AI inference task will use CUDACore computing unit of GPU. Based on the Serverless platform, heterogeneous computing tasks need Serverless platform to provide a method, so that the customers of Serverless platform can choose appropriate GPU indicators according to their own heterogeneous computing workload type, so as to cope with the elastic expansion during traffic peak and the elastic contraction during traffic low.

[0046] Based on this, in an elastic scaling scheme provided in the present specification, the Serverless platform usually performs elastic scaling on GPU computing instances based on request concurrency, such as: when the concurrency of the function is set to 1, 100 concurrent requests will create 100 GPU instances; when the concurrency of the function is set to 10, 100 concurrent requests will create 10 GPU instances. This elastic scaling method based on request concurrency does not truly reflect whether the GPU hardware is fully used, such as: when the concurrency of the function is set to 10, 100 concurrent requests will create 10 GPU instances, and each GPU instance has different hardware parts inside the GPU, such as: GPU computing unit, GPU storage unit, GPU interconnection bandwidth resource, GPU component utilization may still maintain at a low level, resulting in resource waste and cost waste.

[0047] Further, referring to Figure 1 , Figure 1 is a Serverless platform scheduling framework diagram provided in the present specification, which includes a request access system, a scheduling system, a inventory system, and a GPU instance. When a user completes the Serverless service function writing, a function call request will be initiated to the request access system of the Serverless platform. Referring to Figure 1 , Figure 1The processing flow of the function request after reaching inside the Serverless platform is described, specifically including: the access request system provides HTTP, HTTPS or other access protocols, and the request is dispatched to the scheduling system through the access protocol; the scheduling system is responsible for applying for GPU instances to the inventory system, and is responsible for scheduling the user's function request to different GPU instances for running corresponding functions. The inventory system is responsible for the management of GPU instances; when the scheduling system perceives that the current GPU instance cannot serve the function call, a new GPU instance is applied to the inventory system, and the inventory system creates a GPU instance based on the application of the scheduling system. The GPU instance is responsible for the specific execution of the function.

[0048] Based on this, referring to Figure 2 , Figure 2 The present specification provides a schematic diagram of a Serverless scheduling algorithm based on request concurrency, which is based on the Serverless platform scheduling framework provided by the above Figure 1 The specific steps include:

[0049] Step 202: Determine whether the function request has a surviving GPU instance.

[0050] Specifically, Serverless can provide a scheduling entrance to users, and when the user initiates a function request to the Serverless platform through the scheduling entrance, the Serverless platform determines whether the function request has a corresponding surviving GPU instance, so as to schedule the function request to the GPU instance.

[0051] If yes, step 206 is performed, and if no, step 204 is performed.

[0052] Step 204: Apply for a new GPU instance to the inventory system.

[0053] Specifically, the Serverless platform applies for a new GPU instance to the inventory system. If the application is successful, a surviving GPU instance is obtained, and step 202 is performed, and if the application fails, step 210 is performed.

[0054] Step 206: Traverse all GPU instances and determine whether the request concurrency of any GPU instance is less than the user-configured request concurrency.

[0055] Specifically, after determining that the function request has a surviving GPU instance, the Serverless platform traverses all GPU instances and determines whether the request concurrency of any GPU instance is less than the user-configured request concurrency. If yes, step 208 is performed; if no, step 204 is performed.

[0056] Step 208: scheduling the request to the GPU instance that meets the condition.

[0057] Specifically, the Serverless platform schedules the function request to the GPU instance that meets the condition.

[0058] Step 210: scheduling termination.

[0059] Based on this, the default request concurrency-based elastic scaling scheduling method of the Serverless platform, although it realizes the scheduling of requests, does not comprehensively consider the utilization of each component inside the GPU hardware, which can lead to cost waste for users when the utilization of each component inside the GPU hardware is maintained at a low level due to the elastic expansion triggered by the request concurrency being higher than the user setting. And when the utilization of each component inside the GPU hardware is maintained at a high level, the performance loss for users caused by the elastic scaling triggered by the request concurrency being lower than the user setting.

[0060] In summary, the default request concurrency-based elastic scaling scheduling method of the Serverless platform does not consider the utilization of each component inside the GPU hardware, resulting in cost and performance loss.

[0061] Based on the defects in the above-mentioned scheme, a system structure for the elastic scaling of Serverless heterogeneous computing tasks is provided in a task processing method, thereby solving the performance and cost problems caused by the elastic scaling of Serverless heterogeneous hardware.

[0062] Specifically, in the present specification, a task processing method is provided, and the present specification also relates to a task processing apparatus, a computing device, a computer-readable storage medium, and a computer program, which are described in detail one by one in the following embodiments.

[0063] Figure 3 A flowchart of a task processing method provided according to one embodiment of the present specification is shown, which specifically includes the following steps.

[0064] Step 302: determining the current state information of the initial virtual node based on the received target task, wherein the current state information is determined based on the physical computing unit corresponding to the initial virtual node.

[0065] The target task can be understood as a heterogeneous computing task that needs to be processed by a heterogeneous hardware device; for example, the heterogeneous computing task includes but is not limited to audio and video production tasks, graphic image processing tasks, AI training tasks, AI inference tasks, scene rendering tasks, etc.

[0066] The initial virtual node can be understood as a node capable of running heterogeneous computing tasks. For example, the initial virtual node can be a general-purpose computing node, a GPU instance, a virtual machine, a container, etc.

[0067] The physical computing unit can be understood as a physical device supporting the initial virtual node implementation. For example, the physical computing unit can be a GPU, a GPGPU, a VPU, a TPU, etc.

[0068] In the case of an initial virtual node being a GPU instance, the current state information can be understood as the utilization rate of each hardware component in the GPU corresponding to the GPU instance. It should be noted that the task processing method provided in the present specification can be applied to a Serverless platform or a scheduling system in the Serverless platform. The scheduling system can be understood as a system for scheduling a target task to a corresponding GPU instance.

[0069] Specifically, after receiving the target task, the scheduling system can determine the current state information of all initial virtual nodes based on the received target task, wherein the current state information of the initial virtual node is determined based on the physical computing unit corresponding to the initial virtual node.

[0070] In actual application, in the case of an initial virtual node being a GPU instance and the physical computing unit being a GPU, the current state information of the initial virtual node determined based on the physical computing unit corresponding to the initial virtual node can be understood as determining the utilization rate indicators of each hardware component inside the GPU as the current utilization rate of the GPU instance corresponding to each hardware component, so as to facilitate subsequent task scheduling based on the utilization rate. The specific implementation manner is as follows.

[0071] The determination of the current state information of the initial virtual node based on the received target task comprises:

[0072] Based on the received target task, determining a physical computing subunit in the physical computing unit and current running information of the physical computing subunit;

[0073] Determining a target physical computing subunit corresponding to the initial virtual node from the physical computing subunit;

[0074] Taking the current running information of the target physical computing subunit as the current state information of the initial virtual node.

[0075] The physical computing subunit can be understood as various hardware components inside the GPU hardware, including but not limited to a hardware encoder, a hardware decoder, an SP, a CUDA Core, a Tensor Core, and the like. The current running information of the physical computing subunit can be understood as a utilization rate index of each hardware component. Based on this, the scheduling system determines the current running information of each physical computing subunit in the physical computing unit, and can use the current running information as the current state information of the initial virtual node. For example, the scheduling system can obtain the utilization rate index of each component inside the GPU hardware, and determine the utilization rate index as the utilization rate corresponding to the GPU instance.

[0076] The target physical computing subunit corresponding to the initial virtual node can be understood as a hardware device in the GPU corresponding to the GPU instance.

[0077] The following describes the determination of the current state information of the initial virtual node based on the physical computing unit, taking the application of the task processing method provided in the specification in the Serverless scenario as an example, wherein the physical computing unit is a GPU, the target task is a graphics image processing task, and the target physical computing subunit is a hardware encoder and a hardware decoder.

[0078] Based on this, the scheduling system of the Serverless platform determines the current utilization rate of each hardware unit in the GPU when receiving the target task, and determines the hardware unit corresponding to the GPU instance from a plurality of hardware units, wherein the GPU instance processing the graphics image processing task corresponds to the hardware encoder and the hardware decoder. The GPU instance processing the AI inference processing task corresponds to the CUDA Core and the Tensor Core. The current utilization rate of each hardware unit in the GPU is used as the utilization rate of the corresponding GPU instance.

[0079] In the embodiments of the specification, the current state information of the initial virtual node is determined based on the current running information of the physical computing subunit in the physical computing unit when receiving the target task, thereby facilitating the subsequent determination of the corresponding target virtual machine node for the target task based on the utilization rate.

[0080] Further, in an embodiment provided in the specification, before determining the physical computing unit corresponding to the initial virtual node based on the received target task, the method further comprises:

[0081] The current running information of the physical computing subunit in the physical computing unit sent by the information collection module, wherein the information collection module is a module for monitoring the current running information of the physical computing subunit in the physical computing unit.

[0082] The information collection module can be understood as any module that can realize the function of collecting the current running information of the physical computing unit, such as a GPU monitor.

[0083] Specifically, the information collection module can monitor the current running information of each physical computing subunit in the physical computing unit in real time, and send the current running information to the scheduling system; so that the scheduling system can receive the current running information of each physical computing subunit in the physical computing unit sent by the information collection module. For example, the GPU monitor can obtain the utilization rate index of each hardware component in the GPU corresponding to each GPU instance, and periodically synchronize the utilization rate index to the scheduling system.

[0084] In the embodiments of the present application, that is, the current running information of the physical computing unit sent by the information collection module can be received, so that the current state information of the initial virtual node can be determined based on the current running information. The current running information of the physical computing unit can be the current running information of each physical computing unit in the physical computing unit.

[0085] Further, in an embodiment provided in the present application, the physical computing unit is a GPU;

[0086] Correspondingly, determining the current state information of the initial virtual node based on the received target task comprises:

[0087] determining the hardware components of the GPU and the current utilization rate of the hardware components based on the received target task;

[0088] determining the target hardware component corresponding to the initial virtual node from the hardware components, and taking the current utilization rate of the target hardware component as the current state information of the initial virtual node.

[0089] The hardware components of the GPU include but are not limited to hardware encoders, hardware decoders, SPs, CUDA cores, Tensor cores, etc.

[0090] Continuing with the above example, the GPU monitor can obtain the utilization rate index of each hardware component in the GPU corresponding to each GPU instance, and periodically synchronize the utilization rate index to the scheduling system. Then, the scheduling system of the Serverless platform determines the current utilization rate of each hardware unit in the GPU when receiving the target task, and determines the hardware unit corresponding to the GPU instance from the plurality of hardware units, and takes the current utilization rate of each hardware unit in the GPU as the utilization rate of the corresponding GPU instance. Thus, the corresponding GPU instance can be determined for the target task based on the utilization rate subsequently.

[0091] Step 304: determining, based on the task type information of the target task, a candidate virtual node corresponding to the task type information from the initial virtual node.

[0092] The task type information of the target task can be understood as information representing the type of the target task, such as character, number, etc. In the case of an AI training task, the task type information of the target task can be character, number, etc. representing the type of the AI training task.

[0093] The candidate virtual node can be understood as all virtual nodes in the initial virtual node that can process the target task. For example, in the case of a graphics image processing task, the candidate virtual node is a GPU instance that can process the graphics image processing task, wherein the GPU instance that processes the graphics image processing task corresponds to a hardware encoder and a hardware decoder in the GPU.

[0094] Specifically, after determining the current state information of the initial virtual node, the scheduling system can determine the task type information of the target task, and based on the task type information, determine the candidate virtual node corresponding to the task type information from the initial virtual node, that is, all virtual nodes in the initial virtual node that can process the target task.

[0095] Step 306: determining a target virtual node corresponding to the target task based on the current state information of the candidate virtual node, and executing the target task through the target virtual node.

[0096] The target virtual node can be understood as a GPU instance to which the heterogeneous computing task needs to be scheduled.

[0097] Specifically, the scheduling system can add or select a target virtual node corresponding to the target task based on the current state information of the candidate virtual node, and execute the target task through the target virtual node.

[0098] In an embodiment provided in the present specification, a corresponding GPU instance can be added for the heterogeneous computing task by expanding the GPU instance; or a GPU instance with better hardware performance can be selected from the existing GPU instances for the heterogeneous computing task, thereby realizing flexible scheduling of the heterogeneous computing task and saving hardware resources. Based on this, the GPU instance is expanded for the heterogeneous computing task as follows.

[0099] The determining of the target virtual node corresponding to the target task based on the current state information of the candidate virtual node comprises:

[0100] The adding of the target virtual node corresponding to the target task based on the current state information of the candidate virtual node.

[0101] Specifically, the scheduling system can add a target virtual node for the target task again in a case where it is determined, based on the current state information of the candidate virtual node, that the one or more candidate virtual nodes cannot handle the target task, that is, a virtual node is re-applied or created for the target task.

[0102] Further, in the embodiments provided in the specification, the adding of the target virtual node for the target task based on the current state information of the candidate virtual node comprises:

[0103] determining a target computing ratio of the candidate virtual node based on the current state information of the candidate virtual node;

[0104] adding the target virtual node for the target task in a case where the target computing ratio is greater than or equal to a first ratio threshold.

[0105] In a case where the current state information is the utilization rate of the hardware encoding unit, the target computing ratio can be understood as the utilization rate of the subsequent virtual node. The utilization rate can be set according to the actual application scenario, for example, the utilization rate can be any value in the interval of 0% to 100%, or any value in the interval of [0, 1], etc.

[0106] The first ratio threshold can be set according to the actual application scenario, which is not limited in the specification, for example, 70%, 0.7, etc.

[0107] In the above example, the target task can be an audio-video production task, the current state information is the utilization rate of the hardware encoding unit, and the first ratio threshold can be 70%. Based on this, the scheduling system determines the utilization rate of the hardware decoding unit corresponding to the GPU instance, and takes the utilization rate of the hardware decoding unit as the utilization rate of the GPU instance corresponding to the hardware decoding unit, where the utilization rate can be 80%. Then, the scheduling system determines that the remaining computing capacity of the GPU instance is too low to possibly execute the current audio-video production task in a case where the utilization rate is greater than the first ratio threshold (70%). Therefore, a new GPU instance is created for the audio-video production task by means of GPU instance expansion, so as to ensure the normal execution of the audio-video production task.

[0108] In the embodiments provided in the specification, the scheduling system can achieve the purpose of expanding the GPU instance by applying for a GPU instance to the inventory system, so as to further ensure the normal execution of the heterogeneous computing task. The specific implementation manner is as follows.

[0109] The adding of the target virtual node for the target task comprises:

[0110] generating a virtual node obtaining request based on the target task, and sending the virtual node obtaining request to a virtual node providing module;

[0111] receiving a to-be-determined virtual node sent by the virtual node providing module based on the virtual node obtaining request, and determining the to-be-determined virtual node as a target virtual node corresponding to the target task.

[0112] The virtual node providing module can be understood as a module capable of providing a virtual node for a target task. For example, an inventory system in Serverless. Accordingly, the to-be-determined virtual node can be understood as a virtual node provided by the inventory system. For example, a newly created GPU instance of the inventory system.

[0113] In the above example, the scheduling system can send a GPU instance obtaining request to the inventory system to apply for a new GPU instance when it is determined that the utilization rate of the existing GPU instance is insufficient to run the audio and video production task, and the inventory system will create a new GPU instance based on the application of the scheduling system and send the new GPU instance to the scheduling system. The scheduling system takes the newly applied GPU instance as the GPU instance for processing the audio and video production task, and subsequently the scheduling system can schedule the audio and video production task to the new GPU instance for running.

[0114] Further, the scheduling system matches a GPU instance with better hardware performance for the heterogeneous computing task from the existing GPU instances in the following manner.

[0115] The method further includes:

[0116] selecting a target virtual node corresponding to the target task from the candidate virtual nodes based on the current state information of the candidate virtual nodes.

[0117] Specifically, after determining the candidate virtual nodes, the scheduling system can select a target virtual node corresponding to the target task from the candidate virtual nodes based on the current state information of the candidate virtual nodes when it is determined that the one or more candidate virtual nodes can process the target task.

[0118] Further, in the embodiments provided in the present specification, the method further includes:

[0119] determining a target computing ratio of the candidate virtual nodes based on the current state information of the candidate virtual nodes;

[0120] In a case where the target calculation ratio is less than a first ratio threshold, determining a minimum target calculation ratio from the target calculation ratio;

[0121] Based on the candidate virtual nodes corresponding to the minimum target calculation ratio, determining a target virtual node corresponding to the target task.

[0122] Continuing with the above example, the scheduling system determines the utilization rate of one or more hardware decoding units corresponding to the GPU instance. In a case where it is determined that the utilization rate is less than or equal to a first ratio threshold 70%, it is determined that there is a GPU instance in the current GPU instance that can run the audio and video production task. Then the scheduling system determines the minimum utilization rate from the utilization rates less than or equal to the first ratio threshold 70%.

[0123] In a case where the minimum utilization rate is one, the GPU instance corresponding to the minimum utilization rate is determined as the GPU instance running the audio and video production task.

[0124] In a case where the minimum utilization rate is multiple, one or more GPU instances are randomly determined from the GPU instances corresponding to the multiple minimum utilization rates as the GPU instances running the audio and video production task.

[0125] In actual applications, in a case where it is determined that the utilization rate is less than or equal to the first ratio threshold 70%, determining the GPU instance with better performance from the utilization rates less than or equal to the first ratio threshold 70% further includes: sorting the candidate virtual nodes based on the target calculation ratio to obtain a sorting result of the candidate virtual nodes, wherein the candidate virtual nodes include at least two.

[0126] Based on the sorting result, determining a target virtual node corresponding to the target task from the candidate virtual nodes.

[0127] Continuing with the above example, in a case where the scheduling system determines that the utilization rate is less than or equal to the first ratio threshold 70%, it determines the GPU instances with utilization rates less than or equal to the first ratio threshold 70% and sorts the GPU instances in descending order based on the utilization rates to obtain a sorting result of the GPU instances, wherein the earlier the GPU instance is in the sorting result, the smaller the utilization rate is, that is, the better the performance of the GPU instance is. Based on this, the scheduling system schedules the audio and video production task to the first GPU instance in the sorting result or the first specific number of GPU instances in the sorting result, wherein the specific number can be set according to actual application scenarios, for example, the first three or the first ten.

[0128] Further, in the embodiments provided in the specification, the physical computing unit is a GPU. In this case, after the target virtual node executes the target task, it further includes:

[0129] In a case where it is determined that the target task is completed based on the current utilization of the target hardware component, the target virtual node is deleted.

[0130] Specifically, the scheduling system can monitor the current utilization of the target hardware component in real time, for example, by obtaining the utilization indicators of the hardware components inside each GPU through a GPU monitor component (GPU Monitor), and in a case where it is determined that the target task is completed based on the utilization indicators, the virtual machine node to be deleted is deleted, thereby saving hardware resources.

[0131] In addition, referring to Figure 4 , Figure 4 is a flowchart of task scheduling in a task processing method provided by an embodiment of the present specification. Among them, the function call request can be understood as the target task described above. Based on this, after the scheduling system receives the instruction of requesting scheduling, it can determine whether there is a live GPU instance for the function request, that is, whether there is an instance that can run the function request. Among them, the way to determine whether there is an instance that can run the function request can be determined by judging whether the utilization of the GPU instance is less than the first ratio threshold (that is, the preset maximum utilization of the GPU hardware).

[0132] If not, the scheduling system applies for a new GPU instance to the inventory system, and after the application is successful, it continues to determine whether there is an instance that can run the function request.

[0133] If yes, the scheduling system determines that there is a GPU instance that can run the function call request, and loads the function call request to all GPU instances through the RR polling mode, so as to execute scheduling termination. Among them, loading the function call request to all GPU instances can adopt the above-mentioned way of sorting GPU instances based on utilization to determine the GPU instance with better performance, and realize load balancing.

[0134] Based on this, when the function request arrives, the Serverless scheduling system loads the request to each GPU instance after scaling up and scaling down in turn through the RR polling mode, thereby ensuring that the overall GPU cluster is fully used.

[0135] In an embodiment provided by the present specification, the task processing method provided by the present specification can sample the utilization of each component inside the GPU hardware, and allow users to set different elastic scaling indicators according to different scenarios of heterogeneous computing tasks, thereby solving the cost waste caused by excessive scaling and the performance loss caused by premature narrowing,

[0136] In the embodiments provided in the specification, considering that the serverless default request-based concurrency-based elastic scaling scheduling method does not comprehensively consider the utilization of each component inside the GPU hardware, the problem of cost waste caused by over-scaling and performance loss caused by premature narrowing cannot be solved. The task processing method provided in the specification can solve the problem of cost waste caused by over-scaling and performance loss caused by premature narrowing by sampling the utilization of each component inside the GPU hardware and allowing users to set different elastic scaling indicators according to different heterogeneous computing tasks. The GPU instance is periodically scaled based on the hardware utilization of the GPU instance. The specific implementation manner is as follows.

[0137] The task processing method further includes:

[0138] Based on the physical computing unit corresponding to the initial virtual node, the current state information of the initial virtual node is determined.

[0139] In a case where it is determined based on the current state information that the initial virtual node meets a preset node addition condition, an added virtual node is added; or

[0140] In a case where it is determined based on the current state information that the initial virtual node meets a preset node deletion condition, an idle virtual node in the initial virtual node is deleted.

[0141] Specifically, the scheduling system can periodically determine the physical computing unit corresponding to the initial virtual node, and determine the current state information of the initial virtual node based on the physical computing unit. For example, the utilization of each hardware component in the GPU is monitored in real time, and the utilization of the GPU instance corresponding to each hardware component is determined based on the utilization of each hardware component.

[0142] Then, in a case where it is determined based on the current state information that the initial virtual node meets a preset node addition condition, it is determined that the number of the current initial virtual node is too small, and an added virtual node is added; or

[0143] In a case where it is determined based on the current state information that the initial virtual node meets a preset node deletion condition, it is determined that the number of the initial virtual node is too small, and the utilization of many virtual nodes is low, and thus an idle virtual node in the initial virtual node is deleted. Therefore, the virtual nodes are flexibly and accurately added and deleted, and the problem of cost waste caused by over-scaling and performance loss caused by premature narrowing is solved.

[0144] Further, in the embodiments provided in the specification, the adding of the added virtual node in the case where it is determined based on the current state information that the initial virtual node meets a preset node addition condition includes:

[0145] determine a target computing ratio of the initial virtual node based on the current state information of the initial virtual node;

[0146] determine that the initial virtual node meets a preset node addition condition in a case where the target computing ratio is greater than a node load threshold value;

[0147] add a new virtual node based on a virtual node providing module in a case where the initial virtual node meets the preset node addition condition.

[0148] The node load threshold value can be understood as a threshold value set in advance for each initial virtual node, which represents that the utilization rate of the initial virtual node has reached a load state or is about to reach a load state. In addition, the node load threshold value can be set according to an actual application scenario. For example, the node load threshold value can be set to 80%.

[0149] Specifically, the scheduling system can determine a target computing ratio of the initial virtual node based on the current state information of the initial virtual node, and determine that the initial virtual node meets a preset node addition condition in a case where the target computing ratio is greater than a node load threshold value. In addition, the scheduling system can apply to add a new virtual node to the virtual node providing module in a case where the initial virtual node meets the preset node addition condition, so as to add the virtual node provided by the virtual node providing module as the new virtual node.

[0150] In the above example, the task processing method provided in the specification can allow a user to set different elasticity indexes according to different scene heterogeneous computing tasks. The elasticity indexes include an elasticity expansion index and an elasticity contraction index. The elasticity expansion index can be understood as a utilization threshold value, that is, a node load threshold value. Based on this, after the scheduling system schedules the audio and video production task to the GPU instance with better performance, the scheduling system can monitor the utilization rate of the GPU instance determined based on the hardware decoding unit in real time. In a case where the utilization rate is greater than or equal to the elasticity expansion index (node load threshold value) set by the user, the scheduling system can actively apply to the inventory system for a new GPU instance, and based on the new GPU instance and the GPU instance determined for the audio and video production task based on the utilization rate, the scheduling system can jointly run the video production task, thereby ensuring the normal running of the video production task. In addition, the user can set different elasticity indexes according to different scene heterogeneous computing tasks.

[0151] Further, in the embodiments provided in the specification, the deleting the idle virtual node in the initial virtual node in a case where the initial virtual node meets a preset node deletion condition based on the current state information includes:

[0152] determine a target computing ratio of the initial virtual node based on the current state information of the initial virtual node;

[0153] In a case where the target computing ratio is less than the node idle threshold, it is determined that the initial virtual node meets a preset node deletion condition.

[0154] In a case where the initial virtual node meets the preset node deletion condition, an idle virtual node in the initial virtual node is deleted.

[0155] The node idle threshold can be understood as a threshold value set in advance for each initial virtual node, indicating that the utilization rate of the initial virtual node has reached an idle state. In addition, the node idle threshold can be set according to actual application scenarios. For example, the node load threshold can be set to 0% or 5%.

[0156] In the above example, the task processing method provided by the present specification can monitor the utilization rate of the hardware decoding unit in the GPU instance in real time after the scheduling system schedules the audio and video production task to the GPU instance with better performance. In a case where the utilization rate is less than the elastic scaling index (node idle threshold) set by the user, the scheduling system can actively delete the redundant GPU instance from the inventory system, thereby ensuring the normal operation of the video production task and saving hardware resources.

[0157] In actual applications, in the process of deleting the GPU instance, the scheduling system needs to delete the GPU instance after the task corresponding to the GPU instance is completed, because the GPU instance can still run the audio and video production task. The specific manner is as follows.

[0158] The idle virtual node in the initial virtual node is deleted, including:

[0159] Monitoring task execution state information of the idle virtual node;

[0160] In a case where it is determined that a target task is completed based on the task execution state information, the idle virtual node is deleted.

[0161] The task execution state information can be understood as information indicating the task execution progress.

[0162] Specifically, the scheduling system can monitor the task execution state information of the idle virtual node in real time, and delete the idle virtual node in a case where it is determined that a target task is completed based on the task execution state information, thereby saving hardware resources.

[0163] In actual applications, the scheduling system can periodically perform capacity expansion detection. The capacity expansion detection can be understood as the content in the above embodiments, in which the scheduling system performs elastic scaling according to different elastic indexes set by the user for different heterogeneous computing tasks in different scenarios. Referring to Figure 5 , Figure 5is a schematic diagram of elastic scaling in a task processing method provided by an embodiment of the present specification, wherein the elastic scaling is implemented based on GPU utilization, and a user can configure a GPU elastic scaling indicator for each function. Referring to Figure 5 The scheduling system can periodically perform capacity expansion checking to determine whether the aggregated utilization of all GPU implementations of the function (i.e., the function invocation request) is higher than the user configuration, that is, to determine whether the aggregated utilization of all GPUs running the function invocation request is higher than the user configuration of the GPU elastic scaling indicator for each function invocation request. For example, in an audio / video scenario, the GPU elastic scaling indicator can be configured to expand when the hardware encoding utilization of the GPU instance is > 80%; in an AI scenario, the GPU elastic scaling indicator can be configured to contract when the CUDACORE hardware utilization of the GPU instance is < 20%.

[0164] Based on this, if yes, that is, if the scheduling system determines that the aggregated utilization is higher than the user configuration, it applies to the inventory system for a new GPU instance and performs scheduling termination. If no, that is, if the scheduling system determines that the aggregated utilization is lower than the user configuration, it returns the GPU instance that has been applied to the inventory system and performs scheduling termination.

[0165] It should be noted that when the task processing method is applied in a Serverless scenario, for the elastic scaling of GPU-based heterogeneous computing tasks, the current state information can further include multi-dimensional mixed indicators, so as to facilitate subsequent comprehensive scheduling decisions based on the multi-dimensional mixed indicators, to better adapt to the scaling requirements of heterogeneous computing tasks in various scenarios.

[0166] In addition, the GPU instance capacity expansion in the task processing method provided by the present specification adopts a more aggressive strategy to ensure the service performance of the user function, while the GPU instance capacity contraction adopts a more lazy strategy to take into account the cost of the user function. The aggressive coefficient and the lazy coefficient will take effect when Figure 5 the aggregated utilization of all GPU instances of the function is higher than the user configuration.

[0167] The task processing method provided by the present specification determines the corresponding target virtual node for the target task based on the current state information of the initial virtual node and the task type information of the target task when the target task is received, and executes the target task through the target virtual node, thereby meeting the demand for accurately determining the target virtual node for the target task.

[0168] The following will be described in conjunction with the accompanying Figure 6 The task processing method provided by the present specification will be further described by taking the application of the task processing method in a GPU utilization-based elastic scaling scenario as an example.Figure 6 A process flow chart of a task processing method provided by one embodiment of the present specification is shown, Figure 6 A system framework for realizing elastic scaling based on GPU utilization is provided, wherein the system framework comprises a request access system, a scheduling system, a stock system, a GPU instance, and a GPU monitor (GPU Monitor). The GPU monitor component (GPU Monitor) in the system framework is used to obtain the internal hardware component utilization indicators of each GPU instance. It should be noted that the hardware components are different according to different application scenarios of the task processing method provided by the present specification. For example, in the case that the task processing method is applied to an audio / video production scenario, the hardware components can be hardware encoding units and hardware decoding units. In the case that the task processing method is applied to an AI production scenario, the hardware components can be Cuda Core, etc. Accordingly, the hardware component utilization indicators include but are not limited to hardware encoding utilization in an audio / video production scenario, hardware decoding utilization in an audio / video production scenario, Cuda Core utilization in an AI production scenario, Tensor Core utilization in an AI production scenario, NVLINK bandwidth utilization in an AI production scenario, and memory utilization in all scenarios, etc.

[0169] Based on this, the GPU monitor periodically synchronizes the GPU hardware part utilization to the scheduling system. When a user completes the service function writing of Serverless and initiates a function call request to the request access system of the Serverless platform, the access request system schedules the request to the scheduling system through an access writing protocol. The Serverless scheduling system will realize the elastic scaling of the GPU instance based on the GPU hardware utilization and the corresponding scheduling strategy. For example, the Serverless scheduling system will realize the elastic scaling of the GPU instance by applying or returning the GPU instance to the stock system based on the GPU hardware utilization and the corresponding scheduling strategy. It should be noted that the scheduling strategy can be set according to the actual application scenario, and the present specification does not make specific limitations thereon, for example Figure 4 The scheduling strategy shown.

[0170] At the same time, after the Serverless scheduling system realizes the elastic scaling of the GPU instance, it schedules the function request of the user to different GPU instances to run the corresponding functions, and the GPU instance is responsible for the specific execution of the function.

[0171] The task processing method provided by the embodiments of the present specification provides an elastic scaling control method based on different dimension indicators of a GPU in a Serverless scenario, so that different scenarios of heterogeneous computing tasks (audio / video production, AI production, and graphics / image production) can be set with different elastic scaling indicators, thereby achieving an elastic scaling strategy that takes both performance and cost into account.

[0172] Corresponding to the method embodiments described above, the present specification also provides task processing device embodiments, Figure 6 A structural schematic diagram of a task processing device provided by one embodiment of the present specification is shown. As shown in the figure, Figure 6 The device comprises:

[0173] The receiving module 702 is configured to determine the current state information of the initial virtual node based on the received target task, wherein the current state information is determined based on the physical computing unit corresponding to the initial virtual node;

[0174] The determining module 704 is configured to determine the candidate virtual node corresponding to the task type information from the initial virtual node based on the task type information of the target task;

[0175] The execution module 706 is configured to determine the target virtual node corresponding to the target task based on the current state information of the candidate virtual node, and execute the target task through the target virtual node.

[0176] Optionally, the execution module 706 is further configured to:

[0177] select the target virtual node corresponding to the target task from the candidate virtual node based on the current state information of the candidate virtual node.

[0178] Optionally, the execution module 706 is further configured to:

[0179] add the target virtual node corresponding to the target task based on the current state information of the candidate virtual node.

[0180] Optionally, the task processing device further comprises a node processing module configured to:

[0181] determine the current state information of the initial virtual node based on the physical computing unit corresponding to the initial virtual node;

[0182] add a new virtual node in a case where the initial virtual node meets a preset node addition condition based on the current state information; or

[0183] In a case where it is determined, based on the current state information, that the initial virtual node meets a preset node deletion condition, an idle virtual node in the initial virtual node is deleted.

[0184] Optionally, in the task processing apparatus, the physical computing unit is a GPU.

[0185] Correspondingly, the receiving module 702 is further configured to:

[0186] determine, based on the received target task, a hardware component of the GPU and a current utilization of the hardware component;

[0187] determine, from the hardware component, a target hardware component corresponding to the initial virtual node, and take the current utilization of the target hardware component as the current state information of the initial virtual node.

[0188] Optionally, the task processing apparatus further comprises a deletion module configured to:

[0189] in a case where it is determined, based on the current utilization of the target hardware component, that the target task is executed, delete the target virtual node.

[0190] Correspondingly, the execution module 706 is further configured to:

[0191] determine, based on the current state information of the candidate virtual node, a target computing ratio of the candidate virtual node;

[0192] in a case where the target computing ratio is greater than or equal to a first ratio threshold, add a corresponding target virtual node to the target task.

[0193] Correspondingly, the execution module 706 is further configured to:

[0194] generate a virtual node obtaining request based on the target task, and send the virtual node obtaining request to a virtual node providing module;

[0195] receive a to-be-determined virtual node sent by the virtual node providing module based on the virtual node obtaining request, and determine the to-be-determined virtual node as a target virtual node corresponding to the target task.

[0196] Correspondingly, the execution module 706 is further configured to:

[0197] determine, based on the current state information of the candidate virtual node, a target computing ratio of the candidate virtual node;

[0198] in a case where the target computing ratio is less than a first ratio threshold, determine a minimum target computing ratio from the target computing ratio;

[0199] determine a target virtual node corresponding to the target task based on the candidate virtual node corresponding to the minimum target calculation ratio.

[0200] Optionally, the node processing module is further configured to:

[0201] determine a target calculation ratio of the initial virtual node based on the current state information of the initial virtual node;

[0202] determine that the initial virtual node meets a preset node addition condition in a case where the target calculation ratio is greater than a node load threshold;

[0203] add a new virtual node based on a virtual node providing module in a case where the initial virtual node meets the preset node addition condition.

[0204] Optionally, the node processing module is further configured to:

[0205] determine a target calculation ratio of the initial virtual node based on the current state information of the initial virtual node;

[0206] determine that the initial virtual node meets a preset node deletion condition in a case where the target calculation ratio is less than a node idle threshold;

[0207] delete an idle virtual node in the initial virtual node in a case where the initial virtual node meets the preset node deletion condition.

[0208] Optionally, the receiving module 702 is further configured to:

[0209] determine a physical calculation subunit in the physical calculation unit and current running information of the physical calculation subunit based on the received target task;

[0210] determine a target physical calculation subunit corresponding to the initial virtual node from the physical calculation subunit;

[0211] determine the current running information of the target physical calculation subunit as the current state information of the initial virtual node.

[0212] Optionally, the task processing apparatus further includes an information receiving module configured to:

[0213] receive current running information of the physical calculation subunit in the physical calculation unit sent by an information collection module, wherein the information collection module is a module for monitoring the current running information of the physical calculation subunit in the physical calculation unit.

[0214] The task processing apparatus provided in the specification meets the requirement of accurately determining the target virtual node for the target task by determining the corresponding target virtual node for the target task based on the current state information of the initial virtual node and the task type information of the target task when the target task is received, and executing the target task through the target virtual node.

[0215] The above is a schematic scheme of the task processing apparatus of the embodiment. It should be noted that the technical scheme of the task processing apparatus and the technical scheme of the task processing method described above belong to the same concept, and the details of the technical scheme of the task processing apparatus which are not described in detail can be referred to the description of the technical scheme of the task processing method.

[0216] Figure 8 A structural block diagram of a computing device 800 according to an embodiment of the specification is shown. The components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 through a bus 830, and a database 850 is used to save data.

[0217] The computing device 800 also includes an access device 840, which enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 840 can include one or more of any type of network interface (e.g., network interface card (NIC)) such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a worldwide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like, either wired or wireless.

[0218] In an embodiment of the specification, the above-mentioned components of the computing device 800 and other components not shown in the specification can be connected to each other, for example, through a bus. It should be understood that Figure 8 the structural block diagram of the computing device shown is only for the purpose of example, and is not a limitation on the scope of the specification. Other components can be added or replaced as needed by those skilled in the art. Figure 8 the structural block diagram of the computing device shown is only for the purpose of example, and is not a limitation on the scope of the specification. Other components can be added or replaced as needed by those skilled in the art.

[0219] The computing device 800 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or PC. The computing device 800 can also be a mobile or stationary server.

[0220] The processor 820 is configured to execute computer-executable instructions to perform the steps of the task processing method described above.

[0221] The above is a schematic solution of the computing device of the embodiment. It should be noted that the technical solution of the computing device and the technical solution of the task processing method described above belong to the same concept, and the details of the technical solution of the computing device that are not described in detail can be referred to the description of the technical solution of the task processing method.

[0222] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the task processing method described above.

[0223] The above is a schematic solution of the computer-readable storage medium of the embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the task processing method described above belong to the same concept, and the details of the technical solution of the storage medium that are not described in detail can be referred to the description of the technical solution of the task processing method.

[0224] An embodiment of the present specification further provides a computer program, which, when executed in a computer, causes the computer to perform the steps of the task processing method described above.

[0225] The above is a schematic solution of the computer program of the embodiment. It should be noted that the technical solution of the computer program and the technical solution of the task processing method described above belong to the same concept, and the details of the technical solution of the computer program that are not described in detail can be referred to the description of the technical solution of the task processing method.

[0226] The above-described embodiments of the application have several aspects, no single one of which is solely responsible for the application's desirable attributes. Without limiting the scope of the application as expressed by the claims which follow, some further embodiments make these aspects even more useful. Other embodiments can result in less desirable attributes.

[0227] The computer readable medium can include any entity or apparatus capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, software distribution medium, etc. It should be noted that the computer readable medium can include appropriate contents according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0228] It should be noted that for the foregoing method embodiments, the acts described can be performed in a different order than that described, and that various interlocking and / or parallel configurations are also possible according to the certain embodiments of the present specification. Furthermore, certain of the acts can be optional depending upon the particular embodiment of the method. Furthermore, those skilled in the art will recognize that the embodiments described herein can be embodied as a system, method or computer program in instructions, code or code segments.

[0229] In the above embodiments, the description of each embodiment is focused on different aspects, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0230] The above disclosed preferred embodiments of the present specification are only used to help explain the present specification. The alternative embodiments do not describe all the details and do not limit the application to the specific embodiments described. Obviously, according to the content of the embodiments of the present specification, many modifications and changes can be made. The present specification selects and describes these embodiments in order to better explain the principles and practical applications of the embodiments of the present specification, so that those skilled in the art can well understand and use the present specification. The present specification is limited by the claims and their full scope and equivalents.

Claims

1. A task processing method applied to a scheduling system of a serverless platform, the scheduling system being responsible for applying for GPU instances to an inventory system and being responsible for scheduling function requests of users to different GPU instances to run corresponding functions, comprising: determining current state information of an initial virtual node based on a received target task, wherein the current state information is determined based on a physical computing unit corresponding to the initial virtual node, and the physical computing unit comprises GPU hardware; determining a candidate virtual node corresponding to task type information of the target task from the initial virtual node based on the task type information; determining a target virtual node corresponding to the target task based on current state information of the candidate virtual node, and executing the target task through the target virtual node, wherein the current state information is utilization rates of hardware components in a GPU corresponding to the GPU instance; periodically performing elastic scaling of the GPU instance based on the utilization rates of the hardware components in the GPU instance by sampling the utilization rates of the hardware components in the GPU hardware and allowing a user to set different elastic scaling indexes according to different heterogeneous computing tasks in different scenarios.

2. The task processing method of claim 1, wherein the determining the target virtual node corresponding to the target task based on the current state information of the candidate virtual node comprises: selecting the target virtual node corresponding to the target task from the candidate virtual node based on the current state information of the candidate virtual node.

3. The task processing method of claim 1, wherein the determining the target virtual node corresponding to the target task based on the current state information of the candidate virtual node comprises: adding the target virtual node corresponding to the target task based on the current state information of the candidate virtual node.

4. The task processing method of claim 1, further comprising: determining the current state information of the initial virtual node based on the physical computing unit corresponding to the initial virtual node; adding a new virtual node in a case where it is determined based on the current state information that the initial virtual node meets a preset node adding condition; or deleting an idle virtual node in the initial virtual node in a case where it is determined based on the current state information that the initial virtual node meets a preset node deleting condition.

5. The task processing method of claim 1, wherein the physical computing unit is a GPU; accordingly, the determining the current state information of the initial virtual node based on the received target task comprises: determining hardware components of the GPU and current utilization rates of the hardware components based on the received target task; determining a target hardware component corresponding to the initial virtual node from the hardware components, and taking the current utilization rate of the target hardware component as the current state information of the initial virtual node.

6. The task processing method of claim 5, further comprising: ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ In a case that the target task is determined to be completed based on the current utilization of the target hardware component, the target virtual node is deleted.

7. The task processing method of claim 3, wherein the adding the target virtual node corresponding to the target task based on the current state information of the candidate virtual node comprises: determining a target computing ratio of the candidate virtual node based on the current state information of the candidate virtual node; and in a case that the target computing ratio is greater than or equal to a first ratio threshold, adding the target virtual node corresponding to the target task.

8. The task processing method of claim 7, wherein the adding the target virtual node corresponding to the target task comprises: generating a virtual node obtaining request based on the target task, and sending the virtual node obtaining request to a virtual node providing module; and receiving a to-be-determined virtual node sent by the virtual node providing module based on the virtual node obtaining request, and determining the to-be-determined virtual node as the target virtual node corresponding to the target task.

9. The task processing method of claim 2, wherein the selecting the target virtual node corresponding to the target task from the candidate virtual nodes based on the current state information of the candidate virtual nodes comprises: determining a target computing ratio of the candidate virtual node based on the current state information of the candidate virtual node; in a case that the target computing ratio is less than a first ratio threshold, determining a minimum target computing ratio from the target computing ratios; and determining the target virtual node corresponding to the target task based on the candidate virtual node corresponding to the minimum target computing ratio.

10. The task processing method of claim 4, wherein the adding a new virtual node in a case that the initial virtual node is determined to satisfy a preset node adding condition based on the current state information comprises: determining a target computing ratio of the initial virtual node based on the current state information of the initial virtual node; in a case that the target computing ratio is greater than a node load threshold, determining that the initial virtual node satisfies the preset node adding condition; and adding a new virtual node based on a virtual node providing module in a case that the initial virtual node satisfies the preset node adding condition.

11. The task processing method of claim 4, wherein the deleting an idle virtual node in the initial virtual node in a case that the initial virtual node is determined to satisfy a preset node deleting condition based on the current state information comprises: determining a target computing ratio of the initial virtual node based on the current state information of the initial virtual node; in a case that the target computing ratio is less than a node idle threshold, determining that the initial virtual node satisfies the preset node deleting condition; and deleting the idle virtual node in the initial virtual node in a case that the initial virtual node satisfies the preset node deleting condition.

12. The task processing method of claim 1, wherein the determining the current state information of the initial virtual node based on the received target task comprises: ​ ​ ​ determining a physical computing subunit in the physical computing unit and current running information of the physical computing subunit based on the received target task; determining a target physical computing subunit corresponding to the initial virtual node from the physical computing subunits; taking the current running information of the target physical computing subunit as the current state information of the initial virtual node.

13. The task processing method of claim 12, before determining the current state information of the initial virtual node based on the received target task, further comprising: receiving the current running information of the physical computing subunits in the physical computing unit sent by an information collection module, wherein the information collection module is a module for monitoring the current running information of the physical computing subunits in the physical computing unit.

14. A computing device, comprising: a memory and a processor; the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, and the computer executable instructions, when executed by the processor, implement the steps of the task processing method of any one of claims 1 to 13.

Images