Task execution method and device, computer device, and storage medium

Abstract

The present application relates to the technical field of artificial intelligence, and discloses a task execution method and device, computer equipment and a storage medium, comprising: receiving a target request sent by a target device, wherein the target request comprises target task information and identification information of a target model. Obtain configuration information corresponding to a plurality of computing instances deployed by a target platform. According to the identification information of the target model and the configuration information corresponding to each computing instance, select a target computing instance from the plurality of computing instances. When it is determined that the target computing instance has a deployed model, obtain identification information corresponding to each deployed model. According to the identification information of the target model and the identification information corresponding to each deployed model, determine whether there is a task processing model in the deployed model. When there is a task processing model, input the target task information into the task processing model, obtain the output result, and feed back to the target device. The present application can avoid the problem of resource waste.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to task execution methods, apparatus, computer equipment, and storage media. Background Technology

[0002] With the rapid development of artificial intelligence technology, the application of artificial intelligence (AI) models is becoming increasingly widespread. The deployment and operation of AI models place very high demands on computing resources.

[0003] In AI service platforms, to ensure stable operation, an AI model is typically deployed on a single resource component. However, a large single language model may only consume a portion of the resource component's resources, leading to resource waste. Summary of the Invention

[0004] In view of this, the present invention provides a task execution method, apparatus, computer equipment, storage medium, and program product to solve the problem of resource waste in related technologies.

[0005] In a first aspect, the present invention provides a task execution method, comprising:

[0006] Receive a target request sent by the target device, wherein the target request includes target task information and identification information of the target model for processing the target task information;

[0007] Obtain the configuration information of each computing instance among multiple computing instances deployed on the target platform. Each computing instance occupies all resources on a resource component included in the target platform.

[0008] Based on the identification information of the target model and the configuration information corresponding to each computing instance, a target computing instance is selected from multiple computing instances.

[0009] When it is determined that the target computing instance has at least one deployed model, obtain the identification information corresponding to each deployed model;

[0010] Based on the identification information of the target model and the identification information corresponding to each deployed model, determine whether there is a task processing model in at least one deployed model;

[0011] Once a task processing model is identified, the target task information is input into the task processing model, and the output result is obtained.

[0012] The output results are fed back to the target device.

[0013] The task execution method provided by this invention has the following advantages:

[0014] Upon receiving a target request, this solution selects a target computing instance based on the target model's identifier and the configuration information corresponding to each computing instance. One or more models can be deployed on a target computing instance. Therefore, this solution can determine whether a task processing model exists among the deployed models on the target computing instance based on the target model's identifier. If so, the target task information can be input into the task processing model, and the output result can be fed back to the target device. In this way, each computing instance can utilize all the resources of a resource component, and one or more models can be deployed on a single computing instance. When allocating tasks, tasks are scheduled according to the actual deployment of each computing instance, fully utilizing the resources on each resource component and avoiding resource waste.

[0015] In one optional implementation, a target computing instance is selected from multiple computing instances based on the identification information of the target model and the configuration information corresponding to each computing instance, including:

[0016] Based on the identification information of the target model and the configuration information corresponding to each computing instance, at least one candidate computing instance is selected from multiple computing instances.

[0017] Get the number of tasks corresponding to each candidate computation instance;

[0018] Based on the number of tasks corresponding to each candidate computation instance, a target computation instance is selected from at least one candidate computation instance.

[0019] Specifically, firstly, candidate computing instances are selected based on the configuration information of the computing instances and the identification information of the target model. Then, the number of tasks corresponding to each candidate computing instance is obtained. Subsequently, based on the number of tasks corresponding to each candidate computing instance, the target computing instance is selected from at least one candidate computing instance. This can balance the number of tasks on each computing instance, ensure load balancing, and avoid the problem of excessive load on a certain computing instance leading to low task processing efficiency.

[0020] In one optional implementation, the configuration information includes at least one type of identification information corresponding to each model;

[0021] Based on the identification information of the target model and the configuration information corresponding to each computing instance, at least one candidate computing instance is selected from multiple computing instances, including:

[0022] Determine whether the identification information of the target model is the identification information of any model included in the configuration information corresponding to the first computing instance, wherein the first computing instance is any computing instance among multiple computing instances;

[0023] When the identification information of the target model is determined to be the identification information of any model included in the configuration information corresponding to the first computing instance, the first computing instance is determined as a candidate computing instance.

[0024] Specifically, by comparing the identification information of the target model with the identification information included in the configuration information of each computing instance, it can be determined whether each computing instance can deploy the target model. If so, the computing instance can be identified as a candidate computing instance for further screening.

[0025] In one optional implementation, determining whether a task processing model exists among at least one deployed model is based on the identification information of the target model and the identification information corresponding to each deployed model, including:

[0026] Based on the identification information of the target model and the identification information corresponding to each deployed model, determine whether at least one deployed model includes at least one target model;

[0027] When it is determined that at least one deployed model includes at least one target model, obtain the number of tasks corresponding to each target model;

[0028] Based on the number of tasks corresponding to each target model and the preset task number threshold corresponding to the identification information of the target model, determine whether there is a target model that can be used as a task processing model among at least one target model.

[0029] Specifically, first determine whether the target model is included in the deployed models on the target computing instance. If so, then based on the number of tasks being processed by each model and a preset task quantity threshold, determine whether there is a task processing model among the deployed models that can be used to process the target task model. This ensures that the number of tasks processed by each target model meets the requirements, avoiding the problem of target model congestion or failure due to an excessive number of tasks.

[0030] In an optional implementation, the configuration information further includes a model quantity threshold; when it is determined that at least one deployed model does not include the target model, or that at least one target model does not contain a target model serving as a task processing model, the method further includes:

[0031] Determine whether the number of deployed models in the target computing instance is less than the threshold number of models included in the configuration information corresponding to the target computing instance;

[0032] When the number of deployed models in the target computing instance is less than the threshold of the number of models included in the configuration information corresponding to the target computing instance, the target storage path mounted under the target computing instance is obtained.

[0033] Based on the target storage path and the identification information of the target model, obtain the target file corresponding to the target model;

[0034] The target model is deployed on the target computing instance using the target file to process the target task information.

[0035] Specifically, when it is determined that the target model is not deployed on the target computing instance, or when there is no task processing model among the deployed models that can process the target task information, it is first determined whether a model can still be deployed on the target computing instance. If so, the target file is obtained based on the target storage path mounted under the target computing instance and the identification information of the target model, and the target model is deployed on the target computing instance based on the target file. In this way, when the target computing instance has sufficient resources, the target task information can be processed efficiently without waiting for other tasks to be completed before processing the target task information. This can ensure the full utilization of resources and improve task processing efficiency.

[0036] In one optional implementation, the identification information of the target computing instance includes the resource type; the method further includes:

[0037] The number of tasks corresponding to the target computing instance is updated based on the resource type included in the identifier information of the target computing instance.

[0038] Specifically, since different resource types offer significantly different amounts of resources, when calculating the number of tasks for a compute instance, the number of tasks for the target compute instance can be updated based on the resource type. This eliminates the need for real-time monitoring of remaining resources to determine which compute instance to allocate new tasks to; instead, the task quantity is used directly, saving resources.

[0039] In one alternative implementation, the target computing instance may be of a central processing unit type or a graphics processing unit type;

[0040] Based on the resource types included in the identifier information of the target computing instance, the number of tasks corresponding to the target computing instance is updated, including:

[0041] When the resource type is a central processing unit, the number of tasks corresponding to the target computing instance is updated according to the first preset increment.

[0042] or,

[0043] When the resource type is a graphics processor, the number of tasks corresponding to the target computing instance is updated according to the second preset increment, wherein the first preset increment is greater than the second preset increment.

[0044] Specifically, compared to CPU-type computing instances, GPU-type computing instances have more abundant resources. Therefore, when updating the number of tasks after task allocation, if the target computing instance to be allocated is a CPU-type computing instance, a larger first preset increment can be used for updating. This can reduce the probability of being allocated to a CPU-type computing instance and avoid impacting other CPU services.

[0045] In a second aspect, the present invention provides a task execution device, comprising:

[0046] The receiving module is used to receive a target request sent by the target device, wherein the target request includes target task information and identification information of the target model for processing the target task information;

[0047] The acquisition module is used to acquire the configuration information of each computing instance among multiple computing instances deployed on the target platform. Each computing instance occupies all the resources on a resource component included in the target platform.

[0048] The selection module is used to select a target computing instance from multiple computing instances based on the identification information of the target model and the configuration information corresponding to each computing instance.

[0049] The acquisition module is also used to acquire the identification information corresponding to each deployed model when it is determined that the target computing instance has at least one deployed model;

[0050] The determination module is used to determine whether a task processing model exists in at least one deployed model based on the identification information of the target model and the identification information corresponding to each deployed model.

[0051] The acquisition module is also used to input the target task information into the task processing model and obtain the output results when it is determined that a task processing model exists;

[0052] The feedback module is used to send the output results back to the target device.

[0053] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the task execution method described in the first aspect or any corresponding embodiment thereof.

[0054] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the task execution method described in the first aspect or any corresponding embodiment thereof.

[0055] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to execute the task execution method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the specific embodiments or related technologies of the present invention, the drawings used in the description of the specific embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0057] Figure 1 This is a schematic diagram of the system architecture according to an embodiment of the present invention;

[0058] Figure 2 This is a schematic diagram of the deployment architecture of a computing instance on a target platform according to an embodiment of the present invention;

[0059] Figure 3 This is a flowchart illustrating a task execution method according to an embodiment of the present invention;

[0060] Figure 4 This is a structural block diagram of a task execution device according to an embodiment of the present invention;

[0061] Figure 5 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] This invention can be implemented by a system consisting of a target platform and a target device, such as... Figure 1 As shown, the target platform can be used to deploy various artificial intelligence (AI) models to provide a variety of AI services. For example, the AI ​​model can be a large language model (LLM), which can provide services such as customer service, knowledge retrieval, and question answering. The target device can be a client device, such as a computer or mobile phone, where users can input relevant information into the AI ​​service interface, for example, by entering a question in a question-and-answer service.

[0064] like Figure 2 As shown, the target platform can deploy a scheduling component and multiple computing instances, each a separate computing environment for deploying applications (e.g., AI models). For example, computing instances can be virtual machines and containers. The target platform can include multiple resource components, such as graphics processing units (GPUs), central processing units (CPUs), etc. The scheduling component can receive requests from the target device and schedule the corresponding tasks onto various computing instances. A computing instance can consume all the resources of a single resource component included in the target platform. For example, in... Figure 2 Computational instances 1 through 3 each used all the resources of a graphics processor, while computational instance 4 used all the resources of a central processing unit.

[0065] The target platform can store a list of computing instances, which may include identification information for each instance and corresponding configuration information. Additionally, a queue can be pre-created for each computing instance to temporarily store task information from requests. The identification information of a computing instance may include the resource type and number of its corresponding resource component; the resource type can be a central processing unit (CPU) type or a graphics processing unit (GPU) type. For example, the identification information of a computing instance could be GPU-0, GPU-1, GPU-2, GPU-3, and CPU-0.

[0066] This invention provides a task execution method that, by deploying one or more models on each computing instance, schedules tasks according to the actual deployment of each computing instance when allocating tasks, thereby making full use of the resources on each resource component and avoiding resource waste.

[0067] According to an embodiment of the present invention, a task execution method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0068] This embodiment provides a task execution method that can be executed by the aforementioned target platform. Figure 3 This is a flowchart of a task execution method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0069] Step S301: Receive the target request sent by the target device.

[0070] The target request may include target task information and identification information of the target model that processes the target task information. The type of the target computing instance may include a central processing unit type or a graphics processing unit type. The identification information of the target model may include the name and number of parameters of the target model, for example, "Qwen2-1.5B", where "Qwen2" is the name of the target model and "1.5B" is the number of parameters of the target model.

[0071] Specifically, users can select the identifier information of a target model on the target device and input the target task information. For example, the selected identifier information of the target model can be used to indicate a question-and-answer model, and the input target task information is the question. The target device can generate a target request based on the user's operation and send it to the target platform. After receiving the target request, the target platform can parse the target request and obtain the identifier information of the target model and the target task information.

[0072] Step S302: Obtain the configuration information corresponding to each computing instance among the multiple computing instances deployed on the target platform.

[0073] The configuration information corresponding to a computing instance may include the identification information of at least one model that can be deployed on that computing instance, or it may include a model number threshold (the maximum number of models that can be deployed on the corresponding resource component). Accordingly, the identification information and configuration information of each computing instance in the computing instance list can be arranged from most to least according to the model number threshold. The configuration information corresponding to each computing instance can be determined by the technician based on the model's data precision (e.g., float32, BF16, Int4) and parameter quantity, as well as the resource size provided by the resource component corresponding to that computing instance. For example, the configuration information corresponding to each computing instance may be: (1) GPU-0: "Qwen2-1.5B" and "maximum of 6 models can be deployed simultaneously (which may be the number calculated based on the largest model)"; (2) GPU-1: "Qwen2-1.5B, Llama2-7B" and "maximum of 3 models can be deployed simultaneously".

[0074] Step S303: Select a target computing instance from multiple computing instances based on the identification information of the target model and the configuration information corresponding to each computing instance.

[0075] Specifically, step S303 may include the following steps:

[0076] Step 1: Select at least one candidate computing instance from multiple computing instances based on the identification information of the target model and the configuration information corresponding to each computing instance.

[0077] Specifically, the target platform can determine whether the identification information of the target model corresponds to the identification information of any model included in the configuration information corresponding to the first computing instance, where the first computing instance is any one of multiple computing instances. When it is determined that the identification information of the target model corresponds to the identification information of any model included in the configuration information corresponding to the first computing instance, the first computing instance can be identified as a candidate computing instance. In this way, after performing the above judgment for each computing instance, at least one candidate computing instance can be identified.

[0078] Step 2: Obtain the number of tasks corresponding to each candidate computing instance.

[0079] Step 3: Select a target computing instance from at least one candidate computing instance based on the number of tasks corresponding to each candidate computing instance.

[0080] Specifically, each computing instance can correspond to a task count parameter. Each time a task is assigned to a computing instance, the value of the task count parameter corresponding to that computing instance will be updated. Therefore, before each task assignment, the target platform can first obtain the value of the task count parameter corresponding to each candidate computing instance, select the candidate computing instance with the smallest data as the target computing instance, and add the target task information to the queue corresponding to the target computing instance to complete the target task information allocation operation.

[0081] In some optional implementations, in addition to obtaining the number of tasks corresponding to each candidate computing instance, the target platform can also obtain the identification information corresponding to each deployed model on each candidate computing instance, as well as the number of deployed models on each candidate computing instance. Taking the first candidate computing instance (any one of at least one candidate instance) as an example, the target platform can obtain the task quantity threshold corresponding to each deployed model based on the identification information corresponding to each deployed model on the first candidate computing instance. Then, the priority of the first candidate computing instance can be determined based on the number of tasks corresponding to the first candidate computing instance, the number of deployed models, the model quantity threshold included in the configuration information of the first candidate computing instance, and the task quantity threshold corresponding to each deployed model on the first candidate computing instance. Finally, the target platform can select the candidate computing instance with the highest priority as the target computing instance.

[0082] For example, the specific process of determining the priority of the first candidate computing instance based on the number of tasks corresponding to the first candidate computing instance, the number of deployed models, the model number threshold included in the configuration information of the first candidate computing instance, and the task number threshold corresponding to each deployed model on the first candidate computing instance can be as follows:

[0083] The target threshold is obtained by summing the task number thresholds corresponding to all deployed models on the first candidate computing instance. Then, a first ratio is calculated between the number of tasks corresponding to the first candidate computing instance and the target threshold. Furthermore, a second ratio is calculated between the number of deployed models and the number of models included in the configuration information of the first candidate computing instance. Finally, the sum of the first and second ratios is determined, and the difference between one and this sum is used as the priority.

[0084] A larger first ratio indicates that the deployed model processes a larger number of tasks on average. Assigning new tasks will likely require deploying a new model, which takes time. A larger second ratio indicates a heavier load on the first candidate computing instance, with less resource margin available for model deployment. By summing the first and second ratios, we can determine the efficiency of the first candidate computing instance in processing the target task information; a larger sum indicates lower efficiency. Therefore, selecting target computing instances using this method increases the probability of choosing instances with higher efficiency in processing task information, thus improving overall task information processing efficiency.

[0085] In some optional implementations, the target platform may first obtain the number of tasks corresponding to each computing instance, and determine whether each computing instance supports deploying the target model based on the target model's identification information and the model deployment list configuration information corresponding to each computing instance. Then, based on the number of tasks corresponding to each computing instance and the determination result of whether each computing instance supports deploying the target model, a target computing instance can be selected (specifically, the computing instance with the fewest tasks and supporting the target model is selected as the target computing instance).

[0086] Step S304: When it is determined that there is at least one deployed model in the target computing instance, obtain the identification information corresponding to each deployed model.

[0087] Specifically, the target platform can check whether one or more models have been deployed on the target computing instance. When it is determined that a model has been deployed on the target computing instance, it can obtain the identification information corresponding to each deployed model to determine whether the target model exists.

[0088] Alternatively, if it is determined that no model is deployed on the target compute instance, the target storage path mounted under the target compute instance can be obtained. Then, based on the target storage path and the identification information of the target model, the target file corresponding to the target model is obtained from multiple model files stored under the target storage path. The target file is then used to deploy the target model on the target compute instance. After the target model deployment is complete, the target task information is input into the newly deployed target model on the target compute instance to obtain the output result corresponding to the target task information, which is then fed back to the target device.

[0089] Step S305: Based on the identification information of the target model and the identification information corresponding to each deployed model, determine whether there is a task processing model among at least one deployed model.

[0090] Specifically, step S305 may include the following steps:

[0091] Step 1: Based on the identification information of the target model and the identification information corresponding to each deployed model, determine whether at least one deployed model includes at least one target model.

[0092] Step 2: When it is determined that at least one deployed model includes at least one target model, obtain the number of tasks corresponding to each target model.

[0093] Step 3: Based on the number of tasks corresponding to each target model and the preset task number threshold corresponding to the identification information of the target model, determine whether there is a target model that can be used as a task processing model among at least one target model.

[0094] Specifically, when the number of tasks storing at least one target model is less than a preset task number threshold, the target model with the smallest number of tasks can be identified as the task processing model, meaning that a target model that can be used as a task processing model is determined to exist. When the number of tasks for all target models is equal to the preset task number threshold, it is determined that no target model can be used as a task processing model.

[0095] Step 4: When it is determined that at least one deployed model does not include the target model, determine whether the number of deployed models in the target computing instance is less than the threshold of the number of models included in the configuration information corresponding to the target computing instance.

[0096] Step 5: When it is determined that the number of deployed models in the target computing instance is less than the threshold of the number of models included in the configuration information corresponding to the target computing instance, obtain the target storage path mounted under the target computing instance.

[0097] Step 6: Obtain the target file corresponding to the target model based on the target storage path and the identification information of the target model.

[0098] Step 7: Using the target file, deploy the target model on the target computing instance.

[0099] Step 8: When the number of deployed models in the target computing instance is equal to the threshold number of models included in the configuration information corresponding to the target computing instance, processing can be paused. After other models have finished processing their tasks, the other models that have finished processing their tasks are shut down, and then the target storage path mounted under the target computing instance is retrieved. Based on the target storage path and the identifier information of the target model, the target file corresponding to the target model is obtained, and the target model is deployed on the target computing instance.

[0100] Step 9: Input the target task information into the newly deployed target model on the target computing instance and obtain the output results corresponding to the target task information.

[0101] Step S306: When it is determined that a task processing model exists, the target task information is input into the task processing model and the output result is obtained.

[0102] Specifically, once a task processing model is identified, the target task information can be input into the task processing model to obtain the output result.

[0103] In some optional implementations, when it is determined that no task processing model exists, the target platform can perform the specific processing in steps five to seven of step S305, and input the target task information into the newly deployed target model on the target computing instance to obtain the output result.

[0104] In some optional implementations, after steps S303 and S306, the target platform can update the number of tasks corresponding to the target computing instance based on the resource type included in the identifier information of the target computing instance.

[0105] Specifically, after step S303, when the resource type is determined to be a CPU type, the number of tasks corresponding to the target computing instance is updated according to a first preset increment. Alternatively, when the resource type is a GPU type, the number of tasks corresponding to the target computing instance is updated according to a second preset increment, wherein the first preset increment is greater than the second preset increment. After step S306, when the resource type is determined to be a CPU type, the number of tasks corresponding to the target computing instance is updated according to a first preset decrement. Alternatively, when the resource type is a GPU type, the number of tasks corresponding to the target computing instance is updated according to a second preset decrement, wherein the first preset decrement is greater than the second preset decrement.

[0106] For example, the code for selecting the target compute instance and updating the number of tasks corresponding to the target compute instance can be as follows:

[0107]

[0108]

[0109] In the code above, the first preset increment and the first preset decrement are both "10", while the second preset increment and the second preset decrement are both "1".

[0110] Step S307: Feedback the output results to the target device.

[0111] Specifically, after obtaining the output results, the target platform can send the output results to the target device.

[0112] The task execution method provided in this embodiment, upon receiving a target request, selects a target computing instance based on the identifier information of the target model in the request and the configuration information corresponding to each computing instance. One or more models can be deployed on a target computing instance. Therefore, this solution can determine whether a task processing model exists among the deployed models on the target computing instance based on the identifier information of the target model. If so, the target task information can be input into the task processing model, and the output result can be fed back to the target device. In this way, each computing instance can occupy all the resources of a resource component, and one or more models can be deployed on a computing instance. When allocating tasks, tasks are scheduled according to the actual deployment of each computing instance, which can fully utilize the resources on each resource component and avoid resource waste.

[0113] This embodiment also provides a task execution device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0114] This embodiment provides a task execution device, such as... Figure 4 As shown, it includes:

[0115] The receiving module 401 is used to receive a target request sent by the target device, wherein the target request includes target task information and identification information of the target model for processing the target task information;

[0116] The acquisition module 402 is used to acquire the configuration information corresponding to each of the multiple computing instances deployed on the target platform. Here, a computing instance occupies all the resources on a resource component included in the target platform.

[0117] The selection module 403 is used to select a target computing instance from multiple computing instances based on the identification information of the target model and the configuration information corresponding to each computing instance.

[0118] The acquisition module 402 is also used to acquire the identification information corresponding to each deployed model when it is determined that the target computing instance has at least one deployed model;

[0119] The determination module 404 is used to determine whether a task processing model exists in at least one deployed model based on the identification information of the target model and the identification information corresponding to each deployed model.

[0120] The acquisition module 402 is also used to input the target task information into the task processing model and obtain the output result when it is determined that a task processing model exists;

[0121] Feedback module 405 is used to send the output results back to the target device.

[0122] In some alternative implementations, module 403 is selected and is specifically used for:

[0123] Based on the identification information of the target model and the configuration information corresponding to each computing instance, at least one candidate computing instance is selected from multiple computing instances.

[0124] Get the number of tasks corresponding to each candidate computation instance;

[0125] Based on the number of tasks corresponding to each candidate computation instance, a target computation instance is selected from at least one candidate computation instance.

[0126] In some optional implementations, the configuration information includes at least one type of identification information corresponding to each model;

[0127] Select module 403, specifically for:

[0128] Determine whether the identification information of the target model is the identification information of any model included in the configuration information corresponding to the first computing instance, wherein the first computing instance is any computing instance among multiple computing instances;

[0129] When the identification information of the target model is determined to be the identification information of any model included in the configuration information corresponding to the first computing instance, the first computing instance is determined as a candidate computing instance.

[0130] In some alternative implementations, the determining module 404 is specifically used for:

[0131] Based on the identification information of the target model and the identification information corresponding to each deployed model, determine whether at least one deployed model includes at least one target model;

[0132] When it is determined that at least one deployed model includes at least one target model, obtain the number of tasks corresponding to each target model;

[0133] Based on the number of tasks corresponding to each target model and the preset task number threshold corresponding to the identification information of the target model, determine whether there is a target model that can be used as a task processing model among at least one target model.

[0134] In some optional implementations, the configuration information also includes a threshold for the number of models;

[0135] Module 404 is also used to:

[0136] When it is determined that at least one deployed model does not include the target model, or at least one target model does not have a target model as a task processing model, determine whether the number of deployed models in the target computing instance is less than the threshold of the number of models included in the configuration information corresponding to the target computing instance.

[0137] When the number of deployed models in the target computing instance is less than the threshold of the number of models included in the configuration information corresponding to the target computing instance, the target storage path mounted under the target computing instance is obtained.

[0138] Based on the target storage path and the identification information of the target model, obtain the target file corresponding to the target model;

[0139] The target model is deployed on the target computing instance using the target file to process the target task information.

[0140] In some optional implementations, the identification information of the target computing instance includes the resource type; the apparatus also includes an update module 406 for:

[0141] The number of tasks corresponding to the target computing instance is updated based on the resource type included in the identifier information of the target computing instance.

[0142] In some alternative implementations, the type of the target computing instance includes a central processing unit type or a graphics processing unit type;

[0143] Update module 406, specifically for:

[0144] When the resource type is a central processing unit, the number of tasks corresponding to the target computing instance is updated according to the first preset increment.

[0145] or,

[0146] When the resource type is a graphics processor, the number of tasks corresponding to the target computing instance is updated according to the second preset increment, wherein the first preset increment is greater than the second preset increment.

[0147] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0148] In this embodiment, the task execution device is presented in the form of a functional unit. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0149] This invention also provides a computer device having the above-described features. Figure 4 The task execution device shown.

[0150] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 5 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 5 Take a processor 10 as an example.

[0151] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware integrated circuit. The aforementioned hardware integrated circuit may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0152] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0153] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0154] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0155] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0156] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0157] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0158] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A task execution method, characterized in that, The method includes: Receive a target request sent by a target device, wherein the target request includes target task information and identification information of a target model for processing the target task information; Obtain the configuration information corresponding to each of the multiple computing instances deployed on the target platform, wherein one computing instance occupies all resources on a resource component included in the target platform; Based on the identification information of the target model and the configuration information corresponding to each computing instance, at least one candidate computing instance is selected from the plurality of computing instances; the number of tasks corresponding to each candidate computing instance, and the identification information and number of deployed models on each candidate computing instance are obtained; based on the identification information of each deployed model on the first candidate computing instance, the task number threshold of each deployed model on the first candidate computing instance is obtained, wherein the first candidate computing instance is any one of the at least one candidate computing instances; the task number thresholds corresponding to all deployed models on the first candidate computing instance are summed to obtain a target threshold; a first ratio between the number of tasks corresponding to the first candidate computing instance and the target threshold is calculated; and a second ratio between the number of deployed models on the first candidate computing instance and the number of models included in the configuration information corresponding to the first candidate computing instance is calculated; the sum of the first ratio and the second ratio is determined, and the difference between one and the sum is determined as the priority; the candidate computing instance with the highest priority is determined as the target computing instance. When it is determined that the target computing instance has at least one deployed model, the identification information corresponding to each deployed model is obtained; Based on the identification information of the target model and the identification information corresponding to each of the deployed models, determine whether a task processing model exists in at least one of the deployed models; When the existence of the task processing model is confirmed, the target task information is input into the task processing model, and the output result is obtained; The output result is fed back to the target device.

2. The method according to claim 1, characterized in that, The configuration information includes at least one type of identification information corresponding to each model; The step of selecting at least one candidate computing instance from a plurality of computing instances based on the identification information of the target model and the configuration information corresponding to each computing instance includes: Determine whether the identification information of the target model is the identification information of any model included in the configuration information corresponding to the first computing instance, wherein the first computing instance is any one of the multiple computing instances; When the identification information of the target model is determined to be the identification information of any model included in the configuration information corresponding to the first computing instance, the first computing instance is determined as the candidate computing instance.

3. The method according to claim 2, characterized in that, The step of determining whether a task processing model exists among at least one of the deployed models based on the identification information of the target model and the identification information corresponding to each of the deployed models includes: Based on the identification information of the target model and the identification information corresponding to each of the deployed models, determine whether at least one of the deployed models includes at least one target model; When it is determined that at least one of the deployed models includes at least one of the target models, the number of tasks corresponding to each target model is obtained; Based on the number of tasks corresponding to each target model and a preset task number threshold corresponding to the identification information of the target model, it is determined whether there is a target model among at least one of the target models that serves as the task processing model.

4. The method according to claim 3, characterized in that, The configuration information also includes a model quantity threshold; when it is determined that at least one of the deployed models does not include the target model, or when at least one of the target models does not have a target model as the task processing model, the method further includes: Determine whether the number of deployed models in the target computing instance is less than the number of models included in the configuration information corresponding to the target computing instance; When it is determined that the number of deployed models in the target computing instance is less than the threshold of the number of models included in the configuration information corresponding to the target computing instance, the target storage path mounted under the target computing instance is obtained; Based on the target storage path and the identification information of the target model, obtain the target file corresponding to the target model; The target model is deployed on the target computing instance using the target file to process the target task information.

5. The method according to any one of claims 1 to 4, characterized in that, The identification information of the target computing instance includes the resource type; The method further includes: The number of tasks corresponding to the target computing instance is updated based on the resource type included in the identifier information of the target computing instance.

6. The method according to claim 5, characterized in that, The target computing instance can be of either central processing unit type or graphics processing unit type; The step of updating the number of tasks corresponding to the target computing instance based on the resource type included in the identifier information of the target computing instance includes: When the resource type is the central processing unit type, the number of tasks corresponding to the target computing instance is updated according to the first preset increment; or, When the resource type is the graphics processor type, the number of tasks corresponding to the target computing instance is updated according to the second preset increment, wherein the first preset increment is greater than the second preset increment.

7. A task execution device, characterized in that, The device includes: A receiving module is used to receive a target request sent by a target device, wherein the target request includes target task information and identification information of a target model for processing the target task information; The acquisition module is used to acquire configuration information corresponding to each of the multiple computing instances deployed on the target platform, wherein one computing instance occupies all resources on a resource component included in the target platform; A selection module is configured to: select at least one candidate computing instance from a plurality of computing instances based on the identification information of the target model and the configuration information corresponding to each computing instance; obtain the number of tasks corresponding to each candidate computing instance, and the identification information and number of deployed models on each candidate computing instance; obtain a task number threshold for each deployed model on the first candidate computing instance based on the identification information of each deployed model on the first candidate computing instance, wherein the first candidate computing instance is any one of at least one candidate computing instance; sum the task number thresholds corresponding to all deployed models on the first candidate computing instance to obtain a target threshold; calculate a first ratio between the number of tasks corresponding to the first candidate computing instance and the target threshold; and calculate a second ratio between the number of tasks corresponding to the first candidate computing instance and the number of models included in the configuration information corresponding to the first candidate computing instance; determine the sum of the first ratio and the second ratio, and determine the difference between one and the sum as the priority; and determine the candidate computing instance with the highest priority as the target computing instance. The acquisition module is further configured to acquire identification information corresponding to each of the deployed models when it is determined that the target computing instance has at least one deployed model; The determining module is used to determine whether a task processing model exists in at least one of the deployed models based on the identification information of the target model and the identification information corresponding to each of the deployed models. The acquisition module is further configured to, when it is determined that the task processing model exists, input the target task information into the task processing model and obtain the output result; The feedback module is used to send the output results back to the target device.

8. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the task execution method of any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the task execution method according to any one of claims 1 to 6.

Images