Task execution method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a task execution method and device, computer equipment and a storage medium, and the method comprises the steps that a target request sent by target equipment is received, and the target request comprises target task information and identification information of a target model; and obtaining configuration information corresponding to a plurality of calculation instances deployed by the target platform. And selecting a target calculation instance from the plurality of calculation instances according to the identification information of the target model and the configuration information corresponding to each calculation instance. When it is determined that the target calculation instance has the deployed models, obtaining identification information corresponding to each deployed model; and determining whether a task processing model exists in the deployed models or not according to the identification information of the target model and the identification information corresponding to each deployed model. And when the task processing model exists, inputting the target task information into the task processing model, obtaining an output result, and feeding back the output result to the target equipment. The problem of resource waste can be avoided.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a task execution method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of artificial intelligence technology, artificial intelligence (AI) models are being used more and more widely. The deployment and operation of AI models have very high requirements for computing resources.

[0003] In platforms that provide AI services, an AI model is usually deployed on a resource component to ensure stable operation. However, a single large language model may only occupy part of the resources of a resource component, resulting in a waste of resources. Summary of the invention

[0004] In view of this, the present invention provides a task execution method, apparatus, computer equipment, storage medium and program product to solve the problem of resource waste in related technologies.

[0005] In a first aspect, the present invention provides a task execution method, comprising:

[0006] receiving a target request sent by a target device, wherein the target request includes target task information and identification information of a target model for processing the target task information;

[0007] Obtain configuration information corresponding to each of a plurality of computing instances deployed on the target platform, wherein one computing instance occupies all resources on a resource component included in the target platform;

[0008] Selecting a target computing instance from multiple computing instances according to identification information of the target model and configuration information corresponding to each computing instance;

[0009] When it is determined that the target computing instance has at least one deployed model, obtaining identification information corresponding to each deployed model;

[0010] Determining whether there is a task processing model in at least one of the deployed models according to the identification information of the target model and the identification information corresponding to each of the deployed models;

[0011] When it is determined that there is a task processing model, the target task information is input into the task processing model to obtain an output result;

[0012] Feed the output results back to the target device.

[0013] The task execution method provided by the present invention has the following advantages:

[0014] After receiving the target request, this solution selects the target computing instance according to the identification information of the target model in the target request and the configuration information corresponding to each computing instance. One or more models can be deployed on the target computing instance. Therefore, this solution can determine whether there is a task processing model in the deployed models on the target computing instance according to the identification information of the target model. If so, the target task information can be input into the task processing model to obtain the output result and feed it back to the target device. In this way, each computing instance can occupy all the resources of a resource component, and one or more models can be deployed on a computing instance. When assigning tasks, tasks are scheduled according to the actual deployment of each computing instance, so that the resources on each resource component can be fully utilized to avoid resource waste.

[0015] In an optional implementation, selecting a target computing instance from multiple computing instances according to identification information of the target model and configuration information corresponding to each computing instance includes:

[0016] Selecting at least one candidate computing instance from the plurality of computing instances according to the identification information of the target model and the configuration information corresponding to each computing instance;

[0017] Get the number of tasks corresponding to each candidate computing instance;

[0018] A target computing instance is selected from at least one candidate computing instance according to the number of tasks corresponding to each candidate computing instance.

[0019] Specifically, first, candidate computing instances are selected based on the configuration information of the computing instance and the identification information of the target model, and then the number of tasks corresponding to each candidate computing instance is obtained. Furthermore, based on the number of tasks corresponding to each candidate computing instance, a target computing instance is selected from at least one candidate computing instance. This can balance the number of tasks on each computing instance and ensure load balancing, which can avoid the problem of excessive load on a certain computing instance, resulting in low task processing efficiency.

[0020] In an optional implementation, the configuration information includes identification information corresponding to at least one model;

[0021] At least one candidate computing instance is selected from the multiple computing instances according to the identification information of the target model and the configuration information corresponding to each computing instance, including:

[0022] Determining whether the identification information of the target model is identification information corresponding to any one of the models included in the configuration information corresponding to the first computing instance, wherein the first computing instance is any one of the multiple computing instances;

[0023] When it is determined that the identification information of the target model is identification information corresponding to any one of the models included in the configuration information corresponding to the first computing instance, the first computing instance is determined as a candidate computing instance.

[0024] Specifically, by comparing the identification information of the target model with the identification information included in the configuration information of each computing instance, it can be determined whether each computing instance can deploy the target model. If so, the computing instance can be determined as a candidate computing instance for further screening.

[0025] In an optional implementation, determining whether there is a task processing model in at least one deployed model according to identification information of the target model and identification information corresponding to each deployed model includes:

[0026] Determining whether at least one deployed model includes at least one target model according to identification information of the target model and identification information corresponding to each deployed model;

[0027] When it is determined that at least one deployed model includes at least one target model, obtaining the number of tasks corresponding to each target model;

[0028] According to the number of tasks corresponding to each target model and a preset task number threshold corresponding to identification information of the target model, it is determined whether there is a target model serving as a task processing model in at least one target model.

[0029] Specifically, first determine whether the target model is included in the deployed models on the target computing instance. If so, determine whether there is a task processing model in the deployed models that can be used to process the target task model based on the number of tasks being processed by each model and the preset task number threshold. In this way, it can be ensured that the number of tasks processed by each target model meets the regulations, avoiding the problem of a large number of tasks causing blockage or failure of the target model.

[0030] In an optional implementation, the configuration information further includes a model quantity threshold; when it is determined that at least one deployed model does not include a target model, or at least one target model does not include a target model that serves as a task processing model, the method further includes:

[0031] Determine whether the number of deployed models in the target computing instance is less than a model number threshold included in the configuration information corresponding to the target computing instance;

[0032] When it is determined that the number of deployed models in the target computing instance is less than the model number threshold included in the configuration information corresponding to the target computing instance, obtaining the target storage path mounted under the target computing instance;

[0033] According to the target storage path and the identification information of the target model, obtaining the target file corresponding to the target model;

[0034] The target file is used to deploy the target model on the target computing instance to process the target task information.

[0035] Specifically, when it is determined that the target model is not deployed on the target computing instance, or when there is no task processing model in the deployed model that can process the target task information, first determine whether the model can still be deployed on the target computing instance. If so, obtain the target file based on the target storage path mounted under the target computing instance and the identification information of the target model, and deploy the target model on the target computing instance based on the target file. In this way, when the resources of the target computing instance are sufficient, the target task information can be processed efficiently without waiting for other tasks to be completed before processing the target task information. This can not only ensure full utilization of resources but also improve task processing efficiency.

[0036] In an optional implementation, the identification information of the target computing instance includes a resource type; and the method further includes:

[0037] The number of tasks corresponding to the target computing instance is updated according to the resource type included in the identification information of the target computing instance.

[0038] Specifically, since the resource sizes that different resource types can provide vary greatly, when the number of tasks of a computing instance is increased, the number of tasks of the target computing instance can be updated according to the different resource types. In this way, there is no need to monitor the remaining resources in real time and determine which computing instance to assign new tasks to based on the remaining resources. Instead, it can be determined directly based on the number of tasks, which can save resources.

[0039] In an optional implementation, the type of the target computing instance includes a central processing unit type or a graphics processing unit type;

[0040] According to the resource type included in the identification information of the target computing instance, the number of tasks corresponding to the target computing instance is updated, including:

[0041] When the resource type is a central processing unit type, the number of tasks corresponding to the target computing instance is updated according to a first preset incremental number;

[0042] or,

[0043] When the resource type is a graphics processor type, the number of tasks corresponding to the target computing instance is updated according to a second preset incremental number, wherein the first preset incremental number is greater than the second preset incremental number.

[0044] Specifically, compared with CPU type computing instances, image processor type computing instances have more sufficient resources. Therefore, when updating the task quantity after assigning tasks, if the assigned target computing instance is a CPU type computing instance, a larger first preset incremental quantity can be used for updating, which can reduce the probability of being assigned to a CPU type computing instance and avoid affecting other CPU services.

[0045] In a second aspect, the present invention provides a task execution device, comprising:

[0046] A receiving module, used for receiving a target request sent by a target device, wherein the target request includes target task information and identification information of a target model for processing the target task information;

[0047] An acquisition module, used to acquire configuration information corresponding to each of a plurality of computing instances deployed on a target platform, wherein a computing instance occupies all resources on a resource component included in the target platform;

[0048] A selection module, used to select a target computing instance from multiple computing instances according to identification information of a target model and configuration information corresponding to each computing instance;

[0049] The acquisition module is further used to acquire identification information corresponding to each deployed model when it is determined that the target computing instance has at least one deployed model;

[0050] A determination module, configured to determine whether a task processing model exists in at least one deployed model according to identification information of the target model and identification information corresponding to each deployed model;

[0051] The acquisition module is further used to input the target task information into the task processing model and obtain the output result when it is determined that the task processing model exists;

[0052] The feedback module is used to feed back the output results to the target device.

[0053] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the task execution method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0054] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the task execution method of the first aspect or any corresponding embodiment thereof.

[0055] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the task execution method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 is a schematic diagram of a system architecture according to an embodiment of the present invention;

[0058] Figure 2 is a schematic diagram of a deployment architecture of a computing instance on a target platform according to an embodiment of the present invention;

[0059] Figure 3 is a flowchart of a task execution method according to an embodiment of the present invention;

[0060] Figure 4 is a structural block diagram of a task execution device according to an embodiment of the present invention;

[0061] Figure 5 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0063] The present invention can be implemented by a system consisting of a target platform and a target device, such as Figure 1 As shown. Among them, the target platform can be used to deploy various artificial intelligence (AI) models to provide various AI services. For example, the AI ​​model can be a large language model (LLM), which can provide customer service, knowledge retrieval, question and answer services. The target device can be a client device, such as a computer, mobile phone and other terminals, and the user can enter relevant information in the relevant interface of the AI ​​service of the client device, for example, enter a question in the question and answer service.

[0064] like Figure 2 As shown, the target platform can deploy a scheduling component and multiple computing instances (Computing Instance), and the computing instance is an independent computing environment for deploying applications (for example, AI models). For example, computing instances can be virtual machines and containers. The target platform may include multiple resource components, and the resource components may be graphics processing units (GPUs), central processing units (CPUs), etc. Among them, the scheduling component can be used to receive requests from the target device and schedule the tasks corresponding to the requests on each computing instance. A computing instance can occupy all resources on a resource component included in the target platform. For example, in Figure 2 Computing instances 1 to 3 respectively occupy all resources on a graphics processor, and computing instance 4 occupies all resources on a central processing unit.

[0065] The target platform may store a list of computing instances, which may include identification information of each computing instance and configuration information corresponding to the identification information of each computing instance. In addition, for each computing instance, a queue may be created in advance for each computing instance to temporarily store task information in the request. The identification information of the computing instance may include the resource type and number of the corresponding resource component, and the resource type may be a central processing unit type or an image processing unit type. For example, the identification information of the computing instance may be GPU-0, GPU-1, GPU-2, GPU-3, CPU-0.

[0066] An embodiment of the present invention provides a task execution method, which deploys one or more models on each computing instance and schedules tasks according to the actual deployment status of each computing instance when assigning tasks, so as to fully utilize the resources on each resource component and avoid resource waste.

[0067] According to an embodiment of the present invention, a task execution method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0068] In this embodiment, a task execution method is provided, which can be executed by the above-mentioned target platform. Figure 3 is a flowchart of a task execution method according to an embodiment of the present invention. Figure 3 As shown, the process includes the following steps:

[0069] Step S301, receiving a target request sent by a target device.

[0070] The target request may include target task information and identification information of a target model for processing the target task information. The type of the target computing instance may include a central processing unit type or a graphics processing unit type. The identification information of the target model may include a name and parameter quantity of the target model, for example, "Qwen2-1.5B", "Qwen2" is the name of the target model, and "1.5B" is the parameter quantity of the target model.

[0071] Specifically, the user can select the identification information of the target model on the target device and input the target task information. For example, the identification information of the selected target model can be used to indicate the question-answering model, and the input target task information is a question. The target device can generate a target request according to the user's operation and send it to the target platform. After receiving the target request, the target platform can parse the target request and obtain the identification information and target task information of the target model.

[0072] Step S302: Obtain configuration information corresponding to each of the multiple computing instances deployed on the target platform.

[0073] Among them, the configuration information corresponding to the computing instance may include the identification information corresponding to at least one model that can be deployed on the computing instance, or may also include a model quantity threshold (corresponding to the maximum number of models that can be deployed on the resource component). Accordingly, the identification information and configuration information of each computing instance in the computing instance list can be arranged from most to least according to the model quantity threshold. The configuration information corresponding to each computing instance can be determined by the technician based on the data accuracy (for example, float32, BF16, Int4) and parameter quantity of the model, as well as the resource size provided by the resource component corresponding to the computing instance. For example, the configuration information corresponding to the computing instances can be: (1) GPU-0: "Qwen2-1.5B" and "maximum 6 models can be deployed simultaneously (which can be the number calculated based on the largest model)"; (2) GPU-1: "Qwen2-1.5B, Llama2-7B" and "maximum 3 models can be deployed simultaneously".

[0074] Step S303 : selecting a target computing instance from multiple computing instances according to the identification information of the target model and the configuration information corresponding to each computing instance.

[0075] Specifically, step S303 may include the following steps:

[0076] Step 1: Select at least one candidate computing instance from multiple computing instances based on the identification information of the target model and the configuration information corresponding to each computing instance.

[0077] Specifically, the target platform can determine whether the identification information of the target model is the identification information corresponding to any model included in the configuration information corresponding to the first computing instance, wherein the first computing instance is any computing instance among the multiple computing instances. When it is determined that the identification information of the target model is the identification information corresponding to any model included in the configuration information corresponding to the first computing instance, the first computing instance can be determined as a candidate computing instance. In this way, after performing the above judgment for each computing instance, at least one candidate computing instance can be determined.

[0078] Step 2: Obtain the number of tasks corresponding to each candidate computing instance.

[0079] Step three: Select a target computing instance from at least one candidate computing instance according to the number of tasks corresponding to each candidate computing instance.

[0080] Specifically, each computing instance can correspond to a task counting parameter. Each time a task is assigned to a computing instance, the value of the task counting parameter corresponding to the computing instance will be updated. Therefore, before each task is assigned, the target platform can first obtain the value of the task counting parameter corresponding to each candidate computing instance, and select the candidate computing instance with the smallest data as the target computing instance, and add the target task information to the queue corresponding to the target computing instance to complete the allocation operation of the target task information.

[0081] In some optional implementations, in addition to obtaining the number of tasks corresponding to each candidate computing instance, the target platform can also obtain the identification information corresponding to the deployed models on each candidate computing instance, as well as the number of deployed models on each candidate computing instance. Taking the first candidate computing instance (any candidate instance of at least one candidate instance) as an example, the threshold value of the number of tasks corresponding to each deployed model can be obtained based on the identification information corresponding to each deployed model on the first candidate computing instance. Then, the priority of the first candidate computing instance can be determined based on the number of tasks corresponding to the first candidate computing instance, the number of deployed models, the model number threshold included in the configuration information corresponding to the first candidate computing instance, and the task number threshold corresponding to each deployed model on the first candidate computing instance. Finally, the target platform can select the candidate computing instance with the highest priority as the target computing instance.

[0082] For example, according to the number of tasks corresponding to the first candidate computing instance, the number of deployed models, the model number threshold included in the configuration information corresponding to the first candidate computing instance, and the task number threshold corresponding to each deployed model on the first candidate computing instance, the specific process of determining the priority of the first candidate computing instance may be:

[0083] The task quantity thresholds corresponding to all deployed models on the first candidate computing instance are summed to obtain a target threshold, and then a first ratio of the task quantity corresponding to the first candidate computing instance to the target threshold is calculated. In addition, a second ratio of the number of deployed models to the model quantity threshold included in the configuration information corresponding to the first candidate computing instance is calculated. Finally, the sum of the first ratio and the second ratio is determined, and the difference between one and the sum is determined as the priority.

[0084] The larger the first ratio is, the more tasks the deployed model processes on average. If a new task is assigned, it is very likely that a new model needs to be deployed, and model deployment takes time. The larger the second ratio is, the greater the load on the first candidate computing instance is, and the smaller the resource margin of the deployable model is. By summing the first ratio and the second ratio, the efficiency of the first candidate computing instance in processing the target task information can be determined, that is, the larger the sum is, the lower the efficiency is. Therefore, by selecting the target computing instance in the above manner, a computing instance with a higher efficiency in processing task information can be selected with a higher probability, thereby improving the efficiency of task information processing.

[0085] In some optional implementations, the target platform may also first obtain the number of tasks corresponding to each computing instance, and determine whether each computing instance supports the deployment of the target model based on the identification information of the target model and the model deployment list configuration information corresponding to each computing instance. Then, based on the number of tasks corresponding to each computing instance and the determination result of whether each computing instance supports the deployment of the target model, the target computing instance may be selected (specifically, the computing instance with the least number of tasks and supporting the target model is selected as the target computing instance).

[0086] Step S304: when it is determined that the target computing instance has at least one deployed model, obtain identification information corresponding to each deployed model.

[0087] Specifically, the target platform can check whether one or more models have been deployed on the target computing instance. When it is determined that the model has been deployed on the target computing instance, identification information corresponding to each deployed model can be obtained to determine whether the target model exists.

[0088] Alternatively, when it is determined that the model is not deployed on the target computing instance, the target storage path mounted under the target computing instance can be obtained. Then, according to the target storage path and the identification information of the target model, the target file corresponding to the target model is obtained from the multiple model files stored under the target storage path. The target file is further used to deploy the target model on the target computing instance. After the deployment of the target model is completed, the target task information is input into the newly deployed target model on the target computing instance, and the output result corresponding to the target task information is obtained and fed back to the target device.

[0089] Step S305 , determining whether there is a task processing model in at least one deployed model according to the identification information of the target model and the identification information corresponding to each deployed model.

[0090] Specifically, step S305 may include the following steps:

[0091] Step 1: Determine whether at least one deployed model includes at least one target model according to the identification information of the target model and the identification information corresponding to each deployed model.

[0092] Step 2: When it is determined that at least one deployed model includes at least one target model, the number of tasks corresponding to each target model is obtained.

[0093] Step three: determine whether there is a target model serving as a task processing model in at least one target model according to the number of tasks corresponding to each target model and a preset task number threshold corresponding to the identification information of the target model.

[0094] Specifically, when it is determined that the number of tasks storing at least one target model is less than a preset task number threshold, the target model with the smallest number of tasks can be determined as the task processing model, that is, it is determined that there is a target model that can be used as a task processing model. When it is determined that the number of tasks of all target models is equal to the preset task number threshold, it is determined that there is no target model that can be used as a task processing model.

[0095] Step 4: When it is determined that at least one deployed model does not include the target model, determine whether the number of deployed models in the target computing instance is less than a model number threshold included in the configuration information corresponding to the target computing instance.

[0096] Step five: when it is determined that the number of deployed models in the target computing instance is less than the model number threshold included in the configuration information corresponding to the target computing instance, obtain the target storage path mounted under the target computing instance.

[0097] Step six, obtaining the target file corresponding to the target model according to the target storage path and the identification information of the target model.

[0098] Step 7: Use the target file to deploy the target model on the target computing instance.

[0099] Step 8: When it is determined that the number of deployed models in the target computing instance is equal to the model number threshold included in the configuration information corresponding to the target computing instance, the processing can be suspended, and after other models have completed processing tasks, other models that have completed processing tasks are closed, and then the target storage path mounted under the target computing instance is obtained. According to the target storage path and the identification information of the target model, the target file corresponding to the target model is obtained, and the target model is deployed on the target computing instance.

[0100] Step nine: input the target task information into the newly deployed target model on the target computing instance to obtain the output result corresponding to the target task information.

[0101] Step S306: When it is determined that a task processing model exists, the target task information is input into the task processing model to obtain an output result.

[0102] Specifically, when it is determined that a task processing model exists, the target task information may be input into the task processing model to obtain an output result.

[0103] In some optional implementations, when it is determined that there is no task processing model, the target platform can execute the specific processing of steps five to seven in step S305, and input the target task information into the newly deployed target model on the target computing instance to obtain the output result.

[0104] In some optional implementations, after step S303 and step S306, the target platform may update the number of tasks corresponding to the target computing instance according to the resource type included in the identification information of the target computing instance.

[0105] Specifically, after step S303, when it is determined that the resource type is a CPU type, the number of tasks corresponding to the target computing instance is updated according to a first preset incrementing number. Alternatively, when the resource type is a GPU type, the number of tasks corresponding to the target computing instance is updated according to a second preset incrementing number, wherein the first preset incrementing number is greater than the second preset incrementing number. After step S306, when it is determined that the resource type is a CPU type, the number of tasks corresponding to the target computing instance is updated according to a first preset decrementing number. Alternatively, when the resource type is a GPU type, the number of tasks corresponding to the target computing instance is updated according to a second preset decrementing number, wherein the first preset decrementing number is greater than the second preset decrementing number.

[0106] For example, the code for selecting a target compute instance and updating the number of tasks corresponding to the target compute instance can be as follows:

[0107]

[0108]

[0109] In the above code, the first preset increment number and the first preset decrement number are both "10", while the second preset increment number and the second preset decrement number are both "1".

[0110] Step S307: Feedback the output result to the target device.

[0111] Specifically, after obtaining the output result, the target platform may send the output result to the target device.

[0112] The task execution method provided in this embodiment selects a target computing instance according to the identification information of the target model in the target request and the configuration information corresponding to each computing instance after receiving the target request. One or more models can be deployed on the target computing instance. Therefore, this solution can determine whether there is a task processing model in the deployed model on the target computing instance according to the identification information of the target model. If so, the target task information can be input into the task processing model to obtain the output result and feed it back to the target device. In this way, each computing instance can occupy all the resources of a resource component, and one or more models can be deployed on a computing instance. When assigning tasks, tasks are scheduled according to the actual deployment of each computing instance, so that the resources on each resource component can be fully utilized to avoid the problem of resource waste.

[0113] In this embodiment, a task execution device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0114] This embodiment provides a task execution device, such as Figure 4 As shown, including:

[0115] The receiving module 401 is used to receive a target request sent by a target device, wherein the target request includes target task information and identification information of a target model for processing the target task information;

[0116] The acquisition module 402 is used to acquire configuration information corresponding to each of the multiple computing instances deployed on the target platform, wherein one computing instance occupies all resources on a resource component included in the target platform;

[0117] A selection module 403, configured to select a target computing instance from multiple computing instances according to identification information of the target model and configuration information corresponding to each computing instance;

[0118] The acquisition module 402 is further configured to acquire identification information corresponding to each deployed model when it is determined that the target computing instance has at least one deployed model;

[0119] A determination module 404 is used to determine whether there is a task processing model in at least one deployed model according to the identification information of the target model and the identification information corresponding to each deployed model;

[0120] The acquisition module 402 is further used to input the target task information into the task processing model and obtain the output result when it is determined that there is a task processing model;

[0121] The feedback module 405 is used to feed back the output result to the target device.

[0122] In some optional implementations, the selection module 403 is specifically configured to:

[0123] Selecting at least one candidate computing instance from the plurality of computing instances according to the identification information of the target model and the configuration information corresponding to each computing instance;

[0124] Get the number of tasks corresponding to each candidate computing instance;

[0125] A target computing instance is selected from at least one candidate computing instance according to the number of tasks corresponding to each candidate computing instance.

[0126] In some optional implementations, the configuration information includes identification information corresponding to at least one model;

[0127] The selection module 403 is specifically used for:

[0128] Determining whether the identification information of the target model is identification information corresponding to any one of the models included in the configuration information corresponding to the first computing instance, wherein the first computing instance is any one of the multiple computing instances;

[0129] When it is determined that the identification information of the target model is identification information corresponding to any one of the models included in the configuration information corresponding to the first computing instance, the first computing instance is determined as a candidate computing instance.

[0130] In some optional implementations, the determination module 404 is specifically configured to:

[0131] Determining whether at least one deployed model includes at least one target model according to identification information of the target model and identification information corresponding to each deployed model;

[0132] When it is determined that at least one deployed model includes at least one target model, obtaining the number of tasks corresponding to each target model;

[0133] According to the number of tasks corresponding to each target model and a preset task number threshold corresponding to identification information of the target model, it is determined whether there is a target model serving as a task processing model in at least one target model.

[0134] In some optional implementations, the configuration information also includes a model quantity threshold;

[0135] The determination module 404 is further configured to:

[0136] When it is determined that at least one of the deployed models does not include the target model, or at least one of the target models does not include the target model as the task processing model, determining whether the number of deployed models in the target computing instance is less than a model number threshold included in the configuration information corresponding to the target computing instance;

[0137] When it is determined that the number of deployed models in the target computing instance is less than the model number threshold included in the configuration information corresponding to the target computing instance, obtaining the target storage path mounted under the target computing instance;

[0138] According to the target storage path and the identification information of the target model, obtaining the target file corresponding to the target model;

[0139] The target file is used to deploy the target model on the target computing instance to process the target task information.

[0140] In some optional implementations, the identification information of the target computing instance includes a resource type; the apparatus further includes an updating module 406, which is used to:

[0141] The number of tasks corresponding to the target computing instance is updated according to the resource type included in the identification information of the target computing instance.

[0142] In some optional embodiments, the type of the target computing instance includes a central processing unit type or a graphics processing unit type;

[0143] The update module 406 is specifically used for:

[0144] When the resource type is a central processing unit type, the number of tasks corresponding to the target computing instance is updated according to a first preset incremental number;

[0145] or,

[0146] When the resource type is a graphics processor type, the number of tasks corresponding to the target computing instance is updated according to a second preset incremental number, wherein the first preset incremental number is greater than the second preset incremental number.

[0147] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0148] The task execution device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0149] The embodiment of the present invention also provides a computer device having the above Figure 4 The task execution device shown.

[0150] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 5 A processor 10 is taken as an example.

[0151] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware integrated circuit. The hardware integrated circuit may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable logic gate array, a general purpose array logic or any combination thereof.

[0152] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0153] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0154] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0155] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0156] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0157] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0158] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A task execution method, characterized in that: The method comprises: Receiving a target request sent by a target device, wherein the target request includes target task information and identification information of a target model for processing the target task information; Obtain configuration information corresponding to each of the plurality of computing instances deployed on the target platform, wherein one computing instance occupies all resources on a resource component included in the target platform; Selecting a target computing instance from the plurality of computing instances according to the identification information of the target model and the configuration information corresponding to each of the computing instances; When it is determined that the target computing instance has at least one deployed model, obtaining identification information corresponding to each of the deployed models; Determining whether there is a task processing model in at least one of the deployed models according to the identification information of the target model and the identification information corresponding to each of the deployed models; When it is determined that the task processing model exists, inputting the target task information into the task processing model to obtain an output result; The output result is fed back to the target device.

2. The method according to claim 1, characterized in that The selecting a target computing instance from the plurality of computing instances according to the identification information of the target model and the configuration information corresponding to each of the computing instances includes: Selecting at least one candidate computing instance from the plurality of computing instances according to the identification information of the target model and the configuration information corresponding to each of the computing instances; Obtaining the number of tasks corresponding to each of the candidate computing instances; The target computing instance is selected from at least one of the candidate computing instances according to the number of tasks corresponding to each of the candidate computing instances.

3. The method according to claim 2, characterized in that The configuration information includes identification information corresponding to at least one model; The selecting at least one candidate computing instance from the plurality of computing instances according to the identification information of the target model and the configuration information corresponding to each of the computing instances comprises: Determining whether the identification information of the target model is identification information corresponding to any one of the models included in the configuration information corresponding to the first computing instance, wherein the first computing instance is any one of the plurality of computing instances; When it is determined that the identification information of the target model is identification information corresponding to any one of the models included in the configuration information corresponding to the first computing instance, the first computing instance is determined as the candidate computing instance.

4. The method according to claim 3, characterized in that The determining, according to the identification information of the target model and the identification information corresponding to each of the deployed models, whether there is a task processing model in at least one of the deployed models comprises: Determining, according to the identification information of the target model and the identification information corresponding to each of the deployed models, whether at least one of the deployed models includes at least one of the target models; When it is determined that at least one of the deployed models includes at least one of the target models, obtaining the number of tasks corresponding to each of the target models; According to the number of tasks corresponding to each of the target models and a preset task number threshold corresponding to the identification information of the target model, it is determined whether there is a target model serving as the task processing model in at least one of the target models.

5. The method according to claim 4, characterized in that The configuration information also includes a model quantity threshold; when it is determined that at least one of the deployed models does not include the target model, or at least one of the target models does not include a target model that serves as the task processing model, the method further includes: Determine whether the number of deployed models in the target computing instance is less than a model number threshold included in the configuration information corresponding to the target computing instance; When it is determined that the number of deployed models in the target computing instance is less than the model number threshold included in the configuration information corresponding to the target computing instance, obtaining a target storage path mounted under the target computing instance; Acquire a target file corresponding to the target model according to the target storage path and the identification information of the target model; The target file is used to deploy the target model on the target computing instance to process the target task information.

6. The method according to any one of claims 2 to 5, characterized in that: The identification information of the target computing instance includes a resource type; The method further comprises: The number of tasks corresponding to the target computing instance is updated according to the resource type included in the identification information of the target computing instance.

7. The method according to claim 6, characterized in that The type of the target computing instance includes a central processing unit type or a graphics processing unit type; The updating of the number of tasks corresponding to the target computing instance according to the resource type included in the identification information of the target computing instance includes: When the resource type is the central processing unit type, updating the number of tasks corresponding to the target computing instance according to a first preset incremental number; or, When the resource type is the graphics processor type, the number of tasks corresponding to the target computing instance is updated according to a second preset incremental number, wherein the first preset incremental number is greater than the second preset incremental number.

8. A task execution device, characterized in that: The device comprises: A receiving module, configured to receive a target request sent by a target device, wherein the target request includes target task information and identification information of a target model for processing the target task information; An acquisition module, used to acquire configuration information corresponding to each of the plurality of computing instances deployed on the target platform, wherein one computing instance occupies all resources on a resource component included in the target platform; A selection module, configured to select a target computing instance from the plurality of computing instances according to the identification information of the target model and the configuration information corresponding to each of the computing instances; The acquisition module is further configured to, when it is determined that the target computing instance has at least one deployed model, acquire identification information corresponding to each of the deployed models; A determination module, configured to determine whether a task processing model exists in at least one of the deployed models according to the identification information of the target model and the identification information corresponding to each of the deployed models; The acquisition module is further configured to input the target task information into the task processing model to obtain an output result when it is determined that the task processing model exists; A feedback module is used to feed back the output result to the target device.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the task execution method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the task execution method according to any one of claims 1 to 7.

Images