Task execution method, device, electronic device and computer storage medium

Abstract

The present disclosure provides a task execution method, apparatus, electronic device, and computer storage medium, relating to the fields of computer technology, particularly cloud computing, big data, computer vision, and artificial intelligence technologies such as deep learning. A specific implementation scheme comprises: obtaining information about available resources; determining a first parallel number of subtasks for executing a target task based on the available resource information; and adjusting the subtasks based on the first parallel number. Embodiments of the present disclosure can improve resource utilization and task execution efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to artificial intelligence technologies such as cloud computing, big data, computer vision, and deep learning. Background Art

[0002] With the advancement of computer technology, in scenarios such as deep learning, machine learning, image processing, and data computation, computer tasks can be split into multiple subtasks for parallel execution. However, the number of available resources during task execution is limited. How to prioritize resources to execute tasks more quickly and efficiently is a problem that requires continuous improvement. Summary of the Invention

[0003] The present disclosure provides a task execution method, device, electronic device, and computer storage medium.

[0004] According to one aspect of the present disclosure, a task execution method is provided, comprising:

[0005] Obtain information about available resources;

[0006] Determine a first parallel quantity for executing subtasks of the target task based on information about available resources;

[0007] Adjust the subtasks based on the first parallel quantity.

[0008] According to another aspect of the present disclosure, there is provided a task execution device, comprising:

[0009] Resource information acquisition module, used to obtain information about available resources;

[0010] A first parallel quantity determination module is used to determine a first parallel quantity of subtasks for executing a target task according to information about available resources;

[0011] The adjustment module is used to adjust the subtask according to the first parallel quantity.

[0012] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method in any embodiment of the present disclosure.

[0016] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to cause a computer to execute the method in any embodiment of the present disclosure.

[0017] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program / instruction, which implements the method in any embodiment of the present disclosure when the computer program / instruction is executed by a processor.

[0018] According to the technology disclosed in the present invention, it is possible to determine information about available resources, and based on the information about available resources, determine the number of subtasks of the target task that can be executed in parallel, that is, the first parallel number, and adjust the subtasks based on the determined first parallel number. Thus, in various computer task execution scenarios, it is possible to implement elastic scaling adjustment of the parallel number of subtasks based on available resources, thereby maximizing the effective utilization of available resources, and also making use of available resources to adjust the execution speed of subtasks to the maximum.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention.

[0021] Figure 1 is a schematic diagram of a task execution method according to an embodiment of the present disclosure;

[0022] Figure 2 is a schematic diagram of a task execution method according to another embodiment of the present disclosure;

[0023] Figure 3 is a schematic diagram of a task execution method according to an example of the present disclosure;

[0024] Figure 4 is a schematic diagram of a task execution framework according to an example of the present disclosure;

[0025] Figure 5 is a schematic diagram of a task execution device according to an embodiment of the present disclosure;

[0026] Figure 6 is a schematic diagram of a task execution device according to another embodiment of the present disclosure;

[0027] Figure 7 is a schematic diagram of a task execution device according to another embodiment of the present disclosure;

[0028] Figure 8 is a schematic diagram of a task execution device according to another embodiment of the present disclosure;

[0029] Figure 9 is a schematic diagram of a task execution device according to another embodiment of the present disclosure;

[0030] Figure 10 is a schematic diagram of a task execution device according to another embodiment of the present disclosure;

[0031] Figure 11 is a schematic diagram of a task execution device according to another embodiment of the present disclosure;

[0032] Figure 12 It is a block diagram of an electronic device used to implement the task execution method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0034] According to an embodiment of the present disclosure, a task execution method is provided. Figure 1 This is a flowchart of a task execution method according to an embodiment of the present disclosure. The method can be applied to a task execution device. For example, the device can be deployed on a terminal or server or other processing device to execute the steps of obtaining available resource information, determining the number of parallel tasks, etc. The terminal can be a user equipment (UE), a mobile device, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the method can also be implemented by a processor calling computer-readable instructions stored in a memory. For example Figure 1 As shown, the task execution method includes:

[0035] Step S11: Obtain information about available resources;

[0036] Step S12: determining a first parallel number of subtasks for executing the target task based on information about available resources;

[0037] Step S13: adjusting the subtasks according to the first parallel quantity.

[0038] In this embodiment, the available resource information can be used to indicate the amount of available resources, i.e., the amount of available resources. Specifically, it can indicate the number of subtasks that the available resources can support. For example, the current available resources are sufficient to share 1-10 subtasks for execution.

[0039] The available resources may be resources used to run the subtasks, for example, at least one of software resources and hardware resources such as memory, cache, CPU (Central Processing Unit), disk, and hard disk.

[0040] In this embodiment, the target task may be any computer processing task or computer computing task, such as at least one of image processing, data computing, data transmission, data acquisition, data storage, model training, machine learning, deep learning, etc.

[0041] The target task can be one of the tasks currently running in the system, or one of the tasks that will be added to the system. It can also be the only task in the current system that is running, or a task that will be added to the system.

[0042] A subtask for executing a target task can be at least one task run to achieve the target task. The content of the subtask can be equivalent to the target task itself. For example, in one specific implementation, the target task can be a machine learning task. To achieve the purpose of machine learning, at least one subtask can be initiated simultaneously. Each subtask is used to train at least one machine learning model. After training is completed, the model with the best training effect is selected from the at least one machine learning model corresponding to the at least one subtask as the model obtained by executing the target task.

[0043] In another specific implementation, subtasks can also be multiple parts of the target task. For example, the target task includes multiple steps in no particular order, each of which can constitute a subtask. After the multiple steps in no particular order are completed, the final result is obtained based on the execution results of each subtask, which serves as the execution result of the target task.

[0044] The first parallel number of subtasks executing the target task may be the number of subtasks that can be supported by existing available resources to run simultaneously.

[0045] Adjusting the subtask according to the first parallel quantity may include adjusting the parallel quantity of the subtask so that the parallel quantity of the subtask matches the first parallel quantity.

[0046] In this embodiment, the subtask of the target task may be at least one subtask of the target task. When there are multiple subtasks of the target task, the multiple subtasks may belong to the same category or different categories. When the subtasks of the target task do not belong to the same category, the first parallel quantity may include the number of subtasks that can be run for each subcategory.

[0047] In one possible implementation, each subtask of a target task can be run in a specific medium, such as a POD. Multiple PODs can be launched for each target task, each of which can run at least one container, which can then launch a process for executing the subtask.

[0048] In the disclosed embodiments, a POD (as in a whale pod or a pea pod) can be a holding module or a running module for a group of (one or more) containers; these containers share storage, network, and declarations of how to run these containers. The contents in a POD are always colocated and scheduled together, running in a shared context. What a POD models is an application-specific "logical host" that contains one or more application containers that are relatively tightly coupled together. In a non-cloud environment, applications running on the same physical or virtual machine are similar to cloud applications running on the same logical host.

[0049] The embodiments of the present disclosure can be applied to a variety of scenarios that require the execution of computer-related processing tasks, such as model training scenarios such as machine learning and deep learning.

[0050] In machine learning scenarios, traditional machine learning training tasks typically require extensive manual intervention, primarily in areas such as algorithm selection and parameter adjustment. Automatic machine learning (AutoML) can be seen as building a series of advanced control systems to operate machine learning models, enabling them to automatically learn appropriate parameters and achieve good performance without manual intervention, resulting in complete automation.

[0051] Generally, during automated machine learning, the number of concurrent tasks (parallel execution) for a single AutoML task is fixed in advance. For example, this can be user-configured or completely fixed parameters. In most scenarios, fixed parameters are simple to implement, but they completely ignore the resource constraints of the environment, making it difficult to maximize available resources. If the number of concurrent AutoML subtasks is too low, the task may run very slowly. If the number of concurrent AutoML subtasks is too high, the task may be forced to wait due to insufficient cluster resources.

[0052] In this embodiment, information about available resources can be determined, and the number of subtasks of the target task that can be executed in parallel, i.e., the first parallel number, can be determined based on the information about available resources. The subtasks can be adjusted based on the determined first parallel number. Thus, in various computer task execution scenarios, the number of subtasks that can be elastically scaled and adjusted based on available resources, thereby maximizing the effective utilization of available resources. At the same time, the available resources can also be used to adjust the execution speed of the subtasks to the maximum.

[0053] In one embodiment, the first parallel number of subtasks for executing the target task is determined based on the information of available resources, such as Figure 2 As shown, including:

[0054] Step S21: determining a second parallel number of subtasks of the currently executing target task;

[0055] Step S22: if the available resource information indicates that a new subtask can be added, the second parallel quantity is increased according to a preset adjustment step size;

[0056] Step S23: The second parallel quantity after the increase is used as the first parallel quantity.

[0057] In this embodiment, the second parallel quantity may be the number of subtasks in the running state included in the target task.

[0058] The information about available resources indicates that new subtasks can be added. This may be because the information about available resources is sufficient, and more subtasks can be run on the basis of the second number of subtasks in parallel.

[0059] The second parallel number is increased according to a preset adjustment step, specifically, for example, the adjustment step may be increased based on the second parallel number. For example, if the adjustment step is 1, 1 is added to the second parallel number.

[0060] In this embodiment, when new subtasks can be added based on the information of available resources, the second parallel quantity is increased according to a preset adjustment step, thereby avoiding a sudden shortage of available resources caused by adding too many subtasks at one time.

[0061] In one embodiment, determining a first parallel number of subtasks for executing a target task based on information about available resources includes:

[0062] When the available resource information indicates that the current number of subtasks is insufficient, the second parallel quantity is adjusted downward according to a preset adjustment step size;

[0063] The second parallel quantity after the adjustment is used as the first parallel quantity.

[0064] In a specific implementation, the adjustment step size when the second parallel number is adjusted downward may be the same as or different from the adjustment step size when the second parallel number is adjusted upward.

[0065] In this embodiment, when available resources are insufficient, the second parallel number can be adjusted downward, thereby avoiding always using the same parallel number to run more than two subtasks simultaneously, which may lead to insufficient resources or even failure of subtask execution.

[0066] In another possible implementation, the specific amount of data that can be adjusted upward or downward can be determined based on available resource information, thus achieving more precise adjustments without being restricted by step size. Specifically, when adjusting the number of parallel operations upward, a more conservative data estimation strategy can be used. When available resources are sufficient, the second parallel operation number can be increased slightly. When adjusting the number of parallel operations downward, a more general data estimation strategy can be used. When resources are insufficient, the second parallel operation number can be reduced by an appropriate amount.

[0067] In a possible implementation, the first parallel quantity and the second parallel quantity are both positive integers.

[0068] In a specific implementation, when the information of available resources indicates that the current number of subtasks is insufficient, the second parallel number is reduced according to a preset adjustment step, or when the information of available resources indicates that new subtasks can be added, the second parallel number is increased according to a preset adjustment step. It may include: directly determining the first parallel number, that is, directly determining the number of subtasks that the available resources can support running simultaneously, and when the first parallel number and the second parallel number are not equal, determining the second parallel number to be increased or decreased.

[0069] In a specific implementation, the second parallel number is adjusted up or down according to the adjustment step size. The second parallel number may be adjusted once according to the set step size, or may be adjusted multiple times until a set condition is met.

[0070] In one embodiment, adjusting the subtask according to the first parallel quantity includes:

[0071] When the first parallel quantity is greater than the second parallel quantity of the subtask currently executing the target task, a new subtask is started.

[0072] In this embodiment, when it is determined that a subtask can be added, the new subtask is directly started, so that the subtask is added in real time according to the available resources, thereby improving the utilization rate of the available resources.

[0073] In one embodiment, adjusting the subtask according to the first parallel quantity includes:

[0074] When the first parallel number is smaller than the second parallel number of the subtask of the currently executing target task, a new subtask is started according to the first parallel number after the subtask of the currently executing target task is completed.

[0075] In this embodiment, the subtasks of the currently executed target task are completed. This means that at least one of the currently executed subtasks is completed, or all of the currently executed subtasks are completed.

[0076] After the subtasks of the currently executing target task are completed, new subtasks are started according to the first parallel quantity. This can be after a subtask of the previously executing target task is completed, new subtasks are refused to be started until the number of remaining subtasks is less than the second parallel quantity.

[0077] After the subtasks of the currently executing target task are completed, a new subtask is started according to the first parallel quantity. Alternatively, after all subtasks of the currently executing target task are completed, at least one subtask of the target task is restarted according to the second parallel quantity.

[0078] In this embodiment, when it is determined that the number of parallel subtasks needs to be reduced, the currently running subtasks are still allowed to continue running, thereby avoiding repeated execution of subtasks and making it impossible to adjust the number of parallel subtasks to improve resource utilization.

[0079] In one embodiment, the task execution method further includes:

[0080] When a change event of the target task is detected, the target task is added to the task queue;

[0081] When the target task is dequeued from the task queue, a step of determining a first parallel quantity for executing subtasks of the target task according to information of available resources is performed.

[0082] In this embodiment, the target task change event may be any event that may cause the number of target task subtasks to change, such as the addition of subtasks, reduction of subtasks, completion of subtasks, start of target task, stop of target task, or the like.

[0083] In this embodiment, when a change event occurs that may cause a change in the number of subtasks of the target task, the target task is added to the task queue and sorted with multiple other tasks. When the target task is queued, information about available resources is obtained, and the second parallel number of subtasks for the target task is determined, thereby reducing interference with other tasks.

[0084] In one implementation, the target task may be of the same type as or different from the other tasks. When determining the first parallel quantity, the calculation may be performed in combination with at least one of the type of the target task and the type of the subtasks of the target task.

[0085] In one embodiment, adding the target task to the task queue further includes:

[0086] Determine the order of target tasks in the task queue;

[0087] Add the target tasks to the task queue according to the order.

[0088] In this embodiment, the order of the target tasks in the queue may be determined according to the priorities of the target tasks.

[0089] In this embodiment, when a change event occurs, the order of the target task in the task queue can be determined based on priority or other information, so that when the target task is urgent and the change is small, the parallel quantity can be determined for the target task at a faster speed to ensure that important tasks are executed first.

[0090] In one embodiment, the task execution method may further include:

[0091] Updates information about available resources in the cache.

[0092] In a specific example of the present disclosure, the task execution device can implement various necessary functions through the following modules:

[0093] AutoML Controller: This controller manages all AutoML tasks, generates tasks based on individual AutoML jobs, and manages the lifecycle of all AutoML tasks and subtasks.

[0094] Tuner Service (adjustment service module): can be used to start the tuning algorithm service, receive requests from AutoMLController, and provide corresponding model hyperparameter combination recommendations based on different algorithms or configurations.

[0095] Meta Service: This module is used to persist task metadata, such as persistently storing or configuring metadata for AutoML tasks (Experiments) and individual jobs (Runs). An Experiment can be a single AutoML learning task. A Run can be a single AutoML task, typically consisting of multiple jobs. A single Experiment run is called a Run.

[0096] For a single basic AutoML run, the workflow is as follows:

[0097] Create an AutoML task (either manually or through other means). The AutoML Controller is responsible for processing the user's AutoML task request, requesting a specific number of hyperparameter combinations from the corresponding TunerService based on the concurrency number, tuning algorithm, and other configurations set for the AutoML task. It then assembles and sends the actual Runs, maintains the status of the Runs and the corresponding status of the upper-level Experiment to the Meta Service, and reports the results of the Runs to the TunerService as needed to guide the Tuner Service to better provide hyperparameter combination recommendations for the next round. In this public example, the execution process of the target task may include the above process.

[0098] The disclosed embodiments can be applied to automated machine learning scenarios to achieve elastic scaling of automated machine learning. The task execution method of the disclosed examples can be implemented in a containerized environment, building on existing AutoML services and providing the ability to dynamically scale tasks based on cluster resources.

[0099] In a specific implementation, the example scenario of this disclosure can be Kubernetes, a large-scale containerized infrastructure platform. In this example, Kubernetes can be a mainstream cluster management and task distribution management platform. Currently, the underlying layers of mainstream machine learning platforms are also based on or support Kubernetes.

[0100] In this example, the main functions of Kubernetes may include the ability to adjust the concurrency of running AutoML tasks. An interface that exposes the ability to dynamically modify the concurrency of AutoML tasks (hyperparameter search, image tasks, training tasks) can be set up, for example, through HTTP (Hyper Text Transfer Protocol) / RPC (Remote Procedure Call) interfaces.

[0101] In this example, Kubernetes' main functions also include dynamically scaling the number of concurrent AutoML tasks based on cluster resources.

[0102] In a specific example, AutoML Jobs can be a newly added Kubernetes custom resource, corresponding to a single AutoML task, and the user can configure the minimum and / or maximum number of concurrent jobs. Figure 4As shown, the number of concurrent connections can be adjusted through the following components:

[0103] Cache 41: This component maintains a cache layer, which can be a newly added read-only buffer. It operates by monitoring all pods and / or nodes / AutoML jobs in the cluster. Users can send task lists and watch conditions to cache 41 through the Kubernetes API server. The Kubernetes API server can store information using ETCD (a key-value store).

[0104] Session (Session Control Module) 42: This module runs a timed thread that can be used to calculate and / or update the concurrency of AutoML tasks, typically executed on a scheduled basis. The actions actually executed within Session Control Module 42 are called Actions and can be used to calculate the number of concurrent tasks. Session Control Module 42 can manage multiple task queues. After a task is popped from the queue, it calculates the number of parallel tasks (Compute Parallelism), notifies the task's NotifyService of the calculated number, and updates the task's status (Update Job Status).

[0105] Plugins 43: Also known as customizable plugins, these plugins can be extended at various execution stages within each cycle of a session computation. These plugins can be extended through plugin functions, thereby increasing concurrency. In this example, these plugins may include a notification plugin 44, an elastic plugin 45, and a job order plugin 46.

[0106] In this example, the notification plugin 43 can be used to run. After the actions are executed, the notification service (NotifyService) can be used to characterize the target task. The functionality of the notification service can be extended through plugins. For example, plugins can implement the notification service functionality by calling an interface that exposes the ability to dynamically modify the concurrency of AutoML tasks. The notification service can be called back by the AutoML service using a callback function.

[0107] In an example of the present disclosure, the basic workflow of the above components is as follows: Figure 3 As shown, including:

[0108] Step S31: Monitor the available resources required by PODs and / or nodes and / or AutoML tasks through the informer mechanism provided by Kubernetes, and build a unified Cache layer.

[0109] Step S32: For each AutoML task resource event, place it in a priority processing queue and use a custom jobborder plugin (task order plugin) to determine the order of task dequeueing. Usually, the factors that determine the order include the time of entry into the queue, priority, etc. For example, you can use Figure 4 The task sequence plug-in 46 is shown.

[0110] Step S33: Determine the AutoML task to be dequeued from the queue.

[0111] Step S34: Obtain the POD running under the AutoML task, and update the status of the AutoML task resources according to the POD.

[0112] Step S35: Based on the customized elastic plugin, combined with the cluster resource situation in the Cache and the min (minimum) and / or max (maximum) concurrency configuration defined in the AutoML task, the concurrency is calculated and the status of the AutoML task resources is updated. The elastic plugin can be used as follows Figure 4 The elastic insert 45 is shown.

[0113] In this example, when the status of the AutoML job resource changes, the Notify Service can be called to notify the services that need to be notified. For example, you can set Figure 4 The notification plug-in 44 shown is used to notify the existing AutoML service to update the concurrency number of AutoML tasks.

[0114] In general, if you rely on a pre-set number of task concurrency to control the number of tasks that can be initiated simultaneously in a single round, this constraint makes the entire AutoML task unable to perceive the underlying resource situation, that is, it is impossible to dynamically adjust the concurrency according to the resource situation. When cluster resources are insufficient, an excessively large concurrency will cause resource preemption between tasks, making it impossible to complete tasks quickly; and when cluster resources are sufficient, an excessively small concurrency will make it impossible to fully utilize cluster resources. Therefore, according to the method provided in the example of this disclosure, the concurrency of AutoML tasks can be flexibly adjusted, which plays an important role in the rational and efficient use of available resources. Therefore, the example of this disclosure can maximize the resource utilization of the cluster by monitoring cluster resources and dynamically setting a reasonable concurrency for each AutoML.

[0115] The present disclosure also provides a task execution device, such as Figure 5 As shown, including:

[0116] Resource information acquisition module 51, used to obtain information about available resources;

[0117] A first parallel quantity determination module 52 is configured to determine a first parallel quantity for executing subtasks of a target task based on information about available resources;

[0118] The adjustment module 53 is configured to adjust the subtasks according to the first parallel quantity.

[0119] In one embodiment, Figure 6 As shown, the first parallel quantity determination module includes:

[0120] A second parallel quantity determining unit 61 is used to determine a second parallel quantity of the subtasks currently executing the target task;

[0121] an increasing unit 62, configured to increase the second parallel quantity according to a preset adjustment step size when the information of available resources indicates that a new subtask can be added;

[0122] The upward adjustment execution unit 63 is configured to use the increased second parallel quantity as the first parallel quantity.

[0123] In one embodiment, Figure 7 As shown, the first parallel quantity determination module also includes:

[0124] A downward adjustment unit 71 is configured to adjust the second parallel number downward according to a preset adjustment step size when the information of available resources indicates that the available resources are insufficient for the current number of subtasks;

[0125] The downward adjustment execution unit 72 is configured to use the downward adjusted second parallel quantity as the first parallel quantity.

[0126] In one embodiment, Figure 8 As shown, the adjustment module includes:

[0127] The first adjusting unit 81 is configured to start a new subtask when the first parallel number is greater than the second parallel number of the subtask currently executing the target task.

[0128] In one embodiment, Figure 9 As shown, the adjustment module includes:

[0129] The second adjusting unit 91 is configured to start a new subtask according to the first parallel quantity after the subtask of the currently executing target task is completed, if the first parallel quantity is smaller than the second parallel quantity of the subtask of the currently executing target task.

[0130] In one embodiment, Figure 10 As shown, the task execution device also includes:

[0131] The monitoring module 101 is used to monitor the change event of the target task;

[0132] The adding module 102 is configured to add the target task to the task queue when a change event of the target task is detected;

[0133] The first parallel quantity determination triggering module 103 is configured to, when a target task is dequeued from a task queue, execute a step of determining a first parallel quantity of subtasks for executing the target task according to information of available resources.

[0134] In one embodiment, Figure 11 As shown, the joining module includes:

[0135] The sequence unit 111 is used to determine the order of the target tasks in the task queue;

[0136] The sequential execution unit 112 is used to add the target task to the task queue according to the sequence.

[0137] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0138] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0139] Figure 12 A schematic block diagram of an example electronic device 120 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0140] like Figure 12As shown, the device 120 includes a computing unit 121, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 122 or a computer program loaded from a storage unit 128 into a random access memory (RAM) 123. Various programs and data required for the operation of the device 120 can also be stored in the RAM 123. The computing unit 121, the ROM 122, and the RAM 123 are connected to each other via a bus 124. An input / output (I / O) interface 125 is also connected to the bus 124.

[0141] Various components in device 120 are connected to I / O interface 125, including an input unit 126, such as a keyboard and mouse; an output unit 127, such as various types of displays and speakers; a storage unit 128, such as a magnetic disk and optical disk; and a communication unit 129, such as a network card, a modem, a wireless communication transceiver, etc. Communication unit 129 allows device 120 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0142] The computing unit 121 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 121 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 121 performs the various methods and processes described above, such as the task execution method. For example, in some embodiments, the task execution method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 128. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 120 via the ROM 122 and / or the communication unit 129. When the computer program is loaded into the RAM 123 and executed by the computing unit 121, one or more steps of the task execution method described above can be performed. Alternatively, in other embodiments, the computing unit 121 can be configured to perform the task execution method by any other appropriate means (e.g., by means of firmware).

[0143] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0144] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0145] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0146] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0147] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0148] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0149] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0150] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A task execution method, comprising: Obtain information about available resources; Determining a first parallel quantity for executing subtasks of a target task according to the information of the available resources; The target task is one of multiple tasks currently running in the system; The first parallel quantity refers to the number of subtasks that can be supported by existing available resources to run simultaneously; Adjusting the subtask according to the first parallel quantity; Among them, adjusting the subtask according to the first parallel number includes: when the first parallel number is greater than the second parallel number, directly starting a new subtask; when the first parallel number is less than the second parallel number, after the subtask currently executing the target task is completed, starting a new subtask according to the first parallel number; wherein, the second parallel number is the parallel number of the subtask currently executing the target task.

2. The method according to claim 1, wherein The determining, based on the information about the available resources, a first parallel number of subtasks for executing the target task includes: Determine a second parallel quantity of the subtask currently executing the target task; If the available resource information indicates that a new subtask can be added, increasing the second parallel quantity according to a preset adjustment step; The second parallel quantity after the increase is used as the first parallel quantity.

3. The method according to claim 2, wherein: The determining, based on the information about the available resources, a first parallel number of subtasks for executing the target task includes: When the information about the available resources indicates that the current number of subtasks is insufficient, the second parallel quantity is adjusted downward according to a preset adjustment step size; The second parallel quantity after the adjustment is used as the first parallel quantity.

4. The method according to any one of claims 1 to 3, wherein: The method further comprises: When a change event of the target task is detected, the target task is added to the task queue; When the target task is dequeued from the task queue, the step of determining a first parallel number of subtasks for executing the target task according to the information of the available resources is performed.

5. The method according to claim 4, wherein Adding the target task to a task queue includes: Determine the order of the target tasks in the task queue; According to the order, the target task is added to the task queue.

6. A task execution device, comprising: Resource information acquisition module, used to obtain information about available resources; A first parallel quantity determination module is used to determine a first parallel quantity of subtasks for executing the target task according to the information of the available resources; The target task is one of multiple tasks currently running in the system; the first parallel quantity refers to the number of subtasks that can be supported by existing available resources to run simultaneously; an adjusting module, configured to adjust the subtask according to the first parallel quantity; Wherein, the adjustment module includes: a first adjusting unit, configured to directly start a new subtask when the first parallel quantity is greater than the second parallel quantity of the subtask currently executing the target task; The second adjustment unit is configured to start a new subtask according to the first parallel number after the subtask currently executing the target task is completed if the first parallel number is smaller than the second parallel number of the subtask currently executing the target task.

7. The device according to claim 6, wherein The first parallel quantity determination module includes: A second parallel quantity determining unit, configured to determine a second parallel quantity of the subtask currently executing the target task; an increasing unit, configured to increase the second parallel quantity according to a preset adjustment step size when the information about the available resources indicates that a new subtask can be added; The upward adjustment execution unit is configured to use the increased second parallel quantity as the first parallel quantity.

8. The device according to claim 7, wherein The first parallel quantity determination module further includes: a downward adjustment unit, configured to adjust the second parallel quantity downward according to a preset adjustment step size when the information of the available resources indicates that the available resources are insufficient for the current number of subtasks; The downward adjustment execution unit is configured to use the second parallel quantity after the downward adjustment as the first parallel quantity.

9. The device according to any one of claims 6 to 8, wherein: The device further comprises: A monitoring module, used to monitor the change events of the target task; An adding module, configured to add the target task to a task queue when a change event of the target task is detected; The first parallel quantity determination triggering module is configured to execute the step of determining the first parallel quantity of subtasks for executing the target task according to the information of the available resources when the target task is dequeued from the task queue.

10. The device according to claim 9, wherein The joining module includes: A sequence unit, used to determine the order of the target tasks in the task queue; A sequential execution unit is used to add the target task to the task queue according to the sequence.

11. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images