Method and device for executing computing tasks, storage medium and electronic device

Abstract

The present application discloses a method and device for executing a computing task, a storage medium and an electronic device. The method for executing a computing task comprises: receiving a task request; responding to the task request, converting multiple target task operators of the computing task into multiple target operator commands to be executed with a target execution order, and calling multiple second threads to generate target task data for multiple reference task operators in the multiple target task operators in parallel; executing multiple target operator commands according to the target task data in the target execution order, so that the execution idle parameters of the computing device executing the multiple target task operators are less than the idle parameter threshold; receiving the target computing result returned by the computing device as the execution result of the computing task. The above technical solution is adopted to solve the problems of low utilization rate of computing devices during the execution of computing tasks in the related technology, thereby achieving the technical effect of improving the utilization rate of computing devices during the execution of computing tasks.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and more specifically, to a method and device for executing a computing task, a storage medium, and an electronic device. Background Art

[0002] In the field of modern computing, especially in the application of artificial intelligence technologies such as deep learning, recommendation systems and reinforcement learning, the execution efficiency of computing tasks is crucial. These tasks usually involve a large amount of data processing and complex mathematical operations, which require powerful computing resources to ensure performance.

[0003] In the related technology, the traditional way of executing computing tasks often faces the problem of low computing resource utilization. For example, in the scenario where the CPU (Central Processing Unit) and the GPU (Graphics Processing Unit) work together to execute the AI ​​(Artificial Intelligence) model, the CPU is usually responsible for preparing the data required for the execution of the operator of the AI ​​model, and the GPU is responsible for executing the operator after obtaining the operator data. Since multiple operators in the AI ​​model need to be executed in a specific order, the CPU usually uses a single thread to prepare data for each operator in the order of operator execution. Using the above-mentioned traditional single-thread method of preparing operator data, it is easy for the speed of the CPU preparing data to be mismatched with the speed of the GPU executing the operator. For example, when the CPU takes too long to prepare the operator data, due to the high speed of the GPU executing the operator, after executing one operator, the GPU needs to wait for the CPU to prepare the data of the next operator to be executed before it can start executing the next operator. During the waiting period, the GPU is in an idle stage and does not execute any operator, resulting in low GPU utilization and reducing the overall performance of AI model training and reasoning.

[0004] In view of the problems in related technologies such as low utilization of computing equipment during the execution of computing tasks, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method and device for executing a computing task, a storage medium, and an electronic device, so as to at least solve the problem of low utilization rate of computing equipment during the execution of computing tasks in the related art.

[0006] According to an embodiment of the present application, a method for executing a computing task is provided. A first thread and multiple second threads are created on a task execution device. The method is applied to the first thread. The method includes:

[0007] receiving a task request, wherein the task request is used to request execution of a computing task;

[0008] In response to a task request, multiple target task operators of the computing task are converted into multiple target operator commands to be executed with a target execution order, and multiple second threads are called to generate target task data for multiple reference task operators in the multiple target task operators in parallel, wherein the target execution order is the execution order of the multiple target task operators in the computing task, each target operator command is used to control the computing device to execute one target task operator in the multiple target task operators, the task data is the data required when the corresponding task operator is executed, the task operator includes the reference task operator, and the task data includes the target task data;

[0009] Execute multiple target operator commands according to the target task data in a target execution order, so that the execution idle parameters of the computing device executing the multiple target task operators are less than the idle parameter threshold;

[0010] The target computing result returned by the computing device is received as the execution result of the computing task.

[0011] Optionally, converting multiple target task operators of the computing task into multiple target operator commands to be executed with a target execution order includes:

[0012] Obtain the execution order of multiple target task operators in the computing task to obtain the target execution order;

[0013] Convert each target task operator in the multiple target task operators into a corresponding target operator command to be executed, thereby obtaining multiple target operator commands to be executed;

[0014] Add multiple target operator commands to be executed to the first-in-first-out target command queue in sequence according to the target execution order.

[0015] Optionally, converting each target task operator among the multiple target task operators into a corresponding target operator command to be executed includes:

[0016] In the case where the multiple target task operators are N target task operators, the i-th target task operator among the N target task operators is converted into the corresponding i-th target operator command to be executed through the following steps, where N is an integer greater than 1:

[0017] Convert the i-th target task operator into the corresponding i-th initial operator command to be executed, where i is an integer greater than or equal to 1 and less than or equal to N;

[0018] According to the target execution order, the i-th first semaphore and the i-th second semaphore are added to the context structure of the i-th initial operator command to be executed to obtain the i-th target operator command to be executed, wherein the i-th second semaphore is used to indicate the current execution status of the i-th target operator command to be executed. When i is greater than 1, the i-th first semaphore is used to indicate the current execution status of the i-1-th target operator command to be executed. In the target execution order, the i-1-th target operator command to be executed is the previous target operator command to be executed of the i-th target operator command to be executed, and the execution status is the execution completion status or the non-execution status. When i is equal to 1, the execution status indicated by the i-th first semaphore is the execution completion status.

[0019] Optionally, multiple target operator commands are executed according to the target task data in the target execution order, including:

[0020] Invoking multiple second threads to obtain multiple reference operator commands from the target command queue;

[0021] The plurality of second threads are controlled to execute the plurality of reference operator commands according to the first semaphore and the second semaphore in the plurality of reference operator commands.

[0022] Optionally, controlling the plurality of second threads to execute the plurality of reference operator commands according to the first semaphore and the second semaphore in the plurality of reference operator commands includes:

[0023] In the case where the multiple second threads are M second threads, a kth second thread among the M second threads is controlled through the following steps to execute a kth reference operator command according to a kth first semaphore and a kth second semaphore in the kth reference operator command, wherein M is an integer greater than 1, the kth reference operator command is a reference operator command obtained by the kth second thread from N target operator commands, and k is an integer greater than or equal to 1 and less than or equal to M:

[0024] Detect whether the execution state indicated by the kth first semaphore is an execution completion state;

[0025] When it is detected that the execution state indicated by the kth first semaphore is the execution completion state, executing the kth reference operator command according to the target task data;

[0026] When it is detected that the execution state indicated by the kth first semaphore is a non-execution state, the execution of the kth reference operator command is prohibited.

[0027] Optionally, after executing the kth reference operator command according to the target task data, the method further includes:

[0028] The execution state indicated by the kth second semaphore is switched from the unexecuted state to the executed state.

[0029] Optionally, executing the kth reference operator command according to the target task data includes:

[0030] Obtaining the kth target task data of the kth target task operator, wherein the kth target task operator is the target task operator corresponding to the kth reference operator command among the N target task operators;

[0031] Execute the kth reference operator command according to the kth target task data.

[0032] Optionally, executing the kth reference operator command according to the kth target task data includes:

[0033] Get the kth operator function of the kth target task operator;

[0034] The kth operator function and the kth target task data are transmitted to a computing device, wherein the computing device is used to execute the kth target task operator according to the kth operator function and the kth target task data.

[0035] Optionally, calling multiple second threads to generate target task data for multiple reference task operators in the multiple target task operators in parallel includes:

[0036] In a case where the multiple second threads are M second threads, controlling the M second threads to search in parallel whether there is a target operator command in a target command queue created by the first thread, wherein the target command queue is used to store target operator commands to be executed, and M is an integer greater than 1;

[0037] When P second threads among the M second threads find P target operator commands from the target command queue, determine the P target task operators corresponding to the P target operator commands as P reference task operators, where P is an integer greater than or equal to 1 and less than or equal to M;

[0038] The P second threads are controlled to generate target task data for the P reference task operators in parallel.

[0039] Optionally, when P second threads among the M second threads find P target operator commands from the target command queue, determining P target task operators corresponding to the P target operator commands as P reference task operators, including: when the j-th second thread among the P second threads finds the j-th target operator command from the target command queue, determining the j-th target task operator corresponding to the j-th target operator command as the j-th reference task operator among the P reference task operators, where j is an integer greater than or equal to 1 and less than or equal to P;

[0040] Controlling P second threads to generate target task data for P reference task operators in parallel includes: controlling the j-th second thread to generate target task data for the j-th reference task operator.

[0041] Optionally, receiving the target computing result returned by the computing device as the execution result of the computing task includes:

[0042] Generate result synchronization instructions for computing tasks;

[0043] Check whether the execution progress of the computing task reaches the target progress;

[0044] When it is detected that the task execution progress reaches the target progress, a result synchronization instruction is sent to the computing device, wherein the computing device is configured to execute a target task operator corresponding to each target operator command when each target operator command among multiple target operator commands to be executed is executed, and when the result synchronization instruction is received, the computing device returns the operation results of the multiple target task operators currently being executed to the first thread;

[0045] The receiving computing device returns the target computing result in response to the result synchronization instruction.

[0046] Optionally, checking whether the task execution progress of the computing task reaches the target progress includes:

[0047] Get the command execution progress of multiple target operator commands corresponding to the computing task;

[0048] Check whether the command execution progress reaches the target progress.

[0049] Optionally, obtaining the command execution progress of multiple target operator commands corresponding to the computing task includes: detecting whether the last target operator command among multiple target operator commands to be executed in a target execution order has been executed;

[0050] Detecting whether the command execution progress reaches the target progress includes: when it is detected that the last target operator command has been executed, determining that the execution progress has reached the target progress; when it is detected that the last target operator command has not been executed, determining that the execution progress has not reached the target progress.

[0051] Optionally, detecting whether the last target operator command among multiple target operator commands to be executed in a target execution order has been executed includes:

[0052] Detecting a second semaphore of a last target operator command, wherein the second semaphore of the last target operator command is used to indicate a current execution state of the last target operator command;

[0053] In the case where it is detected that the execution state indicated by the second semaphore of the last target operator command is an execution completion state, determining that the last target operator command has been executed;

[0054] In the case where it is detected that the execution state indicated by the second semaphore of the last target operator command is a not-executed state, it is determined that the last target operator command is not executed.

[0055] Optionally, before calling the plurality of second threads to concurrently generate target task data for the plurality of reference task operators in the plurality of target task operators, the method further comprises:

[0056] Obtaining the thread configuration number of the second thread, wherein the thread configuration number is used to indicate the number of second threads currently allowed to be created by the task execution device;

[0057] Spawns a second thread with the configured number of threads.

[0058] Optionally, the thread configuration quantity of the second thread is obtained, including:

[0059] Collect the target operator count of multiple target task operators;

[0060] Generate a target number of threads that matches the target number of operators, where the target number of operators is positively correlated with the target number of threads;

[0061] Determine the target number of threads as the configured number of threads.

[0062] According to another embodiment of the present application, a device for executing a computing task is further provided. A first thread and a plurality of second threads are created on a task execution device. The device is applied to the first thread, and the device includes:

[0063] A first receiving module is used to receive a task request, wherein the task request is used to request the execution of a computing task;

[0064] A response module, used to respond to a task request, convert multiple target task operators of a computing task into multiple target operator commands to be executed with a target execution order, and call multiple second threads to generate target task data for multiple reference task operators in the multiple target task operators in parallel, wherein the target execution order is the execution order of the multiple target task operators in the computing task, each target operator command is used to control the computing device to execute one target task operator in the multiple target task operators, the task data is the data required when the corresponding task operator is executed, the task operator includes the reference task operator, and the task data includes the target task data;

[0065] An execution module, used to execute multiple target operator commands according to target task data in a target execution order, so that execution idle parameters of the computing device executing the multiple target task operators are less than an idle parameter threshold;

[0066] The second receiving module is used to receive the target operation result returned by the operation device as the execution result of the operation task. According to another embodiment of the present application, a computer program product is also provided, including a computer program, and the computer program is executed by a processor to perform the steps in any of the above method embodiments.

[0067] According to another embodiment of the present application, a non-volatile readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0068] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0069] In an embodiment of the present application, a method for executing a computing task is proposed, wherein a first thread and multiple second threads are created on a task execution device, and the method comprises: the first thread first receives a task request, which may request the execution of a computing task; then, in response to the task request, multiple target task operators of the computing task are converted into multiple target operator commands to be executed with a target execution order, and multiple second threads are called in parallel to generate target task data for multiple reference task operators in the multiple target task operators, wherein the target execution order is the execution order of the multiple target task operators in the computing task, each target operator command may control the computing device to execute one of the multiple target task operators, and the task data is the data required when the corresponding task operator is executed, the task operator includes the reference task operator, and the task data includes the target task data; then, according to the target execution order, multiple target operator commands are executed according to the target task data, so that the execution idle parameters of the computing device executing the multiple target task operators are less than the idle parameter threshold; finally, the target computing result returned by the receiving computing device is used as the execution result of the computing task, that is, A first thread and multiple second threads are created on the task execution device. When the first thread needs to execute a computing task, the first thread converts multiple target task operators into multiple target task commands to be executed according to the target execution order of multiple target task operators of the computing task. Each target task command can control the computing device to execute the corresponding target task operator. At the same time, multiple second threads are used to generate target task data for multiple reference task operators in parallel. After preparing the target task data of the target task operator, the first thread can execute the corresponding target task command according to the target execution order of the target task operator to control the computing device to execute the target task operator. By generating target task data for multiple reference task operators in parallel through multiple second threads, the preparation time of the target task data is greatly shortened. After executing a target task operator, the computing device can obtain the target task data of the target task operator indicated by the next target task command to be executed, and execute the corresponding target task operator, so that the execution idle parameters of the computing device executing multiple target task operators are less than the idle parameter threshold, thereby improving the utilization rate of the computing device. The above technical solution solves the problems of low utilization rate of computing devices in the execution of computing tasks in the related technology, and achieves the technical effect of improving the utilization rate of computing devices in the execution of computing tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a hardware structure block diagram of a computer device for executing a computing task according to an embodiment of the present application;

[0071] Figure 2 It is a schematic diagram of the execution process of a computing task in the related art;

[0072] Figure 3is a flowchart of a method for executing a computing task according to an embodiment of the present application;

[0073] Figure 4 is a schematic diagram of a method for executing a computing task according to an embodiment of the present application;

[0074] Figure 5 is a schematic diagram of a context structure of a target operator command according to an embodiment of the present application;

[0075] Figure 6 is a schematic diagram of multi-threaded execution of computing tasks according to an embodiment of the present application;

[0076] Figure 7 is a schematic diagram of a device for executing a computing task according to an embodiment of the present application;

[0077] Figure 8 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0078] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0079] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0080] The method embodiments provided in the embodiments of the present application can be executed in a server device or a similar computing device. Taking running on a server device as an example, Figure 1 1 is a hardware structure block diagram of a computer device for executing a computing task according to an embodiment of the present application. Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned server device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above server device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0081] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the execution methods of computing tasks in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, to implement the above-mentioned methods. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the server device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0082] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0083] The present application proposes a method for executing a computing task. Before describing the optional embodiments of the present application, in order to better understand the inventive concept of the present application and the creativity of the solution, the related technologies are first described:

[0084] Figure 2 It is a schematic diagram of the execution method of the computing task in the related art, such as Figure 2 As shown, in the related technology, the execution of the operator of the AI ​​model is achieved through the collaboration of the CPU and the GPU. The main thread of the CPU performs pre-execution preparations and post-execution operations for each operator in turn. The GPU is responsible for executing the operator after the CPU completes the pre-execution preparations of an operator. Specifically, for operator 1, the main thread of the CPU first needs a certain amount of time to perform the pre-execution preparations of operator 1, including parameter parsing and complex logic processing, etc. After completing the pre-execution preparations of operator 1, operator 1 is executed, that is, one or more kernel functions (kernel function kernel_1) are started in the task queue of the GPU to start execution. At the same time, the CPU needs a certain amount of time to perform the post-execution operations of operator 1. After the CPU completes all operations on operator 1 (operation 1), operation 2 is performed on operator 2 according to the operator execution order. At the same time, after executing operator 1, the GPU needs to wait for the CPU to complete the pre-execution preparations of operator 2 before executing operator 2.

[0085] However, the execution method of computing tasks in related technologies has the following technical problems: when the CPU takes a long time to complete the pre-execution preparation of an operator in a single-threaded manner, since the GPU executes the kernel function of the operator at a faster speed, it may happen that when the GPU completes the execution of the kernel function of one operator and is able to execute the kernel function of the next operator, the CPU has not completed the pre-execution preparation of the next operator to be executed. At this time, the GPU needs to wait for the CPU to complete the pre-execution preparation of the operator before it can execute the kernel function of the next operator, resulting in an idle period of the GPU. During the idle period, the GPU does not execute any program, the utilization rate of the GPU is low, and the overall performance of the AI ​​model is also affected.

[0086] In order to solve the above technical problems, a method for executing a computing task is provided in this embodiment. Figure 3 is a flowchart of a method for executing a computing task according to an embodiment of the present application. A first thread and multiple second threads are created on a task execution device. The method is applied to the first thread, such as Figure 3 As shown, the process includes the following steps:

[0087] Step S12, receiving a task request, wherein the task request is used to request execution of a computing task;

[0088] Step S14, in response to the task request, converting multiple target task operators of the computing task into multiple target operator commands to be executed with a target execution order, and calling multiple second threads to generate target task data for multiple reference task operators in the multiple target task operators in parallel, wherein the target execution order is the execution order of the multiple target task operators in the computing task, each target operator command is used to control the computing device to execute one target task operator in the multiple target task operators, the task data is the data required when the corresponding task operator is executed, the task operator includes the reference task operator, and the task data includes the target task data;

[0089] Step S16, executing multiple target operator commands according to the target task data in the target execution order, so that the execution idle parameters of the computing device executing the multiple target task operators are less than the idle parameter threshold;

[0090] Step S18, receiving the target computing result returned by the computing device as the execution result of the computing task.

[0091] Optionally, in this embodiment, the computing task may be, but is not limited to, an AI model computing task that can be used for training or reasoning, such as a deep learning model, a recommendation system model, and a reinforcement learning model, etc. Each computing task contains multiple target task operators that can represent specific computing operation functions. The target task operator may be, but is not limited to, a function or class that can perform specific operations, such as a convolutional layer in a convolutional neural network. The execution of the target task operator is achieved by calling the execution of one or more kernel functions (kernel functions). In the following embodiment, the AI ​​model is taken as an example of a computing task to describe the execution method of the computing task without limiting the type of computing task.

[0092] Optionally, in this embodiment, the task execution device may be a CPU, the first thread and the second thread may respectively be a main thread and a sub-thread created on the CPU that can complete the AI ​​model execution task, and the kernel functions corresponding to multiple operators in the AI ​​model may be executed on a computing device, wherein the computing device may be but is not limited to a GPU that can execute kernel functions.

[0093] Optionally, in this embodiment, Figure 4 is a schematic diagram of a method for executing a computing task in an embodiment of the present application, such as Figure 4 As shown, when the main thread of the CPU needs to execute the AI ​​model, it first parses the AI ​​model to determine all target task operators (hereinafter referred to as operators) that the AI ​​model needs to execute and the execution order of each operator (for example, operator 1 represents an operator with an execution order of 1, kernel_1 represents the kernel function corresponding to operator 1, operator 2 represents an operator with an execution order of 2, kernel_2 represents the kernel function corresponding to operator 2, operator 3 represents an operator with an execution order of 3, and kernel_3 represents the kernel function corresponding to operator 3). The execution order of the operators is the target execution order; then, multiple operators are converted into target operator commands associated with the execution order of the operators, wherein each target operator command can control the GPU to execute a kernel function corresponding to an operator, that is, after the CPU completes the pre-execution preparation of the operators according to the execution order of the operators, executing the target operator command can start the kernel function corresponding to the operator to be executed in the task queue of the GPU to start execution, the CPU completes the pre-execution preparation for each operator one by one, and executes the target operator command to control the GPU to execute the kernel function corresponding to the operator until all operators are executed, that is, the execution of the AI ​​model is completed.

[0094] Optionally, in this embodiment, if Figure 4As shown, before the main thread of the CPU executes the target operator command in the target execution order, it needs to prepare the target task data of the operator corresponding to the target operator command, where the target task data can be but is not limited to the parameters required when the running device executes a specific computing operation function, such as the need to prepare one or more kernel function input parameters, output parameters, execution configuration, etc. before the GPU executes the kernel function.

[0095] Optionally, in this embodiment, if Figure 4 As shown, the main thread may prepare target task data in a manner including but not limited to: calling multiple sub-threads to generate target task data for a reference task operator in parallel, where the reference task operator represents k operators that can be processed in parallel by the current k sub-threads, where k is an integer greater than or equal to 1 and less than or equal to the total number of operators. Assuming that the AI ​​model has 10 operators (for example, operators 1 to 10), and 1 main thread and 3 sub-threads (for example, sub-threads 1 to 3) are created on the CPU, at the same time, the 3 sub-threads can complete the pre-execution preparation of the 3 operators (i.e., the reference task operators) in parallel (i.e., generate target task data for the 3 operators).

[0096] Optionally, in this embodiment, if Figure 4 As shown, according to the target execution order of the operators, that is, first executing the kernel function kernel_1 corresponding to operator 1, then executing the kernel function kernel_2 corresponding to operator 2, and then executing the kernel function kernel_3 corresponding to operator 3, after the child thread completes the operator execution preparation (generating target task data), the main thread can execute the corresponding target task command to control the GPU to execute the kernel function corresponding to the operator. For example, the three sub-threads currently implement the pre-execution preparation of operator 1, operator 2 and operator 23 in parallel. According to the execution order of the operators, operator 1 is executed first. When sub-thread 1 completes the pre-execution preparation of operator 1, the main thread of the CPU can execute the target task command of operator 1 to control the GPU to execute the kernel function kernel_1 corresponding to operator 1. At this time, if the pre-execution preparation of operator 2 to be executed next is also completed, it is necessary to wait until the GPU completes the execution of kernel_1. Then the main thread initiates the target task command of operator 2 to control the GPU to execute kernel_2, ensuring that the GPU will execute the operators according to the target execution order. The waiting time at this time is the waiting time of sub-thread 2 on the CPU. The GPU is executing kernel_1 at this time, and does not cause the GPU to be idle.

[0097] Optionally, in this embodiment, if Figure 4As shown, the execution of all operator corresponding kernel functions is completed on the GPU in the target execution order. With the above execution method, the execution idle parameter of the GPU when executing multiple operator corresponding kernel functions (that is, the time when the GPU does not execute any operator corresponding kernel function) is less than waiting for a single main thread to generate target task data and then execute the operator corresponding kernel function. That is, when the GPU completes the execution of an operator corresponding kernel function and can execute the next operator corresponding kernel function, it needs to wait for the CPU main thread to generate the target task data of the operator, resulting in execution idle time (that is, execution idle threshold). Compared with the related technology, the utilization rate of the GPU is improved. When the execution of the last operator corresponding kernel function is completed, the GPU returns the target operation result, which is the execution result of the AI ​​model.

[0098] Optionally, in this embodiment, it is assumed that a large convolutional neural network (CNN) model needs to be trained for image recognition tasks. This model contains multiple layers, each of which can be regarded as an operator, including a convolution layer, an activation layer, a pooling layer, etc. During the training process, the first thread (main thread) receives the task request, and is then responsible for converting the various layers of the model (target task operators) into commands to be executed (target operator commands), and arranging them in the forward propagation order of the model (target execution order). At the same time, the main thread calls multiple second threads (sub-threads) to generate the data (target task data) required for each operator to be executed in parallel. After the sub-threads prepare the data, the main thread triggers each target operator command to execute the calculation of the corresponding operator in the target execution order. In this way, when the kernel function corresponding to an operator is calculated on the GPU, the calculation of the kernel function corresponding to the next operator can start immediately. After the calculation of the kernel functions corresponding to all operators is completed, the main thread receives the target operation results returned by the GPU. These results may include the output of the model (such as classification results) and loss values, which are used for subsequent back propagation and model parameter updates. That is, through multi-threaded parallel processing, each sub-thread is responsible for preparing the data of one or more operators, while the main thread is responsible for scheduling and executing the calculation of the kernel functions corresponding to these operators. This division of labor and cooperation method significantly reduces the idle time of the GPU, solves the problem of low utilization of computing equipment during the execution of computing tasks in related technologies, and achieves the technical effect of improving the utilization of computing equipment during the execution of computing tasks.

[0099] As an optional solution, multiple target task operators of the computing task are converted into multiple target operator commands to be executed with a target execution order, including:

[0100] S21, obtaining the execution order of multiple target task operators in the computing task to obtain the target execution order;

[0101] S22, converting each target task operator in the multiple target task operators into a corresponding target operator command to be executed, thereby obtaining multiple target operator commands to be executed;

[0102] S23, adding multiple target operator commands to be executed to a first-in-first-out target command queue in sequence according to the target execution order.

[0103] Optionally, in this embodiment, the method of converting multiple operators of the AI ​​model into target operator commands with a target execution order may be, but is not limited to, as follows: when the AI ​​model needs to be executed, the main thread parses the AI ​​model to obtain all operators and the execution order of each operator; then all operators are converted one by one into target operator commands to be executed according to the execution order, and the converted target operator commands are added to the first-in-first-out target command queue in sequence.

[0104] As an optional solution, each target task operator in the multiple target task operators is converted into a corresponding target operator command to be executed, including:

[0105] S31, when the multiple target task operators are N target task operators, converting the i-th target task operator among the N target task operators into the corresponding i-th target operator command to be executed by the following steps, where N is an integer greater than 1:

[0106] S32, converting the i-th target task operator into the corresponding i-th initial operator command to be executed, where i is an integer greater than or equal to 1 and less than or equal to N;

[0107] S33, according to the target execution order, add the i-th first semaphore and the i-th second semaphore to the context structure of the i-th initial operator command to be executed, and obtain the i-th target operator command to be executed, wherein the i-th second semaphore is used to indicate the current execution status of the i-th target operator command to be executed. When i is greater than 1, the i-th first semaphore is used to indicate the current execution status of the i-1-th target operator command to be executed. In the target execution order, the i-1-th target operator command to be executed is the previous target operator command to be executed of the i-th target operator command to be executed, and the execution status is the execution completion status or the non-execution status. When i is equal to 1, the execution status indicated by the i-th first semaphore is the execution completion status.

[0108] Optionally, in this embodiment, each target operator command may include a context structure, which may include information indicating the execution status of the current target operator command (first semaphore) and the execution status of the target operator command of the previous sequence (second semaphore), wherein the execution status may be an execution completed state or an unexecuted state.

[0109] Optionally, in this embodiment, a semaphore pointer may be used to implement the first semaphore and the second semaphore, each target operator command includes two semaphore pointers, and all used semaphores may be managed using a semaphore pool. Figure 5 is a schematic diagram of a context structure of a target operator command according to an embodiment of the present application, such as Figure 5 As shown, the context structure also encapsulates streams (which can be regarded as a task queue to ensure that operators are executed in a specific order) and handles (handles that can access GPU computing resources) that can be obtained through interface functions. By judging the first semaphore in the context structure of each target operator command, it is ensured that the CPU will execute the target operator command only after the execution status of the target operator command of the previous sequence is completed. Correspondingly, when the execution status of the target operator command of the previous sequence is not executed, it will wait when executing the instruction "pre_op->Wait()".

[0110] Optionally, in this embodiment, the method of converting the i-th operator in the AI ​​model into the corresponding i-th target operator command to be executed may include but is not limited to: assuming that i is equal to 3, first converting the third operator (assuming operator 3) into the corresponding initial operator command (assuming Initial_Op_3), wherein the initial operator command contains an empty context structure; then adding two semaphores to the context structure of Initial_Op_3 according to the target execution order (the execution order of operator 3 is 3), namely the third first semaphore (pre_op_3) and the third second semaphore (cur_op_3), to obtain the target operator command of operator 3 (assuming Target_Op_3), wherein cur_op_3 represents the current execution status of Target_Op_3, and pre_op_3 represents the current execution status of the target operator command to be executed in the previous order (ie, Target_Op_2). Assume that i is equal to 1. Since the execution order of operator 1 is 1, that is, Target_Op_1 is the first target operator command to be executed, the execution state corresponding to the first first semaphore is set to the execution completion state by default.

[0111] As an optional solution, multiple target operator commands are executed according to the target task data in the target execution order, including:

[0112] S41, calling multiple second threads to obtain multiple reference operator commands from the target command queue;

[0113] S42, controlling the plurality of second threads to execute the plurality of reference operator commands according to the first semaphore and the second semaphore in the plurality of reference operator commands.

[0114] Optionally, in this embodiment, the target command queue may be, but is not limited to, a data structure that can store all target operator commands to be executed.

[0115] Optionally, in this embodiment, the main thread calls multiple child threads to obtain multiple reference operator commands that can be currently processed in parallel from the target command queue, which may be but is not limited to: each child thread cyclically queries the target command queue to obtain the target operator command, and waits when the target command queue is empty until the main thread puts the target operator command in. Then, each child thread may obtain the target operator command from the target command queue as a reference operator command.

[0116] As an optional solution, controlling the plurality of second threads to execute the plurality of reference operator commands according to the first semaphore and the second semaphore in the plurality of reference operator commands includes:

[0117] S51, in the case where the multiple second threads are M second threads, control a kth second thread among the M second threads through the following steps to execute a kth reference operator command according to a kth first semaphore and a kth second semaphore in the kth reference operator command, wherein M is an integer greater than 1, the kth reference operator command is a reference operator command obtained by the kth second thread from N target operator commands, and k is an integer greater than or equal to 1 and less than or equal to M:

[0118] S52, detecting whether the execution status indicated by the kth first semaphore is the execution completion status;

[0119] S53, when it is detected that the execution state indicated by the kth first semaphore is the execution completion state, executing the kth reference operator command according to the target task data;

[0120] S54: When it is detected that the execution state indicated by the kth first semaphore is a non-executed state, prohibiting the execution of the kth reference operator command.

[0121] Optionally, in this embodiment, after multiple sub-threads obtain multiple reference operator commands, the execution status indicated by the first semaphore corresponding to each reference operator command (i.e., the execution status of the target operator command to be executed in the previous sequence) is detected. When it is detected that the execution status of the target operator command to be executed in the previous sequence is in the execution completed state, the GPU executes the reference operator command according to the target task data corresponding to the reference operator command generated by the sub-thread. When it is detected that the execution status of the target operator command to be executed in the previous sequence is in the unexecuted state, the execution of the reference operator command is prohibited, and other reference operator commands continue to be detected.

[0122] As an optional solution, after executing the kth reference operator command according to the target task data, the method further includes:

[0123] S61, switching the execution state indicated by the kth second semaphore from an unexecuted state to an executed state.

[0124] Optionally, in this embodiment, after each reference operator command is executed, that is, after all operations requiring the use of a stream or a handle are completed, the execution status indicated by the corresponding second semaphore is updated to an execution completion status, indicating that the next sequential target operator command can use the stream or handle, that is, the corresponding kernel function can be called to be executed by the GPU.

[0125] As an optional solution, executing the kth reference operator command according to the target task data includes:

[0126] S71, obtaining k-th target task data of the k-th target task operator, wherein the k-th target task operator is the target task operator corresponding to the k-th reference operator command among the N target task operators;

[0127] S72, execute the kth reference operator command according to the kth target task data.

[0128] Optionally, in this embodiment, if Figure 4 As shown, for the third operator (i.e., operator 3), when it is detected that the execution status of the target operator command of operator 2 is in the completed state and the GPU obtains the third target task data of the third operator (i.e., operator 3) (the data prepared and generated by the CPU's subthread 3 before executing operator 3), operator 3 starts to execute.

[0129] As an optional solution, executing the kth reference operator command according to the kth target task data includes:

[0130] S81, obtaining the kth operator function of the kth target task operator;

[0131] S82, transmitting the kth operator function and the kth target task data to a computing device, wherein the computing device is used to execute the kth target task operator according to the kth operator function and the kth target task data.

[0132] Optionally, in this embodiment, the operator function may be, but is not limited to, a kernel function corresponding to the operator, which includes one or more functions that can perform specific mathematical or logical operations (such as mathematical operation functions, activation functions, and normalization functions, etc.).

[0133] As an optional solution, calling multiple second threads to generate target task data for multiple reference task operators in multiple target task operators in parallel includes:

[0134] S91, when the multiple second threads are M second threads, control the M second threads to search in parallel whether there is a target operator command in a target command queue created by the first thread, where the target command queue is used to store target operator commands to be executed, and M is an integer greater than 1;

[0135] S92, when P second threads among the M second threads find P target operator commands from the target command queue, determine the P target task operators corresponding to the P target operator commands as P reference task operators, where P is an integer greater than or equal to 1 and less than or equal to M;

[0136] S93, controlling the P second threads to generate target task data for the P reference task operators in parallel.

[0137] Optionally, in this embodiment, Figure 6 is a schematic diagram of multi-threaded execution of computing tasks according to an embodiment of the present application, such as Figure 6 As shown, the method of calling multiple sub-threads to generate target task data for multiple reference task operators in parallel may include but is not limited to: in the initialization step, the main thread may start multiple sub-threads (sub-thread X is used to uniformly represent multiple sub-threads), sub-thread X obtains commands through a loop, and loops to query the target command queue to obtain the target operator command therein, and uses the operator corresponding to the found target operator command as the reference task operator; then, the multiple sub-threads that have found the multiple reference task operators are controlled to generate target task data for the reference task operators in parallel.

[0138] As an optional solution, when P second threads among the M second threads find P target operator commands from the target command queue, the P target task operators corresponding to the P target operator commands are determined as P reference task operators, including:

[0139] S101, when the j-th second thread among the P second threads finds the j-th target operator command from the target command queue, determine the j-th target task operator corresponding to the j-th target operator command as the j-th reference task operator among the P reference task operators, where j is an integer greater than or equal to 1 and less than or equal to P;

[0140] Control P second threads to generate target task data for P reference task operators in parallel, including:

[0141] S102, controlling the j-th second thread to generate target task data for the j-th reference task operator.

[0142] Optionally, in this embodiment, assuming that in the initialization step, the main thread starts 10 child threads, and the 10 child threads find 3 target operator commands in the target command queue by cyclically obtaining commands, the 3 operators corresponding to the 3 target operator commands found are determined as 3 reference task operators, and then the 3 child threads that find the 3 reference task operators are controlled to generate target task data for the reference task operators they respectively find.

[0143] As an optional solution, receiving the target computing result returned by the computing device as the execution result of the computing task includes:

[0144] S111, generating a result synchronization instruction for the computing task;

[0145] S112, detecting whether the task execution progress of the computing task reaches the target progress;

[0146] S113, when it is detected that the task execution progress reaches the target progress, sending a result synchronization instruction to the computing device, wherein the computing device is configured to execute a target task operator corresponding to each target operator command among multiple target operator commands to be executed when each target operator command is executed, and return the operation results of the multiple target task operators currently being executed to the first thread when receiving the result synchronization instruction;

[0147] S114, receiving the target computing result returned by the computing device in response to the result synchronization instruction.

[0148] Optionally, in this embodiment, the method of obtaining the execution result of the AI ​​model may include but is not limited to: first, the CPU generates a result synchronization instruction for the AI ​​model, and the result synchronization instruction may be a signal used to notify the GPU to prepare to return the calculation result of the AI ​​model; then detect whether the task execution progress of the AI ​​model has reached the target progress, and send a result synchronization instruction to the GPU when it is detected that the task execution progress has reached the target progress. When the GPU receives the result synchronization instruction sent by the CPU, it returns the calculation results of the kernel functions corresponding to all operators currently being executed to the main thread; finally, the target calculation result returned by the GPU in response to the result synchronization instruction is received. The target calculation result may be but is not limited to the direct result of the execution of the AI ​​model, such as the predicted value and the loss value.

[0149] As an optional solution, detecting whether the task execution progress of the computing task reaches the target progress includes:

[0150] S121, obtaining the command execution progress of multiple target operator commands corresponding to the computing task;

[0151] S122, detecting whether the command execution progress reaches the target progress.

[0152] Optionally, in this embodiment, the target progress may be, but is not limited to, a preset value, which may represent the degree of completion of the execution of the AI ​​model.

[0153] As an optional solution, the command execution progress of multiple target operator commands corresponding to the computing task is obtained, including:

[0154] S131, detecting whether the last target operator command among a plurality of target operator commands to be executed in a target execution order has been executed;

[0155] Check whether the command execution progress reaches the target progress, including:

[0156] S132: When it is detected that the last target operator command has been executed, it is determined that the execution progress has reached the target progress; when it is detected that the last target operator command has not been executed, it is determined that the execution progress has not reached the target progress.

[0157] Optionally, in this embodiment, when calling the GPU's task queue to synchronize calculation results, it is necessary to ensure that all target operator commands initiated on the main thread have been executed. Therefore, it is necessary to wait for the execution status indicated by the second semaphore corresponding to the last target operator command to be the execution completion status before responding to the response result synchronization instruction and returning the target calculation result to the CPU.

[0158] As an optional solution, detecting whether the last target operator command among multiple target operator commands to be executed in a target execution order has been executed includes:

[0159] S141, detecting a second semaphore of the last target operator command, wherein the second semaphore of the last target operator command is used to indicate a current execution state of the last target operator command;

[0160] S142, when it is detected that the execution state indicated by the second semaphore of the last target operator command is the execution completion state, determining that the last target operator command has been executed;

[0161] S143: When it is detected that the execution state indicated by the second semaphore of the last target operator command is a not-executed state, determine that the last target operator command is not executed.

[0162] Optionally, in the present embodiment, the method for obtaining the command execution progress of multiple target operator commands corresponding to the AI ​​model may include but is not limited to: detecting the execution status indicated by the second semaphore corresponding to the last target operator command in the target operator commands; when it is detected that the execution status indicated by the second semaphore corresponding to the last target operator command is the execution completed state, determining that the execution progress has reached the target progress; when it is detected that the execution status indicated by the second semaphore corresponding to the last target operator command is the unexecuted state, determining that the execution progress has not reached the target progress.

[0163] As an optional solution, before calling multiple second threads to generate target task data for multiple reference task operators in the multiple target task operators in parallel, the method further includes:

[0164] S151, obtaining the thread configuration number of the second thread, wherein the thread configuration number is used to indicate the number of second threads currently allowed to be created by the task execution device;

[0165] S152: Generate a second thread with the configured number of threads.

[0166] Optionally, in this embodiment, the number of thread configurations may be, but is not limited to, the number of sub-threads that the CPU is currently allowed to create.

[0167] As an optional solution, obtaining the thread configuration quantity of the second thread includes:

[0168] S161, collecting the number of target operators of multiple target task operators;

[0169] S162, generating a target number of threads that matches the target number of operators, wherein the target number of operators is positively correlated with the target number of threads;

[0170] S163, determining the target number of threads as the number of configured threads.

[0171] Optionally, in this embodiment, the number of sub-threads can be set according to the total number of operators in the AI ​​model (i.e., the target number of operators), the CPU load, the thread scheduling strategy, and the performance requirements of the application, and then the corresponding number of sub-threads is created after the number of sub-threads is determined. Generally, the more total number of operators in the AI ​​model, the more sub-threads need to be created. The number of thread configurations can also be set in the CPU configuration file. Users can modify the number of thread configurations according to different AI model task requirements and system resources, and find the appropriate number of thread configurations to achieve better AI model execution performance.

[0172] Optionally, in this embodiment, in order to better understand the process of the execution method of the above-mentioned computing task, the process of the execution method of the above-mentioned computing task is described below in combination with an optional embodiment, but it is not used to limit the technical solution of the embodiment of this application.

[0173] In this embodiment, a method for executing a computing task is provided, such as Figure 6 As shown, it mainly includes the following steps:

[0174] Main thread side:

[0175] Step 1: The user starts the program and begins executing the AI ​​model (including computing tasks such as training or inference tasks);

[0176] Step 2: Initialization. Multiple child threads (child threads X) are created according to the number of threads configured. Multiple child threads X start to query the queue (target command queue) in a loop, waiting to obtain the command (target operator command) added to the command queue by the main thread.

[0177] Step 3: Convert the operator of the AI ​​model (target task operator) into a target operator command and add the command to a queue (target command queue), where the command (target operator command) contains the information required for the execution of the corresponding operator.

[0178] Step 4: After adding the command to the queue, wait for all commands to be executed and then obtain the target operation result.

[0179] Child thread X end:

[0180] Step 2.1: Multiple child threads cyclically query the command queue (target command queue) to obtain the command (target operator command) put in by the main thread.

[0181] Step 2.2, execute the command;

[0182] Step 2.2.1: Execute the corresponding operator pre-execution preparation for the acquired command, such as parameter parsing and other logical processing;

[0183] Step 2.2.2, detecting the execution state indicated by the first semaphore in the context structure of the command, and waiting for the previous command to be executed;

[0184] Step 2.2.3, when the detection result is in the execution completion state (valid), start the operator of the stream control GPU start command, that is, control the GPU to execute the corresponding operator;

[0185] Step 2.2.4, after the GPU completes executing the operator, the execution state indicated by the second semaphore in the context structure of the command is updated to the execution completion state, and the updated second semaphore of the command is synchronized to the semaphore pool;

[0186] Step 2.2.5, perform the post-operation of the operator;

[0187] Step 2.3, the acquired command is processed;

[0188] Step 2.4, return to step 2.1, and continue to execute steps 2.1 to 2.3 for the newly acquired commands until all commands are processed, that is, the queue is empty.

[0189] The above process encapsulates the executed operator into a command and uses multiple sub-threads to complete the pre-execution preparation of the operator in parallel, which can reduce the waiting time of the GPU and improve the efficiency of the GPU executing the kernel function corresponding to the operator, thereby improving the technical effect of improving the utilization rate of the computing device during the execution of the computing task. The performance improvement results of the llama2 model executed using the method of this embodiment in the reasoning and training test conditions are shown in Table 1.

[0190] Table 1

[0191]

[0192] The performance improvement results of the llama2 model executed using the method of this embodiment in the case of 100 steps of fine-tuning training test are shown in Table 2.

[0193] Table 2

[0194]

[0195] According to the data in Table 1 and Table 2, it can be seen that multiple sub-threads are used to complete the pre-execution preparation of operators in parallel. Since the idle time of GPU waiting can be reduced, the GPU can be more fully utilized, thereby improving performance. There are different performance improvements under different input conditions. In summary, the use of the method in this embodiment can bring about a 5% performance improvement in both reasoning and training.

[0196] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0197] In the present embodiment, a device for executing a computing task is also provided, and the device is used to implement the above-mentioned embodiment and preferred implementation mode, and the descriptions that have been made are not repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0198] Figure 7 is a structural block diagram of a device for executing a computing task according to an embodiment of the present application; Figure 7 As shown, a first thread and a plurality of second threads are created on the task execution device, and the apparatus is applied to the first thread, and the apparatus includes:

[0199] A first receiving module 702 is used to receive a task request, wherein the task request is used to request to execute a computing task;

[0200] A response module 704 is used to respond to the task request, convert multiple target task operators of the computing task into multiple target operator commands to be executed with a target execution order, and call multiple second threads to generate target task data for multiple reference task operators in the multiple target task operators in parallel, wherein the target execution order is the execution order of the multiple target task operators in the computing task, each target operator command is used to control the computing device to execute one target task operator in the multiple target task operators, the task data is the data required when the corresponding task operator is executed, the task operator includes the reference task operator, and the task data includes the target task data;

[0201] An execution module 706 is used to execute multiple target operator commands according to the target task data in a target execution order, so that the execution idle parameters of the computing device executing the multiple target task operators are less than an idle parameter threshold;

[0202] The second receiving module 708 is used to receive the target computing result returned by the computing device as the execution result of the computing task.

[0203] In an exemplary embodiment, the response module includes:

[0204] An acquisition unit, used to acquire the execution order of multiple target task operators in the computing task, and obtain the target execution order;

[0205] A conversion unit, used for converting each target task operator in the multiple target task operators into a corresponding target operator command to be executed, so as to obtain multiple target operator commands to be executed;

[0206] The adding unit is used to add multiple target operator commands to be executed to the first-in-first-out target command queue in sequence according to the target execution order.

[0207] In an exemplary embodiment, the conversion unit is further configured to:

[0208] In the case where the multiple target task operators are N target task operators, the i-th target task operator among the N target task operators is converted into the corresponding i-th target operator command to be executed through the following steps, where N is an integer greater than 1:

[0209] Convert the i-th target task operator into the corresponding i-th initial operator command to be executed, where i is an integer greater than or equal to 1 and less than or equal to N;

[0210] According to the target execution order, the i-th first semaphore and the i-th second semaphore are added to the context structure of the i-th initial operator command to be executed to obtain the i-th target operator command to be executed, wherein the i-th second semaphore is used to indicate the current execution status of the i-th target operator command to be executed. When i is greater than 1, the i-th first semaphore is used to indicate the current execution status of the i-1-th target operator command to be executed. In the target execution order, the i-1-th target operator command to be executed is the previous target operator command to be executed of the i-th target operator command to be executed, and the execution status is the execution completion status or the non-execution status. When i is equal to 1, the execution status indicated by the i-th first semaphore is the execution completion status.

[0211] In an exemplary embodiment, the execution module includes:

[0212] A calling unit, used for calling a plurality of second threads to obtain a plurality of reference operator commands from a target command queue;

[0213] The first control unit is used to control the plurality of second threads to execute the plurality of reference operator commands according to the first semaphore and the second semaphore in the plurality of reference operator commands.

[0214] In an exemplary embodiment, the first control unit is further configured to:

[0215] In the case where the multiple second threads are M second threads, a kth second thread among the M second threads is controlled through the following steps to execute a kth reference operator command according to a kth first semaphore and a kth second semaphore in the kth reference operator command, wherein M is an integer greater than 1, the kth reference operator command is a reference operator command obtained by the kth second thread from N target operator commands, and k is an integer greater than or equal to 1 and less than or equal to M:

[0216] Detect whether the execution state indicated by the kth first semaphore is an execution completion state;

[0217] When it is detected that the execution state indicated by the kth first semaphore is the execution completion state, executing the kth reference operator command according to the target task data;

[0218] When it is detected that the execution state indicated by the kth first semaphore is a non-execution state, the execution of the kth reference operator command is prohibited.

[0219] In an exemplary embodiment, the apparatus further comprises:

[0220] The switching module is used to switch the execution state indicated by the kth second semaphore from an unexecuted state to an executed state after executing the kth reference operator command according to the target task data.

[0221] In an exemplary embodiment, the first control unit is further configured to:

[0222] Obtaining the kth target task data of the kth target task operator, wherein the kth target task operator is the target task operator corresponding to the kth reference operator command among the N target task operators;

[0223] Execute the kth reference operator command according to the kth target task data.

[0224] In an exemplary embodiment, the first control unit is further configured to:

[0225] Get the kth operator function of the kth target task operator;

[0226] The kth operator function and the kth target task data are transmitted to a computing device, wherein the computing device is used to execute the kth target task operator according to the kth operator function and the kth target task data.

[0227] In an exemplary embodiment, the response module includes:

[0228] a second control unit, configured to, when the plurality of second threads are M second threads, control the M second threads to search in parallel whether a target operator command exists in a target command queue created by the first thread, wherein the target command queue is used to store target operator commands to be executed, and M is an integer greater than 1;

[0229] a first determining unit, configured to determine, when P second threads among the M second threads find P target operator commands from the target command queue, the P target task operators corresponding to the P target operator commands as P reference task operators, where P is an integer greater than or equal to 1 and less than or equal to M;

[0230] The third control unit is used to control the P second threads to generate target task data for the P reference task operators in parallel.

[0231] In an exemplary embodiment, the first determining unit is further used to: when the jth second thread among the P second threads finds the jth target operator command from the target command queue, determine the jth target task operator corresponding to the jth target operator command as the jth reference task operator among the P reference task operators, where j is an integer greater than or equal to 1 and less than or equal to P;

[0232] The third control unit is further used to: control the j-th second thread to generate target task data for the j-th reference task operator.

[0233] In an exemplary embodiment, the second receiving module includes:

[0234] A first generating unit, used for generating a result synchronization instruction for a computing task;

[0235] A detection unit, used to detect whether the task execution progress of the computing task reaches the target progress;

[0236] a sending unit, configured to send a result synchronization instruction to the computing device when it is detected that the task execution progress reaches the target progress, wherein the computing device is configured to execute a target task operator corresponding to each target operator command among multiple target operator commands to be executed when each target operator command is executed, and return the operation results of the multiple target task operators currently being executed to the first thread when the result synchronization instruction is received;

[0237] The receiving unit is used to receive the target computing result returned by the computing device in response to the result synchronization instruction.

[0238] In an exemplary embodiment, the detection unit is used to:

[0239] Get the command execution progress of multiple target operator commands corresponding to the computing task;

[0240] Check whether the command execution progress reaches the target progress.

[0241] In an exemplary embodiment, the detection unit is further used to: detect whether the last target operator command among a plurality of to-be-executed target operator commands having a target execution order has been executed;

[0242] The detection unit is further used to: determine that the execution progress has reached the target progress when it is detected that the last target operator command has been executed; and determine that the execution progress has not reached the target progress when it is detected that the last target operator command has not been executed.

[0243] In an exemplary embodiment, the detection unit is further configured to:

[0244] Detecting a second semaphore of a last target operator command, wherein the second semaphore of the last target operator command is used to indicate a current execution state of the last target operator command;

[0245] In the case where it is detected that the execution state indicated by the second semaphore of the last target operator command is an execution completion state, determining that the last target operator command has been executed;

[0246] In the case where it is detected that the execution state indicated by the second semaphore of the last target operator command is a not-executed state, it is determined that the last target operator command is not executed.

[0247] In an exemplary embodiment, the apparatus further comprises:

[0248] An acquisition module, used for acquiring the thread configuration quantity of the second thread before calling multiple second threads to generate target task data for multiple reference task operators in multiple target task operators in parallel, wherein the thread configuration quantity is used to indicate the number of second threads currently allowed to be created by the task execution device;

[0249] The generating module is used to generate a second thread having a configured number of threads.

[0250] In an exemplary embodiment, the acquisition module includes:

[0251] A collection unit, used for collecting the target operator quantity of multiple target task operators;

[0252] A second generating unit is used to generate a target number of threads matching the target number of operators, wherein the target number of operators is positively correlated with the target number of threads;

[0253] The second determining unit is used to determine the target number of threads as the number of configured threads.

[0254] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0255] An embodiment of the present application also provides a computer program product, including a computer program, which implements the steps of the method in each embodiment of the present application when the computer program is executed by a processor; the computer program product also includes a non-volatile readable storage medium, which stores the computer program, and implements the steps of the method in each embodiment of the present application when the computer program is executed by the processor.

[0256] An embodiment of the present application further provides a non-volatile readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0257] In an exemplary embodiment, the above-mentioned non-volatile readable storage medium may include, but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0258] An embodiment of the present application further provides an electronic device, Figure 8 is a schematic diagram of an electronic device according to an embodiment of the present application, such as Figure 8 As shown, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0259] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0260] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0261] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0262] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for executing a computing task, characterized in that: A first thread and a plurality of second threads are created on a task execution device, and the method is applied to the first thread. The method includes: Receiving a task request, wherein the task request is used to request execution of a computing task; In response to the task request, converting the multiple target task operators of the computing task into multiple target operator commands to be executed with a target execution order, and calling multiple second threads in parallel to generate target task data for multiple reference task operators among the multiple target task operators, wherein the target execution order is the execution order of the multiple target task operators in the computing task, each of the target operator commands is used to control the computing device to execute one of the multiple target task operators, the task data is the data required when the corresponding task operator is executed, the task operator includes the reference task operator, and the task data includes the target task data; Execute the plurality of target operator commands according to the target task data in the target execution order, so that the execution idle parameters of the plurality of target task operators executed by the computing device are less than an idle parameter threshold; Receiving the target computing result returned by the computing device as the execution result of the computing task; The step of converting the multiple target task operators of the computing task into multiple target operator commands to be executed in a target execution order includes: Each of the target task operators in the plurality of target task operators is converted into a corresponding target operator command to be executed by the following steps: in the case where the plurality of target task operators is N target task operators, the i-th target task operator in the N target task operators is converted into the corresponding i-th target operator command to be executed by the following steps, where N is an integer greater than 1: the i-th target task operator is converted into the corresponding i-th initial operator command to be executed, where i is an integer greater than or equal to 1 and less than or equal to N; the i-th first signal is added to the context structure of the i-th initial operator command to be executed according to the target execution order; The i-th target operator command to be executed is obtained by comparing the i-th first semaphore and the i-th second semaphore, wherein the i-th second semaphore is used to indicate the current execution status of the i-th target operator command to be executed, and when i is greater than 1, the i-th first semaphore is used to indicate the current execution status of the i-1-th target operator command to be executed. In the target execution sequence, the i-1-th target operator command to be executed is the previous target operator command to be executed of the i-th target operator command to be executed, and the execution status is the execution completion status or the non-execution status. When i is equal to 1, the execution status indicated by the i-th first semaphore is the execution completion status.

2. The method according to claim 1, characterized in that The converting of the multiple target task operators of the computing task into multiple target operator commands to be executed in a target execution order includes: Obtaining the execution order of the multiple target task operators in the computing task to obtain the target execution order; Convert each of the target task operators in the plurality of target task operators into a corresponding target operator command to be executed, to obtain a plurality of target operator commands to be executed; Adding a plurality of the target operator commands to be executed to a first-in-first-out target command queue in sequence according to the target execution order.

3. The method according to claim 2, characterized in that The executing a plurality of target operator commands according to the target task data in accordance with the target execution order includes: Invoking a plurality of the second threads to obtain a plurality of reference operator commands from the target command queue; The plurality of second threads are controlled to execute the plurality of reference operator commands according to the first semaphore and the second semaphore in the plurality of reference operator commands.

4. The method according to claim 3, characterized in that The controlling the plurality of second threads to execute the plurality of reference operator commands according to the first semaphore and the second semaphore in the plurality of reference operator commands comprises: In the case where the plurality of second threads is M second threads, the kth second thread among the M second threads is controlled through the following steps to execute the kth reference operator command according to the kth first semaphore and the kth second semaphore in the kth reference operator command, wherein M is an integer greater than 1, the kth reference operator command is the reference operator command obtained by the kth second thread from the N target operator commands, and k is an integer greater than or equal to 1 and less than or equal to M: Detecting whether the execution state indicated by the kth first semaphore is the execution completion state; When it is detected that the execution state indicated by the kth first semaphore is the execution completion state, executing the kth reference operator command according to the target task data; When it is detected that the execution state indicated by the kth first semaphore is the unexecuted state, the execution of the kth reference operator command is prohibited.

5. The method according to claim 4, characterized in that After executing the kth reference operator command according to the target task data, the method further includes: The execution state indicated by the kth second semaphore is switched from an unexecuted state to an executed state.

6. The method according to claim 4, characterized in that The executing the kth reference operator command according to the target task data comprises: Obtaining the kth target task data of the kth target task operator, wherein the kth target task operator is the target task operator corresponding to the kth reference operator command among the N target task operators; Execute the kth reference operator command according to the kth target task data.

7. The method according to claim 6, characterized in that The executing the kth reference operator command according to the kth target task data comprises: Obtaining the kth operator function of the kth target task operator; The kth operator function and the kth target task data are transmitted to the computing device, wherein the computing device is used to execute the kth target task operator according to the kth operator function and the kth target task data.

8. The method according to claim 1, characterized in that The calling of the plurality of the second threads to generate target task data for the plurality of reference task operators in the plurality of the target task operators in parallel includes: In a case where the plurality of second threads is M second threads, controlling the M second threads to search in parallel whether an operator command exists in a target command queue created by the first thread, wherein the target command queue is used to store the target operator command to be executed, and M is an integer greater than 1; When P of the second threads among the M second threads find P of the target operator commands from the target command queue, determine the P of the target task operators corresponding to the P of the target operator commands as P reference task operators, where P is an integer greater than or equal to 1 and less than or equal to M; Control P of the second threads to generate target task data for P of the reference task operators in parallel.

9. The method according to claim 8, characterized in that The step of determining the P target task operators corresponding to the P target operator commands as P reference task operators when the P second threads among the M second threads find the P target operator commands from the target command queue comprises: determining the jth target task operator corresponding to the jth target operator command as the jth reference task operator among the P reference task operators when the jth second thread among the P second threads finds the jth target operator command from the target command queue, wherein j is an integer greater than or equal to 1 and less than or equal to P; The controlling P second threads to generate target task data for P reference task operators in parallel includes: controlling the jth second thread to generate the target task data for the jth reference task operator.

10. The method according to claim 1, characterized in that The receiving the target computing result returned by the computing device as the execution result of the computing task includes: generating a result synchronization instruction for the computing task; Detecting whether the task execution progress of the computing task reaches the target progress; When it is detected that the task execution progress reaches the target progress, the result synchronization instruction is sent to the computing device, wherein the computing device is configured to execute one target task operator corresponding to each target operator command among the multiple target operator commands to be executed when each target operator command is executed, and return the operation results of the multiple target task operators currently being executed to the first thread when the result synchronization instruction is received; Receive the target operation result returned by the operation device in response to the result synchronization instruction.

11. The method according to claim 10, characterized in that The detecting whether the task execution progress of the computing task reaches the target progress includes: Obtaining the command execution progress of the plurality of target operator commands corresponding to the computing task; Check whether the command execution progress reaches the target progress.

12. The method according to claim 11, characterized in that The obtaining of the command execution progress of the plurality of target operator commands corresponding to the computing task comprises: detecting whether the last target operator command among the plurality of target operator commands to be executed in the target execution order has been executed; The detecting whether the command execution progress reaches the target progress includes: when it is detected that the last target operator command has been executed, determining that the execution progress has reached the target progress; when it is detected that the last target operator command has not been executed, determining that the execution progress has not reached the target progress.

13. The method according to claim 12, characterized in that The detecting whether the last target operator command among the plurality of target operator commands to be executed in the target execution order has been executed includes: Detecting a second semaphore of the last target operator command, wherein the second semaphore of the last target operator command is used to indicate a current execution state of the last target operator command; In the case where it is detected that the execution state indicated by the second semaphore of the last target operator command is an execution completion state, determining that the last target operator command has been executed; When it is detected that the execution state indicated by the second semaphore of the last target operator command is a not-executed state, it is determined that the last target operator command is not executed.

14. The method according to claim 1, characterized in that Before calling the plurality of second threads to generate target task data for the plurality of reference task operators in the plurality of target task operators in parallel, the method further includes: Obtaining a thread configuration quantity of the second thread, wherein the thread configuration quantity is used to indicate the quantity of the second threads currently allowed to be created by the task execution device; Generate the configured number of second threads.

15. The method according to claim 14, characterized in that The obtaining the thread configuration quantity of the second thread includes: Collecting the target operator quantity of the plurality of target task operators; Generating a target number of threads matching the target number of operators, wherein the target number of operators is positively correlated with the target number of threads; The target number of threads is determined as the configured number of threads.

16. A computing task execution device, characterized in that: A first thread and a plurality of second threads are created on the task execution device, and the apparatus is applied to the first thread, and the apparatus includes: A first receiving module, used for receiving a task request, wherein the task request is used for requesting to execute a computing task; a response module, used to respond to the task request, convert the multiple target task operators of the computing task into multiple target operator commands to be executed with a target execution order, and call multiple second threads in parallel to generate target task data for multiple reference task operators in the multiple target task operators, wherein the target execution order is the execution order of the multiple target task operators in the computing task, each of the target operator commands is used to control the computing device to execute one of the multiple target task operators, the task data is the data required when the corresponding task operator is executed, the task operator includes the reference task operator, and the task data includes the target task data; an execution module, configured to execute the plurality of target operator commands according to the target task data in accordance with the target execution order, so that an execution idle parameter of the computing device executing the plurality of target task operators is less than an idle parameter threshold; A second receiving module, used for receiving the target computing result returned by the computing device as the execution result of the computing task; Wherein, the response module includes a conversion unit, and the conversion unit is further used to convert each of the multiple target task operators into a corresponding target operator command to be executed through the following steps: when the multiple target task operators are N target task operators, the i-th target task operator among the N target task operators is converted into the corresponding i-th target operator command to be executed through the following steps, where N is an integer greater than 1: converting the i-th target task operator into the corresponding i-th initial operator command to be executed, where i is an integer greater than or equal to 1 and less than or equal to N; sending the upper and lower commands of the i-th initial operator command to be executed according to the target execution order. The i-th first semaphore and the i-th second semaphore are added to the text structure to obtain the i-th target operator command to be executed, wherein the i-th second semaphore is used to indicate the current execution status of the i-th target operator command to be executed, and when i is greater than 1, the i-th first semaphore is used to indicate the current execution status of the i-1-th target operator command to be executed. In the target execution order, the i-1-th target operator command to be executed is the previous target operator command to be executed of the i-th target operator command to be executed, and the execution status is the execution completion status or the non-execution status. When i is equal to 1, the execution status indicated by the i-th first semaphore is the execution completion status.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 15 are implemented.

18. A non-volatile readable storage medium, characterized in that: The non-volatile readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 15 when executed by a processor.

19. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 15 are implemented.

Images