Task management method, task management device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a task management method, a task management device, electronic equipment and a storage medium. The task management method is executed by an artificial intelligence processor and comprises the steps that a first kernel function of the artificial intelligence processor obtains a message corresponding to a calculation result of a second kernel function of the artificial intelligence processor through a message passing channel in the artificial intelligence processor, and a task corresponding to the first kernel function is determined according to the message, the first kernel function is a subsequent kernel function executed depending on the calculation result of the second kernel function; the task corresponding to the first kernel function is executed, and the message passing channel is achieved based on the storage space of the artificial intelligence processor. By means of the method, the device can solve the problem of model dynamics under the condition that no host participates in, and the task execution efficiency and the utilization rate of hardware resources are improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of artificial intelligence, and in particular, to a task management method, a task management apparatus, an electronic device, and a storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence models, the runtime dynamics of models have become an important feature to adapt to complex application scenarios. In this context, device kernel function scheduling strategies based on static compilation are facing new adaptability challenges to better adapt to the dynamic changes of runtime operator morphology, data shape, and dependency relationships. Therefore, how to achieve efficient and flexible task scheduling and execution of kernel functions has become one of the technical problems to be solved at present. SUMMARY

[0003] At least one embodiment of the present disclosure provides a task management method executed by an artificial intelligence processor, comprising: a first kernel function of the artificial intelligence processor acquiring a message corresponding to a calculation result of a second kernel function of the artificial intelligence processor through a message passing channel in the artificial intelligence processor, and determining a task corresponding to the first kernel function according to the message, wherein the first kernel function is a successor kernel function dependent on the calculation result of the second kernel function; and executing the task corresponding to the first kernel function, wherein the message passing channel is implemented based on a storage space of the artificial intelligence processor.

[0004] In the task management method provided by at least one embodiment of the present disclosure, the storage space includes a global memory, and the message passing channel is located in a hypervisor workspace pre-allocated by a host in the global memory.

[0005] In the task management method provided by at least one embodiment of the present disclosure, the message passing channel is further used for message passing between kernel functions of different artificial intelligence processors, or message passing between kernel functions of the artificial intelligence processor and the host.

[0006] The task management method provided by at least one embodiment of the present disclosure further comprises: acquiring the message based on the calculation result of the second kernel function, and writing the message into the message passing channel.

[0007] In the task management method provided by at least one embodiment of the present disclosure, the second kernel function comprises a second kernel function hypervisor and a second kernel function user program, the second kernel function user program is used for executing calculation, and the second kernel function hypervisor is used for executing at least one of the following: task scheduling, memory allocation management.

[0008] In the task management method provided in at least one embodiment of the present disclosure, the message is obtained based on the calculation result of the second kernel function, and the message is written into the message passing channel, including: the second kernel function management program determines the message based on the calculation result of the second kernel function user program, and writes the message into the message passing channel.

[0009] In the task management method provided in at least one embodiment of the present disclosure, the message includes at least one of the following: branch selection result, memory size to be allocated, and the task corresponding to the first kernel function is determined according to the message, including at least one of the following: in response to the message including the branch selection result, the task corresponding to the first kernel function includes the calculation task of the target conditional branch corresponding to the branch selection result; in response to the message including the memory size to be allocated, the task corresponding to the first kernel function includes the dynamic memory allocation task.

[0010] In the task management method provided in at least one embodiment of the present disclosure, all potential calculation results of the second kernel function correspond to one or more conditional branches; the first kernel function includes a first kernel function management program and a first kernel function working program, and the first kernel function working program includes one or more first kernel function user programs, and the one or more first kernel function user programs correspond to the one or more conditional branches respectively.

[0011] In the task management method provided in at least one embodiment of the present disclosure, the first kernel function obtains the message through the message passing channel, and determines the task corresponding to the first kernel function according to the message, including: in response to the message existing in the message passing channel, the first kernel function management program reads the message from the message passing channel, and determines the task corresponding to the first kernel function according to the message, wherein the task management method further includes: the first kernel function management program writes the task corresponding to the first kernel function into a task queue.

[0012] In the task management method provided in at least one embodiment of the present disclosure, the task management method further includes: in response to determining that there is a to-be-processed task in the task queue, the first kernel function working program determines the task type of the to-be-processed task, wherein the task type includes at least one of the following: dynamic memory allocation task, exit task, and calculation task.

[0013] In the task management method provided in at least one embodiment of the present disclosure, the execution of the task corresponding to the first kernel function comprises: in response to the task type including a dynamic memory allocation task, the first kernel function management program divides required memory in a management program workspace; in response to the task type including an exit task, the first kernel function is terminated at a preset position; or in response to the task type including a calculation task, a first kernel function user program corresponding to a target conditional branch specified by the to-be-processed task is executed.

[0014] In the task management method provided in at least one embodiment of the present disclosure, the first kernel function worker determines the task type of the to-be-processed task, comprising: in response to the task type not including a dynamic memory allocation task and an exit task, determining that the task type includes a calculation task; or in response to the first kernel function management program having completed dividing required memory in the management program workspace and the task type not including an exit task, determining that the task type includes a calculation task.

[0015] The task management method provided in at least one embodiment of the present disclosure further comprises: in response to the first kernel function user program corresponding to the target conditional branch specified by the to-be-processed task having completed execution, the first kernel function worker determines whether there is a new to-be-processed task in the task queue.

[0016] In the task management method provided in at least one embodiment of the present disclosure, the task queue is located in a shared memory of the artificial intelligence processor.

[0017] The present disclosure provides a task management device applied to an artificial intelligence processor, comprising: a determination module configured to enable a first kernel function of the artificial intelligence processor to acquire a message corresponding to a calculation result of a second kernel function of the artificial intelligence processor through a message passing channel in the artificial intelligence processor, and determine a task corresponding to the first kernel function according to the message, wherein the first kernel function is a successor kernel function dependent on the calculation result of the second kernel function; and an execution module configured to execute the task corresponding to the first kernel function, wherein the message passing channel is implemented based on a storage space of the artificial intelligence processor.

[0018] The present disclosure provides an electronic device, comprising: at least one processor; at least one memory including one or more computer program modules; wherein the one or more computer program modules are stored in the at least one memory and configured to be executed by the at least one processor, and the one or more computer program modules are used to implement the method provided in at least one embodiment of the present disclosure.

[0019] The at least one embodiment of the present disclosure provides a non-transitory computer-readable storage medium, which stores computer-readable instructions, wherein the computer-readable instructions are executed by at least one processor to perform the method provided by the at least one embodiment of the present disclosure.

[0020] The task management method, the task management apparatus, the electronic device and the storage medium provided by at least one embodiment of the present disclosure can directly perform message passing between a plurality of kernel functions on the device side without the host, so that the device can solve the model dynamicity problem without the participation of the host, effectively avoid the time overhead caused by the interaction between the host and the device and the frequent starting of the kernel functions, and improve the task execution efficiency and the utilization rate of hardware resources.

[0021] Further, the device can dynamically select another kernel function to execute according to the result obtained by one kernel function without the participation of the host, or can dynamically apply a memory region at runtime according to the result obtained by one kernel function. That is, dynamic control flow and dynamic memory allocation are supported without the participation of the host, which can effectively improve the overall execution performance of the dynamic neural network model. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, but not limit the present disclosure.

[0023] Figure 1A A schematic block diagram of a monolithic system is provided for at least one embodiment of the present disclosure.

[0024] Figure 1A A schematic block diagram of a distributed system is provided for at least one embodiment of the present disclosure.

[0025] Figure 1A A schematic structural diagram of a general-purpose graphics processor is provided for at least one embodiment of the present disclosure.

[0026] Figure 2A A task scheduling method for control flow dynamicity is provided.

[0027] Figure 2A A task scheduling method for shape dynamicity of data dependency is provided.

[0028] Figure 3 A flowchart of a task management method is provided for at least one embodiment of the present disclosure.

[0029] Figure 4 A flowchart of another task management method is provided for at least one embodiment of the present disclosure.

[0030] Figure 5 A flowchart of another task management method provided in at least one embodiment of this disclosure.

[0031] Figure 6 This is a schematic block diagram of a task management device provided for at least one embodiment of the present disclosure.

[0032] Figure 7 This is a schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure.

[0033] Figure 8 This is a schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure.

[0034] Figure 9 This is a schematic block diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0036] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms “first,” “second,” and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as “comprising” or “including” mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as “connected” or “linked” are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as “upper,” “lower,” “left,” and “right” are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described objects changes.

[0037] The present disclosure will now be described through several specific embodiments. To keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and known components may be omitted. When any component of an embodiment of the present disclosure appears in more than one drawing, the component is represented by the same or similar reference numerals in each drawing.

[0038] Figure 1A This is a schematic block diagram of a monolithic system provided for at least one embodiment of the present disclosure.

[0039] For example, Figure 1A The monolithic system shown may include one or more hosts, each of which can directly connect to one or more devices via an internal bus or dedicated interface. This tightly coupled architecture enables the hosts to directly schedule device resources with extremely low communication latency, allowing them to work together to complete computing tasks.

[0040] Figure 1A This is a schematic block diagram of a distributed system provided for at least one embodiment of the present disclosure.

[0041] In distributed computing scenarios, computing nodes are the basic building blocks of distributed systems, enabling efficient parallel processing of large-scale tasks through collaboration and communication between nodes.

[0042] For example, Figure 1A The distributed system shown includes multiple computing nodes, each computing node may include one or more hosts, and each host may connect to one or more devices. Hosts and devices can collaborate to complete computing tasks. This disclosure does not limit the number of computing nodes included in the distributed system, nor the number of hosts and devices included in each computing node. Within a computing node, data transmission between hosts and devices can be achieved in various ways. Devices (also referred to as "slave devices") can either be directly connected to the host via a hardware interface (e.g., via a PCI Express, PCIe bus) to achieve low-latency data transmission, or they can collaborate with the host via a network in a software-defined manner (such as virtualization technology).

[0043] It should be noted that, Figure 1A As an example only, in the distributed system described above, each computing node may consist of only one host, and each host may be physically connected to only one device. Furthermore, the number of hosts in each node may be unequal, and the number of devices physically connected to each host may also be unequal.

[0044] For example, the host described above may include a central processing unit (CPU), and the device may include a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), an accelerated processing unit (APU), a neural network processing unit (NPU), an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA), etc.

[0045] Figure 1A This is a schematic structural diagram of a general-purpose graphics processor provided for at least one embodiment of the present disclosure.

[0046] like Figure 1A As shown, a general-purpose graphics processor is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a streaming processor cluster (SPC), such as including... Figure 1A The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In a general-purpose graphics processor, one streaming processor cluster processes one computational task, or multiple streaming processor clusters process one computational task. Multiple streaming processor clusters share data through a global cache or global memory. Global memory can be, for example, high-bandwidth memory (HBM).

[0047] like Figure 1A As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 1AThe system is structured as Computation Unit 1, Computation Unit 2, ..., Computation Unit N, where N is a positive integer. Each Computation Unit (CU) performs arithmetic and logical operations, such as accumulation, reduction, and standard addition, subtraction, multiplication, and division. A Computation Unit includes multiple Cores, each containing an Arithmetic Logic Unit (ALU), a floating-point unit, etc., and is used to execute specific computational tasks. Furthermore, the Computation Unit also includes registers (e.g., ...). Figure 1A The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared memory in a computing unit is used to share data between the cores of that computing unit.

[0048] like Figure 2A As shown, each streaming processor cluster also provides a buffer for caching data from the N computing units within the streaming processor cluster.

[0049] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processor (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 2A (Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0050] In each computing unit, there is a thread beam scheduling / distribution module (or thread beam scheduler). Figure 2A (Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple cores of the computing unit can run thread bundles. Multiple thread bundles can be scheduled by hardware on the same core without software overhead. Depending on the number of cores in the computing unit, multiple thread bundles in a thread block can be executed concurrently or in a time-sharing manner. Multiple threads in each thread bundle will execute the same instructions. Memory execution instructions are issued to shared memory in the computing unit or further issued to intermediate cache, global cache, or global memory for read and write operations, etc.

[0051] In modern computing paradigms, such as the field of artificial intelligence neural networks, some operators and model structures exhibit runtime dynamism. For example, their complete topology, execution path, and data format cannot be statically determined during the compilation phase and need to be determined at runtime based on specific input data or intermediate computation results.

[0052] The aforementioned dynamism manifests in various forms; the following two scenarios will be used as examples for illustration:

[0053] (1) Control flow dynamism refers to the fact that the behavior (execution path) of the model depends on the input data or intermediate computation results during runtime. For example, in the Mixture of Experts (MoE) model, specific expert subnetworks are dynamically selected and activated based on the semantic features of tokens; another example is dynamic branching logic based on conditional judgment (e.g., when the intermediate loss value exceeds a preset threshold, branch A is executed, otherwise branch B is executed).

[0054] (2) Data-dependent shape dynamics refers to the fact that the shape (dimensionality) of a tensor depends on the input data or intermediate computation results at runtime. For example, in image object detection tasks, the number of object bounding boxes output by the model depends on the actual number of objects in the image; or in natural language processing, changes in sequence length cause unpredictable changes in the dimensions of intermediate activation tensors.

[0055] It should be noted that the above are only examples of two types of dynamism. Those skilled in the art should understand that runtime dynamism can encompass many more aspects. For example, memory layout dynamism refers to the fact that due to the uncertainty of shape and structure, the optimal physical layout and alignment of tensors in memory cannot be statically planned at compile time and needs to be dynamically allocated and managed at runtime.

[0056] Existing high-performance optimization techniques, such as operator fusion and memory optimization, rely on the assumption of a static computation graph, meaning the complete graph structure, data shape, and control flow path are known at compile time. Faced with the aforementioned dynamism, traditional compilers struggle to construct a deterministic computation graph during the compilation phase. This makes it impossible to effectively implement operator fusion to eliminate redundant memory accesses in intermediate results, and also hinders the generation of highly optimized device code (e.g., GPU code). In extreme cases, traditional AI compilers that rely on static graph constraints may even directly report errors, limiting the deployment and application of dynamic models in efficient inference engines.

[0057] Figure 2A This is a task scheduling method designed for the dynamics of control flow.

[0058] For example, such as Figure 2A As shown, the task scheduling method includes steps S101 to S103.

[0059] In this method, to address the dynamics of control flow, it is necessary to pre-define all possible execution branches ( Figure 2A The example has n execution branches. The compiler generates the corresponding kernel function (e.g., GPU kernel function).

[0060] Step S101: The host receives the calculation results of the pre-kernel function user program on the device side.

[0061] Step S102: The host performs conditional judgment based on the calculation results to obtain the branch selection result.

[0062] Step S103: The host starts the corresponding pre-compiled successor kernel function according to the branch selection result.

[0063] exist Figure 2A In the example, the host selects the successor kernel function (marked with a pentagram) specified by the branch selection result from among several possible successor kernel functions to start and execute the user program in that successor kernel function.

[0064] The execution flow of the above method relies on high-frequency communication between the host and the device, as well as frequent kernel function startups. This mode not only consumes communication bandwidth and increases memory access latency, but also incurs significant time overhead due to the inherent scheduling time cost of kernel function startups, severely limiting the system's throughput and execution efficiency.

[0065] Figure 1A This is a task scheduling method for the shape dynamics of data dependencies.

[0066] For example, such as Figure 1A As shown, the task scheduling method includes steps S201 to S202.

[0067] In this method, a general but inefficient kernel function capable of handling arbitrary shapes can be generated to address the data-dependent shape dynamics. Figure 1A (The successor general kernel function in the series).

[0068] Step S201: The device-side pre-kernel function user program calculates the size of the tensor that the subsequent general kernel function needs to access and sends it to the host side.

[0069] Step S202: The host-side driver dynamically allocates memory at runtime based on the size of the tensor required by the subsequent generic kernel function. For example, a region can be allocated in the device hardware's global memory.

[0070] The execution process of the above method also relies on high-frequency communication between the host and the device, which consumes communication bandwidth, increases memory access latency, and severely restricts the system's throughput and execution efficiency.

[0071] To address the dynamic nature of data-dependent shapes, another task scheduling approach could involve pre-generating multiple versions of kernel functions for various possible shapes at compile time. However, this approach leads to a significant increase in code size and puts pressure on the instruction cache. Yet another task scheduling approach could include inserting runtime shape checks and recompiling on-premises when the shape changes; however, this approach introduces additional overhead from on-premises recompilation.

[0072] This disclosure provides at least one embodiment of a task management method, a task management device, an electronic device, and a storage medium.

[0073] The task management method provided in at least one embodiment of this disclosure is executed by an artificial intelligence processor, comprising: a first core function of the artificial intelligence processor obtaining a message corresponding to the calculation result of a second core function of the artificial intelligence processor through a message passing channel in the artificial intelligence processor, and determining a task corresponding to the first core function based on the message, wherein the first core function is a successor core function executed depending on the calculation result of the second core function; and executing the task corresponding to the first core function, wherein the message passing channel is implemented based on the storage space of the artificial intelligence processor.

[0074] Through the task management method provided in at least one embodiment of this disclosure, multiple kernel functions on the device side (artificial intelligence processor) can directly pass messages without going through the host. Therefore, the device can solve the model dynamics problem without the participation of the host, effectively avoiding the time overhead caused by the interaction between the host and the device and the frequent start of kernel functions, and improving the task execution efficiency and the utilization rate of hardware resources.

[0075] Furthermore, through the task management method provided in at least one embodiment of this disclosure, the device (artificial intelligence processor) can dynamically select and execute another kernel function based on the result calculated by one kernel function without host intervention, and can also dynamically allocate a memory region at runtime based on the result calculated by one kernel function. That is, it supports dynamic control flow and dynamic memory allocation without host intervention, which can effectively improve the overall execution performance of dynamic neural network models.

[0076] The task management method provided in at least one embodiment of this disclosure can be deployed on Figure 1A The single-unit system shown or Figure 3 The example shown is a distributed system. Of course, the application scenarios of this method are not limited to this; it can also be extended to other types of computer systems.

[0077] The task management method provided in at least one embodiment of this disclosure is executed by an artificial intelligence processor. The artificial intelligence processor may include a graphics processing unit (GPU) or a general-purpose graphics processing unit (see example). Figure 3This includes acceleration devices such as tensor processors, deep learning processors, accelerator processors, neural network processors, application-specific integrated circuits (ASICs), or field-programmable gate arrays (FPGAs). In other words, the artificial intelligence processor described in at least one embodiment of this disclosure is the device in the monolithic system or distributed system described above. Therefore, the terms "artificial intelligence processor" and "device" mentioned in the following description will be used synonymously.

[0078] Figure 1A This is a flowchart illustrating a task management method provided in at least one embodiment of the present disclosure. The task management method is executed by an artificial intelligence processor.

[0079] For example, such as Figure 1A As shown, at least one embodiment of the task management method provided in this disclosure includes steps S301 to S302.

[0080] Step S301: The first kernel function of the artificial intelligence processor obtains the message corresponding to the calculation result of the second kernel function of the artificial intelligence processor through the message passing channel in the artificial intelligence processor, and determines the task corresponding to the first kernel function according to the message. The first kernel function is a successor kernel function that depends on the calculation result of the second kernel function.

[0081] Step S302: Execute the task corresponding to the first kernel function.

[0082] In step S301, the message passing channel is implemented based on the storage space of the artificial intelligence processor. The various kernel functions of the artificial intelligence processor can communicate directly through this message passing channel without going through the host.

[0083] The relationship between the preceding kernel function and the succeeding kernel function has been described above; that is, the succeeding kernel function depends on the calculation result of the preceding kernel function for execution. In the embodiments of this disclosure, the first kernel function is used to refer to the succeeding kernel function, and the second kernel function is used to refer to the preceding kernel function corresponding to the first kernel function. It should be noted that "first kernel function" and "second kernel function" here are exemplary terms used to distinguish different kernel functions, and do not refer to a specific kernel function, nor do they indicate a specific order, hierarchical relationship, or number limitation between kernel functions.

[0084] For example, when the pre-kernel function starts, the host-side driver can pre-allocate a region in the AI ​​processor's memory space as a hypervisor workspace. The driver is a program running on the host side that can be used to start kernel functions, manage devices, and allocate memory. The message passing channel mentioned above can reside in the hypervisor workspace.

[0085] For example, the storage space described above may include the global memory of an AI processor. In some examples, the AI ​​processor may include global memory and one or more streaming processor clusters, as described in the relevant descriptions. Figure 4 For example, the global memory of an AI processor can support data sharing among multiple streaming processor clusters within that AI processor, data transfer between the host and the AI ​​processor, and data transfer between the AI ​​processor and other AI processors. Correspondingly, the message passing channel can reside within a hypervisor workspace pre-allocated by the host in the global memory of the AI ​​processor.

[0086] The aforementioned message passing channels can be used not only for message passing between kernel functions within the same AI processor, but also for message passing between kernel functions of different AI processors, or between a kernel function of an AI processor and the host. That is, through the message passing channels in an AI processor, a kernel function within that AI processor can communicate with other kernel functions or the host. These other kernel functions can reside in different streams within the same AI processor, or they can reside in different AI processors. The streams mentioned above are a series of sequentially scheduled operations within the AI ​​processor. Specifically, multiple kernel functions belonging to the same stream are executed serially, while kernel functions belonging to different streams can be executed in parallel.

[0087] In step S301, the "message corresponding to the calculation result of the second kernel function" refers to an intermediate data carrier generated by the preceding kernel function and used to pass control information or configuration parameters to the subsequent kernel function. This message is generated based on the final or intermediate calculation result of the preceding kernel function and is used to dynamically guide the execution strategy of the subsequent kernel function.

[0088] Specifically, the content of the message can take various forms. In one example, when the preceding kernel function needs to determine the target conditional branch from multiple candidate conditional branches, the message may include the branch selection result, which is used to explicitly indicate to the subsequent kernel function the specific conditional branch to be executed. In another example, when the preceding kernel function is used for data preprocessing or computational scale estimation, its computation result may be the size of the tensor that the subsequent kernel function needs to access, and the message may include the size of the memory to be allocated, indicating the amount of memory resources that the subsequent kernel function needs to dynamically request. In some examples, the message may include both the branch selection result and the size of the memory to be allocated, and this disclosure does not limit this.

[0089] It should be understood that all kinds of indication information generated by the preceding kernel function and used to trigger or configure the subsequent kernel function task can be regarded as falling within the scope of "messages corresponding to calculation results" as described in the embodiments of this disclosure, and the embodiments of this disclosure do not limit this.

[0090] An example of "determining the task corresponding to the first kernel function based on the message" in step S301 may include: in response to the message including the branch selection result, determining the task corresponding to the first kernel function includes the calculation task of the target condition branch corresponding to the branch selection result.

[0091] Another example of "determining the task corresponding to the first kernel function based on the message" in step S301 may include: in response to the message including the size of the memory to be allocated, determining that the task corresponding to the first kernel function includes a dynamically allocated memory task.

[0092] For example, when a message carries a branch selection result, the task corresponding to the determined successor kernel function includes: executing the computation task under the target conditional branch corresponding to the branch selection result. For example, when a message carries the size of memory to be allocated, the task corresponding to the determined successor kernel function includes: performing dynamic allocation of memory resources based on the memory size information. When a message carries both a branch selection result and the size of memory to be allocated, the task corresponding to the determined successor kernel function can include both of the above tasks simultaneously.

[0093] In this way, the preceding kernel function and the succeeding kernel function can communicate directly through the message passing channel in the AI ​​processor. The succeeding kernel function only needs to read the message from the message passing channel to determine the task indicated by the preceding kernel function, without host intervention, effectively improving task execution efficiency. Furthermore, the above method can effectively support the dynamics of the model's control flow and the shape dynamics of data dependencies without host intervention, providing streamlined task scheduling and memory allocation functions.

[0094] It should be noted that the above embodiments regarding message types of "branch selection result" or "memory size to be allocated" are merely illustrative examples of the technical solutions disclosed herein and are not intended to limit the message content and the method of determining its corresponding task.

[0095] In other embodiments of this disclosure, the message may also carry various forms of control information or configuration parameters to dynamically adapt to the execution strategy of the subsequent kernel function. For example, the message may include the valid data length or the data starting address, in which case determining the task corresponding to the subsequent kernel function includes tasks of data access or data movement based on the valid data length or the data starting address. As another example, the message may include calculation parameters, in which case determining the task corresponding to the subsequent kernel function includes tasks of executing corresponding calculation logic according to the calculation parameters. Yet another example, the message may include a data layout transformation instruction, in which case determining the task corresponding to the subsequent kernel function includes data rearrangement operations corresponding to the data layout transformation instruction.

[0096] It should be understood that any scheme that uses messages generated by a preceding kernel function to dynamically guide subsequent kernel functions to perform specific computational tasks, resource scheduling, or process control can be considered to fall within the scope of "determining the task corresponding to the first kernel function based on the message" as described in the embodiments of this disclosure, and the embodiments of this disclosure do not impose any limitations on this.

[0097] Furthermore, the message may also include a termination instruction to dynamically truncate the computation process. Specifically, when the preceding kernel function detects that certain preset conditions are met during execution, such as the iterative algorithm converging to the target precision, data verification finding irreparable errors, or determining that subsequent computations do not contribute to the final result, the generated message may include a signal indicating termination of execution. In response to the message including the termination instruction, the task corresponding to the subsequent kernel function can be determined to skip preset computation logic, or the subsequent kernel function can be directly triggered to terminate execution early. Through the above-described message-based execution termination mechanism, the embodiments of this disclosure can effectively avoid unnecessary consumption of computing resources (e.g., GPU computing power) and storage bandwidth in scenarios where it is not necessary to continue executing subsequent computations, thereby improving the execution efficiency and resource utilization of heterogeneous computing systems.

[0098] Prior to step S301, the task management method provided in at least one embodiment of this disclosure may further include step S303.

[0099] Step S303: Obtain the message based on the calculation result of the second kernel function and write the message into the message passing channel.

[0100] In step S303, the "message" is the "message corresponding to the calculation result of the second kernel function (preceding kernel function)" mentioned above.

[0101] In some examples, the second kernel function includes a second kernel function manager and a second kernel function user program. The second kernel function user program is used to perform computations, and the second kernel function manager is used to perform at least one of the following: task scheduling and memory allocation management.

[0102] In other words, the pre-kernel function includes a pre-kernel function management program and a pre-kernel function user program. The pre-kernel function user program is used to carry out and execute specific, domain-related routine computing tasks, such as data preprocessing, feature extraction, and intermediate result iteration. This disclosure does not limit this. The pre-kernel function management program can be used to implement at least one of task scheduling and memory allocation management, or it can be used to implement other tasks.

[0103] In some examples, all potential computation results of the second kernel function correspond to one or more conditional branches; the first kernel function includes a first kernel function manager and a first kernel function worker, the first kernel function worker including one or more first kernel function user programs, each of which corresponds to one or more conditional branches. The first kernel function manager is used to perform at least one of the following: task scheduling, memory allocation management.

[0104] In other words, a successor kernel function includes a successor kernel function manager and a successor kernel function worker. The successor kernel function worker includes one or more successor kernel function user programs. For example, all possible computation results of the preceding kernel function correspond to one or more conditional branches, and the successor kernel function worker is the collection of all successor kernel function user programs prepared for all possible conditional branches.

[0105] In one example, the preceding kernel function corresponds to three branches (branches A through C). If the calculation result of the preceding kernel function is greater than a preset threshold, it is determined that the subsequent branch will be A; if the calculation result is equal to the preset threshold, it is determined that the subsequent branch will be B; and if the calculation result is less than the preset threshold, it is determined that the subsequent branch will be C. In the successor kernel function corresponding to the preceding kernel function, the successor kernel function's working program integrates successor kernel function user program 1, successor kernel function user program 2, and successor kernel function user program 3, which are used to execute the data processing operations corresponding to branches A, B, and C, respectively. It should be noted that the foregoing description of the branch determination conditions and corresponding program scheduling is only a specific implementation example. In practical applications, the setting of the determination conditions, the number of branches, and the mapping relationship of the user programs can all be adjusted according to actual needs, and this disclosure embodiment does not impose any limitations on this.

[0106] In artificial intelligence processors, each kernel function can embed a kernel function manager to provide streamlined task scheduling and memory allocation. Since kernel functions are essentially executed in parallel by multiple thread bundles, a dedicated thread bundle can be designated to carry and run the kernel function manager. A thread bundle scheduler then performs round-robin scheduling between the kernel function manager and the kernel function user program, enabling them to execute collaboratively.

[0107] At least one embodiment of this disclosure achieves coordinated scheduling of kernel function management logic and user program computation logic by embedding a kernel function management program inside a conventional kernel function, thereby reducing kernel function startup overhead and improving hardware resource utilization and task execution flexibility.

[0108] An example of step S303 may include: the second kernel function manager determining a message based on the calculation result of the second kernel function user program, and writing the message into the message passing channel.

[0109] For example, the pre-kernel function manager can receive the calculation results of the pre-kernel function user program and determine the message based on the calculation results. For example, the pre-kernel function manager can determine the subsequent target conditional branch based on the calculation results of the pre-kernel function user program and generate a message carrying the branch selection result. For example, the pre-kernel function manager can determine the size of memory to be dynamically allocated for the subsequent kernel function (i.e., the memory size to be allocated mentioned above) based on the calculation results of the pre-kernel function user program and generate a message carrying the size of the memory to be allocated. For example, the pre-kernel function manager can also generate a message carrying an execution termination instruction based on the calculation results of the pre-kernel function user program. The embodiments of this disclosure are not limited to the above examples; other possible forms of messages have been described above and will not be repeated here.

[0110] For example, a preceding kernel manager can write a defined message to a message passing channel for subsequent kernel managers to read from the message passing channel.

[0111] By using the above method, the work of determining the target condition branch and dynamically allocating memory can be executed directly on the device side without the need for host intervention, reducing unnecessary host-device communication and effectively improving task execution efficiency and hardware resource utilization.

[0112] An example of step S301 may include: in response to the presence of a message in the message passing channel, the first kernel function manager reads the message from the message passing channel and determines the task corresponding to the first kernel function based on the message.

[0113] For example, the successor kernel manager can poll the message passing channel until a message is obtained from it. The specific implementation method for determining the task based on the message has been described above and will not be repeated here. For example, the task determined by the successor kernel manager can carry various information, such as task type, corresponding branch flag, size of memory to be allocated, program termination flag, or program termination timing, etc., and this disclosure does not limit this.

[0114] For example, after determining the task corresponding to the successor kernel function, the successor kernel function manager can write the task corresponding to the successor kernel function into the task queue for the successor kernel function worker to retrieve. Each streaming processor cluster of the artificial intelligence processor in at least one embodiment of this disclosure may include one or more computing units, each computing unit may include shared memory and one or more cores, as described in the relevant description. Figure 4 For example, the host can pre-allocate a region in the shared memory of the AI ​​processor as a task queue; that is, the task queue can reside in the shared memory of the AI ​​processor. The shared memory is configured to support data interaction and sharing between multiple cores within the same computing unit. The kernel functions described above are executed by the cores; a single kernel function can be executed independently by a single core or collaboratively by multiple cores.

[0115] The task management method provided in at least one embodiment of this disclosure may further include: a first kernel function worker program checking whether there are any pending tasks in the task queue; in response to determining that there are pending tasks in the task queue, the first kernel function worker program determining the task type of the pending tasks.

[0116] For example, the subsequent kernel function worker checks if there is a new task in the task queue. If so, it removes the task from the task queue and determines the task type.

[0117] For example, task types can include at least one of the following: dynamic memory allocation task, exit task, and computation task. For instance, a dynamic memory allocation task indicates that the successor kernel function needs to dynamically allocate memory based on the size of the memory to be allocated; an exit task indicates that the successor kernel function needs to terminate at a specified location; and a computation task indicates that the successor kernel function needs to execute the corresponding successor kernel function user program based on a determined target condition branch. It should be noted that a single task can contain multiple task types. For example, a task's task type can be configured as both "dynamic memory allocation task" and "exit task," meaning that the successor kernel function will terminate directly after the dynamic memory allocation operation is completed. The successor kernel function can then execute the corresponding task based on the task type defined above.

[0118] In some examples, when the task type does not include dynamically allocated memory tasks and exit tasks, it can be determined that the task type includes computational tasks.

[0119] In other examples, when the first kernel function manager has completed allocating the required memory in the manager's workspace and the task type does not include exit tasks, it can be determined that the task type includes computation tasks.

[0120] For example, one embodiment of step S302 may include: in response to a task type including a dynamically allocated memory task, the first kernel function manager allocates the required memory in the manager's workspace. The concept of the manager's workspace has been introduced above and will not be repeated here. For example, when a subsequent kernel function worker determines that the task type includes a dynamically allocated memory task, it can notify the subsequent kernel function manager to perform memory allocation. For example, when a subsequent kernel function worker determines that the task type includes a dynamically allocated memory task, the subsequent kernel function manager can obtain the size of the memory to be allocated based on the message and allocate a portion of the host-pre-allocated manager's workspace as working memory.

[0121] For example, one embodiment of step S302 may include terminating the first kernel function at a preset position in response to the task type including an exit task. For example, when the determined task type includes both an exit task and a dynamically allocated memory task, the preset position is the position after the dynamic memory allocation operation is completed. For example, when the determined task type only includes an exit task, the preset position is the current position. It should be noted that the above examples are merely illustrative, and the preset position can be set according to actual needs; this disclosure does not impose any limitations on this.

[0122] For example, one embodiment of step S302 may include: in response to the task type including a computation task, executing the first kernel function user program corresponding to the target conditional branch specified by the task to be processed. For example, the first kernel function user program corresponding to the target conditional branch refers to the post-kernel function user program that needs to be actually executed, selected from the first kernel function working program based on the computation result of the second kernel function.

[0123] In some examples, the task management method provided in this disclosure may further include: in response to the completion of execution of the first kernel function user program corresponding to the target condition branch specified by the task to be processed, the first kernel function worker program determines whether there is a new task to be processed in the task queue.

[0124] In other words, during the execution of the current task, new tasks may be added to the task queue. After the current task is completed, the subsequent kernel function worker can obtain new tasks to be processed and continue execution.

[0125] Figure 3 A flowchart of another task management method provided for at least one embodiment of this disclosure. Figure 4 yes Figure 4 A specific example of the described task management method.

[0126] For example, such as Figure 5 As shown, another task management method provided by at least one embodiment of this disclosure may include steps S401 to S411.

[0127] Step S401: The pre-kernel function management program determines the message based on the calculation result of the pre-kernel function user program.

[0128] Step S402: The pre-kernel function management program writes the message to the message passing channel.

[0129] Step S403: The subsequent kernel function management program polls the message passing channel and obtains messages.

[0130] Step S404: The successor kernel function management program determines the tasks that the successor kernel function needs to execute based on the message and writes them into the task queue.

[0131] Step S405: The subsequent kernel function worker checks if there are any pending tasks in the task queue. If yes, proceed to step S406; otherwise, return to step S405.

[0132] Step S406: The subsequent kernel function worker determines the task type of the task to be processed.

[0133] Step S407: The subsequent kernel function worker determines whether the task type includes dynamically allocated memory tasks. If yes, proceed to step S408; otherwise, proceed to step S409.

[0134] Step S408: The subsequent kernel function manager allocates the required memory in the manager's workspace and continues to execute step S409.

[0135] Step S409: The subsequent kernel function worker determines whether the task type includes an exit task. If yes, proceed to step S410; otherwise, proceed to step S411.

[0136] Step S410: Terminate the successor kernel function.

[0137] Step S411: Execute the user program of the successor kernel function corresponding to the target condition branch specified by the task to be processed, and return to step S405.

[0138] about Figure 5 For a description of each step, please refer to the relevant descriptions in the above embodiments, which will not be repeated here.

[0139] Through the task management method provided in at least one embodiment of this disclosure, an artificial intelligence processor (device) can dynamically select and execute another kernel function based on the result calculated by one kernel function without host intervention, and can also dynamically allocate a memory region at runtime based on the result calculated by one kernel function. That is, it supports dynamic control flow and dynamic memory allocation without host intervention, which can effectively improve the overall execution performance of dynamic neural network models.

[0140] Figure 5 A flowchart of another task management method provided in at least one embodiment of this disclosure.

[0141] In this example, the host-side driver is only used to pre-allocate the hypervisor workspace and task queue, and does not participate in the device-side task scheduling and dynamic memory allocation process. The device-side pre-kernel functions and successor kernel functions pass messages through global memory in the device hardware. The pre-kernel function includes the pre-kernel function manager and the pre-kernel function application; the successor kernel function includes the successor kernel function manager and the successor kernel function worker. The successor kernel function worker is a collection of all successor kernel function user programs prepared for all possible conditional branches. The successor kernel function manager can write tasks to the task queue pre-allocated in the device hardware's shared memory, which are then read by the successor kernel function worker.

[0142] Another task management method provided in at least one embodiment of this disclosure includes steps S501 to S511 ( Figure 5 Only steps S501 to S508 are shown.

[0143] Step S501: The host-side driver pre-allocates a region in the device-side global memory as the management program's workspace, and pre-allocates a region in the device-side shared memory as the task queue.

[0144] Step S502: The user program of the pre-kernel function performs the calculation to obtain the calculation result and passes it to the pre-kernel function management program.

[0145] Step S503: The pre-kernel function management program determines the message based on the calculation result of the pre-kernel function user program and writes the message into the message passing channel in the management program's workspace.

[0146] Step S504: The subsequent kernel function management program polls the message passing channel.

[0147] Step S505: The subsequent kernel function manager reads messages from the message passing channel.

[0148] Step S506: The successor kernel function management program determines the tasks that the successor kernel function needs to execute based on the message and writes them into the task queue.

[0149] Step S507: The subsequent kernel function worker checks whether there are any pending tasks in the task queue. If so, it determines the task type of the pending task.

[0150] Step S508: If the subsequent kernel function worker determines that the task type includes a dynamically allocated memory task, the subsequent kernel function manager allocates the required memory in the manager's workspace.

[0151] Step S509: If the successor kernel function worker determines that the task type includes dynamic memory allocation tasks and exit tasks, then after the successor kernel function manager allocates the required memory in the manager's workspace, the successor kernel function is terminated.

[0152] Step S510: If the successor kernel function worker determines that the task type includes dynamically allocated memory tasks but does not include exit tasks, then after the successor kernel function manager allocates the required memory in the manager's workspace, it executes the successor kernel function user program corresponding to the target condition branch specified by the task to be processed, and returns to step S507.

[0153] Step S511: If the successor kernel function worker determines that the task type does not include dynamically allocated memory tasks and exit tasks, execute the successor kernel function user program corresponding to the target condition branch specified by the task to be processed, and return to step S507.

[0154] For example, in Figure 5 In the example, the user program that follows the target conditional branch specified by the task to be processed is the kernel function. Figure 6 The user program that marks the successor kernel function with a pentagram symbol.

[0155] about Figure 6 For a description of each step, please refer to the relevant descriptions in the above embodiments, which will not be repeated here.

[0156] It should also be noted that the execution order of the steps of the task management method is not limited in the various embodiments of this disclosure. Although the execution process of each step has been described in a specific order above, this does not constitute a limitation on the embodiments of this disclosure. The steps in the task management method can be executed sequentially or in parallel, depending on actual needs.

[0157] For example, compared to the above description, the task management method provided in at least one embodiment of this disclosure may include more or fewer steps, and the embodiments of this disclosure do not limit this.

[0158] Figure 7 This is a schematic block diagram of a task management device provided for at least one embodiment of the present disclosure. The task management device can be applied to an artificial intelligence processor, for example, it can be disposed within the artificial intelligence processor.

[0159] For example, such as Figure 7 As shown, the task scheduling device 600 provided in at least one embodiment of this disclosure includes a determination module 601 and an execution module 602.

[0160] For example, the determining module 601 is configured to enable the first core function of the artificial intelligence processor to obtain the message corresponding to the calculation result of the second core function of the artificial intelligence processor through the message passing channel in the artificial intelligence processor, and determine the task corresponding to the first core function according to the message. Here, the first core function is a successor core function that depends on the calculation result of the second core function and is executed. The message passing channel is implemented based on the storage space of the artificial intelligence processor.

[0161] For example, execution module 602 is configured to execute the task corresponding to the first kernel function.

[0162] In some examples, the storage space includes global memory, and the messaging channel resides within the hypervisor workspace pre-allocated by the host in global memory.

[0163] In some examples, message passing channels are also used for message passing between kernel functions of different AI processors, or between the kernel function of an AI processor and the host.

[0164] In some examples, the task management device also includes an acquisition module. This acquisition module is configured to acquire a message based on the computation result of a second kernel function and write the message to a message passing channel.

[0165] In some examples, the second kernel function includes a second kernel function manager and a second kernel function user program. The second kernel function user program is used to perform computations, and the second kernel function manager is used to perform at least one of the following: task scheduling and memory allocation management.

[0166] In some examples, the acquisition module is configured to cause the second kernel function manager to determine the message based on the computation result of the second kernel function user program and write the message to the message passing channel.

[0167] In some examples, the message includes at least one of the following: branch selection result, size of memory to be allocated, and the determining module is further configured to implement at least one of the following: in response to the message including the branch selection result, determining the task corresponding to the first kernel function includes the computation task of the target condition branch corresponding to the branch selection result; in response to the message including the size of memory to be allocated, determining the task corresponding to the first kernel function includes the task of dynamically allocating memory.

[0168] In some examples, all the potential computation results of the second kernel function correspond to one or more conditional branches.

[0169] The first kernel function includes a first kernel function management program and a first kernel function working program. The first kernel function working program includes one or more first kernel function user programs, and the one or more first kernel function user programs correspond to one or more conditional branches.

[0170] In some examples, the determination module is configured to, in response to the presence of a message in the message passing channel, cause the first kernel function manager to read the message from the message passing channel and determine the task corresponding to the first kernel function based on the message; and cause the first kernel function manager to write the task corresponding to the first kernel function into the task queue.

[0171] In some examples, the determination module is configured to, in response to the determination that there are pending tasks in the task queue, cause the first kernel function worker to determine the task type of the pending task, wherein the task type includes at least one of the following: dynamically allocated memory task, exit task, computation task.

[0172] In some examples, the execution module is further configured to, in response to a task type including a dynamically allocated memory task, cause the first kernel function manager to allocate the required memory in the manager's workspace; in response to a task type including an exit task, terminate the first kernel function at a preset location; or in response to a task type including a computation task, execute the first kernel function user program corresponding to the target conditional branch specified by the task to be processed.

[0173] In some examples, the determination module is further configured to: determine that the task type includes a computation task in response to the task type not including dynamically allocated memory tasks and exit tasks; or determine that the task type includes a computation task in response to the first kernel function manager having completed allocating the required memory in the manager workspace and the task type not including exit tasks.

[0174] In some examples, the determination module is further configured to: in response to the completion of execution of the first kernel function user program corresponding to the target condition branch specified by the pending task, cause the first kernel function worker program to determine whether there are new pending tasks in the task queue.

[0175] In some examples, the task queue resides in the shared memory of the AI ​​processor.

[0176] It should be noted that the above-mentioned modules can be implemented by software, hardware, firmware or any combination thereof. For example, the determination module can be implemented as a determination circuit, and the execution module can be implemented as an execution circuit. The embodiments of this disclosure do not limit their specific implementation methods.

[0177] It should be understood that the task management device provided in at least one embodiment of this disclosure can be used to implement the aforementioned task management method and can also achieve similar technical effects as the aforementioned task management method, which will not be elaborated here.

[0178] It should be noted that, in the embodiments of this disclosure, the task management device may include more or fewer modules or units, and the connection relationship between the various modules or units is not limited and can be determined according to actual needs. The specific configuration of each module or unit is not limited; it can be constructed from analog devices according to circuit principles, or from digital chips, or in other suitable ways.

[0179] Figure 8 This is a schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure.

[0180] For example, such as Figure 8 As shown, the electronic device 700 includes at least one processor 701 and at least one memory 702. The at least one memory 702 includes one or more computer program modules. These computer program modules are stored in the memory 702 and configured to be executed by the at least one processor 701. The one or more computer program modules include instructions for performing the task management method described above. When executed by the at least one processor 701, they can perform one or more steps of the task management method provided in at least one embodiment of this disclosure. The memory 702 and the processor 701 can be interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0181] For example, processor 701 can be a central processing unit (CPU), digital signal processor (DSP), graphics processing unit (GPU), general-purpose graphics processing unit (GPGPU), artificial intelligence (AI) accelerator, or other form of processing unit with data processing and / or program execution capabilities, such as a field-programmable gate array (FPGA); for example, the central processing unit (CPU) can be an x86, ARM, or RISC-V architecture. Processor 701 can be a general-purpose processor or a special-purpose processor, capable of controlling other components in electronic device 700 to perform desired functions.

[0182] For example, memory 702 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc.

[0183] Figure 8 This is a schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure.

[0184] The electronic devices in at least one embodiment of this disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, and fixed terminals such as digital TVs and desktop computers. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0185] The electronic device includes at least one processor and a memory. The processor may be referred to as processing device 801 as described below, and the memory may include at least one of ROM 802, RAM 803, and storage device 808 as described below. The memory is used to store programs for performing the methods described in the various method embodiments above; the processor is configured to execute the programs stored in the memory. The processor may include a central processing unit (CPU) or other forms of processing unit having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0186] like Figure 9 As shown, the electronic device 800 may include a processing unit 801 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in ROM 802 or a program loaded from storage device 808 into RAM 803. RAM 803 also stores various programs and data required for the operation of the electronic device 800. The processing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interfaces are also connected to bus 804.

[0187] Typically, the following devices can be connected to I / O interface 805: input devices 806 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 807 including, for example, displays, speakers, vibrators, etc.; storage devices 808 including, for example, magnetic tapes, hard disks, etc.; and communication devices 809. Communication device 809 allows electronic device 800 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 9 An electronic device 800 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0188] In particular, according to at least one embodiment of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, at least one embodiment of this disclosure includes a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by a processing device 801, it performs the functions defined in the methods of at least one embodiment of this disclosure.

[0189] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In at least one embodiment of this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In at least one embodiment of this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, radio frequency (RF), etc., or any suitable combination thereof.

[0190] The aforementioned computer-readable medium may be included in the aforementioned electronic device 800; or it may exist independently and not assembled into the electronic device 800.

[0191] Figure 7 This is a schematic block diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure.

[0192] For example, such as ​ As shown, a non-transitory computer-readable storage medium 900 stores computer-readable instructions 901, which, when executed by at least one processor, perform one or more steps of the task management method described above.

[0193] For example, the storage medium may include a memory card for a smartphone, a storage component for a tablet computer, a hard drive for a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media. For example, the readable storage medium may also be... ​ The memory 702 in the memory is described in the foregoing content and will not be repeated here.

[0194] Although the present disclosure has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to the embodiments of the present disclosure, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present disclosure are within the scope of protection claimed by the present disclosure.

[0195] The following points should be noted regarding this disclosure:

[0196] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0197] (2) For clarity, the thickness of layers or regions in the drawings used to describe embodiments of the present disclosure is enlarged or reduced, i.e., these drawings are not drawn to actual scale.

[0198] (3) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0199] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure should be determined by the scope of protection of the claims.

Claims

1. A task management method, characterized in that, The task management method is executed by an artificial intelligence processor and includes: The first kernel function of the artificial intelligence processor obtains the message corresponding to the calculation result of the second kernel function of the artificial intelligence processor through the message passing channel in the artificial intelligence processor, and determines the task corresponding to the first kernel function according to the message. The first kernel function is a successor kernel function that depends on the calculation result of the second kernel function. Execute the task corresponding to the first kernel function. The message transmission channel is implemented based on the storage space of the artificial intelligence processor.

2. The task management method according to claim 1, characterized in that, The storage space includes global memory, and the message passing channel is located in the management program workspace pre-allocated by the host in the global memory.

3. The task management method according to claim 2, characterized in that, The message passing channel is also used for message passing between kernel functions of different artificial intelligence processors, or for message passing between the kernel function of the artificial intelligence processor and the host.

4. The task management method according to claim 1, characterized in that, The task management method also includes: The message is obtained based on the calculation result of the second kernel function, and the message is written into the message passing channel.

5. The task management method according to claim 4, characterized in that, The second kernel function includes a second kernel function manager and a second kernel function user program. The second kernel function user program is used to perform calculations. The second kernel function manager is used to perform at least one of the following: task scheduling, memory allocation management.

6. The task management method according to claim 5, characterized in that, The step of obtaining the message based on the calculation result of the second kernel function and writing the message into the message transmission channel includes: The second kernel function management program determines the message based on the calculation result of the second kernel function user program, and writes the message into the message passing channel.

7. The task management method according to claim 1, characterized in that, The message includes at least one of the following: branch selection result, size of memory to be allocated, Determining the task corresponding to the first kernel function based on the message includes at least one of the following: In response to the message including the branch selection result, it is determined that the task corresponding to the first kernel function includes the calculation task of the target condition branch corresponding to the branch selection result; In response to the message including the size of the memory to be allocated, the task corresponding to the first kernel function is determined to include a dynamic memory allocation task.

8. The task management method according to claim 1, characterized in that, All potential computation results of the second kernel function correspond to one or more conditional branches; The first kernel function includes a first kernel function manager and a first kernel function worker. The first kernel function worker includes one or more first kernel function user programs, and the one or more first kernel function user programs correspond to the one or more conditional branches.

9. The task management method according to claim 8, characterized in that, The first kernel function obtains the message through the message passing channel and determines the task corresponding to the first kernel function based on the message, including: In response to the presence of the message in the message passing channel, the first kernel function management program reads the message from the message passing channel and determines the task corresponding to the first kernel function based on the message. The task management method further includes: The first kernel function management program writes the task corresponding to the first kernel function into the task queue.

10. The task management method according to claim 9, characterized in that, The task management method also includes: In response to determining that there are pending tasks in the task queue, the first kernel function worker determines the task type of the pending task, wherein the task type includes at least one of the following: dynamically allocated memory task, exit task, and computation task.

11. The task management method according to claim 10, characterized in that, The task corresponding to executing the first kernel function includes: In response to the task type including dynamically allocated memory tasks, the first kernel function manager allocates the required memory in the manager's workspace; In response to the task type including exiting the task, the first kernel function is terminated at a preset location; or In response to the task type including a computational task, the first kernel function user program corresponding to the target conditional branch specified by the task to be processed is executed.

12. The task management method according to claim 10, characterized in that, The first kernel function worker determines the task type of the task to be processed, including: In response to the fact that the task type does not include dynamically allocated memory tasks and exit tasks, the task type is determined to include computation tasks; or In response to the fact that the first kernel function manager has completed the allocation of the required memory in the manager's workspace, and the task type does not include exit tasks, the task type is determined to include computation tasks.

13. The task management method according to claim 11, characterized in that, The task management method also includes: In response to the completion of the execution of the first kernel function user program corresponding to the target condition branch specified by the task to be processed, the first kernel function worker program determines whether there are any new tasks to be processed in the task queue.

14. The task management method according to claim 9, characterized in that, The task queue is located in the shared memory of the artificial intelligence processor.

15. A task management device, characterized in that, The task management device is applied to an artificial intelligence processor and includes: The determination module is configured to enable the first kernel function of the artificial intelligence processor to obtain a message corresponding to the calculation result of the second kernel function of the artificial intelligence processor through the message passing channel in the artificial intelligence processor, and determine the task corresponding to the first kernel function according to the message, wherein the first kernel function is a successor kernel function that depends on the calculation result of the second kernel function for execution; The execution module is configured to execute the task corresponding to the first kernel function. The message transmission channel is implemented based on the storage space of the artificial intelligence processor.

16. An electronic device, characterized in that, The electronic device includes: At least one processor; At least one memory, including one or more computer program modules; The one or more computer program modules are stored in the at least one memory and configured to be executed by the at least one processor, and the one or more computer program modules are used to implement the method according to any one of claims 1-14.

17. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer-readable instructions, wherein the computer-readable instructions, when executed by at least one processor, perform the method according to any one of claims 1-14.

Images