Task processing method, storage medium and electronic device for artificial intelligence processor

Abstract

The present application discloses a task processing method, storage medium and electronic device for an artificial intelligence processor, and relates to the field of computer technology, wherein the method comprises: obtaining a target task in a task queue of a processor; integrating task instructions of the same operation type executable by the target task into a data packet based on the data dependency relationship between the target tasks; obtaining the task priority of the target task and the real-time load status of multiple computing units in the processor; and allocating the data packet to the target computing unit among the multiple computing units based on the task priority and the real-time load status. Compared with the current existing technology, the present application avoids the need to decode task instructions one by one according to the order of task execution by integrating the data flow trigger mechanism with the dynamic resource allocation strategy, thereby reducing processing time, improving task execution efficiency, and effectively solving the problem of uneven resource utilization caused by static task allocation, thereby achieving more efficient and flexible task scheduling and execution.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a task processing method, storage medium, and electronic device of an artificial intelligence processor. Background Art

[0002] With the development of artificial intelligence technology, the demand for computing performance is becoming increasingly higher. From the early simple machine learning algorithms to today's complex deep learning models, such as deep neural networks, which are widely used in computer vision, speech recognition, natural language processing and other fields, these applications require powerful computing power to process massive amounts of data and complex operations.

[0003] At present, artificial intelligence processors in related technologies usually need to rely on the task execution order to decode task instructions when dynamically adjusting task allocation, which consumes a lot of time and leads to low task execution efficiency. Summary of the Invention

[0004] This disclosure provides a task processing method, storage medium, electronic device, and program product for an artificial intelligence processor. Its primary purpose is to address the problem in related technologies where artificial intelligence processors, when dynamically adjusting task allocation, typically rely on the order in which tasks are executed to decode task instructions, consuming considerable time and resulting in low task execution efficiency.

[0005] In a first aspect, the present application provides a task processing method for an artificial intelligence processor, comprising:

[0006] Get the target task in the processor's task queue;

[0007] Based on the data dependency relationship between the target tasks, integrating the task instructions of the same operation type that can be executed by the target tasks into a data packet;

[0008] Acquire the task priority of the target task and the real-time load status of multiple computing units in the processor;

[0009] The data packet is distributed to a target computing unit among the multiple computing units according to the task priority and the real-time load status, and the target computing unit is used to execute the computing task corresponding to the target task based on the data packet.

[0010] In a second aspect, the present application provides a task processing device for an artificial intelligence processor, comprising:

[0011] A first acquisition module is configured to acquire a target task in a task queue of a processor;

[0012] an integration module configured to integrate task instructions of the same operation type executable by the target tasks into a data packet based on the data dependency relationship between the target tasks;

[0013] a second acquisition module, configured to acquire the task priority of the target task and the real-time load status of the plurality of computing units in the processor;

[0014] The allocation module is configured to allocate the data packet to a target computing unit among the multiple computing units based on the task priority and the real-time load status, and the target computing unit is used to execute the computing task corresponding to the target task based on the data packet.

[0015] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the method of the first aspect when the computer program is executed by a processor.

[0016] In a fourth aspect, the present application provides an electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the method of the first aspect when executing the computer program.

[0017] In a fifth aspect, the present application provides a computer program product having a computer program stored thereon, which implements the method of the first aspect when the computer program is executed by a processor.

[0018] The present disclosure provides a task processing method, storage medium, electronic device and program product for an artificial intelligence processor, wherein the method comprises: first obtaining a target task in a task queue of the processor; based on the data dependency between the target tasks, integrating the task instructions of the same operation type that can be executed by the target task into a data packet; obtaining the task priority of the target task and the real-time load status of multiple computing units in the processor; according to the task priority and the real-time load status, allocating the data packet to a target computing unit among the multiple computing units, wherein the target computing unit is used to execute the computing task corresponding to the target task based on the data packet. Compared with the current existing technology, the present application avoids the need to decode task instructions one by one according to the order of task execution by integrating the data flow trigger mechanism with the dynamic resource allocation strategy, reduces processing time, improves task execution efficiency, and effectively solves the problem of uneven resource utilization caused by static task allocation, realizes more efficient and flexible task scheduling and execution, and can also significantly improve cache hit rate and alleviate memory bandwidth bottleneck.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 A flowchart of a task processing method of an artificial intelligence processor provided in an embodiment of the present application is shown;

[0022] Figure 2 A schematic structural diagram of an example provided in an embodiment of the present application is shown;

[0023] Figure 3 A flowchart of another task processing method of an artificial intelligence processor provided in an embodiment of the present application is shown;

[0024] Figure 4 A structural diagram of a task processing device of an artificial intelligence processor provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0026] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0027] The field of artificial intelligence is rapidly developing, placing extremely high demands on computing performance. From the simple machine learning algorithms of the early days to today's complex deep learning models, such as deep neural networks, which are widely used in fields like computer vision, speech recognition, and natural language processing, these applications require powerful computing power to process massive amounts of data and complex calculations. Traditional processor architectures struggle to cope with AI workloads, prompting the research and development of specialized processors for AI applications. The field of AI chips continues to evolve, moving from general-purpose graphics processing units (GPUs) to more efficient, dedicated AI processors, such as Google's Tensor Processing Units (TPUs). Within this trend, how to better integrate processor technology with AI applications and improve the performance of AI processors in data processing, task scheduling, and execution has become a key research direction in the current technology landscape.

[0028] During the development of artificial intelligence (AI), general-purpose CPUs (CPUs) were the primary computing resource. However, because their architectures were not optimized for AI workloads, they were inefficient when processing large amounts of data and complex neural network operations. With technological advancements, GPUs, leveraging their powerful parallel computing capabilities, have become the primary platform for deep learning training. Their large number of compute cores accelerates highly parallel tasks such as matrix operations. Furthermore, specialized AI chips, such as application-specific integrated circuits (ASICs), are tailored for specific neural network structures or algorithms, providing efficient computing performance in specific application scenarios. Meanwhile, traditional dataflow processors (DFPs) employ a dataflow-driven design combined with control flow. They integrate dataflow and control flow through explicit data transfer requirements and utilize data-triggered compute cores to support parallel execution at various granularities, meeting the needs of both compute- and data-intensive applications. DFPs also utilize a storage architecture including an instruction cache, direct memory access (DMA) controller, and local memory to reduce memory access overhead. They utilize an on-chip interconnect network architecture to enable efficient communication between multiple high-performance cores and support multi-core synchronization through atomic instruction-based synchronization mechanisms. In terms of task scheduling, traditional strategies usually adopt static task allocation, that is, the mapping relationship between tasks and processing units is determined before the tasks are executed; in data stream processors, tasks are dynamically executed according to the order in which the data is ready. Although this method may have a long data dependency chain for tasks with high real-time requirements, it will increase the delay and affect the real-time satisfaction.

[0029] Traditional AI processor architectures suffer from numerous shortcomings. Due to their versatility, general-purpose CPUs leave significant computing resources idle when processing AI workloads, leading to low hardware utilization. While GPUs excel at parallel computing, they face challenges with high power consumption and significant memory bandwidth pressure. This is particularly true during deep learning training, where frequent data reads from memory often create a performance bottleneck. ASICs, while highly efficient in specific application scenarios, lack versatility and require chip redesign when changes to neural network architectures or algorithms occur, resulting in limited flexibility. Traditional dataflow processor architectures also suffer from drawbacks. Regarding hardware utilization, static task allocation can easily overload some processing units while leaving others idle, leading to resource waste. Random memory access patterns triggered by fine-grained dataflows reduce cache hit rates and increase memory bandwidth pressure. Methods that rely on static task partitioning suffer from uneven data dependencies, leading to low resource utilization and a dependency on dynamic data readiness order, making them difficult to meet real-time requirements. Furthermore, explicit dataflow programming requires manual dependency management, making debugging difficult and increasing development costs and complexity. For task scheduling and execution methods in traditional data stream processors, static task allocation strategies cannot be dynamically adjusted according to the runtime system state and cannot fully utilize system resources. For example, when one core is overloaded and other cores are idle, tasks cannot be automatically migrated to balance the load. Relying on the data-ready order to execute tasks may cause critical tasks to be delayed and fail to meet the needs of real-time applications. Traditional methods lack comprehensive consideration of task priority, data locality, and system resource status, making it difficult to achieve efficient task scheduling and execution in complex artificial intelligence application scenarios. In summary, these shortcomings limit the effectiveness and adaptability of existing architectures in handling modern artificial intelligence workloads.

[0030] In order to improve the problem that the artificial intelligence processor in the related art usually needs to rely on the task execution order to decode the task instructions when dynamically adjusting the task allocation, which takes a lot of time and leads to low task execution efficiency. This embodiment provides a task processing method for an artificial intelligence processor, such as Figure 1 As shown, the method comprises the following steps:

[0031] Step 101: Obtain a target task from a task queue of a processor.

[0032] Exemplarily, the processor may monitor all target tasks in the task queue whose input dependencies have been satisfied, wherein the target tasks are sorted according to their priorities and then stored in the task queue.

[0033] Step 102 : Based on the data dependency relationship between the target tasks, task instructions of the same operation type that can be executed by the target tasks are integrated into a data packet.

[0034] In some examples, the main processor assigns tasks to the task queues of processing units (PUs) through a global scheduling mechanism. These tasks are then aggregated into data packets and dispatched to the target PU based on task requirements. Within the PU, local task scheduling methods perform dependency checks, packaging instructions of the same operation type and with ready data into a single data packet.

[0035] For example, by integrating instructions of the same type into data packets (supporting unified integration of multiple tasks), optimized resource allocation and improved task execution efficiency are achieved. This not only reduces the resource consumption of task startup, but also improves the utilization of computing resources by batch processing similar tasks, ensuring efficient task scheduling and execution.

[0036] Step 103: Obtain the task priority of the target task and the real-time load status of multiple computing units in the processor.

[0037] For example, the priority of the target task can be calculated based on the criticality score and the estimated execution time to reflect the importance and urgency of the task. At the same time, by real-time monitoring of the load status of multiple computing units in the processor (such as matrix, vector, floating point, fixed point, etc.), including indicators such as computing utilization, bus bandwidth utilization and queue depth, the busyness of each computing unit can be evaluated. Combining the task priority of the target task and the real-time load status of multiple computing units in the processor, the processor can select the target task with the highest priority and the most suitable for the current load condition for allocation, ensuring that critical tasks can be efficiently executed on computing units with sufficient resources, thereby optimizing the overall task processing efficiency and resource utilization.

[0038] Step 104: Distribute the data packet to a target computing unit among the multiple computing units based on the task priority and the real-time load status.

[0039] For example, the target computing unit is used to execute the computing task corresponding to the target task based on the data packet. When the target computing unit receives the data packet, it automatically triggers the corresponding computing module to perform specific operations. After the calculation is completed, the results are fed back to the data packet generation and dispatch unit, which uses these results to update the task status and guide the subsequent task scheduling process. This ensures that tasks can be executed efficiently and orderly, while optimizing resource allocation and load balancing, making it particularly suitable for complex artificial intelligence computing tasks.

[0040] In some examples, the target task with the highest priority and most suitable for the current processing unit load can be dynamically selected for allocation and execution based on the system's current resource status and load conditions, ensuring that each task in the task queue can be processed efficiently and orderly, optimizing overall resource utilization and task execution efficiency.

[0041] In some examples, such as Figure 2 The high-performance, data-triggered, multi-core AI processor architecture shown here consists of four main components. First, the General Purpose Processor (GPP), available in common architectures such as ARM, RISC-V, and SPARC, runs the operating system and performs various control tasks closely related to the application. During system operation, the GPP handles critical system-level tasks, such as promptly responding to interrupts and precisely controlling various external devices, thereby ensuring stable operation and efficient collaboration across the entire system. In AI application scenarios, the GPP leverages specific framework tools, such as TensorFlow XLA and PyTorch Glow, to decompose AI training programs into a data flow graph (DFG), tailored to the unique characteristics of AI applications. In this DFG, nodes represent specific tasks, while edges represent dependencies between tasks. After completing the task decomposition, the GPP allocates these tasks appropriately to the various processing units. The specific operation process is as follows: First, the main processor creates an initialization thread space to build a basic environment for the execution of subsequent tasks; then, it starts the corresponding processing unit, so that the tasks can be executed in an orderly manner on each PU, thereby achieving efficient processing of the artificial intelligence training program. Secondly, the Data Triggered Multi-core Accelerator (DTMA) processing unit mainly assumes the key responsibility of computing acceleration in the system. When the main processor creates a thread, it will reasonably schedule the task to the PU for execution according to the established scheduling strategy, and one task will be completed on one PU. During operation, the PU interacts with other PUs through email communication, direct memory access transmission, etc. to achieve data sharing and collaborative processing.

[0042] For example, the PU unit is optimized for the specific computing needs of AI applications, integrating a large number of functional modules for matrix calculations, floating-point operations, fixed-point operations, and trigonometric functions. These computing modules are equipped with independent data flow trigger modules, input queues, and output queues, enabling orderly data reception and processing. The DTMA processing unit is the core computing unit of the data-triggered multi-core AI processor. Its architecture is designed to efficiently execute AI tasks and supports dynamic task scheduling and data flow triggering mechanisms. The main components of the PU may include: computing module (including matrix compute unit (MCU), vector compute unit (VCU), floating-point multiply-accumulate unit (floating-point MAC), floating-point arithmetic logic unit (floating-point ALU), fixed-point multiply-accumulate unit (fixed-point MAC), fixed-point arithmetic logic unit (fixed-point ALU), etc.), data flow trigger module, input / output queue, register file, memory interface (connected to high-bandwidth memory, High Bandwidth Memory, HBM), data packet generation and dispatch unit, and content-addressable memory (CAM) for fast identification and reception of data packets.

[0043] For example, within a processing unit (PU), data packets are transmitted to the target compute unit via the PU bus. Content-addressable memory (CAM) performs parallel address comparisons to ensure accurate packet delivery. Once a match is found, the packet is received and its header is parsed to extract key information, such as the operation type, task ID, instruction priority, and data length, to determine the compute unit to which the packet corresponds. The payload data is then distributed to the corresponding module's input queue in the order of the instruction array. Each compute module monitors the data readiness of its input queue and automatically triggers the corresponding computation operation (such as GEMM or vector addition) when conditions are met. The result is written to the output queue, tagged with the task and instruction IDs for subsequent tracking. The computation result is fed back to the packet generation and dispatch unit to update dependency relationships and scheduling logic. If the data required by a subsequent task is ready, the task is marked as executable. Furthermore, the system employs a dynamic priority scheduling strategy that periodically scans the task queue, adjusts the scan period based on the PU load, and selects the appropriate compute unit to dispatch data packets based on the real-time load, achieving dynamic load balancing and hardware acceleration. The register file supports fast data access, the high-bandwidth memory reduces memory access latency through unified addressing by the memory management unit, the DMA controller supports direct memory access to efficiently move data, and the global interconnection network uses an optically variable topology on-chip interconnection network to support multi-node communication. Peripheral device interfaces such as PCI-E and JTAG provide the ability to connect to external devices.

[0044] Compared with the related art, this embodiment first obtains the target task in the task queue of the processor; based on the data dependency between the target tasks, the task instructions of the same operation type that can be executed by the target task are integrated into a data packet; the task priority of the target task and the real-time load status of multiple computing units in the processor are obtained; based on the task priority and the real-time load status, the data packet is allocated to the target computing unit among the multiple computing units, wherein the target computing unit is used to execute the computing task corresponding to the target task based on the data packet. By integrating the data flow trigger mechanism with the dynamic resource allocation strategy, the need to decode the task instructions one by one according to the task execution order is avoided, the processing time is reduced, the task execution efficiency is improved, and at the same time, the problem of uneven resource utilization caused by static task allocation is effectively solved, and more efficient and flexible task scheduling and execution are achieved. It can also significantly improve the cache hit rate and alleviate the memory bandwidth bottleneck. By intelligently allocating critical path tasks, more balanced task execution efficiency is achieved, which brings significant beneficial effects in multiple dimensions such as architecture design, storage management, and task scheduling.

[0045] Further, as a refinement and extension of the above embodiment, in order to specifically illustrate the data processing process, optionally, step 102 may specifically include: identifying task instructions that are executable, have the same operation type, and satisfy the data dependency from the task instructions of the target task; integrating the identified task instructions to generate the data packet.

[0046] For example, during local scheduling (within a PU), the packet generation and dispatch unit receives computation results, performs dependency checks, and integrates task instructions into packets in a specific format for dispatch. When the DTMA processing unit receives task information in the form of an instruction queue generated by the compiler mapping, it fully scans the task queue during each packet generation cycle.

[0047] In some examples, if the data required for a task is not yet available, currently executable instructions of the same type are combined into a single data packet. Based on the real-time load status of each compute unit (such as the input queue status), idle or lightly loaded compute units are selected as the target for transmission. After the data packet is assembled, it is transmitted via the PU bus. At this point, the content-addressable memory hardware in each compute unit performs a parallel comparison of the information transmitted on the bus. Once the data is identified as being destined for it, it performs a receive operation to obtain the corresponding target data. Subsequently, the data flow trigger module initiates the task execution in that compute unit. After the computation is complete, the compute unit feeds the task ID, instruction ID, and result back to the packet generation and dispatch unit. This unit repeats this process according to a fixed working cycle, including scanning the instruction queue, sorting dependencies, organizing data packets, and dispatching them. This ensures efficient and orderly execution of tasks, while optimizing resource utilization and task processing efficiency.

[0048] Optionally, step 103 may specifically include: obtaining the input queue lengths, the number of currently executed tasks, and the computing resource occupancy rates of multiple computing units. Accordingly, step 104 may specifically include: selecting a target computing unit from multiple computing units based on the task priority of the target task and in combination with the input queue lengths, the number of currently executed tasks, and the computing resource occupancy rates of multiple computing units; and allocating the data packet to the target computing unit.

[0049] For example, a dedicated data packet generation and dispatch unit can be set up in each data-triggered multi-core accelerator processing unit, which is responsible for integrating the instructions of multiple tasks into data packets and dispatching them to the PU bus. Its workflow starts with periodic scanning of the task queue to check whether the data dependencies of the task are ready. It supports parallel scanning of 4 to 8 task queues, and performs a full scan every 100 working cycles by default to determine the timing of data packet generation. Once the executable task is confirmed, the instructions of the same type will be integrated into a data packet, which includes a header of control information and a data part containing specific instructions and data. Subsequently, based on the task priority and the real-time load status of the PU, the appropriate target PU is selected and the data packet is efficiently sent out through the star bus structure. This method not only optimizes the task execution process, but also improves resource utilization and task processing efficiency, ensuring that data-intensive tasks can run smoothly under high load conditions.

[0050] As a refinement of this embodiment, before obtaining the target task in the task queue of the processor, the following methods may be used but are not limited to assign the task to the task queue, such as: Figure 3 As shown, Figure 3 A flowchart of a task processing method for an artificial intelligence processor provided in an embodiment of the present disclosure includes:

[0051] Step 201: Determine the task priorities of different tasks based on the label data of different tasks.

[0052] For example, each task can be labeled based on its nature and priority, ensuring that its execution requirements and dependencies are accurately reflected. Dynamic priority task scheduling is then implemented to flexibly adjust the execution order of tasks based on the tag information and the current system load, ensuring that high-priority and critical-path tasks are prioritized. This process not only improves the flexibility and efficiency of task scheduling but also optimizes resource utilization, making it particularly suitable for compute-intensive deep learning applications. This approach allows for efficient management of parallel task execution and improves overall computing performance.

[0053] Optionally, the tag data includes a criticality score and an estimated execution time; accordingly, step 201 may specifically include: calculating the task priorities of different tasks based on the criticality score and the estimated execution time.

[0054] For example, the criticality score and estimated execution time (EstimatedCycles) are two core indicators for evaluating task priority. The criticality score is used to quantify the impact of a task on the overall system performance. It reflects the importance of the task in the data flow graph. A high score means that the task is on the critical path and its delay will directly affect the overall performance of the system; a low score means that the task can be delayed without significantly affecting the total delay. The estimated execution time is the number of task execution cycles predicted based on the hardware performance model, which is used to estimate the time resources required to complete the task. Combining these two indicators, the priority of the task can be calculated through a specific formula to ensure that critical and shorter tasks are executed first, thereby optimizing system resource allocation and reducing total delay or maximizing throughput.

[0055] Optionally, the method of this embodiment may further specifically include: obtaining the path relationship between different tasks in the data flow graph; and performing criticality scoring on different tasks according to the degree of influence of the path relationship on processor performance.

[0056] For example, algorithms and toolchains (such as TensorFlow XLA and PyTorch Glow) can be used to decompose AI training programs into data flow graphs, where nodes represent tasks and edges represent dependencies between tasks. Dynamic task label injection is then performed, adding detailed label information to each task, including task ID, task type, input dependency list, estimated cycle time, and criticality score.

[0057] In some examples, the priority calculation formula is: Priority = Criticality Score (60% weight) + Estimated Execution Time (40% weight). The criticality score, a core metric for task scheduling and resource allocation, quantifies the impact of a task on overall system performance, typically ranging from 0.0 to 1.0. A high score (close to 1.0) indicates that the task is on the critical path, and its delay directly impacts overall system performance, such as the output layer of AI reasoning. A low score (close to 0.0) indicates that the task can be delayed without significantly affecting overall latency, such as in offline batch processing tasks.

[0058] Optionally, the above-mentioned obtaining of the path relationship of different tasks in the data flow graph may further specifically include: dividing different tasks into multiple data processing nodes and multiple data flow paths; and determining the path relationship of different tasks in the data flow graph based on the multiple data processing nodes and multiple data flow paths.

[0059] For example, the data flow graph of an AI program is first input, such as a computational graph generated by TensorFlow or PyTorch. Its nodes represent tasks (such as matrix multiplication and convolution operations) and its edges represent data dependencies (such as the data flow from task A to task B). The first step in the entire process is to construct this data flow graph, ensuring that all tasks and their dependencies are accurately represented. Next, in step two, a critical path analysis is performed, using the Critical Path Method (CPM) or a dynamic programming algorithm to calculate the longest path from the starting point to the end point, known as the critical path. Based on the path relationships, tasks on this critical path are assigned an initial criticality score of 1.0 due to their direct impact on overall performance. Tasks on non-critical paths are assigned decreasing scores based on their distance from the nearest critical path node.

[0060] Optionally, the method of this embodiment may further specifically include: dynamically adjusting the criticality scores according to the actual execution status of different tasks, and updating the estimated execution time of different tasks.

[0061] For example, label injection is divided into two stages to optimize task scheduling: static injection at compile time and dynamic adjustment at runtime. First, during the static injection stage at compile time, the label information of each task is initialized based on data flow graph analysis combined with the hardware model. This includes the task ID, task type, input dependency list, estimated cycle, and initial criticality score, providing a basis for subsequent task execution. Then, during the dynamic adjustment stage at runtime, the system optimizes the criticality score in real time based on feedback from actual execution, ensuring that it accurately reflects the actual impact and urgency of each task in the current environment.

[0062] Optionally, the above-mentioned criticality scores are dynamically adjusted according to the actual execution status of different tasks, and the estimated execution time of different tasks is updated. Specifically, it may also include: based on the estimated execution time, monitoring the actual execution time, resource usage and impact on the overall performance of the processor of different tasks; adjusting the criticality scores according to the monitoring results, and updating the estimated execution time of different tasks.

[0063] For example, during the static analysis phase of the compiler, the DFG is first input and critical path analysis is performed to identify task paths that are critical to system performance. The system then traverses all task nodes, sequentially completing tasks such as task ID generation, dependency analysis, execution cycle estimation based on hardware performance models, initial criticality score determination, and hardware optimization prompts. Finally, the system outputs a DFG with label information injected. During the runtime dynamic adjustment phase, after task execution is complete, the system dynamically updates the label information based on the actual execution status.

[0064] For example, the task importance score accounts for 60% of the weight and is used to quantify the impact of the task on the overall system performance. In particular, tasks on the critical path will receive a higher score (such as output layer calculations in AI reasoning); the estimated execution time accounts for 40% of the weight and tends to prioritize tasks with shorter estimated execution cycles to reduce queuing delays. The specific calculation formula for priority is: Priority = α × Criticality Score + β × (1 / Estimated Cycle), where α and β are 0.6 and 0.4 respectively, and these parameters can be dynamically adjusted according to the system status. The calculation of the task importance score is divided into two stages: the static analysis stage at compile time determines the initial score and initializes the label information based on data flow graph analysis and hardware model; the dynamic adjustment stage at runtime optimizes the score based on actual execution feedback, uses exponential smoothing to dynamically adjust the criticality score and update the estimated cycle to prevent subsequent tasks from starving.

[0065] In some examples, execution data can be recorded by monitoring the actual execution cycle of each task and comparing it with the estimated cycle. If the actual execution cycle of a task exceeds the estimated cycle by 20%, it is considered to have a potential impact on the critical path. Then, positive and negative feedback mechanisms are used to dynamically adjust the criticality score of the task. For tasks that are on the critical path and have increased delays, their scores will be increased; for tasks that are not on the critical path and are completed ahead of schedule, their scores will be reduced. The score is updated using the exponential smoothing formula as follows:

[0066] new_score=α*old_score+(1-α)*adjustment

[0067] Here, α is a smoothing factor, set to 0.8 by default, that controls the magnitude of the adjustment. The adjustment value is calculated based on the task's impact on the overall latency: if a task delay blocks subsequent tasks, the adjustment is set to min(1.0, old_score + 0.1); if the task completes early without affecting dependencies, the adjustment is set to max(0.0, old_score - 0.05). The system then periodically (for example, every 1000 cycles) recalculates the global critical path, analyzing and updating the critical path by traversing the entire data flow graph (DFG) to ensure that the criticality score reflects the actual situation in the latest execution environment, thereby achieving optimal resource allocation and scheduling optimization.

[0068] For example, the algorithm first selects the highest-priority ready task with satisfied dependencies from the head of the priority queue. Then, based on the load status table, it selects the processing unit (PU) with the lowest overall load for task assignment. Specifically, the algorithm evaluates the load of each PU based on metrics such as compute utilization, bus bandwidth utilization, and queue depth, binds tasks to the selected PU, and updates the load status table to reflect this change. For example, task T100 may be assigned to PU2, and the queue depth count for PU2 may be increased accordingly. To further optimize scheduling, the algorithm also includes a dynamic feedback mechanism: weights are adjusted periodically (e.g., every 1000 cycles). If a PU is found to be chronically overloaded (utilization exceeding 90%), its task assignment weight is reduced, reducing the number of new tasks assigned to it. If the overall system throughput decreases, the algorithm adjusts the α / β ratio to increase the weight of the criticality score, ensuring that tasks on the critical path receive priority, thereby improving system responsiveness and execution efficiency.

[0069] Step 202: Allocate different tasks to task queues of at least one processing unit in the processor according to their task priorities.

[0070] Optionally, step 202 may specifically include: by real-time monitoring of the utilization of at least one processing unit, allocating tasks with a task priority higher than a preset priority threshold to a task queue of a processing unit whose current utilization is lower than a preset utilization threshold.

[0071] For example, a load-based resource allocation mechanism aims to efficiently allocate tasks to data-triggered multi-core accelerator processing units based on the real-time load of each processing unit. Each processing unit reports its utilization in real time, including metrics such as the number of instructions executed per cycle and bus bandwidth utilization, so that high-priority tasks can be assigned to the processing unit with the lowest current utilization, thereby optimizing resource usage. Computational utilization is calculated by counting the active cycles of computational units (such as matrix units and vector units) within the processing unit. The specific formula is: utilization = (number of cycles executed by computational unit 1 + number of cycles executed by computational unit 2 + ... + number of cycles executed by computational unit n) / (number of computational units * total number of cycles). This statistical process is implemented using counters and counter registers within the computational units. Bus bandwidth utilization measures the actual bandwidth usage for data transfer between the PU and memory or other PUs. It is calculated using the formula: bandwidth utilization = (actual amount of data transferred) / (maximum theoretical bandwidth). In addition, monitoring the PU input / output queue depth reflects the number of pending tasks to assess backlog conditions. The comprehensive load is obtained by the weighted sum of computing utilization, bandwidth utilization, and queue depth.

[0072] In some examples, a global scheduler (such as a GPP or a dedicated hardware module) is responsible for collecting load metrics from all PUs and maintaining a load status table. This table is used as a basis for dynamic task allocation and load balancing, ensuring that the system can operate efficiently based on real-time load conditions, maximizing the use of available resources and reducing task delays.

[0073] In some examples, a dynamic priority mechanism is used to optimize the order of task allocation, combining the task's criticality score and estimated execution time, ensuring that tasks on the critical path are executed first. Simultaneously, a load-based resource allocation strategy monitors the utilization of processing units in real time, including computing utilization, bandwidth utilization, and queue depth, and dynamically allocates tasks to the PU with the lowest current load, achieving system-wide load balancing. At the local scheduling level, computations are triggered within the PU using packet flows, further improving task execution efficiency based on task priority and the PU's real-time load status.

[0074] In some examples, dynamic tag injection technology and runtime parameter adjustment methods are used to achieve dynamic scheduling at both global and local levels. At the global scheduling level, task allocation is optimized based on the state of system resources to ensure that tasks are properly distributed across processing units. At the local scheduling level, data packet flows are used to trigger the computation process, ensuring that tasks on the critical path can be executed efficiently. In addition, the proposed data flow triggering mechanism not only effectively reduces programming complexity but also significantly reduces the need for manual dependency management in explicit data flow programming, thereby simplifying the development process and improving the overall execution efficiency of the system.

[0075] Optionally, the method of this embodiment may further specifically include: comparing the target address of the data packet in parallel through a content-addressable memory to determine whether the data packet matches the target computing unit; if the data packet matches the target computing unit and the data packet is in a data-ready state, triggering the target computing unit to execute the computing task corresponding to the target task based on the data packet.

[0076] For example, during the task execution and data packet generation process, when the system scans the task queue that merges all task instructions, if it finds that the data required by a pending task is not yet ready, it will aggregate and package the currently executable instructions of the same type into a data packet of a specific format. Once assembled, the data packet is sent to the PU bus for further processing. Each data packet consists of two parts: a header and a payload. The header contains key control information and identifiers that guide the classification, aggregation, and dispatch of the data packet; the payload carries the specific operation instructions and required data. Furthermore, a category field is included in the data packet to accurately classify the data packet according to the type of computing unit (such as matrix computing unit (MCU) or vector computing unit (VCU)), and accordingly assign the data packet to the corresponding computing unit for execution. This mechanism ensures efficient and orderly task execution while optimizing resource utilization and task processing efficiency.

[0077] In some examples, the PU includes multiple specialized compute modules optimized for AI tasks, enabling efficient task processing. The Matrix Computation Unit (MU) supports general-purpose matrix multiplication (GEMM) and convolution operations, utilizing a parallel computing architecture to support large-scale matrix operations. The Vector Computation Unit (VCU) specializes in element-by-element operations such as addition, multiplication, and reductions like summation and maximum calculation. The Float Scalar Computation Unit (FSU) performs floating-point multiplication-addition (MAC) and floating-point arithmetic logic unit (ALU) operations. The Fixed-point Scalar Computation Unit (SCU) supports fixed-point multiplication-addition (MAC) and fixed-point arithmetic logic unit (ALU) operations. Furthermore, a special MU supports complex operations such as trigonometric functions and reductions. The Memory Access Unit (MCU) handles data movement and synchronization operations, such as barriers. Each compute module is equipped with an independent data flow trigger module, receiving data through an input queue and passing computation results to the next stage through an output queue, ensuring efficient and smooth data processing. By improving overall performance through modular multiple computing units (including matrix, vector, floating-point and fixed-point computing units, etc.), it can flexibly adapt to diverse artificial intelligence computing needs, enabling the PU to achieve high-efficiency and high-performance computing capabilities when processing complex artificial intelligence tasks.

[0078] Optionally, the method of this embodiment may further include: managing and converting the address space of the high-bandwidth memory and the main memory in the processor through the memory interface.

[0079] For example, the packet generation and dispatch unit uses a star bus structure to effectively connect all the aforementioned computing modules, establishing an efficient data transmission path. This unit also connects to the register file (128KB capacity) and the on-chip High Bandwidth Memory (HBM). Furthermore, these HBMs are uniformly addressed with the GPP-side Dynamic Random Access Memory (DDR) through the Memory Management Unit (MMU), thereby achieving unified management and efficient utilization of memory resources at the system level.

[0080] Exemplarily, the key components and hardware support of the data flow trigger module include CAM hardware, input / output queues, register files, and high-bandwidth memory. The CAM hardware is used to achieve fast address matching to ensure that data packets can be delivered to the target computing unit accurately. Each computing module is equipped with an independent input / output queue, which not only decouples the computing process from data transmission, but also improves the overall data processing efficiency. The register file provides fast access to intermediate results, effectively improving the data processing speed; and the high-bandwidth memory is uniformly addressed through the memory management unit, reducing memory access latency and further enhancing data access efficiency.

[0081] For example, the use of high-bandwidth memory and unified address management reduces memory access latency and improves data throughput, while the register file supports fast data access and efficient memory utilization. Furthermore, through hardware optimization methods for data-triggered comparison, content-addressable memory and a dedicated queue mechanism, rapid data packet identification and distribution are achieved, effectively reducing communication overhead. This not only enhances the system's computing performance, but also improves data processing speed and task execution efficiency.

[0082] Compared with existing technologies, this embodiment integrates a data flow triggering mechanism with a dynamic resource allocation strategy, improving the processing performance of AI scenarios while solving the hardware resource idleness and overload imbalance caused by traditional static task partitioning. For random memory access patterns triggered by fine-grained data flows, an optimized memory access mechanism is adopted to significantly improve cache hit rates and alleviate memory bandwidth bottlenecks. An intelligent allocation strategy for critical path tasks avoids hotspots and achieves more balanced task execution efficiency. Finally, an implicit data flow management mechanism reduces explicit programming dependencies, simplifies the data dependency management process and debugging complexity, and systematically resolves the core performance and usability contradiction of traditional data flow processors from four dimensions: architecture, storage, scheduling, and programming. The processing unit significantly improves the execution efficiency and hardware utilization of AI tasks by introducing a data flow triggering mechanism, dynamic priority scheduling, and the design of efficient computing modules. The data flow triggering mechanism decouples computation from data, enabling the system to trigger computation on demand based on actual needs, improving flexibility and responsiveness. The dynamic scheduling strategy combines task criticality scores and real-time load status to optimize resource allocation, ensuring that tasks on the critical path are processed first while balancing the workload of each processing unit.

[0083] The embodiment of the present application also provides a task processing device of an artificial intelligence processor, as Figure 1 The specific implementation of the method shown is as follows Figure 4 As shown, the device includes: a first acquisition module 31 , an integration module 32 , a second acquisition module 33 , and an allocation module 34 .

[0084] A first acquisition module 31 is configured to acquire a target task in a task queue of a processor;

[0085] an integration module 32 configured to integrate task instructions of the same operation type executable by the target tasks into a data packet based on the data dependency relationship between the target tasks;

[0086] A second acquisition module 33 is configured to acquire the task priority of the target task and the real-time load status of the multiple computing units in the processor;

[0087] The allocation module 34 is configured to allocate the data packet to a target computing unit among the multiple computing units based on the task priority and the real-time load status, and the target computing unit is used to execute the computing task corresponding to the target task based on the data packet.

[0088] In some examples of this embodiment, the integration module 32 is specifically configured to identify, from the task instructions of the target task, task instructions that are executable, of the same operation type, and satisfy the data dependency; and integrate the identified task instructions to generate the data packet.

[0089] In some examples of this embodiment, the second acquisition module 33 is specifically configured to obtain the input queue length, the number of currently executed tasks, and the computing resource occupancy rate of the multiple computing units; accordingly, the allocation module 34 is specifically configured to select the target computing unit from the multiple computing units based on the task priority of the target task and in combination with the input queue length, the number of currently executed tasks, and the computing resource occupancy rate of the multiple computing units; and allocate the data packet to the target computing unit.

[0090] In some examples of this embodiment, the first acquisition module 31 is specifically configured to determine the task priorities of different tasks based on the label data of different tasks; and assign the different tasks to the task queues of at least one processing unit in the processor according to the task priorities of different tasks.

[0091] In some examples of this embodiment, the tag data includes a criticality score and an estimated execution time; accordingly, the first acquisition module 31 is further configured to calculate the task priorities of the different tasks based on the criticality score and the estimated execution time.

[0092] In some examples of this embodiment, the first acquisition module 31 is further configured to obtain the path relationship of the different tasks in the data flow graph; and perform criticality scores on the different tasks according to the degree of influence of the path relationship on the processor performance.

[0093] In some examples of this embodiment, the first acquisition module 31 is further configured to divide the different tasks into multiple data processing nodes and multiple data flow paths; and determine the path relationship between the different tasks in the data flow graph based on the multiple data processing nodes and the multiple data flow paths.

[0094] In some examples of this embodiment, the first acquisition module 31 is further configured to dynamically adjust the criticality scores according to the actual execution conditions of the different tasks, and update the estimated execution times of the different tasks.

[0095] In some examples of this embodiment, the first acquisition module 31 is further configured to monitor the actual execution time, resource usage and impact on the overall performance of the processor of the different tasks based on the estimated execution time; adjust the criticality score according to the monitoring results, and update the estimated execution time of the different tasks.

[0096] In some examples of this embodiment, the first acquisition module 31 is further configured to allocate the task whose priority is higher than the preset priority threshold to the task queue of the processing unit whose current utilization is lower than the preset utilization threshold by monitoring the utilization of the at least one processing unit in real time.

[0097] In some examples of this embodiment, the allocation module 34 is further configured to compare the target address of the data packet in parallel through a content-addressable memory to determine whether the data packet matches the target computing unit; if the data packet matches the target computing unit and the data packet is in a data-ready state, the target computing unit is triggered to execute the computing task corresponding to the target task based on the data packet.

[0098] In some examples of this embodiment, the allocation module 34 is further configured to manage and convert the address space of the high bandwidth memory and the main memory in the processor through a memory interface.

[0099] It should be noted that for other corresponding descriptions of the functional units involved in the task processing device of an artificial intelligence processor provided in this embodiment, please refer to Figure 1 The corresponding description in will not be repeated here.

[0100] Based on the above Figure 1 and Figure 3 The method shown in FIG. 1 is a method for performing the above-mentioned steps. Accordingly, this embodiment further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program can realize the above-mentioned steps. Figure 1 and Figure 3 The method shown.

[0101] Based on the above Figure 1 and Figure 3 The method shown in FIG. 1 is a method for performing the above-mentioned operations. Accordingly, this embodiment further provides a computer program product having a computer program stored thereon. When the computer program is executed by a processor, the above-mentioned Figure 1 and Figure 3 The method shown.

[0102] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of the present application.

[0103] Based on the above Figure 1 and Figure 3 The method shown, and Figure 4 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides an electronic device, such as a personal computer or a server, which includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 and Figure 3 The method shown.

[0104] In some embodiments, the physical device may further include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a Wi-Fi module, and the like. The user interface may include a display, an input unit such as a keyboard, and the like. Optional user interfaces may also include a USB interface and a card reader interface. In some embodiments, the network interface may include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0105] Those skilled in the art will understand that the above-mentioned physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0106] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device, supporting the execution of information processing programs and other software and / or programs. The network communication module is used to enable communication between components within the storage medium, as well as with other hardware and software within the physical information processing device.

[0107] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by hardware. By applying the solution of this embodiment, compared with the current existing technology, this embodiment proposes a data-triggered multi-core artificial intelligence processor architecture and a dynamic priority and load task scheduling and execution method based on this architecture, aiming to solve the core pain points of traditional data flow processors in terms of performance, hardware utilization, storage bandwidth pressure, task execution efficiency and programming complexity. First, by combining data flow triggering with dynamic resource allocation, the processing performance in artificial intelligence application scenarios is improved, and the problem of some processing units being idle and other processing units being overloaded due to static task partitioning is overcome, thereby improving hardware utilization. Secondly, the low cache hit rate and memory bandwidth bottleneck caused by random memory access caused by fine-grained data flow triggering are optimized, thereby improving the pressure on storage bandwidth. In addition, this embodiment avoids hot spots by optimizing the allocation of tasks on the critical path, implements a more reasonable task allocation strategy, and improves task execution efficiency. Finally, by reducing the reliance on explicit data flow programming, the need for manual management of data dependencies is reduced, the debugging process is simplified, and programming complexity is effectively reduced. This embodiment not only considers the criticality score and estimated execution time of tasks to optimize resource allocation, but also flexibly schedules tasks based on real-time load conditions, ensuring that tasks on the critical path are prioritized. Furthermore, scheduling parameters can be further optimized by incorporating reinforcement learning, or 3D stacked memory technology can be used to reduce data handling overhead, thereby continuously improving system performance. This innovation provides an efficient and flexible task execution paradigm for next-generation heterogeneous computing architectures, laying a solid technical foundation.

[0108] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device that includes the element.

[0109] The above are merely specific embodiments of the present application to enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments herein, but rather is intended to conform to the widest scope consistent with the principles and novel features of the present application.

Claims

1. A task processing method for an artificial intelligence processor, characterized in that: include: Obtaining target tasks in a task queue of a processor, including: decomposing training programs corresponding to different tasks into a data flow graph; obtaining path relationships of the different tasks in the corresponding data flow graph; performing criticality scoring on the different tasks based on the degree of influence of the path relationships on the performance of the processor, and determining label information corresponding to the different tasks; calculating task priorities of the different tasks based on the label information; assigning the different tasks to task queues of at least one processing unit in the processor based on the task priorities of the different tasks; initializing label information of the different tasks based on the data flow graph, and optimizing the label information based on task status feedback information; Based on the data dependency relationship between the target tasks, integrating the task instructions of the same operation type that can be executed by the target tasks into a data packet; Acquire the task priority of the target task and the real-time load status of multiple computing units in the processor; The data packet is distributed to a target computing unit among the multiple computing units according to the task priority and the real-time load status, and the target computing unit is used to execute the computing task corresponding to the target task based on the data packet.

2. The method according to claim 1, characterized in that The step of integrating the task instructions of the same operation type executable by the target tasks into a data packet based on the data dependency relationship between the target tasks includes: identifying, from the task instructions of the target task, task instructions that are executable, have the same operation type, and satisfy the data dependency; The identified task instructions are integrated to generate the data packet.

3. The method according to claim 1, characterized in that Obtaining real-time load status of multiple computing units in the processor includes: Obtaining the input queue lengths, the number of currently executed tasks, and the computing resource occupancy rates of the plurality of computing units; The allocating the data packet to a target computing unit among the plurality of computing units according to the task priority and the real-time load status includes: Selecting the target computing unit from the multiple computing units according to the task priority of the target task and in combination with the input queue lengths, the number of currently executed tasks, and computing resource occupancy rates of the multiple computing units; The data packet is distributed to the target computing unit.

4. The method according to claim 1, wherein The obtaining of the path relationship of the different tasks in the data flow graph includes: Dividing the different tasks into multiple data processing nodes and multiple data flow paths; Based on the multiple data processing nodes and the multiple data flow paths, the path relationship between the different tasks in the data flow graph is determined.

5. The method according to claim 1, wherein The optimizing the label information according to the task status feedback information includes: According to the actual execution status of the different tasks, the criticality scores are dynamically adjusted, and the estimated execution times of the different tasks are updated.

6. The method according to claim 5, characterized in that The dynamically adjusting the criticality scores according to the actual execution status of the different tasks and updating the estimated execution times of the different tasks include: Based on the estimated execution time, monitoring the actual execution time, resource usage, and impact on the overall performance of the processor of the different tasks; The criticality scores are adjusted according to the monitoring results, and the estimated execution times of the different tasks are updated.

7. The method according to claim 1, characterized in that The allocating the different tasks to the task queues of at least one processing unit in the processor according to the task priorities of the different tasks includes: By monitoring the utilization of the at least one processing unit in real time, the task with a priority higher than a preset priority threshold is allocated to the task queue of the processing unit whose current utilization is lower than the preset utilization threshold.

8. The method according to claim 1, characterized in that After allocating the data packet to a target computing unit among the plurality of computing units according to the task priority and the real-time load status, the method further includes: Comparing the target address of the data packet in parallel with the content addressable memory to determine whether the data packet matches the target computing unit; If the data packet matches the target computing unit and the data packet is in a data-ready state, the target computing unit is triggered to execute the computing task corresponding to the target task based on the data packet.

9. The method according to claim 1, characterized in that The method further comprises: The address space of the high-bandwidth memory and main memory in the processor is managed and converted through the memory interface.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

11. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 9 is implemented.

12. A computer program product having a computer program stored thereon, characterized in that: When the computer program product is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images