Workgroup scheduling method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides a workgroup scheduling method and device, electronic equipment and storage medium. First, a target input tensor and a target workgroup grid corresponding to the target input tensor are obtained. Then, the target workgroup grid is split to obtain multiple super workgroups, and the workgroups in the super workgroups are scheduled to each computing die. For each computing die, the super workgroups are split again into multiple cluster workgroups, and the workgroups in the cluster workgroups are scheduled to each block processing cluster. In the scheduling process of the workgroups, each cluster workgroup is used to split the target input tensor to obtain multiple tensor subblocks, and the tensor subblocks are written into the local memory of the corresponding block processing cluster, so that the block processing cluster reads the tensor subblocks corresponding to the cluster workgroup from the corresponding local memory when executing the cluster workgroup. Therefore, the embodiments of the present disclosure can improve the utilization rate of multi-level resources, improve the task scheduling accuracy, and optimize the system processing performance.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and in particular to a workgroup scheduling method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence, high-performance computing and other fields, current graphics processors and AI accelerator architectures are evolving towards multi-core and chiplet integration. However, traditional task scheduling strategies still remain at the level of simple hierarchical architecture strategy construction, and are difficult to adapt to the multi-level architecture brought by multi-chip packaging technology, resulting in an imbalance between scheduling granularity and hardware physical topology, low utilization of multi-level resources, limited parallel efficiency, and difficulty in meeting the growing demand for computing power and energy efficiency. Therefore, there is an urgent need for a scheduling architecture that takes into account multi-level characteristics and fine-grained scheduling. SUMMARY

[0003] Embodiments of the present disclosure provide a workgroup scheduling method and device, electronic equipment and storage medium, which can improve the utilization of multi-level resources and improve the scheduling accuracy of tasks, thereby optimizing the processing performance of the system.

[0004] In a first aspect, embodiments of the present disclosure provide a workgroup scheduling method applied to a data processor, the data processor comprising a plurality of computing dies, the computing dies comprising a plurality of block processing clusters, the workgroup scheduling method comprising:

[0005] obtaining a target input tensor and a target workgroup grid corresponding to the target input tensor, the target workgroup grid comprising a plurality of workgroups;

[0006] splitting the target workgroup grid to obtain a plurality of super workgroups, and scheduling the workgroups in the super workgroups to each of the computing dies;

[0007] for each of the computing dies, performing secondary splitting on the super workgroups to obtain a plurality of cluster workgroups, and scheduling the workgroups in the cluster workgroups to each of the block processing clusters;

[0008] In the scheduling process of the workgroups, each of the cluster workgroups is used to split the target input tensor to obtain a plurality of tensor subblocks, and the tensor subblocks are written into the local memory of the corresponding block processing cluster, so that the block processing cluster reads the tensor subblocks corresponding to the cluster workgroups from the corresponding local memory in the process of executing the cluster workgroups.

[0009] In a second aspect, embodiments of the present disclosure further provide a workgroup scheduling device, the workgroup scheduling device comprising a plurality of computing dies, the computing dies comprising a plurality of block processing clusters, the workgroup scheduling device further comprising:

[0010] a data and task acquisition unit, configured to acquire a target input tensor and a target workgroup grid corresponding to the target input tensor, the target workgroup grid comprising a plurality of workgroups;

[0011] a first workgroup splitting unit, configured to split the target workgroup grid to obtain a plurality of super workgroups, and schedule workgroups in the super workgroups to respective calculation dies;

[0012] a second workgroup splitting unit, configured to, for each of the calculation dies, split the super workgroups again to obtain a plurality of cluster workgroups, and schedule workgroups in the cluster workgroups to respective block processing clusters;

[0013] a data scheduling unit, configured to, in a scheduling process of the workgroups, split the target input tensor according to each of the cluster workgroups to obtain a plurality of tensor subblocks, and write the tensor subblocks into local memories of the corresponding block processing clusters, so that the block processing clusters read the tensor subblocks corresponding to the cluster workgroups from the corresponding local memories in a process of executing the cluster workgroups.

[0014] Further, the embodiment of the present disclosure further proposes that the first workgroup splitting unit is further configured to:

[0015] split the target workgroup grid into a plurality of super workgroups based on a number of the calculation dies, wherein each of the super workgroups corresponds to one of the calculation dies;

[0016] schedule workgroups in the super workgroups to the corresponding calculation dies.

[0017] Further, the embodiment of the present disclosure further proposes that the first workgroup splitting unit is further configured to:

[0018] determine a first position identifier of the super workgroup based on a global position of the super workgroup in the target workgroup grid;

[0019] schedule workgroups in the super workgroups to respective calculation dies, and store the first position identifier corresponding to the super workgroup to the first register of the corresponding calculation die, so that a kernel of the calculation die executes workgroups required by itself through the first position identifier.

[0020] Further, the embodiment of the present disclosure further proposes that the second workgroup splitting unit is further configured to:

[0021] For each of the computing dies, based on preset dimension parameters or the number of the block processing clusters, the super workgroup is secondarily split into a plurality of cluster workgroups, wherein each of the cluster workgroups corresponds to one of the block processing clusters;

[0022] Workgroups in the plurality of cluster workgroups belonging to one of the super workgroups are scheduled to corresponding block processing clusters in one of the computing dies.

[0023] Further, the embodiments of the present disclosure also propose that the second workgroup splitting unit is further configured to:

[0024] determine a second position identifier of the cluster workgroup based on a global position of the cluster workgroup in the super workgroup;

[0025] schedule workgroups in the cluster workgroup to corresponding block processing clusters and store the second position identifier corresponding to the cluster workgroup to the second register of the corresponding block processing cluster, so that a kernel of a computing unit running on the block processing cluster performs a workgroup required to be processed by the kernel through the second position identifier.

[0026] Further, the embodiments of the present disclosure also propose that the data scheduling unit is further configured to:

[0027] In the workgroup scheduling process of the super workgroup, the target input tensor is segmented according to each of the super workgroups to obtain a plurality of large tensor subblocks, and a target computing die to which the super workgroup belongs is determined, and the large tensor subblocks are stored to local memories of the corresponding target computing dies;

[0028] In the workgroup scheduling process of the cluster workgroup, the large tensor subblocks stored in the local memory of the current computing die are segmented according to each of the cluster workgroups to obtain a plurality of tensor subblocks, and a target block processing cluster to which the cluster workgroup belongs is determined, and the tensor subblocks are stored to a local memory of the corresponding target block processing cluster.

[0029] Further, the embodiments of the present disclosure also propose that the first workgroup splitting unit is further configured to:

[0030] receive splitting information provided by a driver, extract starting coordinates of each super workgroup from the splitting information, split the target workgroup grid according to the starting coordinates to obtain a plurality of super workgroups;

[0031] Alternatively, the starting coordinates of each super workgroup are calculated by using the command processor to respond to an external splitting command, the target workgroup grid is split according to the starting coordinates to obtain a plurality of super workgroups.

[0032] Further, the second working group splitting unit is further configured to:

[0033] The cluster working group constructor is used to perform structural analysis on the super working group, generate a plurality of sub-tasks, and encapsulate each sub-task to obtain a plurality of cluster working groups.

[0034] In a third aspect, the embodiments of the present disclosure further provide an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory. The program is executed by the processor to realize the working group scheduling method in the first aspect.

[0035] In a fourth aspect, the embodiments of the present disclosure further provide a computer readable storage medium, which stores one or more programs executable by one or more processors to realize the working group scheduling method in the first aspect.

[0036] The working group scheduling method, device, electronic device and storage medium provided by the present disclosure first obtain a target input tensor and a corresponding target working group grid, split the target working group grid into a plurality of super working groups, and schedule the working groups in the super working groups to each computing die to add a task scheduling at the die level, expose the die level of the hardware to the system programming, ensure that the scheduling level of the task allocation is aligned with the die level of the hardware, thereby making full use of the internal resources of the die, effectively reducing the communication overhead across the dies, improving the parallel efficiency of the die, and avoiding the delay caused by global scheduling. Then, the super working groups are further split for each computing die to obtain a plurality of cluster working groups, and the further fine-grained splitting closely matches the physical level of the block processing cluster, optimizes the load balancing between clusters, and improves the resource utilization. At the same time, the working groups in the cluster working groups are scheduled to each block processing cluster to ensure that the computing unit can process tasks matching the local data and reduce the task migration overhead. In addition, during the scheduling of the working groups, the target input tensor is synchronously divided according to the division logic of each cluster working group to ensure that the data sub-blocks accurately correspond to the task requirements, avoid redundant data transmission, and write the tensor sub-blocks into the local memory of the corresponding block processing cluster, so that the data storage location is physically bound to the execution unit, reducing the remote access delay, which helps the execution unit to directly use the local data to complete the calculation, significantly improving the data access efficiency and overall parallel performance. Therefore, the working group scheduling method proposed in the embodiments of the present disclosure matches the task splitting and data division with the hardware level architecture, so that the scheduling granularity accurately matches the hardware physical topology level, improves the resource utilization and parallel efficiency in the multi-level architecture, and thus optimizes the system processing performance.

[0037] Additional features and advantages of the present disclosure will be set forth in the description that follows, and in part will be apparent from the description, or can be learned by practice of the present disclosure. The objectives and other advantages of the present disclosure will be realized and attained by the structure particularly pointed out in the description and claims. BRIEF DESCRIPTION OF DRAWINGS

[0038] The accompanying drawings are included to provide a further understanding of the technical solutions of the present disclosure, and constitute a part of the specification, and are used to explain the technical solutions of the present disclosure together with the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions of the present disclosure.

[0039] Figure 1 is a hardware hierarchical architecture diagram of a data processor provided by an embodiment of the present disclosure;

[0040] Figure 2 is an optional flowchart of a workgroup scheduling method provided by an embodiment of the present disclosure;

[0041] Figure 3 is a schematic diagram of a workgroup scheduling hierarchy provided by an embodiment of the present disclosure;

[0042] Figure 4 is an optional flowchart of super workgroup splitting provided by an embodiment of the present disclosure;

[0043] Figure 5 is an optional flowchart of super workgroup splitting provided by an embodiment of the present disclosure;

[0044] Figure 6 is a flowchart of cluster workgroup splitting provided by an embodiment of the present disclosure;

[0045] Figure 7 is a flowchart of tensor data splitting provided by an embodiment of the present disclosure;

[0046] Figure 8 is an optional structural schematic diagram of a workgroup scheduling device provided by an embodiment of the present disclosure;

[0047] Figure 9 is an optional structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0048] In order to make the purposes, technical solutions and advantages of the present disclosure clearer, the present disclosure will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure, and do not limit the present disclosure.

[0049] In the embodiments of the present disclosure, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented in whole or in part by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the functions of the module or unit.

[0050] To facilitate understanding of the technical solutions provided by the embodiments of the present disclosure, some key terms used by the embodiments of the present disclosure are explained first:

[0051] Graphics Processing Unit (GPU): A microprocessor designed specifically for processing graphics and image computations; GPUs were originally designed to accelerate the rendering of 2D and 3D graphics, enabling improved performance in games and professional graphics software. With thousands of cores, GPUs are capable of processing large amounts of data simultaneously, making them ideal for parallel computing tasks. As technology has advanced, GPUs are no longer limited to graphics processing; they are now used for a variety of general-purpose computing tasks. GPUs play a crucial role in deep learning, machine learning, and artificial intelligence, as they can quickly process large amounts of data, accelerating the training and inference of neural networks. In scientific computing and data analysis, GPUs are used to speed up complex numerical simulations and data analysis tasks. In cloud computing and data centers, GPUs are used to provide high-performance computing resources, supporting various compute-intensive applications. In specialized applications such as video editing, 3D modeling, scientific visualization, and others, GPUs can provide real-time high-performance rendering.

[0052] Thread: The smallest unit of execution of a processor when performing data processing tasks, each thread can independently perform the same mode of processing on different data, for example, a graphics processor needs to normalize the elements of each matrix row in an M*N matrix, at this time, M threads can be called in the graphics processor, each thread is responsible for normalizing the elements of a matrix row in the matrix, to parallel process the elements of M matrix rows in the matrix through M threads. Alternatively, multiple threads simultaneously perform different modes of processing on different data, for example, when normalizing an M*N matrix, the graphics processing unit needs to copy the data of the matrix from the global memory to the registers of the computing units of the graphics processing unit before performing normalization processing by the computing units, at this time, a part of the threads can be used to copy the data of the matrix from the global memory to the registers of the computing units, and another part of the threads is used to instruct the computing units to normalize the data that has been copied to the registers.

[0053] Work Group: A group of threads formed by multiple threads, which can also be referred to as a thread block or a work item. The threads of a single work group are generally used to perform the same mode of processing on different data.

[0054] Grid: A group of work groups, each of which is a grid point in the grid, and each work group has a corresponding grid point coordinate. The grid can be one-dimensional, two-dimensional, or three-dimensional or higher. It can be understood that the shape of the grid is set by the user during programming.

[0055] Execution Unit (EU): The smallest execution component in a processor that actually executes thread-level computing instructions. The execution unit can directly execute the instruction stream of a thread, undertake bottom-level arithmetic logic operations and data processing tasks, and the execution unit can implement parallel computing, so that the graphics processor can efficiently process data-intensive tasks such as graphics rendering and AI inference.

[0056] Compute Unit (CU): A processing module in a graphics processor. Each compute unit includes multiple execution units. The compute unit can receive a work group scheduled by the upper layer, split it into smaller threads, and assign them to the corresponding execution units for execution. The compute unit integrates multiple execution units to form a parallel computing cluster, which can manage thread data and coordinate load balancing between execution units.

[0057] Block Process Cluster: A cluster formed by multiple compute units in a single graphics processor. In the architecture of some graphics processing units, the block process cluster can also be referred to as a graphics processing cluster (GPC). In a graphics processor, each block process cluster includes an equal number of compute units. For example, a graphics processing cluster includes 32 compute units, and every 4 compute units are integrated into a block process cluster, so the graphics processor includes 8 block process clusters. It can be understood that the communication distance between the compute units in the same block processor cluster is relatively short, while the communication distance between the compute units of different block process clusters is relatively long.

[0058] Compute Die: A kind of core integrated chip in the graphics processor and AI accelerator, which is the basic computing unit in the chiplet architecture, integrating multiple block processing clusters, on-chip memory and memory controllers, as a physical carrier of resource aggregation and communication boundary. A compute die usually contains multiple block processing clusters, each of which integrates multiple computing units and execution units, forming a multi-level parallel architecture of "die, block processing cluster, computing unit, execution unit". As a scheduling unit, the compute die can sense the load state, memory bandwidth distribution and memory access characteristics of multiple block processing clusters in the chip, and can realize global resource optimization. At the same time, in the chiplet architecture, the compute die can cooperate with other dies through interconnection between dies, playing an important role in end-to-end performance.

[0059] Non-Uniform Memory Access (NUMA): A memory architecture under multi-processing nodes, in which each node is provided with a corresponding local memory, each node can access the corresponding local memory at high speed, and cross-node communication can be performed between different nodes to access the local memory of other nodes.

[0060] In the field of artificial intelligence and high-performance computing, processor architecture is accelerating towards multi-core and chiplet integration. The graphics processor and AI accelerator can currently use multi-die packaging technology to build a multi-level structure. Such architecture includes multiple physically separated compute dies, each of which further integrates multiple block processing clusters, forming a complex hardware topology hierarchy. However, the current task scheduling strategy is still based on the design of traditional single-chip model, and the task allocation mechanism cannot adapt to the multi-level physical characteristics in the multi-die environment, resulting in the loss of fine-grained perception of hardware topology hierarchy in the scheduling process. The splitting and scheduling granularity of the workgroup grid cannot match the hierarchical distribution characteristics of the compute die and block processing cluster, causing the data of the input tensor, task segmentation and hardware resource hierarchical layout to be disconnected, which easily causes frequent cross-die data transmission, greatly increasing the memory access delay and communication overhead, resulting in low utilization of multi-level resources and limited parallel efficiency, making it difficult to meet the growing demand for computing power and energy efficiency. Therefore, there is an urgent need for a scheduling architecture that takes into account multi-level characteristics and fine-grained scheduling.

[0061] For example, when performing an inference task of a deep learning model, the graphics processor includes four computing dies, each of which contains eight block processing clusters, the input tensor is 1024x1024 pixel image data, and the target workgroup grid can be composed of 1024 workgroups corresponding to the convolution operation. In the traditional scheduling strategy, the computing die level and the block processing cluster level are considered as the same level, causing level confusion, and the workgroups are directly allocated to the block processing clusters, causing the data of the same workgroup to be scattered in the memories of different computing dies. Therefore, during the task execution process, the block processing clusters need to obtain data across dies, causing an increase in data transmission delay, a decrease in local memory resource utilization, and further affecting the system throughput.

[0062] Based on this, the embodiments of the present disclosure propose a workgroup scheduling method and device, an electronic device, and a storage medium. First, the target input tensor and the corresponding target workgroup grid are obtained, the target workgroup grid is split into multiple super workgroups, and the workgroups in the super workgroups are scheduled to each computing die. A task scheduling at the die level is added to expose the die level of the hardware to the system programming, so as to ensure that the scheduling level of task allocation is aligned with the die level of the hardware, thereby fully utilizing the internal resources of the die, effectively reducing the cross-die communication overhead, improving the die-level parallel efficiency, and avoiding the delay caused by global scheduling. Then, for each computing die, the super workgroups are further split to obtain multiple cluster workgroups, which further finely match the physical level of the block processing cluster, optimize the inter-cluster load balancing, and improve the resource utilization. At the same time, the workgroups in the cluster workgroups are scheduled to each block processing cluster, so as to ensure that the computing unit can directly process the tasks corresponding to the local data and reduce the task migration overhead. In addition, during the scheduling of the workgroups, the target input tensor is synchronously divided according to the division logic of each cluster workgroup, so as to ensure that the data subblocks accurately correspond to the task requirements, avoid redundant data transmission, and write the tensor subblocks into the local memories of the corresponding block processing clusters, so as to physically bind the data storage location and the execution unit, reduce the remote access delay, and help the execution unit to directly use the local data to complete the calculation, thereby significantly improving the data access efficiency and the overall parallel performance. Therefore, the workgroup scheduling method proposed in the embodiments of the present disclosure matches the task splitting and data division with the hardware level architecture, so as to accurately match the scheduling granularity with the hardware physical topology level, improve the resource utilization and parallel efficiency in the multi-level architecture, and thereby optimize the system processing performance.

[0063] In a first aspect, with reference to Figure 1 , Figure 1Figure 1 is a hardware hierarchical architecture diagram of a data processor provided by an embodiment of the present disclosure. As shown in the figure, the hardware architecture of the data processor provided by the embodiment of the present disclosure is a multi-level hierarchical parallel architecture, taking a chiplet architecture as a top-level integration unit, and being subdivided into a computing die, a block processing cluster (BPC), a computing unit (CU), and an execution unit (EU) in a downward direction, forming a five-level hierarchical structure of "chiplet architecture-computing die-block processing cluster-computing unit-execution unit", and realizing resource hierarchical management and collaboration through a communication mechanism and memory design between levels. Specifically, the chiplet is a top-level integration unit of the data processor, and multiple independently manufactured computing dies are packaged into a whole through high-density interconnection (such as a silicon interposer), to realize chip-level integration and efficient interconnection, that is, the chiplet can contain multiple computing dies (the number can be configured according to performance requirements, such as 2 / 4 / 8), and an interconnection network (Die-to-Die Interconnect) in the chiplet, so that multiple computing dies in the same chiplet architecture can realize high-speed communication and data transmission through an inter-chip interconnection link.

[0064] The computing die is a core computing unit in the chiplet architecture, and each computing die contains multiple block processing clusters, an on-chip shared cache, a memory controller, and a die-level interconnection network (BPC-to-BPC Interconnect). The computing die, as an intermediate level for resource scheduling and data management, can sense the load state, memory bandwidth, and NUMA characteristics of multiple BPCs in the chip, realize global resource optimization, multiple BPCs in the same computing die can communicate through a die-level interconnection network (such as a crossbar network or a Mesh network), the communication delay is lower than that of cross-die communication, and the computing die can interface with off-chip storage resources through a memory controller, to realize large-capacity data access. The computing die is integrated with an L2 cache as an on-chip shared memory, which can cache frequently accessed data, and the local memory of the block processing cluster can interact with the L2 cache through an on-chip network, to optimize data locality.

[0065] The block processing cluster is a middle-level parallel cluster in the computing die, composed of multiple computing units and shared resources, and can be understood as a basic unit of fine-grained task scheduling and data cooperation. Each BPC contains a plurality of computing units CU (such as 4 / 8 / 16), local memory (which can be composed of the local memories of subordinate computing units), intra-cluster interconnection network (CU-to-CU Interconnect), and task scheduling controller, responsible for coordinating task allocation and data synchronization of each computing unit in the cluster. Due to the close physical distance of computing units in the same BPC, low-latency communication can be achieved through the intra-cluster interconnection network, while communication between different BPCs requires the die-level interconnection network, which has relatively high latency. To optimize cross-cluster cooperation efficiency, the task scheduling controller can dynamically allocate tasks according to data locality and computing load, preferentially scheduling highly related computing tasks within the same BPC to reduce cross-cluster data access frequency. At the same time, with the help of NUMA-aware scheduling strategy, the system can preferentially allocate data to local memory areas close to the target BPC, reducing remote memory access overhead, thereby effectively balancing parallelism and communication cost and improving overall computing efficiency. Each computing unit is equipped with an independent private local memory for storing thread private data and temporary computing results, reducing latency access. The local memory of all computing units in the BPC is combined to form the local memory of the BPC, which logically forms a shared memory space that can be cooperatively accessed by each computing unit in the block processing cluster.

[0066] The computing unit is an execution cluster within the block processing cluster, composed of multiple execution units, and is the smallest unit of thread parallel management and task execution. Each computing unit can independently schedule multiple threads. Specifically, it can receive the work group allocated by the BPC, split it into threads, and then allocate them to execution units. The execution unit is the smallest execution unit within the computing unit and can directly execute thread-level computing instructions, responsible for completing arithmetic logic operations, address generation, and data loading and storage operations.

[0067] It is worth noting that in the current related technology, the block processing cluster and the computing die are considered as the same system level, completely ignoring the NUMA characteristics, shared cache, and cross-block processing cluster cooperation mechanism of the computing die level. The current "level flattening" scheduling strategy skips the computing die level and cannot perform global resource allocation according to the load state and memory bandwidth distribution of multiple block processing clusters within the computing die. Tasks and data cannot be load balanced at the die level, and the value of the computing die as a natural communication granularity is also hidden. Especially in small chip architectures, die-to-die interconnection becomes a performance bottleneck, and the scheduling strategy in the related technology cannot optimize the cooperation of cross-die and on-die resources due to the lack of exposure of the computing die level, resulting in low resource utilization and inability to fully utilize the computing power potential of the multi-level architecture, restricting further improvement of end-to-end performance.

[0068] Based on the above data processor architecture, and the current related technology in the above architecture exists the problem of missing hierarchical abstraction, the embodiment of the disclosure proposes a new scheduling architecture to solve the problem of low resource utilization and limited parallel efficiency, taking into account the multi-level characteristics and fine-grained scheduling, improving the resource utilization of multi-level, and improving the task scheduling precision, thereby optimizing the system processing performance.

[0069] Specifically, the embodiment of the disclosure proposes a workgroup scheduling method, which can be applied to the data processor 100 as shown in Figure 1 The workgroup scheduling method provided by the embodiment of the disclosure can be applied to the data processor 100 as shown in Figure 2 , Figure 2 An optional flowchart of the workgroup scheduling method provided by the embodiment of the disclosure is shown in the figure, which includes but is not limited to the following steps 201 to 203.

[0070] Step 201: Obtain a target input tensor and a target workgroup grid corresponding to the target input tensor;

[0071] Step 202: Split the target workgroup grid to obtain a plurality of super workgroups, and schedule the workgroups in the super workgroups to each computing die;

[0072] Step 203: For each computing die, perform secondary splitting on the super workgroup to obtain a plurality of cluster workgroups, and schedule the workgroups in the cluster workgroup to each block processing cluster.

[0073] The target input tensor can be a multi-dimensional data group to be processed for a computing task, that is, a data set used by the data processor for arithmetic operation or logical operation. For example, the target input tensor can be a feature map or a weight matrix from the data processor when running a deep learning model. The target input tensor can be read from the system main memory, for example, by loading data through a memory controller; or receiving data through an external device interface. The dimension structure of the target input tensor matches the computing demand of the neural network model, and can be expressed in the format (N, C, H, W), which corresponds to the scale size, the number of channels, the feature map height and the width.

[0074] The target workgroup grid includes a plurality of workgroups, and the target workgroup grid can be understood as a structure defining the logical arrangement of a plurality of workgroups. Each "grid point" corresponds to a workgroup, and each workgroup includes a plurality of threads for parallel processing of corresponding data blocks in the target input tensor. Each thread cooperates to complete a local computing task. The specific arrangement can be determined based on the scheduling demand of the application program, for example, the task provides grid parameters divided according to the computing characteristics of the target work tensor.

[0075] It can be understood that a super work group (SWG) can refer to a work group set formed after the target work group grid is split, and each super work group contains part of the work groups in the original grid. Each super work group can be allocated a plurality of work groups to fully utilize the parallel computing capability of a plurality of processing clusters in a single computing die. For example, the target work group grid contains 1024 work groups, and the target work group grid can be split into 4 super work groups, each containing 256 work groups. Therefore, the super work group can be understood as a “first-level splitting unit”. The scheduling level of the super work group corresponds to the “computing die” level of the hardware, and the plurality of super work groups after splitting are respectively mapped to different computing dies. For example, SWG1 is allocated to the first computing die, SWG2 is allocated to the second computing die, and so on. When scheduling, load balancing and data locality can be considered. Specifically, the super work groups are dynamically allocated according to the computing capability and current load state of each computing die. The super work groups are preferentially allocated to the computing dies with lower current load, or if part of the data of the target input tensor is cached in the local memory of a computing die, the corresponding part of the super work groups is scheduled to the computing die to reduce the cross-die data transmission overhead, thereby improving the overall computing efficiency and resource utilization.

[0076] It can be understood that a cluster work group (CWG) can refer to a work group subset formed after a super work group is further split within a single computing die. Each cluster work group contains part of the work groups in the original super work group at the upper level. As a “second-level splitting unit”, the cluster work group corresponds to the “block processing cluster” level of the hardware in terms of scheduling level, that is, each cluster work group corresponds to a block processing cluster for performing more fine-grained parallel computing tasks. The plurality of cluster work groups after splitting are respectively mapped to different block processing clusters. For example, CWG1 is mapped to the first block processing cluster, CWG2 is mapped to the second block processing cluster, and so on. In the mapping process, the cluster work groups can be preferentially allocated to the currently idle block processing clusters according to the computing capability of the block processing clusters, or matched and scheduled according to the data cached in the local memory to maximize data reuse and reduce repeated loading.

[0077] Therefore, the scheduling manner proposed in the disclosure splits and maps the target workgroup mesh to the computing die and block processing cluster in a hierarchical manner, splits the target workgroup mesh into super workgroups, each of which is a computing die level task package, and is explicitly scheduled to the corresponding computing die, so as to expose the computing die level existing objectively in hardware, then, in the computing die, the super workgroup is split into cluster workgroups again and is scheduled to the corresponding block processing cluster, so as to further expose the block processing cluster level in the computing die, align the scheduling strategy with the multi-level architecture of hardware, and enable the computing die to actively participate in cross-die task cooperation and data interaction as a communication granularity, thereby improving resource utilization.

[0078] It should be noted that in the scheduling process of the workgroup, the target input tensor is split according to each cluster workgroup to obtain a plurality of tensor subblocks, and the tensor subblocks are written into the local memory of the corresponding block processing cluster, so that the block processing cluster reads the tensor subblock corresponding to the cluster workgroup from the corresponding local memory in the process of executing the cluster workgroup.

[0079] In the process of splitting one super workgroup into a plurality of cluster workgroups, the target input tensor is divided into a corresponding number of tensor subblocks according to the number of cluster workgroups and the workgroup responsibilities (such as the tensor region to be processed) contained in each cluster workgroup. The tensor subblock is a small data subset obtained by splitting the target input tensor. The size and dimension of each tensor subblock are strictly aligned with the cluster workgroup computing task allocation, so as to ensure that each block processing cluster can directly load the required data locally, avoid redundant transmission, and for example, if the current cluster workgroup is responsible for processing the “i-th row to j-th row” of the tensor, the tensor subblock corresponding to the cluster workgroup is the data in the row range. Each split tensor subblock is transmitted to the local memory of the target BPC (including the local memory of all computing units in the BPC) through the on-chip network such as the die level interconnection network of the computing die. Therefore, while the cluster workgroup is scheduled to the corresponding block processing cluster, the tensor subblock is also preloaded. The cluster workgroup can independently and directly read the required tensor subblock from the local memory and execute the computing (such as matrix multiplication, convolution) task, without waiting for global memory access or cross-die data transmission, so that multiple BPCs in the same computing die can process different cluster workgroups in parallel, which is helpful for BPC level load balancing, thereby significantly reducing the delay, improving the computing efficiency and enhancing the data locality.

[0080] It should be noted that since the local memory of the computing die also includes local storage resources in the block processing cluster, the preloading of the tensor subblock corresponding to the cluster workgroup is also the data preloading of the super workgroup, that is, the computing die can directly read the tensor data corresponding to the super workgroup from the local memory of the computing die in the process of executing the super workgroup, thereby reducing the cross-die data transmission.

[0081] Referring to Figure 3 , Figure 3 is a schematic diagram of a workgroup scheduling hierarchy provided by an embodiment of the present disclosure, assuming that a chiplet architecture of a data processor includes 4 computing dies, each computing die integrates 4 block processing clusters, each block processing cluster contains 4 computing units, and each computing unit also includes 4 execution units; when the chiplet receives a target workgroup grid of a 16x16 structure, the target workgroup grid can be split into 4 super workgroups (each 4x16), and the 4 super workgroups are correspondingly allocated to the 4 computing dies, then each super workgroup can be split into 4 cluster workgroups (each 4x4) in the computing die, and each cluster workgroup is also correspondingly allocated to the 4 block processing clusters in the computing die; then, each cluster workgroup is split into 4 workgroups (each 4x1) in the block processing cluster, and is scheduled to the 4 computing units in the block processing cluster, further, the threads in the workgroup can be allocated to the 4 execution units by the computing unit, and each execution unit is responsible for processing 1 row of corresponding threads to implement fine-grained parallel computing. At the same time, the target input tensor is also divided into 16 tensor subblocks according to the cluster workgroup, and each tensor subblock is written into the local memory of the corresponding block processing cluster. Since the task division and the data division are strictly aligned, each computing unit can directly access the locally stored tensor subblock when executing, thereby reducing the delay caused by global memory access. As shown in FIG. 2, the target workgroup grid is split into 4 super workgroups (each 4x16), and each super workgroup is correspondingly allocated to the 4 computing dies. Then, each super workgroup is split into 4 cluster workgroups (each 4x4) in the computing die, and each cluster workgroup is also correspondingly allocated to the 4 block processing clusters in the computing die. Then, each cluster workgroup is split into 4 workgroups (each 4x1) in the block processing cluster, and is scheduled to the 4 computing units in the block processing cluster. Further, the threads in the workgroup can be allocated to the 4 execution units by the computing unit, and each execution unit is responsible for processing 1 row of corresponding threads to implement fine-grained parallel computing. At the same time, the target input tensor is also divided into 16 tensor subblocks according to the cluster workgroup, and each tensor subblock is written into the local memory of the corresponding block processing cluster. Since the task division and the data division are strictly aligned, each computing unit can directly access the locally stored tensor subblock when executing, thereby reducing the delay caused by global memory access. Figure 3As shown, the task splitting corresponds to the hardware level one by one, and the super workgroup in the task splitting is a carrier specially designed for the compute die level, and one super workgroup corresponds to only one compute die. The super workgroup division can follow the number of compute dies for even division, and the target workgroup grid is divided into 4 super workgroups (each containing 64 workgroups) along the first dimension (row direction), which corresponds to 4 compute dies one by one, so as to fully utilize the parallel processing capability of all compute dies in the small chip architecture. In the related art, the compute die level and the block processing cluster level are often scheduled together, and the workgroup is directly split into the block processing cluster, which is easy to cause uneven load across dies. For example, if 4 BPCs belong to 2 compute dies (each compute die contains 2 BPCs), it may occur that 2 BPCs of compute die 1 are fully loaded, and 2 BPCs of compute die 2 are idle, which leads to a decrease in resource utilization and an increase in computing delay. Therefore, the present disclosure explicitly introduces a super workgroup as a die-level scheduling unit. When splitting the workgroup grid, each small chip architecture can also dynamically adjust the allocation of the super workgroup according to the load rate of each compute die. In addition, the present disclosure avoids splitting a logically related task to different BPCs of dies, so that all workgroups of a super workgroup are scheduled and executed in only one compute die, and the data interaction in the execution process preferentially uses the interconnection network within the die, reduces cross-die scheduling, and synchronizes the segmentation of the target input tensor with the task scheduling, so as to ensure that each tensor sub-block and the corresponding cluster workgroup remain consistent in physical position. Similarly, the tensor sub-block stored in the local memory of the compute die also corresponds to the super workgroup, so as to ensure that the data can be efficiently reused at the die level and avoid the bandwidth bottleneck caused by frequent access across dies.

[0082] In some possible implementations, the division manner of the super workgroup can include two kinds, one is to divide according to the number of compute dies, and the other is to dynamically divide according to the real-time load of each compute die. Therefore, based on different division manners of the super workgroup, the number of workgroups contained in each super workgroup can be the same or different.

[0083] Among them, the scheme of dividing according to the number of dies is specifically based on the number of compute dies, and the target workgroup grid is split into a plurality of super workgroups, at this time the number of workgroups contained in each super workgroup is the same, and then the workgroups in the super workgroup are scheduled to the corresponding compute die. Therefore, the number of super workgroups after splitting matches the number of actually available compute dies, each super workgroup corresponds to one compute die, and resource waste or overload caused by number imbalance is avoided.

[0084] Specifically, referring to Figure 4 , Figure 4is an optional flow diagram of super workgroup splitting provided by the embodiments of the present disclosure, first count the total number of computing dies actually available in the chiplet architecture (denoted as N), then cut the target workgroup network into N subsets along a certain dimension (such as a dimension that can be divided by N), each subset constitutes a super workgroup, at this time the number of split super workgroups is N, that is, the number of super workgroups is consistent with the number of computing dies, bind the super workgroup and the computing die one by one, simplify the scheduling strategy, and can directly allocate super workgroups according to the physical number order of the computing dies, and all workgroups in the same super workgroup are executed in the same computing die, and data interaction can preferentially use the intra-die interconnection network, reducing cross-die communication.

[0085] Among them, the scheme of dynamically dividing according to the die load is specifically to dynamically adjust the number of workgroups in the super workgroup according to the real-time load of each computing die. Specifically, the computing die can collect real-time load indicators of each die, such as task queue length, memory bandwidth occupancy, execution unit utilization, etc., dynamically adjust the number of workgroups contained in each super workgroup, so that the computing die with lower load is allocated to a super workgroup containing more workgroups, and the computing die with higher load is allocated to a workgroup. Therefore, in the dynamic division based on die load, the number of workgroups contained in the super workgroup can be different.

[0086] Specifically, referring to Figure 5 , Figure 5 is an optional flow diagram of super workgroup splitting provided by the embodiments of the present disclosure, assuming that the chiplet architecture has N computing dies, collecting real-time load indicators of each computing die, obtaining real-time load rates of each die (denoted as L1, L2, …, Ln), and sorting in ascending order according to the real-time load rate, obtaining the sorted die sequence as L1, L3, L2, …, Ln, and distributing the total number of workgroups (T) of the target workgroup grid to each computing die according to the load inverse weight, that is, the computing die with lower load has higher weight, such as load inverse weight wi = 1 / Li, if Li = 0, take the maximum value; calculate the number of workgroups that each computing die should allocate according to the computing die weight, that is, the number of workgroups allocated by the i-th die is Ti = T× (wi / Σwi), and after rounding up or down, a super workgroup of non-equal workgroups is formed, at this time the number of workgroups in each super workgroup matches the real-time load of the corresponding computing die.

[0087] Therefore, the workgroup scheduling method provided by the embodiments of the present disclosure can select different super workgroup division schemes according to different scenarios. For example, in a stable load scenario, the super workgroup can be divided according to the equal number of dies, so as to simplify the scheduling logic; in a complex scenario with fluctuating load, the super workgroup can be divided according to the real-time load of the dies, so as to achieve more fine load balancing. It should be noted that both the two super workgroup division schemes take the computing die level as the core scheduling unit, and take the computing die as an independent level for scheduling, so as to achieve efficient utilization of computing resources under different workloads.

[0088] In some possible implementations, the division of the super workgroup is actually achieved by a continuous area intercepted from the target workgroup grid by a coordinate range, and the coordinate range is determined by the division scheme. The workgroup coordinate is a unique identifier for identifying the position of the workgroup in the target workgroup grid, and the coordinate dimension is consistent with the grid structure. The division of the super workgroup is determined by specifying the "start coordinate and coordinate range", wherein the start coordinate corresponds to the workgroup coordinate of the first workgroup in the division area, and the coordinate range represents the number of workgroups included in each dimension. The start coordinate is calculated by the division scheme, and each super workgroup corresponds to a continuous row and column coordinate interval.

[0089] The data processor can run a driver, and the driver can calculate the corresponding start coordinate and coordinate range of the super workgroup in advance according to a preset rule (such as the number of computing dies, the real-time load of the dies) or the demand of an upper application, and encapsulate the start coordinate and the coordinate range as "split information", that is, the split information is used to indicate the division mode of the super workgroup. After receiving the split information provided by the driver, the start coordinate and the coordinate range of each super workgroup can be extracted from the split information, and the target workgroup grid is split according to the start coordinate, to obtain a plurality of super workgroups. Specifically, the driver can obtain the parameters of the target workgroup grid, such as the grid dimension and the total number of workgroups, obtain the number of computing dies and / or the real-time load rate of each computing die based on the division scheme, generate the start coordinate and the coordinate range of the super workgroup according to the division scheme, encapsulate the start coordinate and the coordinate range as the split information, and then deliver the split information to the command processor or the scheduler. After receiving the split information, all workgroup coordinates of the target workgroup grid can be traversed to determine whether the workgroup coordinates belong to the coordinate range of a super workgroup. If yes, the workgroup is divided into the corresponding super workgroup, the division of the target workgroup grid is completed, and each workgroup belongs to only one super workgroup, so as to avoid resource scheduling conflicts.

[0090] The data processor includes a command processor which can be used to execute a hardware unit of command processing split logic, can receive a split command sent by an external device (such as a driver), and the split command can carry a split strategy (such as dividing according to the number of compute dies or dividing according to the real-time load rate of compute dies) and the dimension of the target workgroup grid. The command processor can obtain the number of compute dies and the real-time load rate, calculate the starting coordinates and coordinate range of each super workgroup based on the split strategy, and directly cut the workgroup from the address space of the target workgroup grid according to the calculation result to generate a super workgroup, and store the starting coordinates of the super workgroup and the corresponding workgroup list in a scheduling queue, waiting for allocation to a compute die.

[0091] In some possible implementations, in the process of dividing the super workgroup, a first position identifier of the super workgroup can be determined based on the global position of the super workgroup in the target workgroup grid; then the workgroups in the super workgroup are scheduled to each compute die, and the first position identifier corresponding to the super workgroup is stored to the first register of the corresponding compute die, so that the kernel of the compute die executes the workgroup required by itself through the first position identifier.

[0092] The compute die includes a first register which can be a storage unit dedicated to the compute die, can be an independently allocated storage area in a register file, and is used for quickly storing and retrieving position information. The first position identifier can be a unique identifier reflecting the absolute coordinate position of the super workgroup in the target workgroup grid, for example, can be the workgroup coordinates and coordinate range of the super workgroup in the target workgroup grid. Storing the first position identifier in the first register synchronously completes the writing operation of the identifier in the scheduling process, so that when the kernel receives the workgroup, the boundary range of the super workgroup in which the compute die is located can be quickly determined by reading the first position identifier in the first register, and then the workgroup to be executed is parsed.

[0093] Specifically, first, when the target workgroup grid is split into multiple super workgroups, for example, assuming that the target workgroup grid is a two-dimensional structure, for the super workgroup located at the starting area of the target workgroup grid, its global position (i.e., the starting coordinates) can be determined as (0, 0), and combined with the number X of workgroups contained in the super workgroup, a unique first position identifier (0, 0, X) is generated, and then in the process of scheduling the workgroup to the computing die, the first position identifier is written to the first register of the corresponding computing die synchronously, so that the first position identifier is bound to the transmission scheduling of the workgroup, so that the kernel of the computing die can immediately analyze the required super workgroup index range such as (0, 0) to (0, X) by directly reading the first position identifier in the local first register, so as to accurately define the processing range in the task execution stage, reduce task execution errors, repetitions or resource idle phenomena, and improve the parallel task execution efficiency and resource utilization rate under the multi-die architecture.

[0094] Wherein, the first position identifier can also be a globally unique serial number of the super workgroup in the target workgroup grid. Specifically, taking the global coordinate system of the target workgroup grid as the reference, all the super workgroups after splitting are sorted according to a preset arrangement rule (such as row priority, column priority), and each super workgroup is assigned a serial number (such as 0, 1, 2…N, N is the total number of super workgroups). Through the grid dimension and the division parameter of the target workgroup grid, this serial number, i.e., the first position identifier, can be converted into the starting coordinates and the coordinate range.

[0095] In one example, assuming that the super workgroups are evenly divided into grids according to the number of computing dies, the first position identifier is sequentially numbered in the order of space sequence with row priority, the target workgroup grid is 16x16, x∈[0, 15], y∈[0, 15], divided by 4 computing dies, cut along the x-axis into 4 super workgroups of 4x16, and the specific division is as follows: the starting coordinates of SWG0 (first position identifier 0) are (0, 0), and the coordinate range is x∈[0, 3], y∈[0, 15]; the starting coordinates of SWG1 (first position identifier 1) are (4, 0), and the coordinate range is x∈[4, 7], y∈[0, 15]; the starting coordinates of SWG2 (first position identifier 2) are (8, 0), and the coordinate range is x∈[8, 11], y∈[0, 15]; the starting coordinates of SWG3 (first position identifier 3) are (12, 0), and the coordinate range is x∈[12, 15], y∈[0, 15]. Therefore, each first position identifier corresponds to a super workgroup at a fixed spatial position, and the first position identifier can be used to locate the corresponding super workgroup, and the starting coordinates and the coordinate range of the super workgroup are converted.

[0096] In one example, it is assumed that the super workgroup is grid-divided according to the real-time load rate of the computing dies, the target workgroup grid is 20x20, x∈[0, 19], y∈[0, 19], but at this time the spatial position and size of the super workgroup are no longer equal, the real-time load rates of the four computing dies are L1=30%, L2=50%, L3=70%, and L4=90%, the dynamic division is as follows: SWG0 (the first position identifier is 0, and is allocated to the low-load computing die 1), 160 workgroups (8x20) are allocated, and the starting coordinates are (0, 0); SWG1 (the first position identifier is 1, and is allocated to the low-load computing die 2), 100 workgroups (5x20) are allocated, and the starting coordinates are (8, 0); SWG2 (the first position identifier is 2, and is allocated to the low-load computing die 3), 80 workgroups (4x20) are allocated, and the starting coordinates are (13, 0); and SWG3 (the first position identifier is 4, and is allocated to the low-load computing die 4), 60 workgroups (3x20) are allocated, and the starting coordinates are (17, 0); at this time, the super workgroups are allocated in ascending order of load, the first position identifiers are also sorted in ascending order of the real-time load rate, and when the super workgroups are divided according to the real-time load rate of the computing dies, the first position identifiers can also include the specific starting coordinates and coordinate range of the super workgroup.

[0097] In some possible implementation manners, the cluster workgroup division is implemented in the computing die, the super workgroup scheduled by the upper level to the computing die is secondarily split into cluster workgroups, and specifically, for each computing die, the super workgroup is secondarily split into multiple cluster workgroups based on a preset dimension parameter or the number of block processing clusters, where each cluster workgroup corresponds to one block processing cluster; and the workgroups in the multiple cluster workgroups belonging to one super workgroup are scheduled to the corresponding block processing cluster in the same computing die.

[0098] The preset dimension parameter can be a user-specified dimension parameter or dimension information of task data, and the division of the cluster workgroup is adjusted according to the preset dimension parameter or the number of block processing clusters, so that the number of split cluster workgroups can be matched with available resources, and resource fragmentation can be avoided.

[0099] It can be understood that a single super workgroup received by the computing die can be split into multiple cluster workgroups according to the preset dimension parameter or the number of block processing clusters, where the number of split cluster workgroups is an integer multiple of the number of block processing clusters of the computing die.

[0100] Specifically, the total number of workgroups of the super workgroup can be evenly divided as a splitting basis in the number of block processing clusters, each portion constituting a cluster workgroup, that is, the total number of workgroups of the super workgroup is counted (denoted as T, such as 64), the number of workgroups Tcwg of a single cluster workgroup is calculated T / M (M is the number of block processing clusters), and then the super workgroup is cut into M cluster workgroups according to the original dimension of the super workgroup. Assuming that M block processing clusters are calculated in a die, and the super workgroup is a two-dimensional grid of WxH, the number of cluster workgroups is (W / a)x(H / b), which needs to satisfy that (W / a)x(H / b) is an integer multiple of M. For example, the super workgroup is 12x12, and if M=4, each block processing cluster can be allocated 4 cluster workgroups of 3x3, forming a continuous task block. At this time, the division of the cluster workgroup adapts to the number of block processing clusters, while ensuring the spatial locality of the task, so that the task data in the same cluster workgroup can be efficiently hit by the cache of the block processing cluster, reducing the memory access delay.

[0101] Specifically, the splitting can be performed based on a preset dimension parameter, and the super workgroup is directly split according to the preset dimension parameter to obtain a plurality of cluster workgroups. The number of cluster workgroups after splitting can be greater than the number of block processing clusters, or can be in a multiple relationship, to ensure that each block processing cluster can undertake the task of one or more cluster workgroups. For example, if the super workgroup is a two-dimensional grid of 16x16, the preset dimension parameter specifies splitting by 4x4 sub-grid, then 16 cluster workgroups can be obtained; if the number of block processing clusters in the calculation die is 8, then each block processing cluster can be allocated 2 adjacent 4x4 cluster workgroups, realizing batch scheduling and parallel processing of tasks. In addition, the division of the cluster workgroup can also dynamically adjust the dimension parameter in combination with the hardware characteristics (such as cache size, number of calculation units) of the block processing cluster, for example, for a block processing cluster with a larger cache capacity, a larger size cluster workgroup (such as 8x8) can be allocated to reduce data transfer overhead and improve local cache utilization.

[0102] It can be understood that the division of the cluster workgroup can also introduce a dynamic adjustment mechanism: the block processing cluster can feed back its load state (such as calculation unit utilization, cache hit rate) in real time, and the calculation die adjusts the size of the cluster workgroup according to these feedbacks: if a block processing cluster has a low load and its cache is idle, then multiple adjacent cluster workgroups can be combined into a larger task block and allocated to it; if the load is high, the existing cluster workgroup can be split into smaller units and distributed to other idle block processing clusters. This dynamic adjustment method can further optimize the utilization of resources in the calculation die, avoid local overload or idleness, and improve the overall task execution efficiency.

[0103] Referring to Figure 6 , Figure 6is a flowchart of cluster workgroup splitting provided by the embodiments of the present disclosure, assuming that a computing die integrates 4 block processing clusters, and the super workgroup assigned to the computing die is a 4x16 structure, the super workgroup is split based on the number of block processing clusters, and the total number of workgroups of the super workgroup is 64, so the number of each cluster workgroup is 16. The super workgroup can be split along the row dimension to obtain 4 cluster workgroups of 1x16 (each cluster workgroup contains 16 workgroups), or split along the column dimension to obtain 4 cluster workgroups of 4x4. Based on the two different splitting methods, the executable thread block structure of the computing unit in the block processing cluster can be preferentially selected. It is worth noting that after splitting the 4 cluster workgroups, they are respectively scheduled to the 4 block processing clusters in the same computing die. Meanwhile, the tensor subblocks corresponding to each cluster workgroup are also written to the local memory of the corresponding block processing cluster, so that the block processing cluster can directly read data from the local memory when executing. Cluster workgroups scheduled to the block processing clusters of the same computing die share intermediate results between workgroups, which can be completed through the die-level interconnection network, reducing cross-die access. The secondary division of cluster workgroups realizes fine-grained resource mapping from the computing die level to the block processing cluster level, so that task scheduling matches the hardware level architecture, improving the parallel processing efficiency in a single die.

[0104] In some possible implementations, based on the division mode of the cluster workgroup, the number of workgroups in each cluster workgroup can be the same or different. Specifically, when the block processing clusters integrated in the computing die are homogeneous, and the number of workgroups in the super workgroup can be divided by the number of block processing clusters, the super workgroup can be uniformly split according to the number of block processing clusters, and the number of workgroups in each cluster workgroup is the same, reducing load calculation and realizing fast splitting and scheduling. When the block processing clusters in the computing die are heterogeneous (such as different numbers of computing units and local memory capacities) or the number of workgroups in the super workgroup cannot be divided by the number of block processing clusters, the number of workgroups in each cluster workgroup needs to be adjusted according to the configuration difference of the block processing clusters to realize on-demand allocation, or the number of workgroups is adjusted according to the real-time load rate of each block processing cluster. For example, there are 4 block processing clusters, and the number of workgroups in the super workgroup is 80, but BPC0 contains 8 computing units, while BPC1 to BPC3 contain only 4 computing units. Therefore, BPC0 can be allocated 32 workgroups of CWG0, and the remaining BPCs are allocated 16 workgroups of CWG1 to CWG3. That is, the high-performance BPC0 processes more workgroups, and the low-performance BPC processes fewer workgroups, thereby improving resource utilization.

[0105] In some possible implementation manners, in the splitting process of the cluster workgroup, the second position identifier of the cluster workgroup can be determined based on the global position of the cluster workgroup in the super workgroup; then the workgroups in the cluster workgroup are scheduled to the corresponding block processing cluster, and the second position identifier corresponding to the cluster workgroup is stored to the second register of the corresponding block processing cluster, so that the kernel running on the computing unit of the block processing cluster performs the workgroup required by itself through the second position identifier.

[0106] The block processing cluster includes the second register, which can be a storage unit dedicated to the block processing cluster and can be implemented by a static random memory register file; the second position identifier can be a unique identifier reflecting the absolute position of the cluster workgroup in the super workgroup. Similarly, the format of the second position identifier can be the same as that of the first position identifier, that is, including the workgroup coordinates and coordinate range of the cluster workgroup in the super workgroup, or the second position identifier can also reflect the sequential number of the cluster workgroup in the super workgroup (such as the i th cluster workgroup in row / column priority). It is worth noting that the second position identifier reflects the relative position of the cluster workgroup in the super workgroup, rather than the position in the entire target workgroup grid.

[0107] It can be understood that, assuming that the super workgroup is a 4*16 structure and includes 64 workgroups, and is split into four cluster workgroups equal to the number of block processing clusters, the starting coordinates and coordinate range of each cluster workgroup in the super workgroup can be used as the second position identifier, or the cluster workgroups can be sequentially numbered in row priority / column priority order, and the number is the second position identifier. The split cluster workgroup is bound to the corresponding block processing cluster in a one-to-one manner, and the workgroups of the cluster workgroup are scheduled to the corresponding block processing cluster. In the cluster workgroup scheduling process, the second position identifier of the cluster workgroup is written into the second register of the corresponding block processing cluster, so that the computing unit in each block processing cluster can read the second position identifier by reading the second register when performing a task, determine the global position information of the workgroup required by itself in the super workgroup based on the second position identifier and the super workgroup provided by the current computing die, realize fast positioning, and then efficiently schedule the task and computing resources.

[0108] In some possible implementation manners, in the calculation die, a cluster workgroup constructor can be integrated, which can be implemented by hardware logic circuit or firmware, and is responsible for dynamically splitting a super workgroup into multiple cluster workgroups according to preset rules. Specifically, the cluster workgroup constructor performs structural analysis on the super workgroup, generates multiple subtasks, and encapsulates each subtask to obtain multiple cluster workgroups. The cluster workgroup constructor first analyzes the metadata of the super workgroup, including logical structures such as dimension size, total workgroup quantity, workgroup layout mode, and the like of the super workgroup, and reads the number of block processing clusters integrated in the current calculation die, divides the super workgroup according to the number of block processing clusters, generates splitting parameters of the cluster workgroup, including splitting quantity, starting coordinates and coordinate range or sequence number of each cluster workgroup, and then based on these splitting parameters, the cluster workgroup constructor can split the workgroup set in the super workgroup into multiple subtasks, each of which corresponds to a cluster workgroup. For example, if the number of block processing clusters is evenly divided, the super workgroup is split along the divisible dimension, each subtask is associated with a continuous grid area in the super workgroup, and the number of workgroups contained in the subtask is the same. For another example, according to the dynamic load adjustment, the cluster workgroup constructor can read the real-time load rate of each block processing cluster, and distribute the workgroups of each subtask according to the load reciprocal weight. After splitting the super workgroup into multiple subtasks, the cluster workgroup constructor can add metadata (such as workgroup coordinates, coordinate range) and a second position identifier to the generated subtasks, and encapsulate them as cluster workgroups.

[0109] In some possible implementation manners, in the scheduling process of the workgroup, the target input tensor can be segmented according to the hardware level to obtain multiple tensor blocks corresponding to different levels, and the tensor blocks are written into the local memory according to the corresponding level, so that the computing tasks executed by the workgroups at different levels and the data stored in the local memory are matched as much as possible, and the data access delay is reduced. Specifically, in the workgroup scheduling process of the super workgroup, the target input tensor is segmented according to each super workgroup to obtain multiple large tensor subblocks, and the target calculation die to which the super workgroup belongs is determined, and the large tensor subblocks are stored in the local memory of the corresponding target calculation die; in the workgroup scheduling process of the cluster workgroup, the large tensor subblocks stored in the local memory of the current calculation die are segmented according to each cluster workgroup to obtain multiple tensor subblocks, and the target block processing cluster to which the cluster workgroup belongs is determined, and the tensor subblocks are stored in the local memory of the corresponding target block processing cluster.

[0110] The large tensor sub-block can refer to an intermediate data block generated in the super workgroup scheduling stage, and the level of the large tensor sub-block is higher than the level of the tensor sub-block, that is, the size of the large tensor sub-block is larger than the size of the tensor sub-block. The large tensor sub-block is a coarse-grained data unit divided based on data required by a computing task executed by the super workgroup, and the tensor sub-block is a fine-grained data unit further subdivided in the cluster workgroup scheduling stage and used to match more fine computing requirements in the block processing cluster. Specifically, after determining the workgroups included in the super workgroup and the target computing die to which the workgroups are allocated, the target input tensor can be divided according to tensor data required by computing tasks corresponding to the workgroups included in the super workgroup, to generate a large tensor sub-block, and the large tensor sub-block is attributed to the super workgroup and written into the local memory of the target computing die corresponding to the super workgroup. The target input tensor is divided into large tensor sub-blocks corresponding to the super workgroup and the computing die level, and the large tensor sub-blocks are stored in the local memory of the corresponding computing die, to establish preliminary data partitioning, so that the computing task executed by the super workgroup matches data in the local memory of the computing die as much as possible, to reduce more fine-grained data cross-die transmission in subsequent task execution and reduce data access delay.

[0111] In the cluster workgroup scheduling process, the tensor sub-block division performed is that, in the current computing die, a stored large tensor sub-block is cut again according to data required by a computing task executed by a cluster workgroup, to obtain multiple tensor sub-blocks with smaller sizes, and the tensor sub-blocks are allocated to the local memory of a block processing cluster under the jurisdiction of the corresponding cluster workgroup, to realize the hierarchical binding of the cluster workgroup, the tensor sub-block, and the block processing cluster. Specifically, after determining the computing tasks of the cluster workgroups and the corresponding block processing clusters in the current computing die, the loaded large tensor sub-block can be finely divided according to the data range required by the task, to generate multiple tensor sub-blocks, and each tensor sub-block is accurately mapped to the local memory of the target block processing cluster. It should be noted that the local memory of the computing die can include the local memory of the block processing cluster, and therefore, in the division and scheduling of the tensor sub-block, the tensor data in the local memory of each block processing cluster can be re-divided and layout-optimized, so that the computing task executed by the cluster workgroup matches data in the local memory of the block processing cluster as much as possible.

[0112] Referring to Figure 7 , Figure 7 is a flowchart of tensor data division provided by the embodiments of the present disclosure. In the AI inference task, when processing an image input tensor, in the super workgroup scheduling stage, the image (target input tensor) is uniformly divided into multiple large tensor sub-blocks according to the column dimension. The division process aligns the task requirements of the super workgroup and the corresponding relationship of the computing die, so that each large tensor sub-block is associated with only a single computing die (for example, the computing die 1 in FIG. 1). Figure 7As shown, according to N computing dies are divided into N large tensor subblocks, and through coordinate mapping, the large tensor subblock is written into the local memory of the computing die to which it belongs, so as to realize preliminary data partition at the die level, and reduce the demand for cross-die data transmission. Subsequently, in the cluster workgroup scheduling stage, the large tensor subblock in the local memory is divided into multiple tensor subblocks according to the row dimension, and according to the task demand of the cluster workgroup and the corresponding relationship between the block processing cluster, the tensor subblock is written into the local memory of the corresponding block processing cluster, so that each block processing cluster processes the data subset in the local die as much as possible, and the data access delay is reduced. The scheduling process of the super workgroup realizes the binding of the task and the computing die on the data through the die-level storage of the large tensor subblock, activates the parallel computing capability of the computing die, and helps load balancing. In the scheduling process of the cluster workgroup, the fine binding of the task and the block processing cluster on the data is realized through the block processing cluster-level storage of the tensor subblock, and the fine-grained data parallelism and data locality in the block processing cluster are further optimized. It can be seen that, in addition to the matching of the task scheduling strategy and the hardware level, the tensor data partition strategy is also matched with the hardware level structure, so that the task scheduling is consistent with the physical distribution of the computing resources, the data flow follows the hierarchical path from the computing die to the block processing cluster, and the access delay of cross-level data transmission is effectively eliminated.

[0113] It can be understood that, although each step in each of the above flowcharts is displayed in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified in this embodiment, the execution of these steps does not have strict order restrictions, and these steps can be executed in other orders. Moreover, at least part of the steps in the above flowcharts can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps.

[0114] Referring to Figure 8 , Figure 8 An optional structural schematic diagram of the workgroup scheduling apparatus 800 provided by the embodiment of the present disclosure is provided, which includes a data and task acquisition unit 810, a first workgroup splitting unit 820, a second workgroup splitting unit 830, and a data scheduling unit 840, specifically:

[0115] The data and task acquisition unit 810 is configured to acquire a target input tensor and a target workgroup grid corresponding to the target input tensor, and the target workgroup grid includes a plurality of workgroups.

[0116] The first workgroup splitting unit 820 is configured to split the target workgroup grid to obtain a plurality of super workgroups, and schedule workgroups in the super workgroups to respective computing dies.

[0117] The second workgroup splitting unit 830 is configured to, for each computing die, perform secondary splitting on the super workgroup to obtain a plurality of cluster workgroups, and schedule workgroups in the cluster workgroups to respective block processing clusters.

[0118] The data scheduling unit 840 is configured to, in the scheduling process of the workgroups, split the target input tensor according to each cluster workgroup to obtain a plurality of tensor subblocks, and write the tensor subblocks into local memories of the corresponding block processing clusters, so that the block processing clusters read the tensor subblocks corresponding to the cluster workgroups from the corresponding local memories in the process of executing the cluster workgroups.

[0119] In some possible implementation manners, the disclosure embodiments further propose that the first workgroup splitting unit 820 is further configured to:

[0120] split the target workgroup grid into a plurality of super workgroups based on the number of computing dies, wherein each super workgroup corresponds to one computing die;

[0121] schedule workgroups in the super workgroup to the corresponding computing die.

[0122] In some possible implementation manners, the disclosure embodiments further propose that the first workgroup splitting unit 820 is further configured to determine a first position identifier of the super workgroup based on a global position of the super workgroup in the target workgroup grid, then schedule workgroups in the super workgroup to respective computing dies, and store the first position identifier corresponding to the super workgroup to a first register of the corresponding computing die, so that the kernel of the computing die executes the workgroup required by itself through the first position identifier.

[0123] In some possible implementation manners, the disclosure embodiments further propose that the second workgroup splitting unit 830 is further configured to, for each computing die, perform secondary splitting on the super workgroup to obtain a plurality of cluster workgroups based on a preset dimension parameter or the number of block processing clusters, wherein each cluster workgroup corresponds to one block processing cluster, and then schedule workgroups in a plurality of cluster workgroups belonging to one super workgroup to the corresponding block processing cluster in the same computing die.

[0124] In some possible implementation manners, the disclosure embodiments further provide that the second workgroup splitting unit 830 is further configured to determine a second position identifier of the cluster workgroup based on a global position of the cluster workgroup in the super workgroup; and then schedule a workgroup in the cluster workgroup to a corresponding block processing cluster and store the corresponding second position identifier of the cluster workgroup to a second register of the corresponding block processing cluster, so that a kernel running on a computing unit of the block processing cluster performs a workgroup required by the kernel itself through the second position identifier.

[0125] In some possible implementation manners, the disclosure embodiments further provide that the data scheduling unit 840 is further configured to, in the workgroup scheduling process of the super workgroup, split a target input tensor according to each super workgroup to obtain a plurality of large tensor subblocks, and determine a target computing die to which the super workgroup belongs, and store the large tensor subblocks to a local memory of the corresponding target computing die; and in the workgroup scheduling process of the cluster workgroup, split the large tensor subblocks stored in the local memory of the current computing die according to each cluster workgroup to obtain a plurality of tensor subblocks, and determine a target block processing cluster to which the cluster workgroup belongs, and store the tensor subblocks to a local memory of the corresponding target block processing cluster.

[0126] In some possible implementation manners, the disclosure embodiments further provide that the first workgroup splitting unit 820 is further configured to receive splitting information provided by a driver, extract a starting coordinate of each super workgroup from the splitting information, and split a target workgroup grid according to the starting coordinate to obtain a plurality of super workgroups; or use a command processor to calculate the starting coordinate of each super workgroup in response to an external splitting command, and split the target workgroup grid according to the starting coordinate to obtain a plurality of super workgroups.

[0127] In some possible implementation manners, the disclosure embodiments further provide that the second workgroup splitting unit 830 is further configured to use a cluster workgroup constructor to perform structural analysis on the super workgroup, generate a plurality of subtasks, and encapsulate each subtask to obtain a plurality of cluster workgroups.

[0128] The disclosure embodiments further provide an electronic device including at least one processor and a memory connected with the at least one processor in communication; wherein the processor and the memory are connected in communication through a data bus, the memory stores a program, and the program is executed by the at least one processor to enable the at least one processor to implement the method in any of the above embodiments of the disclosure.

[0129] The following will be described in combination with Figure 9 The hardware structure of the electronic device is described in detail. The electronic device includes a processor 910, a memory 920, an input / output interface 930, a communication interface 940, and a bus 950.

[0130] The processor 910 can be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present disclosure.

[0131] The memory 920 can be implemented by a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 920 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present disclosure are implemented by software or firmware, the related program codes are stored in the memory 920 and are called and executed by the processor 910 to implement the workgroup scheduling method of the embodiments of the present disclosure.

[0132] The input / output interface 930 is configured to implement information input and output.

[0133] The communication interface 940 is configured to implement the communication interaction between the device and other devices. The communication can be implemented by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0134] The bus 950 is configured to transmit information between various components (for example, the processor 910, the memory 920, the input / output interface 930, and the communication interface 940) of the device.

[0135] The processor 910, the memory 920, the input / output interface 930, and the communication interface 940 are connected to each other through the bus 950 to realize the communication connection between the device.

[0136] The embodiments of the present disclosure further provide a computer readable storage medium, which is configured to store a computer program. The computer program is configured to execute the workgroup scheduling method of the above-mentioned various embodiments.

[0137] The embodiments of the present disclosure further provide a computer program product, which includes a computer program stored in a computer readable storage medium. A processor of a computer device reads the computer program from the computer readable storage medium, and the processor executes the computer program to make the computer device execute the workgroup scheduling method.

[0138] The terms "first", "second", "third", "fourth" and the like in the description of the disclosure and the above drawings, if any, are used to distinguish similar objects, and do not necessarily have to be described in a particular order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the disclosure described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "contain" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device containing a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0139] It should be understood that in the present disclosure, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0140] It should be understood that in the description of the embodiments of the present disclosure, the meaning of multiple (or multiple items) is two or more, and greater than, less than, more than, etc. are not included in the number, and above, below, etc. are included in the number.

[0141] In several embodiments provided by the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division, and actual implementation can have another division manner. For example, multiple units or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0142] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0143] In addition, each functional unit in various embodiments of the present disclosure can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0144] It should also be understood that various embodiments provided by the present disclosure can be combined in any manner to achieve different technical effects.

[0145] The above is a specific description of the embodiments of the present disclosure, but the present disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present disclosure, and these equivalent modifications or replacements are included in the scope defined by the claims of the present disclosure.

Claims

1. A workgroup scheduling method, characterized in that, Applied to a data processor, the data processor including multiple computing dies, each computing die including multiple block processing clusters, each block processing cluster including a second register, the workgroup scheduling method includes: Obtain the target input tensor and the target workgroup grid corresponding to the target input tensor, wherein the target workgroup grid includes multiple workgroups; The target workgroup grid is split into multiple super workgroups, and the workgroups in the super workgroups are scheduled to each of the computing dies; For each of the aforementioned computing dies, the super workgroup is split a second time to obtain multiple cluster workgroups, and the second position identifier of the cluster workgroup is determined based on the global position of the cluster workgroup in the super workgroup; The workgroup in the cluster workgroup is scheduled to the corresponding block processing cluster, and the second location identifier corresponding to the cluster workgroup is stored in the second register of the corresponding block processing cluster, so that the kernel of the computing unit running in the block processing cluster executes the workgroup it needs to process through the second location identifier; In the scheduling process of the working groups, the target input tensor is divided into multiple tensor quantum blocks according to each cluster working group, and the tensor quantum blocks are written into the local memory of the corresponding block processing cluster, so that the block processing cluster can read the tensor quantum blocks corresponding to the cluster working group from the local memory during the execution of the cluster working group.

2. The workgroup scheduling method according to claim 1, characterized in that, The step of splitting the target workgroup grid into multiple super workgroups and scheduling the workgroups in the super workgroups to the respective computing dies includes: Based on the number of computing dies, the target working group grid is divided into multiple super working groups, wherein each super working group corresponds to one computing die; The workgroups in the super workgroup are scheduled to the corresponding computing dies.

3. The workgroup scheduling method according to claim 1, characterized in that, The computing die includes a first register, and the step of scheduling the workgroups in the super workgroup to each of the computing dies includes: The first location identifier of the super workgroup is determined based on the global location of the super workgroup in the target workgroup grid; The workgroups in the super workgroup are scheduled to each of the computing dies, and the first location identifier corresponding to the super workgroup is stored in the first register of the corresponding computing die, so that the kernel of the computing die executes the workgroup it needs to process through the first location identifier.

4. The workgroup scheduling method according to any one of claims 1 to 3, characterized in that, The number of working groups in each of the superworking groups may be the same or different.

5. The workgroup scheduling method according to claim 1, characterized in that, For each of the aforementioned computing dies, the super workgroup is further split into multiple cluster workgroups, and the workgroups within the cluster workgroups are scheduled to each of the aforementioned block processing clusters, including: For each of the aforementioned computing dies, based on a preset dimension parameter or the number of the block processing clusters, the super workgroup is further divided into multiple cluster workgroups, wherein each cluster workgroup corresponds to one of the block processing clusters; Workgroups belonging to the same super workgroup are scheduled to the corresponding block processing cluster in the same computing die.

6. The workgroup scheduling method according to claim 1 or 5, characterized in that, The number of working groups in each cluster working group may be the same or different.

7. The workgroup scheduling method according to claim 1, characterized in that, During the scheduling process of the workgroup, the target input tensor is divided according to each cluster workgroup to obtain multiple tensor quantum blocks, and the tensor quantum blocks are written into the local memory of the corresponding block processing cluster, including: During the scheduling process of the super workgroup, the target input tensor is divided according to each super workgroup to obtain multiple large quantum blocks, and the target computing die to which the super workgroup belongs is determined. The large quantum blocks are then stored in the local memory of the corresponding target computing die. During the workgroup scheduling process of the cluster workgroup, the large quantum blocks stored in the local memory of the current computing die are divided according to each cluster workgroup to obtain multiple quantum blocks, and the target block processing cluster to which the cluster workgroup belongs is determined. The quantum blocks are then stored in the local memory of the corresponding target block processing cluster.

8. The workgroup scheduling method according to claim 1, characterized in that, The data processor includes a command processor, which splits the target workgroup grid to obtain multiple super workgroups, including: Receive the splitting information provided by the driver, extract the starting coordinates of each super workgroup from the splitting information, and split the target workgroup grid according to the starting coordinates to obtain multiple super workgroups; Alternatively, the command processor can respond to external splitting commands, calculate the starting coordinates of each super workgroup, and split the target workgroup grid according to the starting coordinates to obtain multiple super workgroups.

9. The workgroup scheduling method according to claim 1, characterized in that, The computing die also includes a cluster workgroup constructor, which performs a secondary split on the super workgroup to obtain multiple cluster workgroups, including: The super workgroup is structured using the cluster workgroup constructor to generate multiple subtasks, and each subtask is encapsulated to obtain multiple cluster workgroups.

10. A workgroup scheduling device, characterized in that, The workgroup scheduling device includes multiple computing dies, each computing die including multiple block processing clusters, each block processing cluster including a second register, and the workgroup scheduling device further includes: The data and task acquisition unit is used to acquire a target input tensor and a target workgroup grid corresponding to the target input tensor, wherein the target workgroup grid includes multiple workgroups; The first workgroup splitting unit is used to split the target workgroup grid into multiple super workgroups, and schedule the workgroups in the super workgroups to each of the computing dies; The second workgroup splitting unit is used to split the super workgroup into multiple cluster workgroups for each computing die, and determine the second position identifier of the cluster workgroup based on the global position of the cluster workgroup in the super workgroup; schedule the workgroup in the cluster workgroup to the corresponding block processing cluster, and store the second position identifier of the cluster workgroup in the second register of the corresponding block processing cluster, so that the kernel of the computing unit running in the block processing cluster can execute the workgroup it needs to process through the second position identifier; The data scheduling unit is used to divide the target input tensor according to each of the cluster work groups during the scheduling process of the work groups, obtain multiple tensor quantum blocks, and write the tensor quantum blocks into the local memory of the corresponding block processing cluster, so that the block processing cluster can read the tensor quantum blocks corresponding to the cluster work group from the local memory during the execution of the cluster work group.

11. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. The program is executed by the processor to implement the workgroup scheduling method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the workgroup scheduling method as described in any one of claims 1 to 9.

Images