Method and apparatus for verifying a computing unit based on a multi-level operator generation

Abstract

This application provides a method and apparatus for verifying computing units based on multi-level operator generation, relating to the field of chip design and manufacturing technology. The method includes: dividing a tensor to be processed into multiple sub-tensors based on a preset number of execution units, and dividing the sub-tensors into multiple data blocks based on a preset data transfer specification; dividing the execution resources of the computing unit into a first execution group and a second execution group; executing transfer instructions based on the first execution group to transfer the tensor to be processed from the global memory buffer to the group shared memory in units of data blocks; and executing calculation instructions based on the second execution group to perform calculations on each data block in the group shared memory in units of preset calculation data specifications. The method and apparatus provided in this application, by performing multi-level segmentation of tensors, associate tensor operations with hardware execution, enabling the automatic generation of complex operators that conform to the actual working mode of the hardware, greatly improving the verification efficiency and completeness of the computing unit.

Description

Technical Field

[0001] This application relates to the field of chip design and manufacturing technology, and in particular to a method and apparatus for verifying computing units based on multi-level operator generation. Background Technology

[0002] To ensure the correctness of chip functionality, performance compliance, and overall stability before tape-out, comprehensive and rigorous testing and verification of the General Computing Unit (GCU), including the Tensor Core, must be conducted during the front-end verification phase of chip design. One of the core aspects of this process is generating and executing various Tensor Core operators, which are combinations of instructions used to simulate real-world application scenarios and cover various boundary conditions.

[0003] Related technologies generate tensor core operators through the following methods: chip verification engineers manually construct operators based on their understanding of chip operating scenarios; upper-level software development engineers provide operators for actual applications; and operators are generated through automated scripts or simple frameworks. Manually constructed operators typically only cover a portion of the anticipated verification scenarios, resulting in severely insufficient verification coverage; operators using actual applications lack the necessary randomness; and operators generated through scripts or simple frameworks often have rudimentary functionality and typically cannot support complex verification scenarios.

[0004] Therefore, how to generate a large number of operators with randomness and complexity to verify computing units and improve the completeness and efficiency of computing unit verification has become a technical problem that the industry urgently needs to solve. Summary of the Invention

[0005] This application provides a method and apparatus for verifying computing units based on multi-level operator generation, which solves the technical problem of how to generate a large number of operators with randomness and complexity to verify computing units, thereby improving the verification completeness and efficiency of computing units.

[0006] This application provides a method for verifying computational units based on multi-level operator generation, including: The tensor to be processed is divided into multiple sub-tensors based on a preset number of execution units, and the sub-tensors are divided into multiple data blocks based on a preset data transport specification. The execution resources of the computing unit are divided into a first execution group for executing transport instructions and a second execution group for executing computation instructions. Based on the first execution group, the transport instructions are executed to transport the tensors to be processed from the global memory buffer to the group shared memory in units of the data blocks; Based on the second execution group, the calculation instructions are executed to perform calculations on each data block in the shared memory of the group in units of preset calculation data specifications, and the calculation results of the tensor to be processed are generated.

[0007] In some embodiments, at least one of the following: the size of the tensor to be processed, the size of the sub-tensor, the size of the data block, the preset number of execution units, the preset data transfer specification, or the preset computational data specification, can be configured by the user or generated randomly within a preset constraint range.

[0008] In some embodiments, the method further includes: When the number of the second execution group is equal to one, the calculations are performed serially on the data block; If the number of the second execution group is greater than one, the computation is performed on the data block in parallel.

[0009] In some embodiments, the execution resource is a thread bundle; the first execution group and the second execution group are thread bundle groups.

[0010] In some embodiments, the method further includes: Data synchronization is performed between the first execution group and the second execution group based on at least one barrier buffer.

[0011] In some embodiments, the data synchronization between the first execution group and the second execution group based on at least one barrier buffer includes: After the first execution group completes the data transfer of the current data block, it updates the state of the first barrier buffer and indicates to the second execution group that the data of the current data block is ready. Upon detecting a state update of the first barrier buffer, the second execution group executes computation instructions on the current data block; When the second execution group completes the calculation of the current data block, it updates the state of the second barrier buffer, indicates to the first execution group that the group shared memory region of the current data block can be reused, and triggers the first execution group to perform data transfer for the next data block.

[0012] In some embodiments, the method further includes: The number of calculation instructions is determined based on the size of the data block and the preset calculation data specifications; The number of thread-level registers required to process the data block is determined based on the number of computation instructions and the number of thread-level registers required by the computation instructions.

[0013] This application provides a computational unit verification device based on multi-level operator generation, comprising: The tensor segmentation module is used to segment the tensor to be processed into multiple sub-tensors based on a preset number of execution units, and to segment the sub-tensors into multiple data blocks based on a preset data transfer specification; The resource partitioning module is used to divide the execution resources of the computing unit into a first execution group for executing transport instructions and a second execution group for executing computing instructions; The data transfer module is used to execute transfer instructions based on the first execution group, and transfer the tensor to be processed from the global memory buffer to the group shared memory in units of the data block; The data calculation module is used to execute calculation instructions based on the second execution group, and to perform calculations on each data block in the shared memory of the group in units of preset calculation data specifications, so as to generate the calculation result of the tensor to be processed.

[0014] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the aforementioned computational unit verification method based on multi-level operator generation.

[0015] This application provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned computational unit verification method based on multi-level operator generation.

[0016] The computational unit verification method and apparatus based on multi-level operator generation provided in this application achieves the connection between macroscopic tensor operations and low-level hardware execution (data transport and core computation) through two intermediate levels: sub-tensors and data blocks by multi-level segmentation of the tensor to be processed. It also manages execution resources through functional grouping. This overcomes the shortcomings of related technologies, such as low operator coverage when manually constructing operators and the inability of simple scripts to handle complex data dependencies and memory management. It makes the operator generation logic clear and management efficient, and can automatically generate complex operators that conform to the actual working mode of the hardware, thereby greatly improving the verification efficiency and completeness of the computational unit. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the computational unit verification method based on multi-level operator generation provided in this application.

[0020] Figure 2 This is a schematic diagram of the first-level segmentation of the tensor to be processed provided in this application.

[0021] Figure 3 This is a schematic diagram of the second-level segmentation of the tensor to be processed provided in this application.

[0022] Figure 4 This is a schematic diagram of the execution resource grouping provided in this application.

[0023] Figure 5 This is a schematic diagram of the computation performed within the data block provided in this application.

[0024] Figure 6 This is a schematic diagram of the barrier buffer provided in this application.

[0025] Figure 7 This is a schematic diagram of the structure of the computing unit verification device based on multi-level operator generation provided in this application.

[0026] Figure 8 This is a schematic diagram of the structure of the electronic device provided in this application.

[0027] Figure label: 710: Tensor partitioning module; 720: Resource partitioning module; 730: Data transfer module; 740: Data calculation module; 810: Processor; 820: Communication interface; 830: Memory; 840: Communication bus. Detailed Implementation

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps, units, or modules is not necessarily limited to those explicitly listed, but may include other steps, units, or modules not explicitly listed or inherent to such processes, methods, products, or devices.

[0030] In order to address the shortcomings of related technologies, Figure 1 This is a flowchart illustrating the computational unit verification method based on multi-level operator generation provided in this application, as shown below. Figure 1 As shown, the method includes steps 110, 120, 130 and 140.

[0031] Step 110: Divide the tensor to be processed into multiple sub-tensors based on the preset number of execution units, and divide the sub-tensors into multiple data blocks based on the preset data transfer specifications.

[0032] Specifically, the execution entity of the computational unit verification method based on multi-level operator generation provided in this application is a computational unit verification device or system. This device can be implemented in software, such as a computational unit verification program; or it can be a device that executes the computational unit verification method based on multi-level operator generation, such as a processor, terminal, computer, or server.

[0033] A computing unit refers to a general computing unit (GCU), a streaming multiprocessor (SM) in a graphics processing unit (GPU), or any other hardware unit containing a programmable execution core, which contains resources for performing computing tasks.

[0034] The tensor to be processed, as input data, typically refers to a multidimensional array. For example, in the matrix multiplication operation D = A * B, the input matrices A and B, as well as the final output matrix D, can all be considered tensors to be processed, each with a dimensional size (Dim size). The dimension of matrix D is M * N, the dimension of matrix A is M * K, and the dimension of matrix B is K * N. M, K, and N are all positive integers.

[0035] The preset number of thread groups (tg num) is a parameter used to guide the first-level splitting of the tensor to be processed. This number can be a fixed value or an array, or it can be a value or array predetermined based on the hardware specifications of the computing units. For example, the preset number of thread groups can include values ​​in three dimensions, corresponding to the three dimensions of the tensor to be processed, and can be represented as an array (m, n, k).

[0036] Figure 2 This is a schematic diagram of the first-level segmentation of the tensor to be processed provided in this application, as shown below. Figure 2 As shown, in this embodiment, the tensor to be processed has dimensions (M, N, K). After being divided according to the preset number of execution units (m, n, k), multiple sub-tensors can be obtained. The dimensions of each sub-tensor are (M / m, N / n, K / k). Through this division, a large computational task can be decomposed into multiple sub-tasks that can be processed on different execution units.

[0037] Next, a second-level decomposition of the tensor to be processed is performed based on the sub-tensor. The preset data transfer specification refers to the amount of data that a single data transfer instruction (e.g., a load instruction) can process. Based on this specification, this embodiment further divides each sub-tensor into multiple smaller-granularity data tiles. The preset data transfer specification can be a fixed value or an array, or a pre-determined value or array. For example, the preset data transfer specification may include values ​​in three dimensions, corresponding to the three dimensions of the sub-tensor, and can be represented as an array (p, q, t). Based on the size of the sub-tensor, corresponding storage space can be allocated for each sub-tensor in the global memory buffer.

[0038] Figure 3 This is a schematic diagram of the second-level segmentation of the tensor to be processed provided in this application, as shown below. Figure 3 As shown, the subtensor has dimensions (M / m, N / n, K / k). After being divided according to the preset data transfer specifications (p, q, t), it yields multiple data tiles. Each data tile has dimensions (M / m / p, N / n / q, K / k / t). The size of each data tile corresponds to the processing capacity of one data transfer instruction. Therefore, a data tile can be understood as the basic unit for data transfer between memory and the computing core. Based on the size of the data tile, corresponding storage space can be allocated for each data tile in the shared memory.

[0039] Through the above two-level segmentation, the embodiments of this application logically decompose a complex, large-size tensor operation into a series of more manageable atomic operation sequences for standardized data blocks.

[0040] Step 120: Divide the execution resources of the computing unit into a first execution group for executing transport instructions and a second execution group for executing computing instructions.

[0041] Specifically, execution resources are the physical or logical units within a computing unit that actually execute instructions. In a specific embodiment, an execution resource can be a warp in a chip, that is, a set of threads that can synchronously execute the same instructions. An execution group is a logical grouping of these execution resources, specifically manifested as a warp group.

[0042] In this embodiment of the application, not all execution resources perform the same task, but rather they are functionally divided.

[0043] Figure 4 This is a schematic diagram of the execution resource grouping provided in this application, such as... Figure 4 As shown, multiple thread bundles (e.g., 32 thread bundles, thread bundle 0 to thread bundle 31) within a computing unit are divided into multiple thread bundle groups (e.g., 8 thread bundle groups, thread bundle group 0 to thread bundle group 7).

[0044] The first execution group is specifically designated for executing transport instructions, such as load instructions LDA (Load Matrix A), LDB (Load Matrix B), and LDC (Load Matrix C). Transport instructions are responsible for moving data, such as loading data from one level of storage (e.g., global memory) to another level of storage (e.g., shared memory), or writing computation results from shared memory back to global memory. In specific implementations, this first execution group can consist of one or more thread bundles, such as thread bundle 0.

[0045] The second execution group is specifically designated for executing computational instructions, such as matrix multiplication and accumulation (MMA) instructions. These instructions are responsible for performing core mathematical operations on the data that has already been moved to the designated location. In specific implementations, this second execution group can consist of one or more thread bundles, such as thread bundle 1 to thread bundle 7.

[0046] This functional division allows for the decoupling of data flow (managed by the first execution group) and computation flow (managed by the second execution group), enabling more efficient pipelined operations and more flexible scheduling management, laying the foundation for subsequent implementation of complex serial-parallel control and synchronization mechanisms.

[0047] Step 130: Execute the transport instructions based on the first execution group, and transport the tensors to be processed from the global memory buffer to the group shared memory in units of data blocks.

[0048] Specifically, global memory buffers (GLM) typically refer to a large but relatively high-latency main storage area within a computing unit, used to store raw, complete tensor data to be processed. Group shared memory (GSM), on the other hand, is a smaller but extremely fast on-chip cache shared by a group of execution resources, used to temporarily store data about to be computed.

[0049] The first execution group, designated for data movement, begins its work. It executes movement instructions, reading the appropriate portions of the tensors to be processed from the global memory buffer in blocks and writing them into the group's shared memory. Each movement operation targets a single data block. This process prepares operands for subsequent computational steps.

[0050] Step 140: Execute calculation instructions based on the second execution group, perform calculations on each data block in the group's shared memory in units of preset calculation data specifications, and generate the calculation results of the tensor to be processed.

[0051] Specifically, after the first execution group moves one or more data blocks to the group shared memory, the second execution group, which has been designated for computation, begins to execute computation instructions.

[0052] Preset computation data specifications refer to the dimensions of operands that a single computation instruction (such as an MMA instruction) can process.

[0053] Figure 5 This is a schematic diagram illustrating the computation performed within the data block provided in this application, such as... Figure 5 As shown, within a data tile, multiple computation instructions may be required to complete the computation of the entire data tile. The second execution group reads data from the shared memory of the group, performs computations, and stores intermediate results in lower-level registers (such as thread-level registers), using a preset computation data specification as the unit. When all computations for a data tile are completed, its result can be written out (e.g., by executing a store instruction from the first execution group), thus forming part of the final result of the tensor to be processed. This process is performed sequentially or in parallel for all data tiles until the computation of the entire tensor to be processed is completed.

[0054] The computing unit verification method based on multi-level operator generation provided in this application embodiment achieves the association between macroscopic tensor operations and low-level hardware execution (data transport and core computation) through two intermediate levels: sub-tensors and data blocks by multi-level segmentation of the tensor to be processed. It also manages execution resources by functional grouping, overcoming the shortcomings of related technologies such as low operator coverage when manually constructing operators and the inability of simple scripts to handle complex data dependencies and memory management. This makes the operator generation logic clear and management efficient, and can automatically generate complex operators that conform to the actual working mode of the hardware, thereby greatly improving the verification efficiency and completeness of the computing unit.

[0055] It should be noted that each implementation method of this application can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0056] In some embodiments, at least one of the following can be configured by the user or generated randomly within a preset constraint range: the size of the tensor to be processed, the size of the subtensor, the size of the data block, the preset number of execution units, the preset data transfer specification, or the preset computation data specification.

[0057] Specifically, user-defined configuration refers to allowing engineers or users performing computational unit verification to manually specify a precise value for a parameter based on a specific test objective. Randomization within preset constraints means that, without the user specifying a specific value, a value can be automatically and randomly selected as a parameter within a legal range determined by hardware specifications, project requirements, or other rules. The following parameters support user-defined configuration or randomization within preset constraints.

[0058] The size of the tensor to be processed refers to the dimension of the top-level input or output tensor. Users can customize the size of the tensor to be processed to simulate computational tasks of different scales. If the user does not configure it, the system can randomly generate it within a range (e.g., the maximum size supported by the hardware) based on project specifications to explore various possible task scales, thereby improving the coverage of validation.

[0059] Subtensor size or preset number of execution units: These two parameters are interrelated. The size of the subtensor is determined by both the size of the tensor to be processed and the preset number of execution units. Users can directly configure the preset number of execution units to control how many subtasks a large task is broken down into. For example, users can configure the preset number of execution units to 1 to simulate a single-task execution scenario, or configure it to a larger value to simulate a high-concurrency scenario. If the user does not configure it, the system can randomly select a value within the range supported by the hardware (e.g., between 1 and the maximum number of execution units) to automatically generate test cases with different degrees of parallelism.

[0060] Data block size or preset data transfer specification: These two parameters are also interrelated. The size of the data block is determined by the specification of a single data transfer instruction. Users can define a custom preset data transfer specification, specifying the size of the data block transferred in a single load instruction. This is crucial for verifying the hardware's support for data transfer instructions of different granularities. For example, a very large transfer specification can be configured to test memory bandwidth stress, or a very small transfer specification can be configured to test instruction scheduling overhead. If the user does not configure it, the system can randomly select one from several transfer specifications supported by the hardware to ensure that all valid instruction specifications are fully verified.

[0061] Preset computation data specifications refer to the data dimensions processed by a single computation instruction (such as an MMA instruction). Modern computing units typically support multiple MMA instruction specifications (e.g., processing matrix blocks of different sizes). Users can specify which specification of computation instruction to use to precisely verify the functionality and performance of a particular instruction. If the user does not configure it, the system can randomly select a valid computation instruction specification from the instruction set supported by the hardware, ensuring that all computation modes are tested.

[0062] The computational unit verification method based on multi-level operator generation provided in this application can not only meet the user's needs for accurate scene reproduction and targeted testing, but also automatically explore a large number of unknown boundary conditions and combined scenarios through randomization mechanisms, thereby making up for the deficiency of insufficient coverage of manually constructed operators; it greatly improves the completeness and efficiency of verification, ensuring that the generated operators not only meet the user's specific verification intentions, but also widely cover the actual working scenarios of the hardware.

[0063] In some embodiments, the method further includes: When the number of execution groups is equal to one, computations are performed serially on the data blocks; If the number of execution groups is greater than one, computations are performed on the data blocks in parallel.

[0064] Specifically, the number of the second execution group becomes a key setting parameter controlling the execution mode of the entire computing process. This number can be configured by the user according to verification needs, or it can be randomly generated by the system within a preset range.

[0065] When the number of execution groups is equal to one, only one execution group is designated for executing computation instructions within the entire computation unit. This means that at any given time, at most one data block is being computed.

[0066] For example, depending on user or system configuration, the first execution group (warp group 0) is responsible for data movement; the second execution group (warp group 1) is responsible for data computation. In the specific execution flow, the first execution group (Warp group 0) moves the data of the first data block (Tile 0) from global memory to the group's shared memory. The sole second execution group (Warp group 1) performs computation on data block (Tile 0). During or after the computation by the second execution group (Warp group 1), the first execution group (Warp group 0) moves the data of the second data block (Tile 1). After completing the computation on data block (Tile 0), the second execution group (Warp group 1) then performs computation on data block (Tile 1). This cycle continues until all data blocks have been processed sequentially by the second execution group (Warp group 1).

[0067] This execution method is known as serial computation. It is suitable for verifying the correctness of computational units in single-task, sequential processing scenarios, such as verifying the correctness of basic functions like data dependencies, synchronization logic, and resource reuse.

[0068] When the number of second execution groups is greater than one, multiple execution groups are designated to execute computation instructions within the entire computing unit.

[0069] For example, depending on user or system configuration, the first execution group (warp group 0) is responsible for data movement; the second execution group (warp group 1 through warp group 3) is responsible for data computation. In the specific execution flow, the first execution group (Warp group 0) prepares data for multiple data blocks (e.g., Tile 0, Tile 1, Tile 2) simultaneously or sequentially, and moves them to different areas of the group's shared memory. Once the data is ready, multiple second execution groups can start computation simultaneously. Warp group 1 is responsible for computing Tile 0, Warp group 2 for computing Tile 1, and Warp group 3 for computing Tile 2. The computation processes of these three data blocks overlap in time, achieving parallel computation. When a second execution group completes its computation task, it can immediately begin computing the next available data block.

[0070] This parallel execution mode is ideal for performance verification, testing the computing unit's processing power, resource contention, power consumption, and heat dissipation under high load and high concurrency conditions. By adjusting the number of the second execution groups (e.g., increasing from 2 to 7), different levels of parallel load can be simulated, thereby comprehensively evaluating the computing unit's performance and scalability.

[0071] The computing unit verification method based on multi-level operator generation provided in this application directly associates the number of the second execution group with the serial-parallel mode of computation, enabling verifiers to easily switch between functional verification (which typically focuses on the correctness of serial logic) and performance verification (which focuses on the ability to process in parallel) by adjusting only one parameter. This not only greatly enriches the verifiable scenarios that can be generated, but also makes it possible to systematically analyze and evaluate the behavior of computing units under different load pressures, thereby significantly improving the completeness of the verification.

[0072] In some embodiments, the execution resource is a thread bundle; the first execution group and the second execution group are thread bundle groups.

[0073] Specifically, execution resources correspond to thread bundles. A thread bundle typically consists of a fixed number (e.g., 32) of threads. Under the management of the hardware scheduler, these threads execute the same instruction synchronously in a single-instruction, multi-threaded manner, but each thread can process different data. The thread bundle is the basic scheduling unit for instruction dispatch and execution in a computing unit.

[0074] Accordingly, execution groups specifically correspond to thread bundle groups. A thread bundle group is a logical grouping of one or more thread bundles. This is achieved by dividing different thread bundles into different thread bundle groups and assigning specific tasks to each thread bundle group.

[0075] The computing unit verification method based on multi-level operator generation provided in this application clearly defines execution resources and their groups as thread bundles and thread bundle groups, making full use of the hardware hierarchy that is common in modern computing units. This division based on the inherent hardware hierarchy allows the operator generation logic to be directly mapped to the scheduling instructions of the underlying hardware, ensuring the effectiveness and execution efficiency of the generated operators.

[0076] In some embodiments, the method further includes: Data synchronization is performed between the first execution group and the second execution group based on at least one barrier buffer.

[0077] Specifically, the first execution group moves data from the global memory buffer to the group's shared memory; the second execution group retrieves data from the group's shared memory for computation. To ensure the correctness of the overall computation result, it is essential to ensure that the second execution group begins computation only after the first execution group has completed data preparation, and also to prevent the first execution group from prematurely overwriting data while the second execution group is still using it. Without an effective synchronization mechanism, computational errors are highly likely, rendering verification meaningless.

[0078] A barrier buffer (MBAR) is a hardware or software synchronization primitive used in embodiments of this application. It can be understood as one or a set of special status registers or memory addresses, whose states (e.g., values ​​of 0 or 1) can be read and written by different execution groups. By agreeing on the meaning of different state values, one execution group can send a signal to another execution group by changing the state of the barrier buffer, while the other execution group monitors the state to wait for a specific event to occur, thus forming an execution barrier to ensure the order of operations.

[0079] By setting up a barrier buffer, data synchronization is achieved between the first and second execution groups. For example, the first execution group executes a data transfer instruction (read) to complete the transfer. After the transfer is complete, a signal is sent to the second execution group by changing the state of the barrier buffer. The second execution group then executes a calculation instruction to begin matrix operations. Alternatively, the second execution group executes a calculation instruction, completes the matrix operation, and then sends a signal to the first execution group by changing the state of the barrier buffer. The first execution group then executes a data transfer instruction (write) to complete the transfer. Only after the transfer is complete can the storage space be released for the next matrix operation.

[0080] The computational unit verification method based on multi-level operator generation provided in this application establishes a reliable coordination barrier between the first execution group responsible for data transport and the second execution group responsible for data computation based on the barrier buffer synchronization mechanism. This effectively solves the data competition and timing problems that may occur in pipeline or parallel processing, ensuring that computation is always performed only after the data is fully ready, thereby guaranteeing the determinism and correctness of the execution result of each operator.

[0081] In some embodiments, data synchronization between a first execution group and a second execution group is performed based on at least one barrier buffer, including: Once the first execution group has completed the data transfer of the current data block, it updates the state of the first barrier buffer and indicates to the second execution group that the data of the current data block is ready. Upon detecting a state update in the first barrier buffer, the second execution group executes computation instructions on the current data block. Once the second execution group has completed the computation of the current data block, it updates the state of the second barrier buffer, instructs the first execution group that the group shared memory region of the current data block can be reused, and triggers the first execution group to perform data transfer for the next data block.

[0082] Specifically, to achieve efficient execution between data transfer and data computation—that is, to allow the first execution group to begin preparing data for the next data block while the second execution group is performing computation—a more refined synchronization mechanism is needed to manage the state of the buffers in the shared memory of the groups. A single data-ready signal is insufficient; a buffer-free signal is also required. Therefore, embodiments of this application employ a method where the first barrier buffer and the second barrier buffer work in concert.

[0083] Figure 6 This is a schematic diagram of the barrier buffer provided in this application, as shown below. Figure 6 As shown, the first barrier buffer Mbar 0 is used to send a data ready signal; the second barrier buffer Mbar 1 is used to send a buffer ready signal. The following explanation uses the example of the first execution group executing a transport instruction (LDA) and the second execution group executing a calculation instruction (MMA).

[0084] In the initial state, it is assumed that the initial values ​​of the first barrier buffer Mbar 0 and the second barrier buffer Mbar 1 are both 0.

[0085] When processing the first data block, Tile 0, the first execution group begins executing its data transfer instructions, moving the matrix A data required for Tile 0 from global memory to a designated area in the group's shared memory. Once the first execution group has completed all data transfer for Tile 0, it updates the state of the first barrier buffer, Mbar 0, for example, by setting the value of Mbar 0 from 0 to 1. This action explicitly indicates to the second execution group that the data for Tile 0 is ready in the group's shared memory.

[0086] Before executing the computation instructions, the second execution group continuously monitors the state of Mbar 0. When it observes that the value of Mbar 0 becomes 1, it knows that it is safe to begin computation. The second execution group then begins reading the data from Tile 0 in the group shared memory and executes the matrix multiplication and accumulation computation instructions.

[0087] In practical applications, computation instructions often require additional input data (such as matrix B), meaning they need to simultaneously monitor another first-barrier buffer associated with the LDB instruction. For simplicity, this section focuses on the interaction between LDA and MMA.

[0088] Once the second execution group has executed the computation instructions and completed all calculations for Tile 0 (meaning it no longer needs to read the area in the shared memory where Tile 0 data is stored), the second execution group will update the state of the second barrier buffer Mbar 1. For example, it will set the value of Mbar 1 from 0 to 1. This action is equivalent to sending a buffer ready signal to the first execution group, indicating that the memory area storing Tile 0 data is now free and can be overwritten.

[0089] Meanwhile, the first execution group executes the data transfer instructions. After completing the transfer of Tile 0, it does not immediately begin transferring data from Tile 1. Instead, it monitors the status of Mbar 1. When it observes that the value of Mbar 1 becomes 1, it knows that the buffer it previously used has been released by the calculation instructions. Only then can it safely start performing data transfer for the next data block, Tile 1, writing the new data into this reusable memory area.

[0090] To handle continuous data block streams, the barrier buffer's state is periodically toggled. For example, when processing the second data block, Tile 1, after the first execution group completes the transfer of Tile 1, it resets the value of Mbar 0 from 1 back to 0. The second execution group, upon detecting that the value of Mbar 0 has become 0, begins calculating Tile 1. After the second execution group completes the calculation of Tile 1, it resets the value of Mbar 1 from 1 back to 0. The first execution group, upon detecting that the value of Mbar 1 has become 0, begins transferring Tile 2.

[0091] Through this alternation of 0 and 1, a stable and efficient synchronization mechanism is formed between the first and second execution groups, enabling data transfer and computation to be tightly connected like a pipeline, thereby maximizing the hiding of memory access latency and improving the overall throughput of the computing unit.

[0092] In this embodiment, the first execution group executes LDA instructions, and the second execution group executes MMA instructions, requiring two barrier buffers to maintain data synchronization. In some embodiments, if the computation unit performs more complex calculations, such as matrix D = matrix A * matrix B + matrix C, the computation instructions may require other transport instructions, such as LDB, LDC, and STD (StoreMatrix D), which could require eight barrier buffers to maintain data synchronization.

[0093] The computing unit verification method based on multi-level operator generation provided in this application realizes efficient reuse of group shared memory resources by setting a first barrier buffer and a second barrier buffer, and enables data transfer and computation to be executed with high overlap; it greatly improves the performance of the entire verification process and can more accurately simulate and verify the real performance of computing units in high-throughput scenarios.

[0094] In some embodiments, the method further includes: The number of computation instructions is determined based on the size of the data block and the preset computation data specifications. The number of thread-level registers required to process a data block is determined based on the number of computation instructions and the number of thread-level registers required for those instructions.

[0095] Specifically, when a computing unit executes a computational task, in addition to needing shared memory to temporarily store operands, it also requires lower-level, faster storage resources to store intermediate results and final output. These resources are typically thread-level registers (TLRs), which are register files private to each thread. Because this resource is limited, the required number of thread-level registers must be precisely calculated and allocated when generating operators to ensure successful execution without failure due to insufficient resources.

[0096] The size of the data block is the result of the second-level data decomposition. For example, the data block is a matrix block of size (M / m / p, N / n / q).

[0097] The default calculation data specification refers to the size of the matrix block that a single calculation instruction (such as an MMA instruction) can process, such as 16x16.

[0098] The number of computation instructions can be obtained by dividing the size of the data block by the preset computation data specification.

[0099] Suppose a data block is 64x64 in size, while the hardware supports a default computational data size of 16x16. Therefore, to fully compute this 64x64 data block, 64 / 16 = 4 MMA instructions need to be executed in both dimensions. Thus, the total number of computational instructions required is 4 * 4 = 16 MMA instructions.

[0100] The number of thread-level registers required for a computation instruction is a fixed value determined by the hardware design, referring to how many TLRs are needed to execute the output of a single computation instruction. For example, the output of an MMA instruction may occupy 1, 2, or 4 TLR units, depending on the hardware design and the instruction specification.

[0101] Multiplying the number of computation instructions by the number of thread-level registers required for each computation instruction yields the number of thread-level registers required to process a data block.

[0102] For example, in one scenario, each MMA instruction's output requires one TLR. Since processing one data block requires 16 MMA instructions, the total number of TLRs required is 16 * 1 = 16. In another scenario, the project specifications change, or the user chooses a different MMA instruction, whose output requires two TLRs. Again, 16 MMA instructions are needed, so the total number of TLRs required becomes 16 * 2 = 32.

[0103] During the operator generation phase, the method in this embodiment performs the above calculations and generates corresponding resource request instructions to ensure that the computing unit has reserved a sufficient number of thread-level registers before the operator is actually executed. This process is automated; the operator generator dynamically calculates and requests resources based on user-configured or randomly generated size specifications.

[0104] The computing unit verification method based on multi-level operator generation provided in this application closely links the upper-level data organization logic (data block size) with the lower-level hardware resource requirements (thread-level registers). It can accurately calculate and request the required number of registers according to different data splitting granularity and instruction specifications, avoiding verification failures caused by incorrect or insufficient resource estimation.

[0105] The apparatus provided in the embodiments of this application is described below. The apparatus described below can be referred to in correspondence with the method described above.

[0106] Figure 7 This is a schematic diagram of the structure of the computing unit verification device based on multi-level operator generation provided in this application, as shown below. Figure 7 As shown, the device includes: Tensor segmentation module 710 is used to segment the tensor to be processed into multiple sub-tensors based on a preset number of execution units, and to segment the sub-tensors into multiple data blocks based on a preset data transfer specification; The resource partitioning module 720 is used to partition the execution resources of the computing unit into a first execution group for executing transport instructions and a second execution group for executing computing instructions. Data transport module 730 is used to execute transport instructions based on the first execution group, and transport tensors to be processed from the global memory buffer to the group shared memory in units of data blocks; The data calculation module 740 is used to execute calculation instructions based on the second execution group, and to perform calculations on each data block in the group shared memory in units of preset calculation data specifications, and generate the calculation results of the tensor to be processed.

[0107] The computing unit verification device based on multi-level operator generation provided in this application embodiment achieves the association between macroscopic tensor operations and low-level hardware execution (data transport and core computation) through two intermediate levels: sub-tensors and data blocks, by performing multi-level segmentation of the tensor to be processed. It also manages the execution resources in functional groups, overcoming the shortcomings of related technologies such as low operator coverage when manually constructing operators and the inability of simple scripts to handle complex data dependencies and memory management. This makes the operator generation logic clear and management efficient, and can automatically generate complex operators that conform to the actual working mode of the hardware, thereby greatly improving the verification efficiency and completeness of the computing unit.

[0108] Figure 8 This is a schematic diagram of the structure of the electronic device provided in this application, such as... Figure 8 As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communications bus 840, wherein the processor, communications interface, and memory communicate with each other via the communications bus. The processor can invoke logical commands stored in the memory to execute the methods described in the above embodiments, for example: The tensor to be processed is divided into multiple sub-tensors based on a preset number of execution units, and the sub-tensors are divided into multiple data blocks based on a preset data transport specification. The execution resources of the computing unit are divided into a first execution group for executing transport instructions and a second execution group for executing computation instructions. The transport instructions are executed based on the first execution group, and the tensor to be processed is transported from the global memory buffer to the group shared memory in units of data blocks. The computation instructions are executed based on the second execution group, and the computation is performed on each data block in the group shared memory in units of a preset computation data specification to generate the computation result of the tensor to be processed.

[0109] Furthermore, the logical commands in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several commands to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] The processor in the electronic device provided in this application embodiment can call logical instructions in the memory to implement the above method. Its specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effect, which will not be repeated here.

[0111] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods provided in the above embodiments.

[0112] The specific implementation method is the same as the aforementioned method implementation method and can achieve the same beneficial effects, so it will not be repeated here.

[0113] This application provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0114] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0115] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for verifying computational units based on multi-level operator generation, characterized in that, include: The tensor to be processed is divided into multiple sub-tensors based on a preset number of execution units, and the sub-tensors are divided into multiple data blocks based on a preset data transport specification. The execution resources of the computing unit are divided into a first execution group for executing transport instructions and a second execution group for executing computation instructions. Based on the first execution group, the transport instructions are executed to transport the tensors to be processed from the global memory buffer to the group shared memory in units of the data blocks; Based on the second execution group, the calculation instructions are executed to perform calculations on each data block in the shared memory of the group in units of preset calculation data specifications, and the calculation results of the tensor to be processed are generated.

2. The method for verifying computational units based on multi-level operator generation according to claim 1, characterized in that, The size of the tensor to be processed, the size of the sub-tensor, the size of the data block, the number of preset execution units, the preset data transport specification, or the preset calculation data specification can be at least one of the following: user-defined configuration or randomized generation within preset constraints.

3. The method for verifying computational units based on multi-level operator generation according to claim 1, characterized in that, The method further includes: When the number of the second execution group is equal to one, the calculations are performed serially on the data block; If the number of the second execution groups is greater than one, the computations on the data blocks are performed in parallel.

4. The method for verifying computational units based on multi-level operator generation according to claim 1, characterized in that, The execution resource is a thread bundle; the first execution group and the second execution group are thread bundle groups.

5. The method for verifying computational units based on multi-level operator generation according to claim 1, characterized in that, The method further includes: Data synchronization is performed between the first execution group and the second execution group based on at least one barrier buffer.

6. The method for verifying computational units based on multi-level operator generation according to claim 5, characterized in that, The data synchronization between the first execution group and the second execution group based on at least one barrier buffer includes: After the first execution group completes the data transfer of the current data block, it updates the state of the first barrier buffer and indicates to the second execution group that the data of the current data block is ready. Upon detecting a state update of the first barrier buffer, the second execution group executes computation instructions on the current data block; When the second execution group completes the calculation of the current data block, it updates the state of the second barrier buffer, indicates to the first execution group that the group shared memory region of the current data block can be reused, and triggers the first execution group to perform data transfer for the next data block.

7. The method for verifying computational units based on multi-level operator generation according to claim 1, characterized in that, The method further includes: The number of calculation instructions is determined based on the size of the data block and the preset calculation data specifications; The number of thread-level registers required to process the data block is determined based on the number of computation instructions and the number of thread-level registers required by the computation instructions.

8. A verification device for computing units based on multi-level operator generation, characterized in that, include: The tensor segmentation module is used to segment the tensor to be processed into multiple sub-tensors based on a preset number of execution units, and to segment the sub-tensors into multiple data blocks based on a preset data transfer specification; The resource partitioning module is used to divide the execution resources of the computing unit into a first execution group for executing transport instructions and a second execution group for executing computing instructions; The data transfer module is used to execute transfer instructions based on the first execution group, and transfer the tensor to be processed from the global memory buffer to the group shared memory in units of the data block; The data calculation module is used to execute calculation instructions based on the second execution group, and to perform calculations on each data block in the shared memory of the group in units of preset calculation data specifications, so as to generate the calculation result of the tensor to be processed.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the computational unit verification method based on multi-level operator generation as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the computational unit verification method based on multi-level operator generation as described in any one of claims 1 to 7.

Images