Tensor core pre-simulation verification method and device, electronic equipment and storage medium

Abstract

The application discloses a tensor core pre-simulation verification method and device, electronic equipment and a storage medium. The tensor data carrying module is subjected to first unit test verification; the matrix operation unit is subjected to second unit test verification; the tensor core integrated with the tensor data carrying module and the matrix operation unit is subjected to module-level verification; the stream multiprocessor integrated with the tensor core is subjected to first integrated verification; the collaborative work between the tensor cores in a plurality of stream multiprocessors integrated with the tensor cores is subjected to second integrated verification; and in response to the first unit test verification, the second unit test verification, the module-level verification, the first integrated verification and the second integrated verification all passing, it is determined that the pre-simulation verification of the tensor core passes. An efficient and comprehensive pre-simulation verification method of the tensor core is provided, which is helpful to find and solve problems in the early stage of chip design.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of simulation verification, in particular to a pre-simulation verification method and device for a tensor core, an electronic device and a storage medium. BACKGROUND

[0002] In current graphics processing unit (GPU) artificial intelligence chip architectures, a tensor core is usually integrated in a stream multi-processor (SM) level and used to perform high-performance matrix operations to accelerate tasks such as deep learning and scientific computing.

[0003] In related technologies, there is no related method for verifying a tensor core in a pre-simulation stage. SUMMARY

[0004] In view of the above, the present application provides a pre-simulation verification method and device for a tensor core, an electronic device and a storage medium, which can provide an efficient and comprehensive pre-simulation verification method for a tensor core, thereby helping to find and solve problems at an early stage of chip design.

[0005] To solve the above technical problems, the technical solution of the present application is as follows:

[0006] In one embodiment, a pre-simulation verification method for a tensor core is provided, which comprises the following steps in a simulation environment:

[0007] performing first unit test verification on a tensor data handling module;

[0008] performing second unit test verification on a matrix operation unit;

[0009] performing module-level verification on a tensor core integrated with the tensor data handling module and the matrix operation unit;

[0010] performing first integrated verification on a stream multi-processor integrated with the tensor core;

[0011] performing second integrated verification on the collaborative work between tensor cores in a plurality of stream multi-processors integrated with the tensor core;

[0012] in response to the first unit test verification, the second unit test verification, the module-level verification, the first integrated verification and the second integrated verification all passing, determining that the pre-simulation verification of the tensor core passes.

[0013] Before the module-level verification on the tensor core integrated with the tensor data handling module and the matrix operation unit, the method further comprises:

[0014] if it is determined that the first unit test is passed and the second unit test is passed, performing module-level verification on the tensor core integrated with the tensor data transfer module and the matrix operation unit; otherwise, determining that the pre-simulation verification on the tensor core is failed;

[0015] After the module-level verification on the tensor core integrated with the tensor data transfer module and the matrix operation unit, and before the first integration verification on the stream multi-processor integrated with the tensor core, the method further comprises:

[0016] if it is determined that the module-level verification is passed, performing the first integration verification on the stream multi-processor integrated with the tensor core; otherwise, determining that the pre-simulation verification on the tensor core is failed;

[0017] After the first integration verification on the stream multi-processor integrated with the tensor core, and before the second integration verification on the collaborative work between the tensor cores in the plurality of stream multi-processors integrated with the tensor core, the method further comprises:

[0018] if it is determined that the first integration verification is passed, performing the second integration verification on the collaborative work between the tensor cores in the plurality of stream multi-processors integrated with the tensor core; otherwise, determining that the pre-simulation verification on the tensor core is failed;

[0019] After the second integration verification on the collaborative work between the tensor cores in the plurality of stream multi-processors integrated with the tensor core is completed, the method further comprises:

[0020] if it is determined that the second integration verification is failed, determining that the pre-simulation verification on the tensor core is failed.

[0021] The first unit test verification on the tensor data transfer module comprises:

[0022] constructing a verification environment of the tensor data transfer module, the verification environment comprising a first instruction sequence generator, a first reference model, a first result checker, a first performance monitor and a first coverage collector;

[0023] generating and sending a test instruction sequence to the tensor data transfer module through the first instruction sequence generator, and performing function test and performance test on the tensor data transfer module;

[0024] comparing, through the first result checker, an output result generated by the tensor data transfer module after executing the test instruction sequence with an expected output result generated by the first reference model based on the same input, to verify the functional correctness of the tensor data transfer module;

[0025] The performance of the tensor data transfer module is determined by collecting and analyzing data through the first performance monitor and confirming whether the analyzed performance data meets the first preset performance target.

[0026] The first coverage collector collects the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process; and determines the completeness of the first unit test verification of the tensor data transfer module based on the functional coverage and the performance coverage.

[0027] The second unit test verification of the matrix operation unit includes:

[0028] A verification environment for the matrix operation unit is constructed, the verification environment including a matrix data generator, a second reference model, a second result checker, a second performance monitor, and a second coverage collector;

[0029] The matrix data generator generates multiple sets of test matrix data, and inputs the test matrix data into the matrix operation unit so that the matrix operation unit can perform matrix operations; wherein, the test matrix data is used to perform operation mode testing, data format testing, matrix dimension testing, special value testing, and accumulation testing on the matrix operation unit;

[0030] The output matrix generated by the matrix operation unit after performing the operation is compared with the expected output matrix generated by the second reference model based on the same input by the second result checker to verify the functional correctness of the matrix operation unit under dynamic simulation.

[0031] During the functional testing process, the second performance monitor collects and analyzes data to determine whether the performance of the matrix operation unit meets the second preset performance target.

[0032] The second coverage collector collects the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process; and determines the completeness of the second unit test verification performed on the matrix operation unit based on the functional coverage and performance coverage.

[0033] The module-level verification of the tensor core, which integrates the tensor data transport module and the matrix operation unit, includes:

[0034] A tensor core verification environment integrating a tensor data transport module and a matrix operation unit is constructed. This verification environment includes a second instruction sequence generator, a third reference model, a third result checker, a third performance monitor, and a third coverage collector.

[0035] The second instruction sequence generator sends a test instruction sequence to the tensor core to enable the tensor core to perform matrix calculation tasks; the test instruction sequence is used to perform instruction testing, control flow testing, data flow testing, instruction parallelism testing, and performance testing on the tensor core.

[0036] The final output of the tensor core after executing the test instruction sequence is compared with the expected output of the third reference model based on the same input by the third result checker to verify the functional correctness of the tensor core.

[0037] The performance of the tensor core is determined by collecting and analyzing data through a third performance monitor to confirm whether the obtained performance data meets the third preset performance target.

[0038] The third coverage collector collects the functional coverage achieved during functional testing and the performance coverage achieved during performance testing; and judges the completeness of the module-level verification of the Tensor Core based on the functional coverage and performance coverage.

[0039] The first integration verification of the streaming multiprocessor integrating the tensor core includes:

[0040] A stream multiprocessor-level verification environment is constructed, integrating the tensor core with a shared memory module, a load storage unit, and a vector core; the verification environment includes a first stimulus generator, a fourth reference model, a fourth result checker, a fourth coverage collector, and a fourth performance monitor;

[0041] The first stimulus generator applies verification stimuli to the streaming multiprocessor to enable the tensor core to work collaboratively with the shared memory module, the loading storage unit, and the vector core; wherein, the verification stimuli are used to perform data path testing, concurrent operation testing, synchronization mechanism testing, and end-to-end scenario testing on the streaming multiprocessor;

[0042] The fourth result checker compares the final output of the streaming multiprocessor after applying verification stimulus with the expected output of the fourth reference model based on the same input to verify the functional correctness of the tensor core integrated in the streaming multiprocessor.

[0043] The fourth performance monitor collects and analyzes data to determine whether the performance meets the fourth preset performance target.

[0044] The fourth coverage collector collects the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process; and determines the completeness of the first integration verification of the streaming multiprocessor with integrated tensor cores based on the functional coverage and performance coverage.

[0045] The second integration verification of the collaborative operation among tensor cores in multiple streaming multiprocessors integrating the tensor cores includes:

[0046] A multi-stream multiprocessor cluster-level verification environment is constructed to perform collaborative verification on multiple stream multiprocessors integrating tensor cores; the verification environment includes a second stimulus generator, a fifth reference model, a fifth result checker, a fifth performance monitor, and a fifth coverage collector;

[0047] The second incentive generator applies multi-threaded workload incentives to multiple streaming multiprocessors, enabling tensor cores in the multiple streaming multiprocessors to execute distributed matrix computation tasks; wherein, the multi-threaded workload incentives are used to perform data broadcasting and reuse testing, block parallel computation testing, and global synchronization mechanism testing on the cluster;

[0048] The fifth result checker compares the final output of multiple streaming multiprocessor clusters after applying multi-threaded workload incentives with the expected output of the fifth reference model based on the same input to verify the correctness of the collaborative work between tensor cores in multiple streaming multiprocessors with integrated tensor cores.

[0049] The fifth performance monitor collects and analyzes data to determine whether the performance meets the fifth preset performance target.

[0050] The fifth coverage collector collects the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process; and judges the completeness of the second integration verification of the collaborative work between tensor cores in multiple stream multiprocessors with integrated tensor cores based on the functional coverage and performance coverage.

[0051] In another embodiment, a pre-simulation verification device for tensor cores is provided, wherein the device includes:

[0052] The first verification unit is used to perform the first unit test verification of the tensor data transfer module.

[0053] The second verification unit is used to perform second unit testing and verification on the matrix operation unit.

[0054] The third verification unit is used to perform module-level verification on the tensor core that integrates the tensor data transport module and the matrix operation unit;

[0055] The fourth verification unit is used to perform the first integration verification on the stream multiprocessor that integrates the tensor core;

[0056] The fifth verification unit is used to perform a second integration verification on the collaborative operation among tensor cores in multiple stream multiprocessors that integrate the tensor cores.

[0057] A determination unit is used to determine that the pre-simulation verification of the tensor core has passed in response to the first unit test verification, the second unit test verification, the module-level verification, the first integration verification, and the second integration verification all passing verification.

[0058] In another embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a pre-simulation verification method for a tensor core.

[0059] In another embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements a pre-simulation verification method for a tensor core.

[0060] As can be seen from the above technical solution, in the above embodiments, the tensor data transfer module and the matrix operation unit are individually tested and verified in the simulation environment; the tensor core integrating the tensor data transfer module and the matrix operation unit is verified at the module level; integration verification is performed at the streaming multiprocessor level; and integration verification is performed between multi-stream multiprocessor cores. When all these verifications pass, the pre-simulation verification of the tensor core is confirmed to be successful. This solution can provide an efficient and comprehensive pre-simulation verification method for tensor cores, which helps to discover and solve problems in the early stages of chip design. Attached Figure Description

[0061] Figure 1 This is a schematic diagram of the pre-simulation verification process of a tensor core in an embodiment of this application;

[0062] Figure 2 This is a schematic diagram of the pre-simulation verification process for another tensor core in this application embodiment;

[0063] Figure 3 This is a schematic diagram of the tensor core structure integrating the tensor data transport module and the matrix operation unit in an embodiment of this application;

[0064] Figure 4 This is a schematic diagram of a streaming multiprocessor structure integrating a tensor core in an embodiment of this application;

[0065] Figure 5 This is a schematic diagram of the integrated structure of a stream multiprocessor with multiple integrated tensor cores in an embodiment of this application;

[0066] Figure 6 This is a schematic diagram of the pre-simulation verification device for the tensor core in the embodiments of this application;

[0067] Figure 7 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0068] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0069] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe the order or sequence of objects. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0070] The technical solution of the present invention will be described in detail below with reference to specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0071] In the relevant technologies, no methods have been disclosed for verifying tensor cores in the pre-simulation stage, making it difficult to identify and solve problems in the early stages of chip design.

[0072] To address the aforementioned technical issues, this application provides a pre-simulation verification method for tensor cores. In a simulation environment, the method involves performing unit testing on the tensor data transfer module (TMA) and the matrix operation unit (GMC) separately; performing module-level verification on the tensor core integrating the TMA and GMC; performing integration verification at the streaming multiprocessor (SM) level; and performing integration verification between multi-stream multiprocessor cores. When all these verifications pass, the pre-simulation verification of the tensor core is confirmed as successful. This approach provides an efficient and comprehensive pre-simulation verification method for tensor cores, thereby helping to identify and resolve problems early in the chip design process.

[0073] Tensor cores are dedicated hardware units within GPUs used to accelerate matrix multiplication and addition operations. Their core characteristic is mixed-precision computing, enabling throughput and energy efficiency far exceeding traditional computing units when performing AI tasks. Essentially, they are hardware circuits customized for specific computing modes and are a key technology supporting modern artificial intelligence computing. A single tensor core can complete a 4x4 or 16x16 matrix block operation in one clock cycle. A modern GPU integrates hundreds or thousands of tensor cores, working in parallel to provide an astonishing TFLOPS (trillion floating-point operations per second) of computing power.

[0074] In this embodiment of the application, during verification, in the simulation environment, it is necessary to perform five parts of verification on the content related to the tensor core, namely:

[0075] The tensor data transfer module was tested using the first unit test (UT) to verify its ability to decode instructions and transfer data as the front end of the control unit.

[0076] The second unit test verification of the matrix operation unit mainly verifies the correctness of the algorithm for calculating the core array and accumulator register.

[0077] The tensor core, which integrates a tensor data transport module and a matrix operation unit, is verified at the module level (BT). The main verification focuses on the collaborative operation between the control and scheduling unit and the computing unit.

[0078] The first integration verification of the streaming multiprocessor with the Tensor Core is performed, which mainly tests the cooperation between the Tensor Core interface and other modules in the interconnected streaming multiprocessor, such as shared memory.

[0079] A second integration verification is performed on the collaborative operation between tensor cores in multiple stream multiprocessors with integrated tensor cores; that is, integration verification is performed between multi-stream multiprocessor cores.

[0080] The pre-simulation verification process of the tensor core in the embodiments of this application is described in detail below with reference to the accompanying drawings. The pre-simulation verification of the tensor core under test is implemented through simulation.

[0081] See Figure 1 , Figure 1 This is a schematic diagram of the pre-simulation verification process for a tensor core in an embodiment of this application. The specific steps are as follows:

[0082] Step 101: Perform the first unit test verification on the tensor data transfer module.

[0083] Step 102: Perform a second unit test and verification on the matrix operation unit.

[0084] Step 103: Perform module-level verification on the tensor core that integrates the tensor data transfer module and the matrix operation unit.

[0085] Step 104: Perform the first integration verification on the stream multiprocessor with integrated tensor cores.

[0086] Step 105: Perform a second integration verification on the collaborative operation between tensor cores in multiple stream multiprocessors that integrate tensor cores.

[0087] Step 106: In response to the successful verification of the first unit test, the second unit test, the module-level verification, the first integration verification, and the second integration verification, it is determined that the pre-simulation verification of the tensor core has passed.

[0088] In this embodiment, the tensor data transfer module and the matrix operation unit are individually tested and verified; the tensor core integrating the tensor data transfer module and the matrix operation unit is verified at the module level; integration verification is performed at the streaming multiprocessor level; and integration verification is performed between multi-stream multiprocessor cores. When all these verifications pass, the pre-simulation verification of the tensor core is confirmed to be successful. This approach provides an efficient and comprehensive pre-simulation verification method for tensor cores, which helps to identify and solve problems in the early stages of chip design.

[0089] In another example,

[0090] The first unit test verification of the tensor data transfer module includes:

[0091] A verification environment for the tensor data transfer module is constructed. The verification environment includes a first instruction sequence generator, a first reference model, a first result checker, a first performance monitor, and a first coverage collector.

[0092] The first instruction sequence generator generates and sends a test instruction sequence to the tensor data transfer module to perform functional and performance tests on the tensor data transfer module.

[0093] The output of the tensor data transfer module after executing the test instruction sequence is compared with the expected output of the first reference model based on the same input by the first result checker to verify the functional correctness of the tensor data transfer module.

[0094] The performance of the tensor data transfer module is determined by collecting and analyzing data through the first performance monitor to confirm whether the obtained performance data meets the first preset performance target.

[0095] The first coverage collector collects the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process; and judges the completeness of the first unit test verification of the tensor data transfer module based on the functional coverage and performance coverage.

[0096] In another example,

[0097] The second unit test verification of the matrix operation unit includes:

[0098] A verification environment for the matrix operation unit is constructed, which includes a matrix data generator, a second reference model, a second result checker, a second performance monitor, and a second coverage collector.

[0099] Multiple sets of test matrix data are generated by a matrix data generator and input into the matrix operation unit so that the matrix operation unit can perform matrix operations; the test matrix data is used to test the operation mode, data format, matrix dimension, special value, and accumulation of the matrix operation unit.

[0100] The second result checker compares the output matrix generated by the matrix operation unit after performing the operation with the expected output matrix generated by the second reference model based on the same input to verify the functional correctness of the matrix operation unit under dynamic simulation.

[0101] During the functional testing process, the second performance monitor collects and analyzes data to confirm whether the obtained performance data meets the second preset performance target, thereby determining whether the performance of the matrix operation unit meets the standard.

[0102] The second coverage collector collects the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process; and judges the completeness of the second unit test verification of the matrix operation unit based on the functional coverage and performance coverage.

[0103] In another example,

[0104] Module-level verification was performed on the tensor core, which integrates a tensor data transfer module and a matrix operation unit, including:

[0105] A tensor core verification environment integrating a tensor data transport module and a matrix operation unit is constructed. This verification environment includes a second instruction sequence generator, a third reference model, a third result checker, a third performance monitor, and a third coverage collector.

[0106] The second instruction sequence generator sends a test instruction sequence to the tensor core to enable the tensor core to perform matrix calculation tasks; the test instruction sequence is used to perform instruction testing, control flow testing, data flow testing, instruction parallelism testing, and performance testing on the tensor core.

[0107] The final output of the tensor core after executing the test instruction sequence is compared with the expected output of the third reference model based on the same input by the third result checker to verify the functional correctness of the tensor core.

[0108] The performance of the tensor core is determined by collecting and analyzing data through a third performance monitor to confirm whether the obtained performance data meets the third preset performance target.

[0109] The third coverage collector collects the functional coverage achieved during functional testing and the performance coverage achieved during performance testing; and judges the completeness of the module-level verification of the Tensor Core based on the functional coverage and performance coverage.

[0110] In another example,

[0111] The first integration verification of the streaming multiprocessor with integrated tensor cores includes:

[0112] A streaming multiprocessor-level verification environment is constructed, integrating the tensor core with a shared memory module, a load storage unit, and a vector core; the verification environment includes a first stimulus generator, a fourth reference model, a fourth result checker, a fourth coverage collector, and a fourth performance monitor;

[0113] The first stimulus generator applies verification stimuli to the streaming multiprocessor to enable the tensor core to work collaboratively with the shared memory module, load storage unit, and vector core; wherein, the verification stimuli are used to perform data path testing, concurrent operation testing, synchronization mechanism testing, and end-to-end scenario testing on the streaming multiprocessor;

[0114] The fourth result checker compares the final output of the streaming multiprocessor after applying verification stimulus with the expected output of the fourth reference model based on the same input to verify the functional correctness of the tensor core integrated in the streaming multiprocessor.

[0115] The fourth performance monitor collects and analyzes data to confirm whether the performance data obtained from the analysis meets the fourth preset performance target, thereby determining whether the performance meets the standard.

[0116] The fourth coverage collector collects the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process; and judges the completeness of the first integration verification of the streaming multiprocessor with integrated tensor cores based on the functional coverage and performance coverage.

[0117] In another example,

[0118] A second integration verification is performed on the collaborative operation between tensor cores in multiple integrated tensor core streaming multiprocessors, including:

[0119] A multi-stream multiprocessor cluster-level verification environment is constructed to perform collaborative verification of multiple stream multiprocessors integrating tensor cores; the verification environment includes a second stimulus generator, a fifth reference model, a fifth result checker, a fifth performance monitor, and a fifth coverage collector;

[0120] A second incentive generator applies multi-threaded workload incentives to multiple stream multiprocessors, enabling the tensor cores in the multiple stream multiprocessors to execute distributed matrix computation tasks; wherein, the multi-threaded workload incentives are used to test data broadcasting and reuse, block parallel computation, and global synchronization mechanism of the cluster.

[0121] The fifth result checker compares the final output of multiple streaming multiprocessor clusters after applying multi-threaded workload stimulation with the expected output of the fifth reference model based on the same input to verify the correctness of the collaborative work between tensor cores in multiple streaming multiprocessors with integrated tensor cores.

[0122] The fifth performance monitor collects and analyzes data to determine whether the performance meets the fifth preset performance target.

[0123] The fifth coverage collector collects the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process; and based on the functional coverage and performance coverage, it determines the completeness of the second integration verification of the collaborative work between tensor cores in multiple stream multiprocessors with integrated tensor cores.

[0124] See Figure 2 , Figure 2 This is a schematic diagram of the pre-simulation verification process for another tensor core in this application embodiment. The specific steps are as follows:

[0125] Step 201: Perform the first unit test verification on the tensor data transfer module.

[0126] Unit testing verification is the lowest and most basic level in the chip verification pyramid; it involves performing complete verification on an independent, small-scale hardware module in an isolated environment. In this embodiment, the verification is performed through simulation before the hardware module is formed.

[0127] The tensor data transfer module is a unit specifically designed for efficiently managing the data required by the tensor core. Performing separate unit tests on the tensor data transfer module allows for a more comprehensive independent functional and performance verification of this tensor memory accelerator.

[0128] The primary function of the tensor data transport module is to replace traditional general-purpose processors or DMA controllers, intelligently and efficiently moving large blocks of matrix data (tensors) required for computation from off-chip memory or various levels of cache to the tensor core's local buffer, while simultaneously writing the computation results back. It understands the multidimensional structure of tensors and can perform efficient data rearrangement, transpose, and other operations to maximize data reuse and memory bandwidth.

[0129] The first unit test verification of the tensor data transfer module includes the following:

[0130] First, a verification environment for the tensor data transport module is constructed. This verification environment includes a first instruction sequence generator, a first reference model, a first result checker, a first performance monitor, and a first coverage collector. The first instruction sequence generator is used to systematically generate test cases, traversing all opcodes in the tensor data transport module's instruction set. The first reference model is a C language reference model (cmodel) written in C language based on the tensor data transport module's design document. It is used to compare the input with the output of the RTL simulation tensor data transport module to determine the correctness of its function.

[0131] Then conduct the test:

[0132] The first instruction sequence generator generates and sends a test instruction sequence to the tensor data transfer module to perform functional and performance tests on the tensor data transfer module.

[0133] The embodiments of this application provide the following test content, but are not limited to the following test items:

[0134] Functional testing:

[0135] Corner testing: testing in special time-series scenarios, full-load testing, etc.

[0136] Instruction traversal: Systematically generates test cases and traverses all opcodes in the instruction set of the tensor data transfer module.

[0137] Domain testing: For each operand field of each instruction, such as address, step size, data format, thread configuration, etc., perform boundary value, typical value and random value tests.

[0138] Combinatorial testing: Using constrained random testing methods, test scenarios with multiple combinations of different operand domains are generated to cover the complex configuration space. Functional coverage is used to determine whether all scenarios have been traversed.

[0139] Anomaly testing: Injecting illegal instructions, out-of-bounds addresses, and other abnormal situations to verify the robustness, error handling mechanism, and interrupt reporting mechanism of the tensor data transfer module.

[0140] Concurrency testing: Check for any anomalies when multiple tensor data transfer module instructions are sent simultaneously.

[0141] Performance testing:

[0142] Single instruction latency: measures the number of clock cycles required from instruction issuance to completion of execution.

[0143] Throughput testing: Construct a dense instruction stream to test the maximum throughput of the tensor data transfer module when executing instructions back-to-back; and issue instruction streams simultaneously to conduct concurrent testing to test the maximum throughput of the tensor data transfer module when executing concurrent instructions.

[0144] The output of the tensor data transfer module after executing the test instruction sequence is compared with the expected output of the first reference model based on the same input by the first result checker to verify the functional correctness of the tensor data transfer module.

[0145] The performance of the tensor data transfer module is determined by collecting and analyzing data through the first performance monitor to confirm whether the obtained performance data meets the first preset performance target.

[0146] The data collected for single instruction latency testing can be: the timestamps of instruction issuance and completion, and the performance data obtained from the analysis can be the single instruction end-to-end latency (number of cycles).

[0147] The data collected for back-to-back and concurrent throughput tests can include: total test time, number of completed instructions, total amount of data transmitted, and concurrent conflict events. The performance data obtained from the analysis can include: instruction throughput (instructions / cycle), steady-state data bandwidth (GB / s), and concurrency efficiency (%).

[0148] This is just an example; the above restrictions are not applied to the specific data collected and analyzed. The specific data should be determined based on the actual testing needs.

[0149] Finally, the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process are collected by the first coverage collector; and the completeness of the first unit test verification of the tensor data transfer module is determined based on the functional coverage and performance coverage.

[0150] The coverage of performance testing can be determined based on the coverage of the first preset performance target.

[0151] The first unit test verification of the tensor data transfer module will only end when both functional and performance test coverage reach 100%. If the functional and / or performance test coverage fails to reach 100%, the test instruction sequences corresponding to the uncovered functions and / or performance will be applied again for testing.

[0152] Step 202: Perform a second unit test and verification on the matrix operation unit.

[0153] This step involves a second unit test and verification of the matrix operation unit, specifically including:

[0154] First, a verification environment for the matrix operation unit is constructed, which includes a matrix data generator, a second reference model, a second result checker, a second performance monitor, and a second coverage collector. The matrix data generator is used to generate multiple sets of test matrix data. The second reference model is a cmodel written in C language based on the design document of the matrix operation unit, which is used to compare the input with the output of the simulated matrix operation unit.

[0155] Then, conduct the test:

[0156] Multiple sets of test matrix data are generated by a matrix data generator and input into the matrix operation unit so that the matrix operation unit can perform matrix operations; the test matrix data is used to test the operation mode, data format, matrix dimension, special value, and accumulation of the matrix operation unit.

[0157] When performing arithmetic pattern testing, the arithmetic pattern that needs to be traversed is: D = A B and D = A Two core operation modes: B and C.

[0158] When performing data format testing, it is necessary to traverse the data formats and test the calculation correctness under different input / output precisions and mixed precision combinations; data formats include FP32, FP16, FP8, FP4, BF16, INT8, etc.

[0159] Matrix Dimension Testing: Tests range from extremely small sizes, such as 8x8, to typical sizes, where A The cumulative dimension K of matrix B is traversed in the form of 32 / 64 / 96 / 128 / 256 / 512B.

[0160] Special value test: Input a matrix including zero, infinity, NaN, and nonnormal numbers, and verify that the operators comply with IEEE standards or design specifications.

[0161] Cumulative Test: A B can be accumulated with the result of the previous calculation, or not accumulated.

[0162] A B+C can be added to the result of the previous calculation, or not added at all.

[0163] The second result checker compares the output matrix generated by the matrix operation unit after performing the operation with the expected output matrix generated by the second reference model based on the same input to verify the functional correctness of the matrix operation unit under dynamic simulation.

[0164] During functional testing, the second performance monitor collects and analyzes data to determine whether the performance of the matrix operation unit meets the second preset performance target.

[0165] During the testing of the matrix operation unit under specific operation modes and data formats, data such as the computational activity cycle, the total number of completed computational operations, and key task timestamps were collected. Based on the collected data, the following performance data were analyzed and obtained:

[0166] Peak computing throughput: For different data formats, verify whether they reach the theoretical peak computing power of the design specifications.

[0167] Computing array utilization: Evaluates the efficiency of hardware resource utilization under typical loads.

[0168] Result delay and computation completion delay: Verify whether they meet the timing requirements of the real-time computing pipeline.

[0169] The data collected and the performance data obtained are merely examples, and the actual implementation is not limited to these.

[0170] Finally, the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process are collected by the second coverage collector; and the completeness of the second unit test verification of the matrix operation unit is judged based on the functional coverage and performance coverage.

[0171] The coverage of performance testing can be determined based on the coverage of the second preset performance target.

[0172] The second unit test verification for the matrix operation unit will only end when both functional and performance test coverage reach 100%. If the functional test coverage and / or performance test coverage fail to reach 100%, the test matrix data corresponding to the uncovered functions and / or performance will be applied again for testing.

[0173] The second unit test verification for the matrix operation unit will only end when the functional test coverage reaches 100%. If the functional test coverage fails to reach 100%, the matrix data generator will be used to generate test matrix data for the uncovered functions for testing.

[0174] The performance test coverage can be determined directly based on the second preset performance target to determine whether the performance test has been completed.

[0175] For matrix operation units, formal tools can also be used for verification:

[0176] The RTL design of the matrix operation unit is verified for equivalence with a second reference model, and its functionality is rigorously proven mathematically under all possible input combinations. The verification is performed by multiplying a specified local computation module A by a point in B.

[0177] To multiply two data items, you need to iterate through the following format:

[0178] FP32, FP16, FP8, FP4, BF16, INT8;

[0179] And the following format combinations: Each format can be combined with itself, and FP8 and BF8, FP16 and BF16 can also be combined.

[0180] In specific implementation, the first unit test verification of the tensor data transfer module and the second unit test verification of the matrix operation unit can be executed in parallel or sequentially. This application embodiment does not impose any restrictions on this, and sequential execution is taken as an example here.

[0181] Step 203: Determine whether the first unit test and the second unit test have both passed. If yes, proceed to step 204; otherwise, proceed to step 211.

[0182] In this step, module-level verification of the tensor core, which integrates the tensor data transfer module and the matrix operation unit, will only be performed when both the first and second verification units pass verification. If only one verification unit passes verification, or if neither verification unit passes verification, the pre-simulation verification of the tensor core is determined to have failed, and no further related verification is required, which can greatly save verification time and verification costs.

[0183] Step 204: Perform module-level verification on the tensor core, which integrates the tensor data transfer module and the matrix operation unit.

[0184] This step involves module-level verification of the tensor core, which integrates a tensor data transfer module and a matrix operation unit. Specifically, this includes:

[0185] First, a tensor core verification environment integrating a tensor data transport module and a matrix operation unit is constructed. (See...) Figure 3 , Figure 3 This is a schematic diagram of the tensor core structure that integrates the tensor data transfer module and the matrix operation unit in the embodiments of this application. Figure 3 The tensor data transport module is specifically responsible for data transport, loading data from outside the tensor core into the internal L1 cache, and writing the calculation results from the internal L1 cache back to the corresponding external memory; the matrix operation unit is responsible for matrix calculations, performing matrix multiplication and addition operations (A...). B or A The B+C buffer handles accumulation operations; the L1 buffer acts as a bridge, storing the input matrix and output results, reducing frequent access to external memory, and supporting data reuse.

[0186] The verification environment includes a second instruction sequence generator, a third reference model, a third result checker, a third performance monitor, and a third coverage collector. The second instruction sequence generator is used to generate test instruction sequences, traversing all opcodes of the Tensor Core instruction set. The third reference model is a cmodel written in C language based on the Tensor Core design document, used to compare the input with the output of the simulated Tensor Core.

[0187] Then conduct the test:

[0188] The second instruction sequence generator sends a test instruction sequence to the tensor core to enable the tensor core to perform matrix calculation tasks; the test instruction sequence is used to perform instruction testing, control flow testing, data flow testing, instruction parallelism testing, and performance testing on the tensor core.

[0189] Instruction testing: Testing different opcodes of matrix multiplication instructions and switching between different multiplication instructions; boundary values ​​and typical values ​​of different fields of a single multiplication instruction, such as whether to perform addition of matrix C, whether to accumulate ACC, different data formats, different matrix multiplication accumulation dimensions, etc.

[0190] Control flow test: Verify that the tensor core correctly parses instructions and generates control signals, accurately controlling the start-up, execution and completion of the matrix operation unit;

[0191] Data flow test: Verify that the matrix data read from memory / cache by the tensor data transfer module can be correctly and accurately transferred to the input buffer of the matrix operation unit;

[0192] Instruction parallelism test:

[0193] Parallel transmission: Simultaneously send multiple independent multiplication instructions to the tensor data transfer module to verify whether the input interface can correctly handle concurrent requests.

[0194] Serial execution: Execution instructions are received in parallel and stored within the tensor core. Individual instructions within the same set of computation instructions are executed sequentially to ensure the correctness of the computation results.

[0195] Performance testing: Test A B, A The output bandwidth of the computing module in two typical scenarios (B+C), traversing different data formats, and all K-accumulated dimensions under different data formats.

[0196] The final output of the tensor core after executing the test instruction sequence is compared with the expected output of the third reference model based on the same input by the third result checker to verify the functional correctness of the tensor core.

[0197] The performance of the tensor core is determined by collecting and analyzing data through a third performance monitor to confirm whether the obtained performance data meets the third preset performance target.

[0198] The data collected for this test can include: the active cycle of the computing array, the amount of output data, and the task time. The performance data analyzed and obtained can include key performance indicators such as the computing throughput, computing efficiency, and end-to-end latency of the tensor core. The relevant data here is just an example, and it is not limited to this in actual applications.

[0199] Finally, the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process are collected by a third coverage collector; and the completeness of the module-level verification of the tensor core is judged based on the functional coverage and performance coverage.

[0200] The coverage of performance testing can be determined based on the coverage of a third preset performance target.

[0201] The module-level verification of the tensor core will only end when both functional and performance test coverage reach 100%. If functional and / or performance test coverage fails to reach 100%, the corresponding test instruction sequences for the uncovered functions and / or performance will be applied again for testing.

[0202] Step 205: Determine whether the module-level verification has passed. If yes, proceed to step 206; otherwise, proceed to step 211.

[0203] If the module-level verification of the tensor core integrating the tensor data transfer module and the matrix operation unit passes, the first integration verification of the stream multiprocessor integrating the tensor core is performed; if the module-level verification of the tensor core integrating the tensor data transfer module and the matrix operation unit fails, it is determined that the pre-simulation verification of the tensor core has failed, and no further related verification is required, which can greatly save verification time and verification costs.

[0204] Step 206: Perform the first integration verification on the stream multiprocessor with integrated tensor cores.

[0205] In this step, the first integration verification of the streaming multiprocessor with integrated tensor cores is performed, specifically including:

[0206] First, a streaming multiprocessor-level verification environment is constructed, integrating the tensor core with the shared memory module, load storage unit, and vector core; see [link to documentation]. Figure 4 , Figure 4 This is a schematic diagram of a streaming multiprocessor architecture integrating a tensor core in an embodiment of this application. The streaming multiprocessor integrates a tensor core with a shared memory module, a load storage unit, and a vector core. The tensor core is responsible for performing high-performance matrix multiplication and addition operations; the vector core performs scalar and vector operations, collaborating with the tensor core to complete complex tasks; the load storage unit is responsible for loading data; and the shared memory module serves as a fast cache within the streaming multiprocessor, allowing the tensor core, vector core, and load storage unit to share data.

[0207] The verification environment includes a first stimulus generator, a fourth reference model, a fourth result checker, a fourth coverage collector, and a fourth performance monitor. The first stimulus generator is used to apply verification stimuli to the streaming multiprocessor. The fourth reference model is a C model written in C language based on the design document of the streaming multiprocessor with integrated tensor cores. It is used to compare the input with the output of the simulated streaming multiprocessor with integrated tensor cores.

[0208] Then conduct the test:

[0209] The first stimulus generator applies verification stimuli to the streaming multiprocessor to enable the tensor core to work collaboratively with the shared memory module, load storage unit, and vector core; wherein, the verification stimuli are used to perform data path testing, concurrent operation testing, synchronization mechanism testing, and end-to-end scenario testing on the streaming multiprocessor;

[0210] Data path verification:

[0211] Load path: The path test for loading data from L2 or HBM into the shared memory module (GSM) via the tensor core, including functional and bandwidth performance.

[0212] Store path: Test the path from the tensor core to L2 or HBM for computation results in registers or GSM, including functionality and bandwidth performance.

[0213] Test the functionality and bandwidth performance of the path during load / store parallel operation;

[0214] Concurrency testing: Simultaneously perform Load, Store, matrix operation unit operations, and Vector Core calculations to test resource sharing, arbitration, and performance bandwidth.

[0215] Synchronization mechanism verification: Test synchronization mechanisms such as data readiness or space buffer readiness. This includes data readiness when loading into GSM, matrix operation unit operation results readiness, vector core operation results readiness, and space buffer readiness when storing data.

[0216] End-to-end scenario testing: Build a complete workflow of "data loading → tensor core calculation → result writing back" and conduct functional and performance testing.

[0217] The above complete workflow under multi-process concurrency functional and performance testing;

[0218] The fourth result checker compares the final output of the streaming multiprocessor after applying verification stimulus with the expected output of the fourth reference model based on the same input to verify the functional correctness of the tensor core integrated in the streaming multiprocessor.

[0219] The fourth performance monitor collects and analyzes data to confirm whether the obtained performance data meets the fourth preset performance target, thus determining whether the performance meets the standard.

[0220] Regarding data path performance, the fourth performance monitor can collect data such as the timestamps and data volume of Load / Store operations, and analyze the performance data such as the measured bandwidth and utilization of concurrent bidirectional operations.

[0221] For concurrent execution efficiency: Collect the active state cycle and resource conflict events of the tensor core, and analyze and calculate IO overlap efficiency, resource contention overhead and system throughput under mixed load.

[0222] For end-to-end task performance: Collect the timestamps and completion amounts of the complete workflow and its multi-process concurrent execution, and analyze single-task latency, multi-process concurrent throughput and system scalability.

[0223] Regarding the overhead of the synchronization mechanism: collect the time difference before and after the synchronization operation and the waiting period of each unit, and analyze the delay and idle overhead introduced by the synchronization operation.

[0224] The data collected and analyzed above are merely examples, and no specific limitations are imposed on them.

[0225] Finally, the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process are collected by the fourth coverage collector; and the completeness of the first integration verification of the streaming multiprocessor with integrated tensor cores is determined based on the functional coverage and performance coverage.

[0226] The coverage of performance testing can be determined based on the coverage of the fourth preset performance target.

[0227] The first integration verification test for the streaming multiprocessor with integrated tensor cores will only end when both functional and performance test coverage reach 100%. If the functional and / or performance test coverage fails to reach 100%, the verification stimuli corresponding to the uncovered functions and / or performance will be applied again for testing.

[0228] Step 207: Determine whether the first integration verification has passed. If yes, proceed to step 208; otherwise, proceed to step 211.

[0229] If the first integration verification of the stream multiprocessor with integrated tensor cores passes, a second integration verification is performed on the collaborative work between tensor cores in multiple stream multiprocessors with integrated tensor cores. If the first integration verification of the stream multiprocessor with integrated tensor cores fails, it is determined that the pre-simulation verification of the tensor cores has failed, and no further related verification is required, which can greatly save verification time and verification costs.

[0230] Step 208: Perform a second integration verification on the collaborative operation between tensor cores in multiple stream multiprocessors that integrate tensor cores.

[0231] In this step, a second integration verification is performed on the collaborative operation between tensor cores in multiple stream multiprocessors that integrate tensor cores. Specifically, this includes:

[0232] First, a multi-stream, multi-processor cluster-level verification environment is constructed to perform collaborative verification of multiple stream multiprocessors integrating tensor cores. See [link / reference]. Figure 5 , Figure 5This is a schematic diagram of the integrated structure of a stream multiprocessor with multiple integrated tensor cores in an embodiment of this application. Figure 5 Taking three streaming multiprocessors as an example, each streaming multiprocessor integrates a tensor core. As the core computing unit of the GPU, the streaming multiprocessor is responsible for executing specific parallel computing instructions and processing tasks. The L2 cache serves as a shared data transfer station for the streaming multiprocessor cluster, accelerating the streaming multiprocessor's access to global data. The memory controller acts as a bridge between the GPU and the video memory, responsible for data exchange between all computing units and external video memory.

[0233] The verification environment includes a second stimulus generator, a fifth reference model, a fifth result checker, a fifth performance monitor, and a fifth coverage collector. The second stimulus generator is used to generate thread workload stimuli. The fifth reference model is a C model written in C language based on the design document of a streaming multiprocessor with multiple integrated tensor cores. It is used to compare the input with the output of the simulated streaming multiprocessor with multiple integrated tensor cores. The fifth performance monitor is used for performance compliance verification.

[0234] Then conduct the test:

[0235] A second incentive generator applies multi-threaded workload incentives to multiple stream multiprocessor clusters, enabling tensor cores in multiple stream multiprocessors to execute distributed matrix computation tasks; wherein, the multi-threaded workload incentives are used to test data broadcasting and reuse, block parallel computation, and global synchronization mechanism of the clusters.

[0236] On chip-level or cluster-level verification platforms, verify whether the tensor cores of multiple streaming multiprocessor cores can work correctly and efficiently to complete complex distributed matrix computation tasks. The core is to verify the performance of interconnect networks, shared caches, and synchronization mechanisms under real-world loads.

[0237] Among them, data broadcasting and multiplexing tests:

[0238] Scenario description: A streaming multiprocessor (source streaming multiprocessor) loads a common data set, such as a weight matrix W, from L2 or HBM, and then broadcasts it to multiple other streaming multiprocessors (target streaming multiprocessors) in the cluster for their respective computation.

[0239] Verification process:

[0240] Functional testing:

[0241] The source stream multiprocessor first loads data W to the local G stream multiprocessor.

[0242] Source-stream multiprocessors initiate broadcast operations through specific instructions or memory address mappings. Weighted data W is broadcast to the GSMs of other stream multiprocessors within the same cluster via the broadcast path.

[0243] The fifth result checker's checkpoint: verifies whether data W has been correctly received in the shared memory of all target stream multiprocessors, and whether the data content is consistent with the W data received by other stream multiprocessors in the same cluster.

[0244] Synchronization test: In order to ensure the correctness of the collaborative computing function between streaming multiprocessors, test whether the synchronization mechanism function of W data in all GSM is correct;

[0245] Performance testing:

[0246] Test the total latency from sending data to the source stream multiprocessor to receiving data ready on all target stream multiprocessors.

[0247] Test the bandwidth of the broadcast path during the process of a multiprocessor source broadcasting full-bandwidth data.

[0248] The performance and bandwidth benefits of quantization broadcasting are compared to each stream multiprocessor loading data independently.

[0249] Block-based parallel computing test

[0250] Scenario description: Perform a large matrix multiplication C = A B is decomposed into multiple sub-blocks (Tile), which are then distributed to tensor cores of different stream multiprocessors for parallel computation, and finally the results are aggregated.

[0251] Verification process:

[0252] Functional testing:

[0253] Data partitioning: The host driver divides the input matrices A and B into blocks, such as by row, by column, or in a chessboard pattern, and assigns a corresponding subtask to each stream multiprocessor.

[0254] Parallel execution: All streaming multiprocessors begin executing their respective tensor core computation tasks simultaneously.

[0255] Results collection: Each stream multiprocessor writes the calculated result submatrix back to the shared L2 cache or video memory.

[0256] The fifth result checkpoint compares the final summarized results with the complete results calculated by the reference model to ensure that the calculation is 100% correct, that is, to confirm consistency.

[0257] Verify whether concurrent access to the L2 cache and memory controller by different stream multiprocessors during computation leads to data corruption or loss. Test the synchronization mechanism for collaborative computation between different stream multiprocessors to ensure its correctness and consistency with the final overall computation result and the c model output.

[0258] Performance testing:

[0259] Measure the total time taken from the start of the task to the completion of the computation by the last streaming multiprocessor.

[0260] Analyze congestion in the Internet when multiple streams and multiprocessors access data simultaneously, and record bandwidth utilization.

[0261] Analyze congestion when multiple streams of multiprocessors broadcast data to each other, and record bandwidth utilization.

[0262] Strong scalability test: With the total matrix calculation size fixed, the number of streaming multiprocessors involved in the calculation is increased, and the improvement in calculation speed is observed.

[0263] Weak scaling test: Keep the computational scale of each stream multiprocessor constant, increase the number of stream multiprocessors, and observe whether the system can handle the increased computational scale.

[0264] Data synchronization function test:

[0265] Scenario description: All streaming multiprocessors must wait until all broadcast data is ready before they can simultaneously enter the next stage.

[0266] Verification process:

[0267] Barrier synchronization test:

[0268] Set a global barrier that requires all participating streaming multiprocessors to reach that point before they can continue.

[0269] The fifth result checker's checkpoints verify whether any streaming multiprocessors are unable to reach the barrier due to various reasons (such as incomplete computation or instruction blocking), leading to system deadlock.

[0270] Measure the delay of synchronous operation.

[0271] Semaphore / Atomic Operation Test:

[0272] Use atomic operations, such as atomicAdd, atomicCAS, or cluster bar, to implement semaphores in global memory to protect shared resources or to pass "task completed" signals.

[0273] The fifth result checker's checkpoints verify the correctness of atomic operations, ensuring there are no data race conditions. For example, multiple streaming multiprocessors simultaneously atomically increment a counter, and the final result must be correct.

[0274] Inject extreme scenarios, such as multiple stream processors competing for the same lock simultaneously, to verify the system's fairness and livelock / deadlock avoidance mechanisms.

[0275] Enhanced stress testing in single-stream multiprocessor scenarios

[0276] Scenario Description: Running all the complex scenarios defined in "Single-Stream Multiprocessor Integration Test" concurrently on multiple stream multiprocessors.

[0277] Verification process:

[0278] At the same time, a full path test of "data load → matrix multiplication → result store" is launched on all streaming multiprocessors.

[0279] Or all other scenarios tested in a single-stream multiprocessor test environment;

[0280] The fifth result checkpoint: observe whether the concurrent data flow creates bottlenecks at the internet, L2 cache, and memory controller, or whether it will cause functional problems in concurrent scenarios.

[0281] This test is designed to expose system-level resource contention and performance bottlenecks that cannot be detected in single-stream multiprocessor tests.

[0282] The fifth result checker compares the final output of multiple streaming multiprocessor clusters after applying multi-threaded workload stimulation with the expected output of the fifth reference model based on the same input to verify the correctness of the collaborative work between tensor cores in multiple streaming multiprocessors with integrated tensor cores.

[0283] The fifth performance monitor collects and analyzes data to determine whether the performance meets the fifth preset performance target.

[0284] The data analyzed in this embodiment includes total broadcast latency, path bandwidth, total task duration, and internet utilization. After the test is completed, the performance data obtained from the actual measurement and analysis is compared with the fifth preset performance target to verify whether the system performance meets the standard.

[0285] Finally, the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process are collected by the fifth coverage collector; and the completeness of the second integration verification of the collaborative work between tensor cores in multiple stream multiprocessors with integrated tensor cores is judged based on the functional coverage and performance coverage.

[0286] The coverage of performance testing can be determined based on the coverage of the fifth preset performance target.

[0287] Testing of the tensor cores in this multi-stream multiprocessor with integrated tensor cores will only conclude when both functional and performance test coverage reach 100%. If functional and / or performance test coverage fails to reach 100%, the corresponding verification stimuli for the uncovered functions and / or performance will be applied again for testing.

[0288] Step 209: Determine whether the second integration verification has passed. If yes, proceed to step 210; otherwise, proceed to step 211.

[0289] Step 210: Confirm that the pre-simulation verification of the tensor core has passed. End this process.

[0290] Step 211: It is determined that the pre-simulation verification of the tensor core failed.

[0291] This embodiment provides a specific verification process. Based on the aforementioned different levels of pre-simulation verification, the completeness of the functionality and performance of the tensor core can be guaranteed. It can systematically ensure that the tensor core maintains functional robustness and excellent performance even in the most challenging multi-core collaborative working scenarios. This embodiment provides an efficient and comprehensive pre-simulation verification method for tensor cores, thereby helping to identify and resolve problems early in the chip design process.

[0292] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0293] Based on the same inventive concept, this application also provides a pre-simulation verification device for tensor cores. See also Figure 6 , Figure 6 This is a schematic diagram of the pre-simulation verification device for the tensor core in this embodiment of the application. The device includes:

[0294] The first verification unit 601 is used to perform the first unit test verification on the tensor data transfer module.

[0295] The second verification unit 602 is used to perform second unit test verification on the matrix operation unit;

[0296] The third verification unit 603 is used to perform module-level verification on the tensor core, which integrates a tensor data transport module and a matrix operation unit.

[0297] The fourth verification unit 604 is used to perform the first integration verification on the streaming multiprocessor with integrated tensor cores;

[0298] The fifth verification unit 605 is used to perform a second integration verification of the collaborative operation between tensor cores in multiple stream multiprocessors that integrate tensor cores.

[0299] Unit 606 is determined to be used in response to the first unit test verification, the second unit test verification, the module-level verification, the first integration verification and the second integration verification all passing the verification, thus confirming that the pre-simulation verification of the tensor core has passed.

[0300] In another example,

[0301] The determination unit 606 is further configured to, before the verification is performed by the third verification unit 603, execute the verification of the third verification unit 603 if it is determined that the first unit test verification has passed and the second unit test verification has passed; otherwise, determine that the pre-simulation verification of the tensor core has failed; after the verification by the third verification unit 603 and before the verification by the fourth verification unit 604, execute the verification of the fourth verification unit 604 if it is determined that the module-level verification has passed; otherwise, determine that the pre-simulation verification of the tensor core has failed; after the verification by the fourth verification unit 604 and before the verification by the fifth verification unit 605, execute the verification of the fifth verification unit 605 if it is determined that the first integration verification has passed; otherwise, determine that the pre-simulation verification of the tensor core has failed; if the second integration verification has failed, determine that the pre-simulation verification of the tensor core has failed.

[0302] In another example,

[0303] The first verification unit 601 is specifically used to construct a verification environment for the tensor data transport module. The verification environment includes a first instruction sequence generator, a first reference model, a first result checker, a first performance monitor, and a first coverage collector. The first instruction sequence generator generates and sends test instruction sequences to the tensor data transport module to perform functional and performance tests. The first result checker compares the output results generated by the tensor data transport module after executing the test instruction sequences with the expected output results generated by the first reference model based on the same input to verify the functional correctness of the tensor data transport module. The first performance monitor collects and analyzes data to confirm whether the obtained performance data meets the first preset performance target to determine whether the performance of the tensor data transport module meets the standard. The first coverage collector collects the functional coverage achieved during the functional test and the performance coverage achieved during the performance test. Based on the functional coverage and performance coverage, the completeness of the first unit test verification of the tensor data transport module is determined.

[0304] In another example,

[0305] The second verification unit 602 is specifically used to construct the verification environment for the matrix operation unit. The verification environment includes a matrix data generator, a second reference model, a second result checker, a second performance monitor, and a second coverage collector. The matrix data generator generates multiple sets of test matrix data and inputs them into the matrix operation unit to perform matrix operations. The test matrix data is used to perform operation mode testing, data format testing, matrix dimension testing, special value testing, and accumulation testing on the matrix operation unit. The second result checker compares the output matrix generated by the matrix operation unit after performing the operations with the expected output matrix generated by the second reference model based on the same input to verify the functional correctness of the matrix operation unit under dynamic simulation. During functional testing, the second performance monitor collects and analyzes data to confirm whether the obtained performance data meets the second preset performance target, thus determining whether the performance of the matrix operation unit meets the standard. The second coverage collector collects the functional coverage achieved during functional testing and the performance coverage achieved during performance testing, and judges the completeness of the second unit test verification of the matrix operation unit based on the functional coverage and performance coverage.

[0306] The third verification unit 603 is specifically used to construct a tensor core verification environment integrating a tensor data transport module and a matrix operation unit. This verification environment includes a second instruction sequence generator, a third reference model, a third result checker, a third performance monitor, and a third coverage collector. The second instruction sequence generator sends test instruction sequences to the tensor core to enable it to execute matrix computation tasks. These test instruction sequences are used to perform instruction testing, control flow testing, data flow testing, instruction parallelism testing, and performance testing on the tensor core. The third result checker compares the final output of the tensor core after executing the test instruction sequences with the expected output of the third reference model based on the same input to verify the functional correctness of the tensor core. The third performance monitor collects and analyzes data to confirm whether the obtained performance data meets the third preset performance target, thus determining whether the tensor core's performance meets the standard. The third coverage collector collects the functional coverage achieved during functional testing and the performance coverage achieved during performance testing, and judges the completeness of the module-level verification of the tensor core based on the functional coverage and performance coverage.

[0307] In another example,

[0308] The fourth verification unit 604 is specifically used to construct a verification environment at the streaming multiprocessor level, integrating the tensor core with the shared memory module, load storage unit, and vector core. The verification environment includes a first stimulus generator, a fourth reference model, a fourth result checker, a fourth coverage collector, and a fourth performance monitor. Verification stimuli are applied to the streaming multiprocessor through the first stimulus generator to enable the tensor core to work collaboratively with the shared memory module, load storage unit, and vector core. The verification stimuli are used to perform data path testing, concurrent operation testing, synchronization mechanism testing, and end-to-end scenario testing on the streaming multiprocessor. The fourth result checker integrates the tensor core with the shared memory module, load storage unit, and vector core. The final output of the processor after applying verification stimuli is compared with the expected output of the fourth reference model based on the same input to verify the functional correctness of the tensor core integrated in the streaming multiprocessor. Data is collected and analyzed by the fourth performance monitor to confirm whether the obtained performance data meets the fourth preset performance target to determine whether the performance is up to standard. The functional coverage achieved during the functional test and the performance coverage achieved during the performance test are collected by the fourth coverage collector. The completeness of the first integration verification of the streaming multiprocessor with the tensor core is judged based on the functional coverage and performance coverage.

[0309] In another example,

[0310] The fifth verification unit 605 is specifically used to construct a verification environment at the multi-stream multiprocessor cluster level to perform collaborative verification of multiple stream multiprocessors integrating tensor cores. The verification environment includes a second stimulus generator, a fifth reference model, a fifth result checker, a fifth performance monitor, and a fifth coverage collector. The second stimulus generator applies multi-threaded workload stimuli to the multiple stream multiprocessors, enabling the tensor cores in the multiple stream multiprocessors to execute distributed matrix computation tasks. The multi-threaded workload stimuli are used to perform data broadcasting and reuse testing, block parallel computation testing, and global synchronization mechanism testing on the cluster. The fifth result checker applies multi-threaded workload stimuli to the multiple stream multiprocessor cluster. The final output generated after excitation is compared with the expected output generated by the fifth reference model based on the same input to verify the correctness of the collaborative work between tensor cores in multiple stream multiprocessors with integrated tensor cores. The fifth performance monitor collects and analyzes data to confirm whether the obtained performance data meets the fifth preset performance target to determine whether the performance is up to standard. The fifth coverage collector collects the functional coverage achieved during the functional test and the performance coverage achieved during the performance test. Based on the functional coverage and performance coverage, the completeness of the second integration verification of the collaborative work between tensor cores in multiple stream multiprocessors with integrated tensor cores is judged.

[0311] The units in the above embodiments can be integrated into one unit or deployed separately; they can be merged into one unit or further divided into multiple sub-units.

[0312] In another embodiment, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a pre-simulation verification method for a tensor core.

[0313] In another embodiment, a computer-readable storage medium is also provided, on which computer instructions are stored, which, when executed by a processor, implement a pre-simulation verification method for a tensor core.

[0314] Figure 7 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of the present invention. Figure 7 As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute the following methods:

[0315] The first unit test was conducted to verify the tensor data transfer module.

[0316] The matrix operation unit was tested and verified in the second unit.

[0317] Module-level verification was performed on the tensor core, which integrates a tensor data transfer module and a matrix operation unit.

[0318] First integration verification of a streaming multiprocessor with integrated tensor cores;

[0319] A second integration verification is performed on the collaborative operation between tensor cores in multiple stream multiprocessors with integrated tensor cores;

[0320] Since the first unit test verification, the second unit test verification, the module-level verification, the first integration verification and the second integration verification all passed, it is confirmed that the pre-simulation verification of the tensor core has passed.

[0321] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0322] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0323] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0324] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments disclosed in this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings. For example, two blocks shown connected together may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0325] Those skilled in the art will understand that the features described in the various embodiments and / or claims disclosed in this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, without departing from the spirit and teachings of this application, the features described in the various embodiments and / or claims of this application can be combined and / or combined in various ways, and all such combinations and / or combinations fall within the scope of this application.

[0326] This document uses specific embodiments to illustrate the principles and implementation methods of the present invention. The descriptions of these embodiments are merely illustrative of the method and core concepts of the present invention and are not intended to limit this application. Those skilled in the art can make changes to the specific implementation methods and application scope based on the ideas, spirit, and principles of the present invention. Any modifications, equivalent substitutions, or improvements made should be included within the scope of protection of this application.

Claims

1. A pre-simulation verification method for tensor cores, characterized in that, In a simulation environment, the method includes: The first unit test was conducted to verify the tensor data transfer module. The matrix operation unit was tested and verified in the second unit. Module-level verification was performed on the tensor core, which integrates the tensor data transport module and the matrix operation unit. A first integration verification was performed on the streaming multiprocessor that integrates the tensor core; A second integration verification was performed on the collaborative operation among tensor cores in multiple streaming multiprocessors that integrate the tensor cores. In response to the first unit test verification, the second unit test verification, the module-level verification, the first integration verification, and the second integration verification all passing, it is determined that the pre-simulation verification of the tensor core has passed.

2. The method according to claim 1, characterized in that, Before performing module-level verification on the tensor core integrating the tensor data transport module and the matrix operation unit, the method further includes: If the first unit test verification is passed and the second unit test verification is passed, then the module-level verification of the tensor core integrating the tensor data transfer module and the matrix operation unit is performed; otherwise, it is determined that the pre-simulation verification of the tensor core has failed. After performing module-level verification on the tensor core integrating the tensor data transport module and the matrix operation unit, and before performing the first integration verification on the stream multiprocessor integrating the tensor core, the method further includes: If the module-level verification is determined to be successful, then the first integration verification of the stream multiprocessor integrating the tensor core is performed; otherwise, the pre-simulation verification of the tensor core is determined to be unsuccessful. After performing the first integration verification on the streaming multiprocessor integrating the tensor cores, and before performing the second integration verification on the collaborative work among the tensor cores in multiple streaming multiprocessors integrating the tensor cores, the method further includes: If the first integration verification is determined to be successful, then the second integration verification is performed on the collaborative work between tensor cores in multiple stream multiprocessors that integrate the tensor cores; otherwise, the pre-simulation verification of the tensor cores is determined to be unsuccessful. After the second integration verification of the collaborative operation among tensor cores in multiple streaming multiprocessors integrating the tensor cores is completed, the method further includes: If the second integration verification is determined to have failed, the pre-simulation verification of the tensor core is also determined to have failed.

3. The method according to claim 1, characterized in that, The first unit test verification of the tensor data transfer module includes: A verification environment for the tensor data transfer module is constructed, the verification environment including a first instruction sequence generator, a first reference model, a first result checker, a first performance monitor, and a first coverage collector; The first instruction sequence generator generates and sends a test instruction sequence to the tensor data transfer module to perform functional and performance tests on the tensor data transfer module. The output of the tensor data transfer module after executing the test instruction sequence is compared with the expected output of the first reference model based on the same input by the first result checker to verify the functional correctness of the tensor data transfer module. The performance of the tensor data transfer module is determined by collecting and analyzing data through the first performance monitor and confirming whether the analyzed performance data meets the first preset performance target. The first coverage collector collects the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process; and determines the completeness of the first unit test verification of the tensor data transfer module based on the functional coverage and the performance coverage.

4. The method according to claim 1, characterized in that, The second unit test verification of the matrix operation unit includes: A verification environment for the matrix operation unit is constructed, the verification environment including a matrix data generator, a second reference model, a second result checker, a second performance monitor, and a second coverage collector; The matrix data generator generates multiple sets of test matrix data, and inputs the test matrix data into the matrix operation unit so that the matrix operation unit can perform matrix operations; wherein, the test matrix data is used to perform operation mode testing, data format testing, matrix dimension testing, special value testing, and accumulation testing on the matrix operation unit; The output matrix generated by the matrix operation unit after performing the operation is compared with the expected output matrix generated by the second reference model based on the same input by the second result checker to verify the functional correctness of the matrix operation unit under dynamic simulation. During the functional testing process, the second performance monitor collects and analyzes data to determine whether the performance of the matrix operation unit meets the second preset performance target. The second coverage collector collects the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process; and determines the completeness of the second unit test verification performed on the matrix operation unit based on the functional coverage and performance coverage.

5. The method according to claim 1, characterized in that, The module-level verification of the tensor core, which integrates the tensor data transport module and the matrix operation unit, includes: A tensor core verification environment integrating a tensor data transport module and a matrix operation unit is constructed. This verification environment includes a second instruction sequence generator, a third reference model, a third result checker, a third performance monitor, and a third coverage collector. The second instruction sequence generator sends a test instruction sequence to the tensor core to enable the tensor core to perform matrix calculation tasks; the test instruction sequence is used to perform instruction testing, control flow testing, data flow testing, instruction parallelism testing, and performance testing on the tensor core. The final output of the tensor core after executing the test instruction sequence is compared with the expected output of the third reference model based on the same input by the third result checker to verify the functional correctness of the tensor core. The performance of the tensor core is determined by collecting and analyzing data through a third performance monitor to confirm whether the obtained performance data meets the third preset performance target. The third coverage collector collects the functional coverage achieved during functional testing and the performance coverage achieved during performance testing; and judges the completeness of the module-level verification of the Tensor Core based on the functional coverage and performance coverage.

6. The method according to claim 1, characterized in that, The first integration verification of the streaming multiprocessor integrating the tensor core includes: A stream multiprocessor-level verification environment is constructed, integrating the tensor core with a shared memory module, a load storage unit, and a vector core; the verification environment includes a first stimulus generator, a fourth reference model, a fourth result checker, a fourth coverage collector, and a fourth performance monitor; The first stimulus generator applies verification stimuli to the streaming multiprocessor to enable the tensor core to work collaboratively with the shared memory module, the loading storage unit, and the vector core; wherein, the verification stimuli are used to perform data path testing, concurrent operation testing, synchronization mechanism testing, and end-to-end scenario testing on the streaming multiprocessor; The fourth result checker compares the final output of the streaming multiprocessor after applying verification stimulus with the expected output of the fourth reference model based on the same input to verify the functional correctness of the tensor core integrated in the streaming multiprocessor. The fourth performance monitor collects and analyzes data to determine whether the performance meets the fourth preset performance target. The fourth coverage collector collects the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process; and determines the completeness of the first integration verification of the streaming multiprocessor with integrated tensor cores based on the functional coverage and performance coverage.

7. The method according to claim 1, characterized in that, The second integration verification of the collaborative operation among tensor cores in multiple streaming multiprocessors integrating the tensor cores includes: A multi-stream multiprocessor cluster-level verification environment is constructed to perform collaborative verification on multiple stream multiprocessors integrating tensor cores; the verification environment includes a second stimulus generator, a fifth reference model, a fifth result checker, a fifth performance monitor, and a fifth coverage collector; The second incentive generator applies multi-threaded workload incentives to multiple streaming multiprocessors, enabling tensor cores in the multiple streaming multiprocessors to execute distributed matrix computation tasks; wherein, the multi-threaded workload incentives are used to perform data broadcasting and reuse testing, block parallel computation testing, and global synchronization mechanism testing on the cluster; The fifth result checker compares the final output of multiple streaming multiprocessor clusters after applying multi-threaded workload incentives with the expected output of the fifth reference model based on the same input to verify the correctness of the collaborative work between tensor cores in multiple streaming multiprocessors with integrated tensor cores. The fifth performance monitor collects and analyzes data to determine whether the performance meets the fifth preset performance target. The fifth coverage collector collects the functional coverage achieved during the functional testing process and the performance coverage achieved during the performance testing process; and judges the completeness of the second integration verification of the collaborative work between tensor cores in multiple stream multiprocessors with integrated tensor cores based on the functional coverage and performance coverage.

8. A pre-simulation verification device for a tensor core, characterized in that, In a simulation environment, the device includes: The first verification unit is used to perform the first unit test verification of the tensor data transfer module. The second verification unit is used to perform second unit testing and verification on the matrix operation unit. The third verification unit is used to perform module-level verification on the tensor core that integrates the tensor data transport module and the matrix operation unit. The fourth verification unit is used to perform the first integration verification on the stream multiprocessor that integrates the tensor core; The fifth verification unit is used to perform a second integration verification on the collaborative operation among tensor cores in multiple stream multiprocessors that integrate the tensor cores. A determination unit is used to determine that the pre-simulation verification of the tensor core has passed in response to the first unit test verification, the second unit test verification, the module-level verification, the first integration verification, and the second integration verification all passing verification.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method of any one of claims 1 to 7.

Images