Graphics processor hardware simulation method and computer-readable storage medium

Abstract

The present invention relates to a hardware simulation method for a graphics processor and a computer-readable storage medium. The hardware simulation method comprises: based on a simplified function and actual parameters in a programming language interface corresponding to a second-type graphics processor called in a user program, introducing instructions corresponding to a first-type graphics processor through an existing GPU programming interface, thereby using the first-type graphics processor to simulate the simplified function in the programming language interface. The simplified function is originally used to drive the second-type graphics processor to perform a specified function. By introducing the instructions corresponding to the first-type graphics processor as described above, the present invention can produce approximate simulation results even before a finished second-type graphics processor is available for testing.

Description

Technical Field

[0001] The present invention relates to a graphics processor used in the field of general computing, and in particular to a hardware simulation method of a graphics processor and a computer-readable storage medium. Background Art

[0002] Currently, computing systems that include a graphics processing unit (GPU) also include a central processing unit (CPU). The CPU is suitable for handling numerical calculations in traditional applications. As new-generation GPUs are under development and before they are mass-produced and released to the market, GPU developers need to build simulation environments to execute and adapt to the program code for these new-generation GPUs. Furthermore, during the execution of these program codes, design issues with the new-generation GPUs can be discovered early. Therefore, the present invention provides a hardware simulation method and computer-readable storage medium for GPUs. Summary of the Invention

[0003] Therefore, how to provide an efficient simulation environment is an important issue in designing the next generation of graphics processors.

[0004] The present invention relates to a graphics processor hardware simulation method, comprising: based on a simplified function and actual parameters in a programming language interface corresponding to a second-type graphics processor called in a user program, introducing instructions corresponding to the first-type graphics processor through a programming interface of a first-type graphics processor, and using the first-type graphics processor to simulate the simplified function in the programming language interface. The simplified function is originally used to drive the second-type graphics processor to perform a specified function.

[0005] The present invention also relates to a computer-readable storage medium for storing a program code that can be executed by a processing unit, and implementing the hardware simulation method of a graphics processor as described above when the program code is executed by the processing unit.

[0006] One of the advantages of the above embodiment is that by introducing the instructions corresponding to the first type of graphics processor as described above, it is possible to avoid wasting a lot of time and manpower on developing a hardware simulator corresponding to the second type of graphics processor.

[0007] Another advantage of the above embodiment is that, by introducing the instructions corresponding to the first type of graphics processor as described above, similar simulation results can be generated before the second type of graphics processor is available for testing.

[0008] Another advantage of the above embodiment is that, by introducing the instructions corresponding to the first type of graphics processor as described above, better execution efficiency can be achieved compared to using a software emulator of a central processing unit, and problems arising from concurrent operations can be exposed.

[0009] Other advantages of the present invention will be explained in more detail with reference to the following description and accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0011] Figure 1 A schematic diagram of an example general programming interface protocol stack.

[0012] Figure 2 FIG. 1 is a schematic diagram of a programming interface protocol stack adapted to an existing graphics processor according to an embodiment of the present invention.

[0013] Figure 3 FIG. 4 is a diagram of a hardware architecture for running an adaptive hardware simulator according to an embodiment of the present invention.

[0014] Figure 4 is a schematic diagram of an exemplary three-dimensional tensor according to an embodiment of the present invention.

[0015] Figure 5 FIG. 1 is a schematic diagram of an exemplary small slice plane according to an embodiment of the present invention.

[0016] Figure 6 FIG. 4 is a flow chart of a pooling operation method according to an embodiment of the present invention.

[0017] Figure 7 FIG. 1 is a flow chart of a generalized instruction simulation method according to an embodiment of the present invention.

[0018] Figure 8 FIG. 1 is a schematic diagram illustrating using an adaptive hardware simulator to simulate a quarter of the shared memory in a computing core of a next-generation graphics processor according to an embodiment of the present invention.

[0019] Figure 9 FIG2 is a schematic diagram of using an adaptive hardware simulator to simulate the reduction computing hardware of a computing unit in a computing core of a new generation graphics processor and a reduction buffer according to an embodiment of the present invention.

[0020] Figure 10 FIG2 is a schematic diagram illustrating using an adaptive hardware simulator to simulate general matrix multiplication hardware and matrix registers in the tensor cores of a next-generation graphics processor according to an embodiment of the present invention.

[0021] Figure 11 FIG. 4 is a schematic diagram of matrix multiplication and accumulation according to an embodiment of the present invention.

[0022] Figure 12 FIG. 1 is a flow chart of a generalized instruction simulation method according to an embodiment of the present invention.

[0023] Figure 13 FIG. 4 is a simulation diagram of an adaptive hardware simulator according to an embodiment of the present invention.

[0024] Among them, the brief description of the symbols in the accompanying drawings is as follows:

[0025] 100: Universal programming interface protocol stack; 110: User program; 120: Next-generation GPU programming language interface; 130: Compiler; 140: Binary code of GPU instructions; 150: GPU execution environment; 200: Programming interface protocol stack adapted to existing GPUs; 210: Existing GPU programming interface; 220: Compiler for existing GPUs; 230: Binary code of existing GPU instructions; 240: Existing GPU execution environment; 30: Electronic device; 310: Graphics processor; 320: Command processor; 330: Computing unit; 332: Execution unit; 335: Universal shared memory; 335-0 to 335-3: Part of universal shared memory; 338: Shared memory; 355-0 to 355-3: Regions within the part of universal shared memory; 350: Direct memory access controller; 360 : global memory; 380: central processing unit; 40: three-dimensional tensor; 410: a whole column; 430: a whole row; 450: a small tile; 50: small tile plane; S610-S650: method steps; S710-S770: method steps; 80: operation core; 820-0-820-3: computing unit; 825-0-825-3: general shared memory; 920: general reduction buffer; 920-0 to 920-3: Sub-universal reduction buffer; 1000: Tensor core; 1010: Matrix A buffer; 1020: Matrix B buffer; 1030: Matrix C buffer; 1050: Universal matrix multiplication unit; 1110: Matrix A; 1120: Matrix B; 1130: Matrix C; S1210 to S1270: Method steps; 1310: Existing GPU instructions; 1320: Simulation results. DETAILED DESCRIPTION

[0026] The embodiments of the present invention will be described below with reference to the accompanying drawings. In these drawings, the same reference numerals represent the same or similar components or method flows.

[0027] It must be understood that the words "comprise", "include" and the like used in this specification are used to indicate the existence of specific technical features, values, method steps, job processing, components and / or components, but do not exclude the addition of more technical features, values, method steps, job processing, components, components, or any combination of the above.

[0028] The terms "first", "second", "third", etc. used in the present invention are used to modify the components in the claims and are not used to indicate a priority order, a precedence relationship, or that one component precedes another, or a temporal sequence in executing method steps. They are only used to distinguish components with the same name.

[0029] It should be understood that when a component is described as being "connected" or "coupled" to another component, it can be directly connected or coupled to the other component, and intervening components may be present. Conversely, when a component is described as being "directly connected" or "directly coupled" to another component, there are no intervening components. Other words used to describe the relationship between components should be interpreted in a similar manner, for example, "between" versus "directly between," or "adjacent" versus "directly adjacent," etc.

[0030] In the development process of the new generation of graphics processing units (GPUs), in order to verify their performance in various applications, developers need to set up a simulation environment so that the program codes written by users for the new generation of GPUs can run in simulation. Figure 1 In the example of a general programming interface protocol stack (General Programming Interface Layer Stack) 100, developers can use high-level programming languages ​​(such as C, C++, etc.) to write user programs 110, such as the Depth-Wise Convolution algorithm, which includes instructions (such as _depth_wise_conv(...)) that call the new generation GPU programming language interface (Programming Language Interface) 120. The simulation environment also includes a compiler 130 for compiling the data structures, variables, and instructions used in the user program 110, as well as the functions of the new generation GPU programming language interface 120 called by it, into binary code (GPU Instruction Binary) 140 of GPU instructions. The simulation environment also includes a GPU execution environment 150 for driving the GPU hardware to execute the binary code 140 of GPU instructions.

[0031] In some previous implementations, developers could create a hardware emulator in a simulation environment, such as Zebu, which includes a compiler 130 and an execution environment 150 adapted to the new generation of GPUs. However, this would consume a significant amount of time and manpower to develop the hardware emulator, and it would not be feasible until the new generation of GPUs were available for testing.

[0032] In other previous implementations, developers can create a software simulator (SoftwareSimulator) within a simulation environment. This simulator includes a behavioral model (e.g., CModel, instruction simulator, etc.) adapted for new-generation GPUs, and is used to simulate the operation of new-generation GPUs using the computing resources of a central processing unit (CPU). The software simulator also includes a compiler 130 adapted for the new-generation GPUs and a GPU execution environment 150. The GPU execution environment 150 also includes a complete software stack (e.g., a driver, etc.) for running the compiled binary code 140. However, because the CPU clock frequency (typically measured in kHz) is much slower than the GPU clock frequency (typically measured in GHz), approximately 1 / 1000 of the GPU clock frequency, using the CPU's computing resources to simulate GPU instructions is inefficient. Furthermore, CPUs typically have only about 10 cores, far fewer than the hundreds or thousands of cores in GPUs. Therefore, the software simulator requires a large amount of sequential logic to perform concurrent operations, making problems arising from concurrent operations unresolved when simulating using the CPU.

[0033] In order to solve the above-mentioned technical problems, an embodiment of the present invention proposes an adaptive hardware emulator that uses an existing GPU (also referred to as a first-type GPU) configured in a computing system to simulate the program code of a new generation GPU (also referred to as a second-type GPU). Figure 2The following figure shows a programming interface protocol stack 200 adapted for existing GPUs. A user program 110 for a new-generation GPU can call streamlined functions in the new-generation GPU programming language interface 120 and provide actual parameters. However, the compiler 220 for existing GPUs cannot compile the call instructions and actual parameters for these streamlined functions. The user program 110 for a new-generation GPU can also use data structures and variables corresponding to the new hardware components of the new-generation GPU, provided in the new-generation GPU programming language interface 120. However, the compiler 220 for existing GPUs cannot compile these data structures and variables. The user program 110 for a new-generation GPU can also call functions corresponding to the new hardware components of the new-generation GPU, provided in the new-generation GPU programming language interface 120, and provide actual parameters. However, the compiler 220 for existing GPUs cannot compile these function call instructions and actual parameters. It should be noted that these new hardware components are not included in the new-generation GPU. To enable a bridge between the user program 110 and the new-generation GPU programming language interface 120 and the compiler 220 for existing GPUs, the adaptive hardware simulator includes a GPU instruction converter. The GPU instruction converter is implemented using software code and performs the following functions when the processing unit (which can be a central processing unit or a graphics processing unit) loads and executes the appropriate code. The GPU instruction converter can import instructions corresponding to the existing GPU through the existing GPU programming interface 210 based on the simplified functions and actual parameters in the GPU programming language interface 120 called by the user program 110 of the new generation GPU, thereby using the existing GPU to simulate the simplified functions in the GPU programming language interface 120, which are originally used to drive the new generation GPU to perform specified functions. The GPU instruction converter can also import instructions corresponding to the existing GPU through the existing GPU programming interface 210 based on the functions and actual parameters in the GPU programming language interface 120 called by the user program 110 of the new generation GPU, as well as the data structures and variables in the user program 110 of the new generation GPU corresponding to the new hardware components in the new generation GPU, thereby using the hardware architecture of the existing GPU to simulate the functions to be performed by the new hardware components in the new generation GPU. Next, the existing GPU compiler 220 compiles the converted existing GPU instructions into binary code 230 that can be parsed by the existing GPU, and the existing GPU execution environment 240 drives the hardware of the existing GPU to execute the binary code 230 of the existing GPU instructions.Compared to the aforementioned hardware simulator, developers do not need to develop compilers and next-generation GPU execution environments within the adaptive hardware simulator. Furthermore, before the next-generation GPU is available for testing, developers can use existing GPUs to debug functions within the next-generation GPU programming language interface 120. Furthermore, the adaptive hardware simulator offers significantly higher execution performance than the aforementioned software simulator, and developers can use it to detect whether functions within the next-generation GPU programming language interface 120 are experiencing concurrent execution issues.

[0034] refer to Figure 3 The hardware architecture used to run the adaptive hardware simulator is shown. The electronic device 30 can be implemented as a mainframe, workstation, personal computer, laptop PC, tablet computer, or other electronic product. The electronic device 30 can be configured with a computing system 300 including a graphics processing unit (GPU) 310, a central processing unit (CPU) 380, and random access memory 390. The computing system 300 can be programmed to perform various application tasks, including but not limited to: linear and nonlinear data conversion, database operations, big data computing, artificial intelligence calculations, audio and video data encoding, decoding, modeling operations, image rendering operations, etc. Unless otherwise specified, the GPU 310 is the existing GPU referred to in the adaptive hardware simulator, not the next-generation GPU. The GPU 310 and CPU 380 are interconnected via a bus architecture 370, such as a Peripheral Component Interconnect Express (PCI-E), to transmit data, addresses, control signals, and the like. Generally speaking, CPU 380 and GPU 310 are suited for different tasks. CPU 380 is more suitable for processing complex serial logic, complex control flow, and interaction with the system's input and output devices, while GPU 310 is more suitable for concurrent single-instruction, multiple-data operations, such as Single Instruction Multiple Data (SIMD) instructions and Single Instruction Multiple Thread (SIMT) technology. Although the hardware architecture and operational performance of GPU 310 are inferior to those of new-generation GPUs, due to the similar hardware architecture and the similar functionality provided, it can still be used to simulate and execute functions in the new-generation GPU programming language interface 120. Although the embodiments of the adaptive hardware simulator describe the use of an existing GPU to simulate and execute functions in the new-generation GPU programming language interface 120, those skilled in the art can also use an existing GPU to simulate and execute functions in the programming language interface of a previous-generation GPU or other generations of GPUs, and the present invention is not limited thereto.

[0035] The CPU 380 can issue a series of commands to the command processor (CP) 320 in the GPU 310 based on the compiled binary code of the GPU instructions, requesting the execution of the specified operation. The CP 320 can assign the operation task to one of the multiple compute units (CU) 330. Each CU 330 contains multiple execution units (EU) 332, and each EU 332 can process the data of a warp. Each warp is composed of a group of threads, and a thread is the smallest unit that runs using hardware and has its own life cycle. For example, a warp contains 32 threads. Each EU 332 contains an arithmetic logic unit (ALU) and general-purpose registers (GPR). The ALU can perform a variety of operations, such as integer and floating-point addition and multiplication, comparison operations, Boolean operations, bit shifting, and algebraic functions (e.g., planar interpolation, trigonometric functions, exponential functions, and logarithmic functions). During execution, the ALU can read data from a specified location in a general-purpose register (also known as a source address) and write back execution results to a specified location in a general-purpose register (also known as a destination address). During parallel operations, the ALU can access non-overlapping portions of the general-purpose registers for each thread, allowing operations between threads to be independent of each other. The CP 320 can drive the Direct Memory Access (DMA) Controller 350 via the bus architecture 370 to read data from a specified address in the RAM 390 and store it in a specified general-purpose register in the EU 332, as well as read computation results from a specified general-purpose register in the EU 332 and store them in a specified address in the RAM 390. Each CU 330 also includes shared memory 338, allowing each EU within that CU 330 to access data in the shared memory 338. In other words, the shared memory 338 within a CU 330 is visible to each EU within that CU. The physical size of the shared memory 338 is typically much larger than the physical size of the general-purpose registers within each EU, allowing for flexible use by threads running within those EUs. The shared memory 338 within each CU 330 can typically store tens of kilobytes (KB) of data.Since the raw data required for some calculations consumes more storage space, GPU 310 can set up a global memory (GlobalMemory) 360, which can usually store hundreds of KB of data, and allow each CU and each EU in GPU 310 to access the data in global memory 360. In other words, the global memory 360 in this GPU 310 is visible to each CU and each EU therein.

[0036] In some embodiments, the adaptive hardware simulator can be used to simulate the data loading function for the warp in the next-generation GPU programming language interface 120. An example of the application programming interface (API) of the data loading function for the warp in the GPU programming language interface 120 is as follows:

[0037] __device__void__load_matrix(__short_vector<E,SVN> *dst,Matrix3D<E,MemType,Layout,N,H,W> In, Coordinate3D coord);

[0038] References Figure 4The three-way tensor 40 shown in FIG. 4 has dimensions {N, H, W, C}: “N” represents the batch size; “H” represents the number of rows; “W” represents the number of columns; and “C” represents the number of channels. The user program 110 can call the function “__load_matrix()” in the new generation GPU programming language interface 120 and give the definition information of the three-way tensor (such as “Matrix3D<E,MemType,Layout,N,H,W> In”), starting coordinates “coord” and a pointer to the destination address “*dst”, etc., are used to allow the new generation GPU (not shown) to continuously read a whole column (also called a target column) 410, a whole row (also called a target row) 430 or a small tile (also called a target small tile) 450 of data from the specified address of the random access memory 390 according to the starting coordinates “coord”, and store the data to the specified address in the general-purpose register (GPR, not shown) of the new generation GPU according to the pointer to the destination address “*dst”. A whole column, a whole row or a whole small tile can contain a matrix that can be processed using 32 threads. However, the compiler 220 of the existing GPU cannot compile the data loading function and actual parameters of the thread bundle in the GPU programming language interface 120 called by the user program 110 to generate the GPU 310. Therefore, the converter of the existing GPU instruction generates an instruction that can be compiled by the existing GPU compiler 220 based on the calling instruction and actual parameters of the data loading function of the thread bundle. The instruction includes executing a loop 32 times, and in each iteration, reading data from a specified address in the random access memory 390 for each thread and storing it in a specified address in the general-purpose register (GPR) of the specified CU 330. Assume that the user program 110 calls the function "__load_matrix()" to continuously load data from 4x8 tiles to 32 threads: the instruction generated by the converter of the existing GPU instruction includes a repeatedly executed loop, and in each iteration, calculates the source coordinates and destination address of a thread (associated with a unit in the 4x8 tile). The source coordinates are used to associate with a specific address in the random access memory 390, and the destination address is used to point to a specific general-purpose register of the specified CU 330. The code example is as follows:

[0039] / / Get the real source coordinates of the current thread

[0040] Unsigned x_off=warp_thread_idx%8;

[0041] Unsigned y_off=warp_thread_idx / 8;

[0042] Coordinate3D realC(coord.n,coord.h+y_off,coord.w=x.off);

[0043] / / Get the destination address of the current thread

[0044] E*p=mapCoordToAddr(realC);

[0045] *dst=*p;

[0046] Next, the compiler 220 of the existing GPU can compile the above code example into binary code 230 of existing GPU instructions, and the binary code 230 is executed by the execution environment 240 of the existing GPU to simulate the continuous loading of 4x8 small slices of data into the general registers configured for 32 threads.

[0047] In other embodiments, the adaptive hardware simulator can be used to simulate the data storage function for thread bundles in the new generation GPU programming language interface 120, and is used to continuously read an entire column (also called a target column) 410, an entire row (also called a target row) 430 or a small tile (also called a target small tile) 450 of data from a specified address in the general register of the new generation GPU, and store it at a specified address in the random access memory 390 according to the pointer of the destination address.

[0048] Based on the call instruction and actual parameters of the data storage function of the thread warp, the converter of the conventional GPU instruction imports instructions that can be compiled by the conventional GPU compiler 220 through the conventional GPU programming interface 210. The instructions include executing a loop 32 times. In each iteration, data is read from a specified address in the general register of the designated CU 330 for each thread and stored to a specified address in the random access memory 390. Assume that the user program 110 calls the function "__store_matrix()" to continuously store data of 32 threads (4x8 tiles) to the random access memory 390: the instructions generated by the converter of the conventional GPU instruction include a repeatedly executed loop. In each iteration, the source coordinates and destination address of a thread (associated with a unit in the 4x8 tile) are calculated. The source coordinates are used to associate with a specific general register of the designated CU 330, and the destination address is used to point to a specific address in the random access memory 390.

[0049] In some other embodiments, the adaptive hardware simulator can be used to simulate the warp-level pooling function in the next-generation GPU programming language interface 120. An example of the application program interface of the warp-level pooling function in the GPU programming language interface 120 is as follows:

[0050] __device__void__pool(__short_vector<E_DST,2> *out,bf162*in,uint in_center_idx,bf162*left_padding,uint left_center_idx,bf162*right_padding,uintright_center_idx,int padX,int padY,uint stride,uint dilation,ConvPackconvPack);

[0051] References Figure 5The tile surface 50 shown includes nine 4x8 tiles R0-R3, R5-R7, and R10-R12. The user program 110 can call the function "__pool()" in the new generation GPU programming language interface 120 and assign the target tile identifier (such as the central 4x8 tile R6) and input pointer "*in", the left adjacent tile identifier (such as the 4x8 tile R1) and pointer "*left_padding", the right adjacent tile identifier (such as the 4x8 tile R10) and pointer "*right_padding", the output pointer "*out" of the calculation result, etc. as parameters, so that the new generation GPU (not shown) can read the data of the target tile (such as the central 4x8 tile R6) and the adjacent tiles (such as the 4x8 tiles R0-R3, R5, R7, R10-R12) from the specified address of the random access memory 390, perform pooling operations on the target tile based on the read data, and store the operation results. The ALU in the specified EU 332 can use 32 threads in parallel to perform operations on 32 groups of data in the target tile. The data in the target small slice can be called activation data, and the data in the adjacent small slice can be called padding data. For example, the first thread is used to process the T0 group data in the target small slice R6, the second thread is used to process the T1 group data in the target small slice R6, and so on. For each group of data in the target small slice R6, the ALU needs to refer to the data of the adjacent 8 groups to perform pooling operations. For example, in the first thread, the ALU will perform pooling operations based on the T31 group data in the small slice R0, the T7 and T15 group data in the small slice R1, the T24 and T25 group data in the small slice R5, and the T0, T1, T8, and T9 group data in the small slice R6 to generate the calculation results of the T0 group in the target small slice R6. In the second thread, the ALU performs pooling operations based on the data in groups T24 to T26 in small slice R5 and the data in groups T0 to T2 and T8 to T10 in small slice R6 to generate the calculation results for group T1 in the target small slice R6. The pooling operations for groups T2 to T31 in the target small slice R6 can be deduced similarly and will not be described in detail for the sake of simplicity. The pooling operations can use algorithms well known to those skilled in the art, such as max pooling and average pooling.

[0052] However, the existing GPU compiler 220 cannot compile the warp-level pooling operation function and actual parameters in the GPU programming language interface 120 called by the user program 110 to generate binary code 230 that can be executed by the GPU 310. Therefore, the converter of the existing GPU instruction introduces instructions that can be compiled by the existing GPU compiler 220 through the existing GPU programming interface 210 based on the calling instructions and actual parameters of the warp-level pooling operation function. The instructions include: configuring enough storage for the target small slice (such as Figure 5 R6 shown) and adjacent small fragments (such as Figure 5 The CPU 380 reads the data of the target small slice and the adjacent small slice from the specified address of the random access memory 390, and stores the read data in the configured space; and performs pooling operations on multiple groups of data in the target small slice in parallel based on the data of the target small slice and the adjacent small slice, and stores the calculation results. Then, the compiler 220 of the existing GPU compiles the introduced instructions into binary code 230. When the CPU 380 sends the compiled binary code 230 to the GPU 310 to drive the GPU 310 to simulate the task of completing the pooling operation function at the thread warp level in the GPU programming language interface 120, the CP 320 executes as follows. Figure 6 The pooling operation method shown is described in detail as follows:

[0053] Step S610: Select one CU 330 from multiple CUs 330. The CP 320 may select a CU 330 that is idle or about to complete a computing task based on the load conditions of all CUs 330.

[0054] Step S620: Allocate enough memory to store the target small slice (e.g. Figure 5 R6 shown) and adjacent small fragments (such as Figure 5 The space for data of R0 to R2, R5, R7, R10 to R12 shown.

[0055] Step S630 : driving the DMA controller 350 to read the data of the target tile and the adjacent tiles from the designated address of the random access memory 390 through the bus architecture 370 , and storing the read data into the shared memory 338 in the selected CU 330 .

[0056] Step S640: The selected CU 330 is driven to cause its EU 332 to perform pooling operations on multiple groups of data in the target small tile in parallel based on the data of the target small tile and neighboring small tiles, and store the calculation results in general registers of the executing EU 332. These calculation results can be further read by the CPU 380 and stored in a specified address of the random access memory 390.

[0057] Step S650 : Release the space originally allocated to the data of the target small slice and the neighboring small slices in the shared memory 338 of the selected CU 330 .

[0058] In other embodiments, in order to simulate data loading and storing of warps, warp-level pooling operations, and other functions in the next-generation GPU programming language interface 120, the CP 320 may execute the following: Figure 7 The generalized instruction simulation method shown is described in detail as follows:

[0059] Step S710: Calculate the target positions on a surface corresponding to all threads in the current warp (e.g. Figure 3 A whole column 410, a whole row 430 or a small slice 450 in, or Figure 4 The location of a specific data group in the target small fragment R6).

[0060] Step S720: Convert warp-uniform input parameters into per-thread-specific input parameters. Because the next-generation GPU programming language interface 120 provides warp-based uniform input parameters, which are not supported by the existing GPU 310, the warp-uniform input parameters must be converted into per-thread-specific input parameters before they can be recognized and processed by the selected CU 330.

[0061] Step S730: Determine whether it is necessary to access adjacent data outside the target location (e.g. Figure 4 If yes, the process continues with step S740; otherwise, the process skips step S740 and directly proceeds to step S750.

[0062] Step S740: Configure the space of the shared memory in the specific EU 332 for storing the data at the target location and the adjacent data outside the target location, drive the DMA controller 350 to read the data at the target location and the adjacent data outside the target location from the specified address of the random access memory 390 through the bus architecture 370, and store them in the space configured in the shared memory.

[0063] Next, the CP 320 drives the designated EU to complete the operations of steps S750 to S770.

[0064] Step S750: The designated EU loads the data required by each thread into the general registers of this EU based on the input parameters specific to each thread. If the shared memory space is not configured for the simulated function (the "No" path in step S730), the designated EU can read the data at the target location from the designated address of the random access memory 390 for each thread through the bus architecture 370 via the DMA controller 350, and store it at the designated address of the general register of this EU. If the shared memory space has been configured for the simulated function (the "Yes" path in step S730), the designated EU can read the data at the target location and adjacent data outside the target location directly from the designated address of the shared memory for each thread, and store it at the designated address of the general register of this EU.

[0065] Step S760: Specify the EU to concurrently calculate the data associated with each thread using the specified algorithm.

[0066] Step S770: The designated EU stores the calculation results of each thread into the general register of the designated EU.

[0067] It should be noted here that if the simulated function does not involve data calculation but only data loading or storage, steps S760 and S770 can be ignored.

[0068] In other embodiments, the adaptive hardware simulator may be used to simulate the operation of the Quartered Shared Memory of a Compute Core of a new generation GPU.

[0069] refer to Figure 8For example, the new generation GPU includes at least four CUs 820-0 to 820-3, and CUs 820-0 to 820-3 each include a general shared memory (GSM) 825-0 to 825-3. The four CUs 820-0 to 820-3 can be combined into one computing core 80 to provide parallel processing of 1024 threads, and a quarter of the space (as shown by the slashed area) is set as shared space in each of the GSMs 825-0 to 825-3. That is, each GSM provides a quarter of the storage space visible to all threads running in the CU to which it belongs, collectively referred to as a quarter of the shared memory. For example, all threads running in CU 820-0 can use the storage space in GSM 825-0, and so on. The user program 110 of the new generation GPU can activate the quarter shared memory of any computing core in the new generation GPU. The code example is as follows:

[0070] __global_mega__void sampleKernel(const void*data){

[0071] __quarter_shared__array

[1024]

[0072] }

[0073] The function form "_global_mega_" indicates that the function "sampleKernel()" starts the operation kernel, and the variable form "_quarter_shared_" indicates that the data of the array "array

[1024] " is stored in the quarter shared memory.

[0074] However, due to the lack of relevant hardware for the new generation GPU, the compiler 220 of the existing GPU cannot compile the function form "_global_mega_" and the variable form "_quarter_shared_" in the user program 110 of the new generation GPU to generate binary code 230 corresponding to GPU 310. Therefore, the converter of the existing GPU instruction introduces instructions that the compiler 220 of the existing GPU can compile based on the function form "_global_mega_" and the variable form "_quarter_shared_" through the programming interface 210 of the existing GPU, thereby simulating the activation of the quarter shared memory of the computing core of the new generation GPU using the hardware architecture of the existing GPU. The compiler 220 of the existing GPU then compiles the newly introduced GPU instructions into binary code 230. When the CPU 380 sends the compiled binary code 230 to the GPU 310, the CP 320 designates a CU 330 to simulate the computing core of the new generation GPU. Specifically, the CP 320 configures the designated CU 330 to execute using a fixed number of thread hardware (e.g., 512 threads), divides all threads into four groups, divides the common shared memory 335 in the designated CU 330 into four local memory sections 335-0 to 335-3, and allows access to a designated portion of threads. For example, threads 0 to 127 are configured to use local memory section 335-0; threads 128 to 255 are configured to use local memory section 335-1; threads 256 to 383 are configured to use local memory section 335-2; and threads 384 to 511 are configured to use local memory section 335-3. The GPU instructions introduced by the existing GPU instruction converter include the following operations for:

[0075] Set a CU to execute using a fixed number of thread hardware;

[0076] Divide the common shared memory in the CU into four non-overlapping parts; and

[0077] The visible range of each thread running in the CU in the universal shared memory is set to restrict each thread to access only one part of the universal shared memory, and each part of the universal shared memory can only be accessed by one quarter of the total number of threads running in the CU.

[0078] In other embodiments, the adaptive hardware simulator can be used to simulate the reduction calculation hardware of the CU in the computing core of the new generation GPU, as well as the operation of the reduction buffer. Figure 9For example, a new generation GPU includes at least four CUs 820-0 to 820-3. These four CUs 820-0 to 820-3 can be combined into a single computing core 80, and each CU includes dedicated hardware for performing reduction calculations. For example, a reduction calculation can be the sum of the calculation results of all threads in a CU, as shown below:

[0079]

[0080] For another example, the reduction calculation can be to find the sum of the squares of the calculation results of all threads in a CU, which can be expressed as follows:

[0081]

[0082] Rst represents the result of the reduction computation, n(t) represents the total number of threads running in the CU, and Ri represents the computation result of the i-th thread. The computing core 80 also includes a general reduce buffer (GRB) 920, which is divided into four local reduce buffers 920-0 through 920-3 (also called sub-general reduce buffers GRB-0 through GRB-3), respectively storing the reduction computation results of CUs 820-0 through 820-3.

[0083] The GPU programming language interface 120 may provide an application program interface for a function to obtain a reduction buffer, an example of which is as follows:

[0084]

[0085]

[0086] The user program 110 can call the reduction buffer acquisition function "__get_reduce_buf()" in the new generation GPU programming language interface 120 to start the reduction computing hardware of the CU in the computing core of the new generation GPU and use the reduction buffer to store the calculation results of the reduction computing hardware.

[0087] However, due to the lack of relevant hardware for the new generation GPU, the compiler 220 of the existing GPU cannot compile the call instructions and actual parameters of the reduction buffer acquisition function in the user program 110 to generate binary code 230 corresponding to the GPU 310. Therefore, the converter of the existing GPU instructions, based on the data structure using the reduction buffer, the call instructions and actual parameters of the reduction buffer acquisition function, imports instructions that the compiler 220 of the existing GPU can compile through the programming interface 210 of the existing GPU. This is used to simulate the reduction computing hardware and reduction buffers of the CU in the computing core of the new generation GPU using the hardware architecture of the existing GPU. The compiler 220 of the existing GPU then compiles the newly imported GPU instructions into binary code 230. When the CPU 380 sends the compiled binary code 230 to the GPU 310, the CP 320 specifies a CU 330 to simulate the reduction computing of the computing core of the new generation GPU. Specifically, CP 320 divides all threads into four groups, configures the general shared memory local 335-0 in the designated CU 330 to be divided into four regions 355-0 to 355-3, which serve as sub-general reduction buffers GRB-0 to GRB-3, and only allows threads in the designated group to access each region. For example, the zeroth group of threads is configured to be able to use region 355-0; the first group of threads is configured to be able to use region 355-1; the second group of threads is configured to be able to use region 355-2; and the third group of threads is configured to be able to use region 355-3. CP 320 drives the designated CU 330 to execute the designated calculations of the four groups of threads in parallel. CP 320 also drives the designated CU 330 to merge the calculation results of the four groups of threads, wherein the corresponding regions in the general shared memory local 335-0 are used to store temporary or final results of the merged operations of one group of threads. The GPU instructions introduced by the converter of existing GPU instructions include the following operations, which are used to:

[0088] Divide all threads into four groups;

[0089] The general shared memory in a CU is divided into four non-overlapping areas, and only threads in a specified group are allowed to access them;

[0090] Using the CU to execute specified calculations of four groups of threads in parallel; and

[0091] The calculation results of each group of threads are merged, wherein a corresponding area in a common shared memory is used to store temporary results or final results of the reduction calculations of the group of threads.

[0092] In some other embodiments, the adaptive hardware simulator can be used to simulate the operation of the General Matrix Multiply (GEMM) hardware and the Matrix Buffer in the TensorCore of the new generation GPU. Figure 10 For example, the new generation GPU includes at least four CUs 820-0 to 820-3, and the four CUs 820-0 to 820-3 can be combined into one computing core 80. The computing core 80 also includes a tensor core 1000, and the tensor core 1000 includes a matrix A buffer (matrix ABuffer, which can be abbreviated as ABuf) 1010, a matrix B buffer (matrix B buffer, which can be abbreviated as BBuf) 1020, a matrix C buffer (matrix C buffer, which can be abbreviated as CBuf) 1030 and a general matrix multiplication unit 1050. The general matrix multiplication unit 1050 is used to perform matrix multiplication and accumulation (MMA) based on the two matrices stored in the matrix A buffer 1010 and the matrix B buffer 1020, and store the calculation results in the matrix C buffer 1030. Matrix multiplication and accumulation is a common algorithm in linear algebra, machine learning, big data statistics, artificial intelligence and other fields. Matrix multiplication and accumulation can be expressed as the following equation:

[0093] C = AxB,

[0094] It represents the generation of a matrix C from two matrices A and B, where the number of columns of matrix A must be equal to the number of rows of matrix B. For example, refer to Figure 11 , matrix A is an mxn matrix 1110, and matrix B is an nxp matrix 1120, which are expressed as follows:

[0095]

[0096] Matrix C is the product of matrix A and matrix B. The result is an mxp matrix 1130, which is represented as follows:

[0097]

[0098] The calculation formula is:

[0099]

[0100] i is any positive integer from 1 to m, and j is any positive integer from 1 to p. m, n, and p may be dynamically adjusted constants.

[0101] The GPU programming language interface 120 may provide an application program interface for a matrix loading function and a matrix multiplication and accumulation function, as shown below:

[0102]

[0103] The user program 110 may call the matrix loading function “__load_input_buf()” in the next-generation GPU programming language interface 120 to read data at a specified location on a three-dimensional tensor from a random access memory via a bus architecture and store the data in the matrix A buffer 1010 or the matrix B buffer 1020 in the tensor core 1000. Next, the user program 110 may call the matrix multiplication and accumulation function “__mma()” in the next-generation GPU programming language interface 120 to perform matrix multiplication and accumulation based on the matrix A stored in the matrix A buffer 1010 and the matrix B stored in the matrix B buffer 1020, and store the calculation results in the matrix C buffer 1030.

[0104] However, due to the lack of relevant hardware for the new generation GPU, the compiler 220 of the existing GPU is unable to compile the call instructions and actual parameters of the matrix load function and the matrix multiplication and accumulation function in the user program 110 to generate binary code 230 corresponding to the GPU 310. Therefore, the converter of the existing GPU instructions introduces instructions that the compiler 220 of the existing GPU can compile through the programming interface 210 of the existing GPU, based on the data structure using the reduction buffer and the call instructions and actual parameters of the matrix load function and the matrix multiplication and accumulation function. This is used to simulate the general matrix multiplication hardware and matrix buffers in the tensor cores of the new generation GPU using the hardware architecture of the existing GPU. The compiler 220 of the existing GPU then compiles the newly introduced GPU instructions into binary code 230. When the CPU 380 sends the compiled binary code 230 to the GPU 310, the CP 320 designates a CU 330 to simulate the matrix multiplication and accumulation calculations of the computation cores of the new generation GPU. Specifically, the CP 320 may allocate space in the global memory 360 as a matrix A buffer 1060, a matrix B buffer 1070, and a matrix C buffer 1080, and drive the DMA controller 350 to read data at a specified location on the three-dimensional tensor from the random access memory 390 via the bus architecture 370 and store the data in the matrix A buffer 1060 and the matrix B buffer 1070 in the global memory 360. Subsequently, the CP 320 may drive the CU 330 to perform matrix multiplication and accumulation on the matrix A stored in the matrix A buffer 1060 and the matrix B stored in the matrix B buffer 1070 using multiple threads, and store the calculation results in the matrix C buffer 1080. The GPU instructions introduced by the converter of the existing GPU instructions include the following operations for:

[0105] Allocate space in the global memory as a simulation matrix A buffer, a simulation matrix B buffer, and a simulation matrix C buffer;

[0106] Reading data at a specified position on the three-dimensional tensor from the random access memory and storing the data in the matrix A buffer and the matrix B buffer in the global memory; and

[0107] Matrix multiplication and accumulation calculations are performed in parallel with multiple threads on the matrix A stored in the matrix A buffer and the matrix B stored in the matrix B buffer, and the calculation results are stored in the matrix C buffer.

[0108] In other embodiments, in order to cope with the new component forms in the next-generation GPU programming language interface 120 (such as operation cores, tensor cores, general matrix multiplication units, reduction computing units, quarter shared memory, reduction buffers, matrix buffers, etc.), as well as the simulation of reduction buffer access, matrix loading, matrix multiplication and accumulation, and other functions, the CP 320 may execute the following: Figure 12 The generalized instruction simulation method shown is described in detail as follows:

[0109] Step S1210: Configure the storage resources required by the hardware architecture of the new generation GPU. Figure 8 For example, in order to cope with the computing core 80 and a quarter of the shared memory of the new generation GPU, the CP 320 can configure the shared memory of a CU 330 of the GPU 310 to simulate a quarter of the shared memory of the new generation GPU. Figure 9 For example, in order to cope with the reduction computing hardware of CU 330 in the computing core 80 of the new generation GPU and the reduction buffer 920, CP 320 can configure the shared memory in one CU 330 of GPU 310 to simulate the reduction buffer 920 of the new generation GPU. Figure 10 Taking the architecture of the GPU as an example, in order to cope with the general matrix multiplication unit 1050 and matrix buffers 1010, 1020 and 1030 in the tensor core 1000 of the new generation GPU, the CP 320 can configure space in the global memory 360 outside the CU 330 to simulate the matrix buffers 1010, 1020 and 1030.

[0110] Step S1220: Determine whether to use shared memory. If yes, the process continues with step S1230; otherwise, the process skips step S1230 and directly proceeds to step S1240. For example, Figure 8 and Figure 9 The hardware architecture shown uses shared memory, Figure 10 The architecture shown does not.

[0111] Step S1230: Divide the shared memory into multiple groups. Figure 8 For example, in order to accommodate the computing cores 80 and a quarter of the shared memory in the new generation GPU, the CP 320 can divide the shared memory in a CU 330 of the GPU 310 into four non-overlapping parts, and set the visible range of each thread running in the CU 330 in the common shared memory, so as to restrict each thread to access only one part of the common shared memory, and each part of the common shared memory can only be accessed by a quarter of the total number of threads running in the CU 330. Figure 9 For example, to accommodate the reduction computing hardware of the CU 330 in the computing core 80 of the new generation GPU and the reduction buffer 920, the CP 320 can partition the shared memory in a CU 330 of the GPU 310 into four non-overlapping areas, and only allow threads in a specified group to access each area.

[0112] Step S1240: For each thread, calculate the target position on a surface (such as Figure 3 A whole column 410, a whole row 430 or a small slice 450 in, or Figure 4 The location of a specific data group in the target small fragment R6).

[0113] Step S1250: Load the data required by each thread. Figure 9 For example, the CP320 drives the DMA controller 350 to read the data at the target location from the specified address of the random access memory 390 through the bus architecture 370 for each thread, and stores it to the specified address of the general register of the CU to which it belongs. Figure 10 For example, when performing matrix multiplication and accumulation calculations using the architecture of CU 330, CP 320 drives DMA controller 350 to read the data of matrix A and matrix B from the specified address of random access memory 390 through bus architecture 370, and stores the data into matrix A buffer 1060 and matrix B buffer 1070 in global memory 360 outside CU 330, respectively.

[0114] Step S1260: Concurrently calculate the data associated with each thread using a specified algorithm.

[0115] Step S1270: store the calculation results of each thread. Figure 9 For example, the architecture of the reducer is used to store the results of the four reduction calculations in the area 355-0 to 355-3 in the local memory 335-0. Figure 10Taking the architecture of as an example to perform matrix multiplication and accumulation calculations, the CU 330 is designated to store the matrix multiplication and accumulation calculations in the matrix C register 1080 in the global memory 360 outside the CU 330.

[0116] References Figure 13 A simulation diagram of an adaptive hardware simulator is shown. In one aspect of the present invention, the GPU instruction converter can import instructions corresponding to the existing GPU through the existing GPU programming interface 1310 based on the streamlined functions and actual parameters in the GPU programming language interface 120 called by the user program 110 of the new-generation GPU. The GPU instruction converter can also import instructions corresponding to the existing GPU through the existing GPU programming interface 1310 based on the functions and actual parameters in the GPU programming language interface 120 called by the user program 110 of the new-generation GPU corresponding to the new hardware components of the new-generation GPU, as well as based on the data structures and variables in the user program 110 of the new-generation GPU corresponding to the new hardware components of the new-generation GPU. The compiler 220 of the existing GPU inputs the application program 110 of the new-generation GPU, the new-generation GPU programming language interface 120, and the existing GPU instructions newly imported through the existing GPU programming interface 1310, and compiles binary code 230 capable of driving the existing GPU 310. The existing GPU 310 completes the specified calculation according to the binary code 230 during operation, which is used to simulate the streamlined functions provided by the new generation GPU programming language interface 120 for driving the new generation GPU, or for running the data structures, variables and functions of the new generation GPU, and generates simulation results 1320.

[0117] All or part of the steps in the method described in the present invention can be implemented by a computer program, such as a driver program. In addition, it can also be implemented in other types of programs as shown above. A person skilled in the art can write the method of the embodiment of the present invention into program code, which will not be described again for the sake of simplicity. The computer program implemented according to the method of the embodiment of the present invention can be stored in an appropriate computer-readable storage medium, such as a DVD, CD-ROM, USB flash drive, hard disk, or can be placed on a network server that can be accessed through a network (e.g., the Internet, or other appropriate media).

[0118] Although Figure 3 、 Figures 8 to 10 The components described above are included in the invention, but it does not exclude the use of more additional components to achieve better technical effects without violating the spirit of the invention. Figure 6 、 Figure 7 、 Figure 12The flowchart is executed in the specified order. However, those skilled in the art may modify the order of the steps without violating the spirit of the invention, provided that the same effect is achieved. Therefore, the present invention is not limited to the sequence described above. Furthermore, those skilled in the art may also combine several steps into one step, or perform more steps sequentially or in parallel in addition to the steps described above, and the present invention should not be limited thereby.

[0119] The above description is only a preferred embodiment of the present invention, but it is not intended to limit the scope of the present invention. Anyone familiar with this technology can make further improvements and changes on this basis without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the scope defined by the claims of this application.

Claims

1. A hardware simulation method for a graphics processor, executed by a processing unit, characterized in that: include: The first-type graphics processor instruction converter imports instructions corresponding to the first-type graphics processor through the programming interface of the first-type graphics processor based on a reduced function in the programming language interface corresponding to the second-type graphics processor called in the user program and actual parameters, wherein the instructions corresponding to the first-type graphics processor are used to use the first-type graphics processor to emulate the reduced function in the programming language interface corresponding to the second-type graphics processor, wherein the reduced function is originally used to drive the second-type graphics processor to perform a specified function; Compiling the instructions corresponding to the first type of graphics processor by a compiler of the first type of graphics processor to generate binary code; and The execution environment of the first-type graphics processor drives the hardware of the first-type graphics processor to execute the binary code.

2. The hardware simulation method of a graphics processor according to claim 1, wherein: The compiler of the first-type graphics processor compiles the instructions corresponding to the first-type graphics processor.

3. The hardware simulation method of a graphics processor according to claim 2, wherein: The compiler of the first-type graphics processor cannot compile the simplified function and the actual parameters in the programming language interface corresponding to the second-type graphics processor called in the user program.

4. The hardware simulation method of a graphics processor according to claim 1, wherein: The streamlined function is a load function of a warp, configured to read a whole column, a whole row, or a small slice of data from a specified address of a random access memory, and store the data to a specified address in a general register of the second-type graphics processor.

5. The hardware simulation method of a graphics processor according to claim 4, wherein: The instructions corresponding to the first type of graphics processor include a repeatedly executed loop, and in each iteration, a source coordinate and a destination address of a thread are calculated, wherein the source coordinate is used to be associated with a specific address in the random access memory, and the destination address points to a specific general register of a computing unit in the first type of graphics processor.

6. The hardware simulation method of a graphics processor according to claim 1, wherein: The compact function is a storage function of a warp, configured to read a whole column, a whole row, or a small slice of data from a specified address in a general register of the second-type graphics processor and store the data in a specified address of a random access memory.

7. The hardware simulation method of a graphics processor according to claim 6, wherein: The instructions corresponding to the first type of graphics processor include a repeatedly executed loop, and in each iteration, a source coordinate and a destination address of a thread are calculated, wherein the source coordinate is used to be associated with a specific general register of a computing unit in the first type of graphics processor, and the destination address points to a specific address in the random access memory.

8. The hardware simulation method of a graphics processor according to claim 1, wherein: The streamlined function is a pooling operation function at the warp level, which is used to read data of a target small slice and adjacent small slices from a specified address of a random access memory, perform a pooling operation on the target small slice according to the read data, and store the operation result.

9. The hardware simulation method of a graphics processor according to claim 8, wherein: The instructions corresponding to the first type of graphics processor include: configuring space sufficient to store data of the target small tile and the neighboring small tiles; reading data of the target small tile and the neighboring small tiles from a specified address of a random access memory, and storing the read data in the configured space; and performing pooling operations on multiple groups of data in the target small tile in parallel based on the data of the target small tile and the neighboring small tiles, and storing the calculation results.

10. A computer-readable storage medium for storing program code executed by a processing unit, characterized in that: When the program code is executed by the processing unit, the hardware simulation method of the graphics processor according to any one of claims 1 to 9 is implemented.

Images