A GPGPU pipeline instruction cache device, GPGPU system and instruction fetching method
By introducing the L0 instruction cache built by FIFO in the GPGPU pipeline, the instructions are cached from the L1 instruction Cache to realize the non-blocking finger fetch of the pipeline, solving the problem of pipeline stagnation caused by the slow access speed of the L1 instruction Cache, and improving computing performance and computing efficiency.
Patent Information
- Application Number
- CN202411151659.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-08-21
AI Technical Summary
During the execution of the pipeline, the pipeline is stagnant due to the slow access speed of the L1 instruction Cache, which reduces the computing speed and efficiency.
The L0 instruction cache device built with FIFO caches instructions from the L1 instruction Cache to realize non-blocking assignment of the pipeline through prefetching the fingers, reducing the time-consuming finger fetching. The cache process of the L0 instruction cache and the execution scheduling process of the pipeline are independent of each other. The instruction execution time that occupies the pipeline.
Improves computing performance, reduces the probability of stagnation of the pipeline, saves the time-consuming finger fetching of the pipeline, and improves the computing speed and efficiency.
Smart Images

Figure CN118656127B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of GPGPU, and in particular to a GPGPU pipeline instruction cache device, a GPGPU system and an instruction fetching method. Background Art
[0002] GPGPU (General-Purpose computing on Graphics Processing Units) is a technology that extends the computing power of graphics processing units (GPUs) to general computing tasks. Through this technology, GPGPU is not only used for graphics rendering, but can also efficiently perform various general computing tasks, including scientific computing, data analysis, deep learning, etc.
[0003] GPGPU storage systems are usually composed of external global memory, SM-level shared memory, and operation unit-level register files. The parallel computing of a large number of GPGPU threads requires high-speed data transmission, but the bandwidth of the external global memory is limited and cannot meet the simultaneous access requirements of a large number of threads. Therefore, GPGPU systems usually have L2 Cache and L1 Cache between global memory and shared memory, and between shared memory and local memory to cache data in global memory and reduce thread access to global memory. However, compared with register-level file access, the access speed to L1 Cache is still slow. During the pipeline execution of instructions, fetching instructions from the L1 instruction cache will inevitably cause the pipeline to stagnate, reducing pipeline efficiency and computing speed.
[0004] GPGPU generally uses a five-stage pipeline to implement instruction execution, including instruction fetch, decoding, emission, execution, and writeback. The instruction fetch stage needs to read instructions from the L1 instruction cache. Although the L1 instruction cache caches instructions from the global memory by merging memory accesses, reducing the memory access time, it is still slower than the register-level execution speed, and there are still cache misses when accessing the L1 instruction cache, which inevitably increases the memory access time, causing the pipeline to stagnate and reducing the computing speed. Summary of the invention
[0005] In view of the defects of the prior art, the present invention provides a GPGPU pipeline instruction cache device, a GPGPU system and an instruction fetch method, which add an L0 instruction cache to the pipeline through FIFO, cache the instructions in the L1 instruction cache, and realize non-blocking assignment of the pipeline through pre-fetching, thereby saving the time of instruction fetching of the pipeline. In addition, the caching process of the L0 instruction cache is independent of the execution scheduling process of the pipeline, does not occupy the instruction execution time of the pipeline, reduces the stagnation probability of the pipeline, and improves the computing performance.
[0006] In order to solve the technical problem, the technical solution adopted by the present invention is: a GPGPU pipeline instruction cache device, which is an L0 instruction cache constructed by FIFO, the L0 instruction cache caches instructions from the L1 instruction cache, and provides register-level instruction reading for the pipeline; it includes FIFO, a read control module, an arbitrator and a cross switch, the FIFO is used to cache instructions read from the L1 instruction cache, the read control module is connected to the FIFO, monitors the full flag signal of the FIFO, if the FIFO is not full, generates an L1 instruction cache read request, and sends the read request to the arbitrator; the arbitrator is connected to the read control module, receives and arbitrates read requests sent by multiple read control modules, and generates a read request queue; the arbitrator is connected to the L1 instruction cache, sends the read request in the read request queue to the L1 instruction cache, the cross switch is connected between the L1 instruction cache and the FIFO, the cross switch receives the instruction returned by the L1 instruction cache, and determines the FIFO to which the instruction belongs according to the tag flag, and then writes the returned instruction into the corresponding FIFO.
[0007] Furthermore, the FIFO and the read control module constitute a hardware cache unit, and the L0 instruction cache includes multiple groups of hardware cache units, each group of hardware cache units is independent of each other and implemented in parallel.
[0008] Furthermore, the L0 instruction cache prefetches instructions from the L1 instruction cache.
[0009] Furthermore, the caching process of the L0 instruction cache is independent of the execution scheduling process of the pipeline.
[0010] Furthermore, the size and number of FIFOs are configured according to the number of cached instructions, pipeline efficiency, and the number of schedulable active warps.
[0011] Furthermore, the FIFO size is positively correlated with the number of cached instructions and negatively correlated with pipeline efficiency.
[0012] Furthermore, the number of FIFOs is equal to the maximum number of scheduled warps supported by the SM unit.
[0013] The present invention also discloses a GPGPU system, including an SM module, the SM module includes a pipeline, the pipeline includes five stages: instruction fetch, decoding, emission, execution and result write-back, and the instruction fetch stage is implemented based on the above-mentioned GPGPU pipeline instruction cache device.
[0014] The present invention also discloses an instruction fetching method based on a GPGPU pipeline instruction cache device, comprising the following steps:
[0015] S01), after the computing task is started, the thread warp scheduling module sends the initial PC to the L0 instruction cache;
[0016] S02), the read control module of the L0 instruction cache detects whether the FIFO is not full. If the FIFO is not full, an L1 instruction cache read request is generated according to the initial PC;
[0017] S03), the arbitrator receives and arbitrates read requests sent by multiple read control modules, generates a read request queue, and sends the first executed read request in the read request queue to the L1 instruction cache;
[0018] S04), after receiving the read request, the L1 instruction cache searches for corresponding data in the cache according to the request address. If the cache hits, the corresponding data is output to the crossbar switch. If the cache does not hit, the request is forwarded to the L2 cache or global memory, waiting for the corresponding memory to return data, and then outputting the data to the crossbar switch;
[0019] S05), after the arbiter determines that the L1 instruction cache processes the first executed read request in the read request queue, it dequeues the next read request to the L1 instruction cache, and the remaining read requests continue to queue, waiting for the current read request to be processed and then dequeuing the next read request, and repeating this process;
[0020] S06), after receiving the instruction returned by the L1 instruction cache, the crossbar switch determines the FIFO to which the current instruction belongs according to the tag, and writes it into the corresponding FIFO;
[0021] S07), when the FIFO is not completely empty, a ready signal is sent to the thread warp scheduling module, and the thread warp scheduling module starts scheduling the thread warp to enter the pipeline for execution;
[0022] S08), when the pipeline executes the instruction fetch stage, the instruction is dequeued from the corresponding FIFO and then enters the decoding stage.
[0023] Furthermore, the pipeline branches when executing instructions, and executes the branch mechanism of the L0 instruction cache, including the following steps:
[0024] S11), when the branch jump instruction reaches the execution stage of the pipeline, the pipeline will generate a branch signal marking the next instruction as a branch instruction and send it to the thread warp scheduling module;
[0025] S12), after the thread warp scheduling module detects the branch signal, it generates a clear signal and sends the clear signal, the thread warp wid and the PC of the branch instruction to the L0 instruction cache;
[0026] S13), after the L0 instruction cache detects the clear signal, the FIFO of the corresponding cache unit group is cleared according to the thread bundle wid, and an L1 instruction cache read request is generated according to the PC of the branch instruction to re-cache the instruction;
[0027] S14), when the thread warp executes the next instruction, the instruction dequeued from the FIFO in the pipeline fetch stage is the branch instruction, and the execution of the pipeline continues.
[0028] Beneficial effects of the present invention: The present invention adds an L0 instruction cache to the pipeline, caches instructions in the L1 instruction cache, implements non-blocking instruction fetching of the pipeline through pre-fetching, and saves the time consumption of instruction fetching of the pipeline.
[0029] The caching process of the L0 instruction cache is independent of the instruction scheduling process of the pipeline, does not occupy the instruction execution time of the pipeline, reduces the probability of pipeline stagnation, and improves computing performance.
[0030] The FIFO-based L0 instruction cache takes up less resources, has flexible control, and is highly scalable. The FIFO size can be reasonably configured according to the number of active thread warps and pipeline efficiency to improve resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 It is a pipeline diagram based on L0 instruction cache;
[0032] Figure 2 Schematic diagram of the GPGPU system. DETAILED DESCRIPTION
[0033] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.
[0034] Example 1
[0035] This embodiment discloses a GPGPU pipeline instruction cache device, which is an L0 instruction cache constructed by FIFO. The L0 instruction cache caches instructions from the L1 instruction cache, provides register-level instruction reading for the pipeline, and flexibly configures the FIFO size and number according to the number of schedulable active thread warps, realizes non-blocking instruction fetching of the pipeline, reduces the probability of pipeline stalling, and improves computing performance. Figure 1 As shown, this embodiment is an L0 instruction cache composed of four FIFOs, which supports the scheduling and execution of a maximum of four active thread warps. The FIFO size is configured according to the pipeline efficiency and the execution unit efficiency to ensure the highest utilization of resources.
[0036] like Figure 1As shown, the GPGPU pipeline instruction cache device described in this embodiment includes a FIFO, a read control module, an arbiter and a cross switch. The FIFO is used to cache instructions read from the L1 instruction cache. The FIFO size determines the number of cached instructions, and the number of FIFOs is determined by the maximum number of scheduled thread warps supported by the SM unit. In this implementation, the FIFO size is positively correlated with the number of cached instructions and negatively correlated with the pipeline efficiency. The number of FIFOs is equal to the maximum number of scheduled thread warps supported by the SM unit.
[0037] The read control module monitors the full flag signal of the FIFO. If the FIFO is not full, it generates an L1 instruction cache read request based on the PC value sent by the thread bundle and sends the read request to the arbitrator. The read request module and the FIFO form a set of hardware cache units. Each set of cache units is independent of each other and implemented in parallel to provide support for multi-thread bundle scheduling. The English expression of the above PC value is Program counter, which means the program counter, which is used to store the address of the instruction. The read control module generates the corresponding read request based on the address value.
[0038] The arbitrator module arbitrates the read requests generated by each read request module, generates a read request queue, and sends the first executed read request in the read request queue to the L1 instruction cache. After the arbitrator determines that the L1 instruction cache processes the request, it selects one from the queued read requests to continue to output to the L1 instruction cache, and the remaining read requests continue to queue, waiting for the current request to be processed and the next read request to be dequeued, and repeating this process. In this embodiment, the arbitrator performs priority sorting according to the identification signal in the read request, thereby generating a read request queue, which can be sorted by maximum priority, minimum priority, or weighted priority. This is a conventional technical means in the art and will not be repeated here.
[0039] After the read request generated by the arbiter module is sent to the L1 instruction cache, the L1 instruction cache accesses the memory according to the corresponding request address. If the memory access hits, the corresponding instruction is returned to the crossbar switch. If the memory access does not hit, the memory access request is forwarded to the L2 cache or global memory and waits for the instruction to return.
[0040] The crossbar switch is used to receive instructions returned by the L1 instruction cache, and determine which group of hardware cache units the instruction belongs to based on the tag flag, and then write the instruction into the FIFO of the corresponding hardware cache unit.
[0041] In this embodiment, the L0 instruction cache pre-fetches instructions from the L1 instruction cache, and the non-blocking assignment of the pipeline is realized by pre-fetching instructions, which saves the time consumed by the pipeline to fetch instructions. In addition, the caching process of the L0 instruction cache is independent of the execution scheduling process of the pipeline, does not occupy the instruction execution time of the pipeline, reduces the probability of pipeline stagnation, and improves the computing performance. At the same time, the FIFO-based L0 instruction cache occupies less resources, has flexible control, and is highly scalable. The FIFO size can be reasonably configured according to the number of active thread bundles and pipeline efficiency, thereby improving resource utilization.
[0042] Example 2
[0043] This embodiment discloses a GPGPU system, which is based on the pipeline instruction cache implementation described in Embodiment 1. Figure 2 As shown, it includes two parts: the host and GPGPU. The host consists of CPU and memory. The CPU is the main processor to complete the establishment, allocation, scheduling and release of tasks. The program written by the developer on the CPU is compiled by the compiler and translated into hardware instructions that can be parsed by GPGPU.
[0044] GPGPU, as a coprocessor, handles computationally intensive tasks that the CPU is not good at. A typical GPGPU consists of a PCIE control module, a thread block scheduling module, a PGPU configuration module, global memory, L2 Cache, constant parameter memory, and SM. The PCIE control module instantiates the PCIE interface, controls the communication between the host CPU and the GPGPU, and establishes a connection between the GPGPU memory and the host memory to achieve data transmission between the host memory and the GPGPU memory; the thread block scheduling module implements the scheduling and allocation of thread blocks, and allocates thread blocks to each SM; the GPGPU configuration module initializes the GPGPU and configures the control parameters of the GPGPU; the L2 Cache is a high-speed cache that pre-fetches data in the global memory by merging memory accesses to reduce pipeline stagnation caused by data memory accesses and improve computing efficiency.
[0045] GPGPU memory is divided into global memory and constant parameter memory. Global memory stores the data to be processed and the hardware control instructions compiled by the compiler. The constant parameter memory is used as read-only memory during the operation of GPGPU to store constant parameters during the operation.
[0046] SM (streaming multiprocessor) is the computing unit of GPGPU, which is composed of warp scheduling module, register file, L1 Cache, shared memory and pipeline. The warp scheduling module implements the scheduling and maintenance of warps, the register file is used to cache data and intermediate calculation results during the operation, the shared memory is used for data exchange between warps and threads, and the L1 Cache pre-fetches data and instructions from the L2 Cache to further reduce the memory access time of the pipeline.
[0047] The pipeline is the key to SM's computing function. It completes instruction fetch, decoding, emission, execution and result writeback through a five-stage pipeline. The efficiency of the pipeline directly determines the computing performance of GPGPU. Among them, the execution stage of the pipeline will assign several threads in a thread warp to each SP for execution.
[0048] SP (streaming processor) is the smallest computing unit of GPGPU, which completes ALU, SFU, FPU and other operations. The number of SPs determines the parallelism of thread execution in a warp. Generally speaking, when the number of SPs is the same as the number of threads in a warp, the parallelism is the highest, and all threads in the warp can be completely parallel during the execution phase.
[0049] In this embodiment, the instruction fetch stage of the pipeline is implemented based on the pipeline instruction cache described in Embodiment 1, and an L0 instruction cache is added here to provide register-level instruction reading for the pipeline by caching instructions from the L1 instruction cache.
[0050] Example 3
[0051] This embodiment discloses an instruction fetching method based on the GPGPU pipeline instruction cache device described in Embodiment 1. The L0 instruction cache implements non-blocking instruction fetching of the pipeline by caching instructions in the L1 instruction cache, thereby reducing the probability of pipeline stalling due to instruction fetching and improving pipeline efficiency and computing performance. The implementation process is as follows:
[0052] 1) After the computing task is started, the thread warp scheduling module sends the initial PC to the L0 instruction cache;
[0053] 2) The read request module of each cache unit of the L0 instruction cache detects whether the FIFO is not full. If the FIFO is not full, an L1 instruction cache read request is generated according to the initial PC;
[0054] 3) The arbiter receives read requests from each group of cache units, generates the first executed L1 instruction cache read request through arbitration, and queues the read requests of the remaining groups;
[0055] 4) After receiving the read request, the L1 instruction cache searches for the corresponding data in the cache according to the request address. If the cache hits, the corresponding data is output to the crossbar switch. If the cache does not hit, the request is forwarded to the L2 cache or global memory, waiting for the corresponding memory to return the data, and then output the data to the crossbar switch;
[0056] 5) After the arbiter determines that the L1 instruction cache processes the request, it queues the next request to the L1 instruction cache, and the remaining read requests continue to queue, waiting for the current request to be processed before the next request is queued, and the process repeats;
[0057] 6) After the crossbar switch receives the instruction returned by the L1 instruction cache, it determines the cache unit group to which the current instruction belongs based on the tag and writes it into the FIFO of the corresponding unit group;
[0058] 7) When the FIFO is not completely empty, a ready signal is sent to the thread warp scheduling module, and the thread warp scheduling module starts scheduling the thread warp to enter the pipeline for execution;
[0059] 8) When the pipeline executes the instruction fetch stage, instructions are dequeued from the FIFO of the corresponding cache unit group and then enter the decoding stage.
[0060] When the pipeline executes instructions, a branch may occur. At this time, the next instruction of the thread warp will no longer be continuous, causing the instructions cached in the FIFO to be invalid. At this time, the branch mechanism of the L0 instruction cache needs to be executed:
[0061] 1) When the branch jump instruction reaches the execution stage of the pipeline, the pipeline will generate a branch signal to mark the next instruction as a branch instruction and send it to the thread warp scheduling module;
[0062] 2) After the warp scheduling module detects the branch signal, it generates a clear signal and sends the clear signal, the warp wid, and the PC of the branch instruction to the L0 instruction cache;
[0063] 3) After the L0 instruction cache detects the clear signal, it clears the FIFO of the corresponding cache unit group according to the thread bundle wid, and generates an L1 instruction cache read request according to the PC of the branch instruction to re-cache the instruction;
[0064] 4) When the thread warp executes the next instruction, the instruction dequeued from the FIFO during the instruction fetch stage of the pipeline is the branch instruction, and the execution of the pipeline continues.
[0065] The English explanations that appear in the above process are:
[0066] ready signal: ready signal;
[0067] branch signal: branch control signal;
[0068] clear signal: cache reload signal;
[0069] Thread bundle wid: thread bundle identifier;
[0070] Cache: Cache.
[0071] The present invention adds an L0 instruction cache to the pipeline through FIFO, caches the instructions in the L1 instruction cache, realizes non-blocking assignment of the pipeline through pre-fetching, saves the time of instruction fetching of the pipeline, and the caching process of the L0 instruction cache is independent of the execution scheduling process of the pipeline, does not occupy the instruction execution time of the pipeline, reduces the stagnation probability of the pipeline, and improves the computing performance. At the same time, the L0 instruction cache based on FIFO occupies less resources, has flexible control and strong scalability, and can reasonably configure the FIFO size according to the number of active thread bundles and pipeline efficiency, thereby improving resource utilization.
[0072] The above description is only the basic principle and preferred embodiments of the present invention. Improvements and substitutions made by those skilled in the art based on the present invention belong to the protection scope of the present invention.
Claims
1. A GPGPU pipeline instruction cache device, characterized in that: It is an L0 instruction cache built by FIFO. The L0 instruction cache caches instructions from the L1 instruction cache and provides register-level instruction reading for the pipeline; it includes FIFO, a read control module, an arbitrator and a cross switch. The FIFO is used to cache instructions read from the L1 instruction cache. The read control module is connected to the FIFO and monitors the full flag signal of the FIFO. If the FIFO is not full, an L1 instruction cache read request is generated and sent to the arbitrator; the arbitrator is connected to the read control module, receives and arbitrates read requests sent by multiple read control modules to generate a read request queue; the arbitrator is connected to the L1 instruction cache and sends read requests in the read request queue to the L1 instruction cache. The cross switch is connected between the L1 instruction cache and the FIFO. The cross switch receives the instruction returned by the L1 instruction cache, and determines the FIFO to which the instruction belongs based on the tag flag, and then writes the returned instruction into the corresponding FIFO.
2. The GPGPU pipeline instruction cache device according to claim 1, characterized in that: The FIFO and the read control module constitute a hardware cache unit. The L0 instruction cache includes multiple groups of hardware cache units. Each group of hardware cache units is independent of each other and implemented in parallel.
3. The GPGPU pipeline instruction cache device according to claim 1, characterized in that: The L0 instruction cache prefetches instructions from the L1 instruction cache.
4. The GPGPU pipeline instruction cache device according to claim 1, characterized in that: The caching process of the L0 instruction cache is independent of the execution scheduling process of the pipeline.
5. The GPGPU pipeline instruction cache device according to claim 1, characterized in that: Configure the FIFO size and number based on the number of cached instructions, pipeline efficiency, and the number of active warps that can be scheduled.
6. The GPGPU pipeline instruction cache device according to claim 5, characterized in that: FIFO size is positively correlated with the number of cached instructions and negatively correlated with pipeline efficiency.
7. The GPGPU pipeline instruction cache device according to claim 5, characterized in that: The number of FIFOs is equal to the maximum number of scheduled thread warps supported by the SM unit, which is a streaming multiprocessor unit.
8. A GPGPU system, comprising an SM module, the SM module comprising a pipeline, the pipeline comprising five stages: instruction fetch, decoding, emission, execution and result write-back, characterized in that: The instruction fetch stage is implemented based on the GPGPU pipeline instruction cache device described in any one of claims 1-7.
9. A method for fetching a finger, characterized in that: The GPGPU pipeline instruction cache device according to any one of claims 1 to 7 is implemented, comprising the following steps: S01), after the computing task is started, the thread warp scheduling module sends the initial PC to the L0 instruction cache, where PC is the program counter; S02), the read control module of the L0 instruction cache detects whether the FIFO is not full. If the FIFO is not full, an L1 instruction cache read request is generated according to the initial PC; S03), the arbitrator receives and arbitrates read requests sent by multiple read control modules, generates a read request queue, and sends the first executed read request in the read request queue to the L1 instruction cache; S04), after receiving the read request, the L1 instruction cache searches for corresponding data in the cache according to the request address. If the cache hits, the corresponding data is output to the crossbar switch. If the cache does not hit, the request is forwarded to the L2 cache or global memory, waiting for the corresponding memory to return data, and then outputting the data to the crossbar switch; S05), after the arbiter determines that the L1 instruction cache processes the first executed read request in the read request queue, it dequeues the next read request to the L1 instruction cache, and the remaining read requests continue to queue, waiting for the current read request to be processed and then dequeuing the next read request, and repeating this process; S06), after receiving the instruction returned by the L1 instruction cache, the crossbar switch determines the FIFO to which the current instruction belongs according to the tag, and writes it into the corresponding FIFO; S07), when the FIFO is not completely empty, a ready signal is sent to the thread warp scheduling module, and the thread warp scheduling module starts scheduling the thread warp to enter the pipeline for execution; S08), when the pipeline executes the instruction fetch stage, the instruction is dequeued from the corresponding FIFO and then enters the decoding stage.
10. The method for fetching a finger according to claim 9, characterized in that: The pipeline branches when executing instructions, and executes the branch mechanism of the L0 instruction cache, including the following steps: S11), when the branch jump instruction reaches the execution stage of the pipeline, the pipeline will generate a branch signal marking the next instruction as a branch instruction and send it to the thread warp scheduling module; S12), after the thread warp scheduling module detects the branch signal, it generates a clear signal and sends the clear signal, the thread warp wid and the PC of the branch instruction to the L0 instruction cache; S13), after the L0 instruction cache detects the clear signal, the FIFO of the corresponding cache unit group is cleared according to the thread bundle wid, and an L1 instruction cache read request is generated according to the PC of the branch instruction to re-cache the instruction; S14), when the thread warp executes the next instruction, the instruction dequeued from the FIFO in the pipeline fetch stage is the branch instruction, and the execution of the pipeline continues.
