Method for improving cache access unit on assembly line in GPGPU (General Purpose Graphics Processing Unit)
By integrating Dcache into the memory access unit and managing data within the pipeline using specialized modules, the method addresses cross-level transmission and synchronization issues, improving GPGPU pipeline efficiency and resource utilization.
Patent Information
- Application Number
- CN202411325993.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-23
- Publication Date
- 2024-12-13
AI Technical Summary
Existing GPGPU architectures suffer from inefficiencies in memory access due to cross-level data transmission and synchronization issues, particularly affecting the execution efficiency of the pipeline due to operations like bursts and the location of Dcache and shared memory outside the pipeline.
Integrate Dcache into the memory access unit, utilize an input module, MSHR module, address calculation module, buffer, and pipeline response module to manage data within the pipeline, ensuring high-speed read and write operations without cross-level transmission, and handle misses by accessing higher-level caches when necessary.
Enhances pipeline efficiency by reducing address and synchronization problems, allowing uninterrupted data transmission and optimizing resource utilization within the pipeline.
Smart Images
Figure CN119127735A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of GPGPU technology, and in particular to a method for improving a cache access unit on a pipeline in a GPGPU. Background Art
[0002] With the development of contemporary science and technology, image recognition, artificial intelligence and other fields, people's demand for computing large amounts of data is increasing, and GPGPU is a technical means for processing big data computing. GPGPU is derived from GPU. It uses the high parallel computing capability of GPU to perform non-graphic tasks. Compared with GPU, GPGPU cancels the graphics computing unit and other parts of GPU that are specifically used for graphics processing, and instead focuses on parallel processing of big data.
[0003] In recent years, the architecture of GPGPU has been constantly updated with the GPU, but there are very few optimizations for the memory access part. The commonly used GPGPU memory access unit only records the instructions of the pipeline and passes the information of the data to be extracted to various modules, such as shared memory, L1 Cache, and L2 Cache, etc., but for the operation of the storage space address, only the offset processing (offset) is performed, that is, the original address is offset converted to meet the address allocation requirements of the storage space. However, in this process, the read and write data may be interrupted by operations such as bursts, which in turn affects the execution efficiency of the entire pipeline. At the same time, Dcache and shared memory are often located outside the entire pipeline, acting as independent storage and then acting as the storage module of the entire unit.
[0004] As attached Figure 1 As shown, after receiving the request from the core, the icache of the pipeline in the existing pipeline architecture diagram starts to process the data. The processed data passes through the decoding module, the buffer module and the operand collector module, and enters the emission module to be emitted to each functional unit for processing the corresponding function. Among them, the scoreboard, the thread number scheduling module and the SIMT stack module are used to better coordinate the resources of each thread in the pipeline to assist the entire pipeline to be more efficient. The write-back module collects the last operation information, performs secondary calculations on the unfinished resources, or waits for a higher-level cache to send the corresponding information back, so it is necessary to wait for another round of pipeline execution, and the corresponding resources need to be redistributed. In it, it can be seen that the shared storage units sharemem and Dache are both outside the pipeline. One of the major functions of the write-back module is to collect the previously processed data, and then re-send it through the emission module, and re-calculate it with the data extracted from the shared storage unit and Dcache. It can be seen that such a process is quite a waste of pipeline resources and execution cycles. Summary of the invention
[0005] In order to overcome the deficiencies of the above technologies, the present invention provides a method for improving the cache access unit on the pipeline in GPGPU to avoid cross-level transmission and reduce the address problem and the synchronization problem of read and write status caused by cross-level transmission.
[0006] The technical solution adopted by the present invention to overcome the technical problems is:
[0007] A method for improving a cache access unit on a pipeline in a GPGPU, wherein the cache access unit is composed of an input module, an MSHR module, an address calculation module, a Buffer, a Dcache, a related information write-back module, and a pipeline response module;
[0008] The method by which the cache access unit extracts and writes data within the pipeline is:
[0009] S1. The input module collects the requests assigned by the pipeline emission module;
[0010] S2. The MSHR module determines whether the request is a hit or a miss. If it is a hit, the operation is completed in the cache access unit. If it is a miss, the current read and write request is transmitted to a higher-level cache unit.
[0011] S3. The address calculation module and the buffer jointly complete the setting of high-speed read and write space;
[0012] S4. The address calculation module completes the setting of the address space offset value.
[0013] Furthermore, in step S1, the input module is composed of FIFO.
[0014] Furthermore, after the input module in step S1 completes the collection, it is typed out in sequence using a unified clock.
[0015] Furthermore, in step S2, if the address space required by the request is within the Dcache, it is defined as a hit; if the address space required by the request is within the sharemem or higher level cache, it is defined as a hit; if the request for the L2cache and higher level main memory is not within the Dcache and sharemem range, it is defined as a miss.
[0016] Furthermore, in step S2, when the current read / write request is transmitted to the high-level cache unit, the high-level cache unit is waited for to send the data, and after the data is sent, the missing records in the MSHR module are erased. Furthermore, in step S3, the address calculation module sets a high-speed read / write space, and pushes the data required for high-speed execution into the high-speed space in sequence, and the result of the data execution is temporarily stored in the Buffer; after the data execution is completed, the resources are released.
[0017] Further, step S3 includes the following steps:
[0018] S3-1. The high-speed read and write space set by the address calculation module is the same as the address space of Dcache;
[0019] S3-2. When the address calculation module receives a write instruction or a read instruction and the Hi gn Speed signal is pulled high, it checks whether there is a task being executed in the high-speed read and write space. If there is a task being executed, the cache current operation is suspended and the task is completed;
[0020] S3-3. When the busy signal is pulled low and the ready signal is pulled high, the state of the new task can be accepted, the current operation is put into the high-speed read-write space for operation, the counter is decremented according to the address space to be operated, and the address space to be operated is obtained by subtracting the minimum bit from the maximum bit of the address space to be operated. If the current operand is larger than the set address space, the buffer module is started to cache the previously calculated data;
[0021] S3-4. Every time the address space is changed, the counter is decremented by 1. Before the counter returns to zero, the buffer module is in the receiving state. If the counter returns to zero, the buffer changes to the read mode and outputs the corresponding data to the Dcache;
[0022] S3-5. The maximum bit of the address space to be operated is subtracted from the minimum bit to obtain the value of the counter recorded at the beginning, and the value of the counter recorded at the beginning and the initial address of the operation are recorded;
[0023] S3-6. When Dcache needs to release data, it restores the address of the corresponding data.
[0024] Further, step S4 includes the following steps:
[0025] S4-1. The shared data between multiple pipelines is placed in a high-level cache unit for data sharing. The H i gh Level signal of the shared data is pulled high to add a bias value to the address space in the pipeline;
[0026] S4-2. When the address range exceeds the current range, perform N cumulative subtraction operations.
[0027] Furthermore, if there is no hit in step S2, the read / write request is upgraded to a read / write request for a higher-level cache unit. After the read / write operation is completed, the higher-level cache unit is waited for to send the data back. When the pipeline is waiting to execute this instruction again in the next cycle, the data is sent back and returned to the pipeline through the relevant information write back module of the cache access unit.
[0028] Furthermore, when the High Level signal of the shared data is pulled high, the relevant information write back module of the cache access unit retains the current read and write operation instructions and feeds them back to the pipeline response module of the cache access unit. The pipeline response module repeats the last read and write tasks again, retrieves the data from the high-level cache unit and erases the missing mark in the MSHR module.
[0029] The beneficial effects of the present invention are: adding a high-speed read and write space, that is, specifying an address space, the data in the space is not allowed to be inserted, and it is only triggered by read and write instructions, which ensures that the data transmission process is not interrupted, and further ensures that the memory access unit extracts the required data at full speed, integrates Dcache into the memory access unit, and only retains the shared storage unit sharememory outside the pipeline for temporary storage of multiple pipelines and multi-level SM cores, and for storing inter-core data. The location of Dcahce in the pipeline can avoid cross-level transmission, reduce address problems and synchronization problems of read and write states caused by cross-level transmission. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 This is the existing pipeline architecture diagram;
[0031] Figure 2 is a flow chart of the present invention;
[0032] Figure 3 It is an execution flow chart of the address calculation module of the present invention. DETAILED DESCRIPTION
[0033] The following is combined with Figure 1 , Attachment Figure 2 , Attachment Figure 3 The preferred embodiments of the present invention are described in detail so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.
[0034] In the description of the present invention, it should be noted that the embodiments described in the present invention are only part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.
[0035] The terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.
[0036] As attached Figure 2 As shown, a method for improving a cache access unit on a pipeline in a GPGPU, the cache access unit is composed of an input module, an MSHR module, an address calculation module, a Buffer, a Dcache, a related information write-back module, and a pipeline response module;
[0037] The method by which the cache access unit extracts and writes data within the pipeline is:
[0038] S1. The input module collects the requests assigned by the pipeline emission module. The function of the input module is to ensure the consistency of timing and the stability of the signal.
[0039] The S2.MSHR module (full name is Miss Status Handling Register) determines whether the request is a hit or a miss. If it is a hit, the operation is completed in the cache access unit. If it is a miss, the current read and write request is transmitted to a higher-level cache unit.
[0040] S3. The address calculation module and the buffer jointly complete the setting of high-speed read and write space.
[0041] S4. The address calculation module completes the setting of the address space offset value.
[0042] Regarding the functional expansion and optimization of the address calculation module, first of all, the module has the function of initialization, that is, to zero and reset the relevant offset address, and the module can set the preset high-speed channel. After the setting is completed, the address space only receives the first read and write request instruction, and can accept new instructions only after completing this instruction, that is, during the execution of the current instruction, bursts and changes in operation requests are not allowed. At the beginning of the read and write request, it is first necessary to determine whether the current address space hits the address space of the current pipeline. If it hits, it means that the data is stored in the Dcache of the current pipeline. If it does not hit, it means that the data is stored in a higher-level cache unit. If it hits, the Dcache is read and written directly. If it does not hit, the request will be upgraded to a higher-level cache for reading and writing, and the missing information will be recorded in the mshr. After the data reaches the pipeline, the mshr will erase the corresponding missing information. Finally, with respect to the optimization of Dcache, the present invention integrates Dcache into the memory access unit. Therefore, when the memory access unit extracts and writes data within the pipeline, it does not need to process requests across layers, which greatly improves the utilization efficiency of this unit.
[0043] In one embodiment of the present invention, the input module in step S1 is composed of FIFO. After the input module completes the collection, it prints out in sequence according to a unified clock.
[0044] In one embodiment of the present invention, in step S2, if the address space required by the request is within Dcache, it is defined as a hit; if the address space required by the request is within sharemem or a higher level cache, it is defined as a hit; if the request for L2 cache and a higher level main memory is not within the Dcache and sharemem interval, it is defined as a miss. For a hit request, the operation can be completed in the memory access unit; if it is a miss request, the current read and write request needs to be upgraded and transmitted to a higher level cache unit. For a higher level cache unit, however, at this time, it needs to be recorded in the MSHR module that the current data has not yet been installed, and it is necessary to wait for the higher level cache unit to send the data. After the data is sent, the missing record in the MSHR module is erased to complete the access process. The content recorded in the MSHR module refers to the address of the missing data. The module will wait for the confirmation (va lid) signal from the higher level cache to complete the erase operation.
[0045] In one embodiment of the present invention, when the current read and write request is transmitted to the higher-level cache unit in step S2, the higher-level cache unit is waited to send data down, and after the data is sent down, the missing records in the MSHR module are erased.
[0046] In one embodiment of the present invention, the address calculation module in step S3 sets a high-speed read-write space, which only accepts the first instruction operation and does not accept burst instructions and insertion instructions on the pipeline. The current space can be released only after the current task is completed. However, it should be noted that if the address space required for execution is insufficient, it will be used in conjunction with the buffer module to push the data required for high-speed execution into the high-speed space in sequence, and the results of the data execution are temporarily stored in the Buffer; after the data execution is completed, the resources are released.
[0047] As attached Figure 3 As shown, in one embodiment of the present invention, step S3 includes the following steps:
[0048] S3-1. Assuming that the high-speed execution space is 0000_0000~1FFF_FFFF, the high-speed read and write space set by the address calculation module is the same as the address space of Dcache, which can ensure that each read and write to Dcache is efficient.
[0049] S3-2. When the address calculation module receives a write instruction or a read instruction and the Hi gn Speed signal is pulled high, it checks whether there is a task being executed in the high-speed read / write space (i.e. whether the busy signal is in a pulled high state). If there is a task being executed, the current cache operation is paused and the task is completed. The High Speed signal refers to the application for the use of the high-speed channel mode.
[0050] S3-3. When the busy signal is pulled low and the ready signal is pulled high, the state of the new task can be accepted, and the current operation is placed in the high-speed read-write space for operation. The counter is decremented according to the address space that needs to be operated. The set address space is obtained by subtracting the minimum bit from the maximum bit of the address space that needs to be operated. If the current operand is larger than the set address space, the buffer module is started to cache the data calculated in advance.
[0051] S3-4. Every time the address space is changed, the counter is decremented by 1. Before the counter returns to zero, the buffer module is in the receiving (writing) state. If the counter returns to zero, the buffer changes to the read mode and outputs the corresponding data to the Dcache, completing the entire high-speed channel process.
[0052] S3-5. But there is another part that needs to be solved. Although the address part enters the high-speed space in sequence by pushing it into the stack, for the address that exceeds the maximum address space, the maximum bit of the address space to be operated is subtracted from the minimum bit to get the value of the counter recorded at the beginning (it can be subtracted repeatedly, so the larger the setting range of the high-speed channel, the higher the execution efficiency, but it will also take up a lot of address space. This space does not support burst and insertion requests, so it is necessary to manually set the address space of the high-speed channel) to ensure that the address always meets the requirements, and record the value of the counter recorded at the beginning (the maximum bit of the address space to be operated minus the minimum bit) and the initial address of the operation.
[0053] S3-6. When Dcache needs to release data, it restores the address of the corresponding data.
[0054] In one embodiment of the present invention, step S4 includes the following steps:
[0055] S4-1. The shared data between multiple pipelines are placed in a high-level cache unit for data sharing. The High Level signal of the shared data is pulled high, and the address space will be special (pre-set address space of a higher-level cache, assuming that the address range of the shared storage unit is 8FFF_FFFF~9FFF_FFFF), but this address space is not reachable in the pipeline, so it is necessary to add a certain value to the address space in the pipeline to meet the address range requirements of the shared unit. The process of adding the value is offset. Assuming that the current address space value is 3FFF_FFFF~4100_0000, it is necessary to use 9FFF_FFFF minus 4100_0000 to get the offset value, and then perform the offset operation on the original address.
[0056] S4-2. When the address range exceeds the current range, N cumulative subtraction operations are performed. For the reading and writing of the advanced cache, there is also an internal high-speed channel. The range of the high-speed channel is consistent with the address space set for the advanced cache. The implementation of the address counter is not described in detail.
[0057] In one embodiment of the present invention, if a read / write request does not hit in step S2, the read / write request is upgraded to a read / write request for a higher-level cache unit. After the read / write operation is completed, the higher-level cache unit is waited for to send the data back. When the pipeline is waiting to execute this instruction again in the next cycle, the data is sent back and returned to the pipeline through the relevant information write back module of the cache access unit.
[0058] In one embodiment of the present invention, when the High Level signal of the shared data is pulled high, the relevant information write-back module of the cache access unit retains the current read and write operation instructions and feeds them back to the pipeline response module of the cache access unit. The pipeline response module repeats the last read and write task again, takes out the data from the high-level cache unit and erases the missing mark in the MSHR module. At this time, the relevant information write-back module pulls the High Level signal low to end the current read and write operation task. For the pipeline response module, its function is to receive the instructions of the transmission module and send them to the input module (FIFO). At the same time, for the feedback signal of the write-back module, it will retain the current operation for the HL signal and stop retaining the last operation for the high-level cache after waiting for the execution to be completed.
[0059] It should also be understood that in the embodiments of this article, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0060] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this article.
[0061] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0062] In the several embodiments provided herein, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.
[0063] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this article.
[0064] In addition, each functional unit in each embodiment of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional unit.
[0065] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-On l yMemory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0066] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for improving a cache access unit on a pipeline in a GPGPU, wherein the cache access unit is composed of an input module, an MSHR module, an address calculation module, a Buffer, a Dcache, a related information write-back module, and a pipeline response module. Features: The method by which the cache access unit extracts and writes data within the pipeline is: S1. The input module collects the requests assigned by the pipeline emission module; S2. The MSHR module determines whether the request is a hit or a miss. If it is a hit, the operation is completed in the cache access unit. If it is a miss, the current read and write request is transmitted to a higher-level cache unit. S3. The address calculation module and the buffer jointly complete the setting of high-speed read and write space; S4. The address calculation module completes the setting of the address space offset value.
2. The method for improving the cache access unit on the pipeline in GPGPU according to claim 1, Features: In step S1, the input module is composed of FIFO.
3. The method for improving the cache access unit on the pipeline in GPGPU according to claim 1, Features: After the input module in step S1 completes the collection, it is typed out in sequence using a unified clock.
4. The method for improving the cache access unit on the pipeline in GPGPU according to claim 1, Features: In step S2, if the address space required by the request is within the Dcache, it is defined as a hit; if the address space required by the request is in sharemem or a higher level cache, it is defined as a hit; if the request for L2 cache and a higher level main memory is not within the Dcache and sharemem range, it is defined as a miss.
5. The method for improving the cache access unit on the pipeline in GPGPU according to claim 1, Features: When the current read and write request is transmitted to the higher-level cache unit in step S2, the higher-level cache unit is waited for to send the data down, and after the data is sent down, the missing records in the MSHR module are erased.
6. The method for improving the cache access unit on the pipeline in GPGPU according to claim 1, Features: In step S3, the address calculation module sets a high-speed read-write space, pushes the data required for high-speed execution into the high-speed space in sequence, and temporarily stores the result of the data execution in the buffer; After the data is executed, the resources are released.
7. The method for improving the cache access unit on the pipeline in GPGPU according to claim 6, It is characterized in that Step S3 includes the following steps: S3-1. The high-speed read and write space set by the address calculation module is the same as the address space of Dcache; S3-2. When the address calculation module receives a write instruction or a read instruction and the Hign Speed signal is pulled high, it checks whether there is a task being executed in the high-speed read / write space. If there is a task being executed, the current cache operation is paused and the task is waited for completion; S3-3. When the busy signal is pulled low and the ready signal is pulled high, the state of the new task can be accepted, the current operation is put into the high-speed read-write space for operation, the counter is decremented according to the address space to be operated, and the address space to be operated is obtained by subtracting the minimum bit from the maximum bit of the address space to be operated. If the current operand is larger than the set address space, the buffer module is started to cache the previously calculated data; S3-4. Every time the address space is changed, the counter is decremented by 1. Before the counter returns to zero, the buffer module is in the receiving state. If the counter returns to zero, the buffer changes to the read mode and outputs the corresponding data to the Dcache; S3-5. The maximum bit of the address space to be operated is subtracted from the minimum bit to obtain the value of the counter recorded at the beginning, and the value of the counter recorded at the beginning and the initial address of the operation are recorded; S3-6. When Dcache needs to release data, it restores the address of the corresponding data.
8. The method for improving the cache access unit on the pipeline in GPGPU according to claim 1, It is characterized in that Step S4 includes the following steps: S4-1. The shared data between multiple pipelines is placed in a high-level cache unit for data sharing. The High Level signal of the shared data is pulled high, and an offset value is added to the address space in the pipeline; S4-2. When the address range exceeds the current range, perform N cumulative subtraction operations.
9. The method for improving the cache access unit on the pipeline in GPGPU according to claim 1, Features: If there is no hit in step S2, the read / write request is upgraded to a read / write request for a higher-level cache unit. After the read / write operation is completed, the higher-level cache unit is waited for to send the data back. When the pipeline is waiting to execute this instruction again in the next cycle, the data is sent back and returned to the pipeline through the relevant information write back module of the cache access unit.
10. The method for improving the cache access unit on the pipeline in GPGPU according to claim 8, Features: When the High Level signal of the shared data is pulled high, the relevant information write-back module of the cache access unit retains the current read and write operation instructions and feeds them back to the pipeline response module of the cache access unit. The pipeline response module repeats the last read and write tasks again, takes out the data from the high-level cache unit and erases the missing mark in the MSHR module.