A multi-granularity cache management system for GPGPU

Abstract

The present invention discloses a multi-granularity cache management system applied to GPGPU, which belongs to the field of cache management technology. The technical problem to be solved is how to dynamically adjust the cache granularity according to the memory access mode and improve the performance of GPGPU in processing irregular memory access mode applications. It includes: a label decoding module, which is used to generate a matching label for the address information specified in the original label; a label management module, which is used to adjust the block label based on the current cache granularity and the data size specified in the original label to obtain the processed block label; a hit judgment module, which is used to search for the corresponding block label from the block label array based on the data address, and compare and match the searched block label with the processed block label; a hit recording module, which is used to judge whether the current granularity meets the requirements based on the hit information; and a granularity controller, which is used to adjust the current cache granularity.

Description

Technical Field

[0001] The present invention relates to the technical field of cache management, in particular to a multi-granularity cache management system applied to GPGPU. Background Art

[0002] In the existing technology, the existing GPGPU cache management system mainly adopts fixed-granularity cache lines, which performs well when processing traditional applications with regular data structures and continuous memory access patterns. However, as computing tasks become more diverse and complex, the existing technology shows obvious limitations when processing applications with irregular and non-continuous memory access patterns. Fixed-granularity cache lines cannot effectively predict and adapt to non-continuous memory access patterns, resulting in a large number of memory requests failing to find the required data in the cache, increasing the frequency of access to the main memory and thus increasing the cache miss rate.

[0003] In addition, due to irregular memory access patterns, data in cache lines are often replaced before they are fully utilized, resulting in a waste of cache space and frequent cache consistency maintenance operations. This not only leads to a decrease in GPGPU processing speed, especially when performing computationally intensive tasks with high parallelism but irregular memory access, but also makes memory bandwidth a bottleneck, increasing energy consumption because more data transfer requires more power.

[0004] At the same time, the fixed-granularity cache management system cannot be dynamically adjusted according to the specific access patterns of the application, cannot flexibly adapt to changing computing needs, and lacks flexibility. This limits GPGPU in processing emerging applications such as deep learning and graphics computing, where memory access patterns may change at any time, resulting in suboptimal performance. Fixed-size cache lines are usually used to organize cache data. This design is effective when processing regular memory access patterns because it can reduce access latency and improve bandwidth utilization by merging memory requests. However, for irregular memory access patterns, this fixed-granularity cache management will cause cache lines to contain a large amount of unaccessed data, thereby increasing cache miss rates and wasting memory bandwidth.

[0005] How to dynamically adjust cache granularity according to memory access patterns and improve GPGPU performance in processing applications with irregular memory access patterns is a technical problem that needs to be solved. Summary of the invention

[0006] The technical task of the present invention is to address the above shortcomings and provide a multi-granularity cache management system applied to GPGPU to solve the technical problem of how to dynamically adjust the cache granularity according to the memory access pattern and improve the performance of GPGPU in processing applications with irregular memory access patterns.

[0007] In a first aspect, the present invention provides a multi-granularity cache management system applied to GPGPU, characterized in that it is applied between a memory and a cache for partitioning and storing data in the form of data blocks, and comprises a label decoding module, a label management module, a hit judgment module, a hit recording module, and a granularity controller;

[0008] The tag decoding module is used to generate a matching tag for the address information specified in the original tag, wherein the original tag is information sent to the tag decoding module when the Load / Store unit receives a memory access request from the application program, the address information is the address of the data specified in the memory access request in the memory, and the matching tag includes a set tag for identifying a data set where the data in the cache is located, a row tag for identifying a cache line where the data in the data set is located, a block tag for identifying a data block where the data in the cache line is located, and a byte index for identifying the position of a specific byte in the cache line;

[0009] The tag management module is used to receive the block tag sent by the tag decoding module, the current cache granularity sent by the granularity controller, and the original tag sent by the Load / Store unit, and adjust the block tag based on the current cache granularity and the specified data size in the original tag to obtain a processed block tag;

[0010] The hit judgment module maintains a block tag array corresponding to all data block addresses in the cache, and is used to receive the processed block tag sent by the tag management module and the row tag, set tag and byte index sent by the tag decoding module, determine the data address based on the row tag, set tag and byte index, search for the corresponding block tag from the block tag array based on the data address, and compare and match the searched block tag with the processed block tag to obtain hit information;

[0011] The hit recording module is used to receive the hit information sent by the hit judgment module, and judge whether the current granularity meets the requirements based on the hit information. If not, it is used to send a granularity non-compliance signal to the granularity controller;

[0012] The granularity controller is used to receive a granularity non-conformance signal and an original tag sent by a Load / Store unit, and adjust the current cache granularity based on the current cache granularity and a data size specified in the original tag.

[0013] Preferably, the data in the memory is divided into a plurality of data blocks of fixed size, and the data blocks are mapped to the cache based on a predefined mapping rule;

[0014] The cache is divided into multiple data sets, each data set contains multiple cache lines, and each cache line is divided into multiple data blocks;

[0015] Correspondingly, the label decoding module is used to decode and obtain a matching label corresponding to the address information in the original label based on the data bits corresponding to the original label.

[0016] Preferably, the tag management module is used to perform the following to adjust the block tag based on the current cache granularity and the data size specified in the original tag:

[0017] Selecting a granularity mode based on the block tag, the current cache granularity, and the data size specified in the original tag;

[0018] Based on the selected granularity mode, the data block size and the data size specified in the original label, the block label is assigned a value, and the block label is assigned a value directly, assigned a value of +n, or not assigned a value.

[0019] When assigning a value to a block tag, the value is assigned to the block tag in the block tag array corresponding to the data block address.

[0020] Preferably, if the block tag found is the same as the processed block tag, it is determined as a hit, otherwise it is determined as a miss;

[0021] When it is judged as a hit, if the memory access request is for data reading, the address corresponding to the hit block tag is sent to the cache storage for value retrieval; if the memory access request is for data writing, check whether the data at the address corresponding to the hit block tag is consistent with the data specified in the memory access request. If they are consistent, the data write operation is not performed. If they are inconsistent, the hit block tag is written into the block tag array and the data is written into the cache storage.

[0022] As a preferred method, the calculation formula of the hit rate is:

[0023] Hit rate = number of hits / (number of hits + number of misses).

[0024] Preferably, the granularity controller is used to detect the frequency of data size changes, and adjust the cache granularity based on the frequency of data size changes and the current cache granularity. If the data change frequency is higher than a threshold and the current data size is smaller than the current cache granularity, the cache line is divided into smaller data blocks. If the data change frequency is lower than a threshold and the current data size is larger than the current cache granularity, the cache line is expanded by merging data blocks.

[0025] Preferably, the system further comprises a conflict detection module, the conflict detection module being used to identify and synchronize concurrent memory access requests to the cache line, and lock the data block being operated in the cache line through a lock mechanism;

[0026] When multiple concurrent memory access requests write data to the same cache line at the same time, a write lock is added to the data block being written by the current thread's memory access request. After the current thread's memory access request is executed, the write lock is released, allowing the next thread's memory access request to write to the data block.

[0027] When multiple concurrent memory access requests perform data read operations on the same cache line at the same time, a shared lock is added to the corresponding data block, allowing all thread concurrent memory requests to perform read operations on the data block at the same time.

[0028] Preferably, the conflict detection module is configured with a request queue and a first-in-first-out scheduling strategy, and memory access requests for the same cache line are queued through the request queue, and concurrent access to the cache line is managed through the first-in-first-out scheduling strategy.

[0029] Preferably, the conflict detection module manages data through a dirty data management mechanism of the cache; dirty data is the state of data when it is updated in the cache but not written to the cache. The data updated to the cache is marked as dirty data through the dirty data management mechanism, and the data marked as dirty data is written to the cache during the non-access phase of the cache, and the block label and status information corresponding to the data are updated.

[0030] The multi-granularity cache management system applied to GPGPU of the present invention has the following advantages:

[0031] 1. Allow the system to dynamically select the most appropriate cache granularity based on the specific needs of the application, thereby optimizing cache utilization efficiency and data access speed. Whether facing large-scale data access requiring high throughput or processing a large number of small but irregular memory requests, multi-granularity cache can provide effective support;

[0032] 2. Combined with dynamic granularity adjustment, the system can respond to changes in the memory access pattern of applications in real time and automatically optimize the granularity. This adaptive capability means that the cache system is not only efficient at startup, but also always maintains optimal performance during the execution of applications. For example, for applications that initially use coarse-grained access and then switch to fine-grained access, dynamic granularity adjustment can reduce the cache granularity in a timely manner to reduce cache miss rates and improve data access speed;

[0033] 3. The combination of multi-granularity cache architecture and dynamic granularity adjustment can significantly improve the energy efficiency of the system. By reducing unnecessary data migration and lowering cache miss rates, the system can complete more computing work with less energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] The present invention is further described below in conjunction with the accompanying drawings.

[0036] Figure 1 A schematic diagram of the structure of a multi-granularity cache management system applied to GPGPU according to an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of a data partitioning technology in a multi-granularity cache management system applied to GPGPU according to an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of a matching tag format in a multi-granularity cache management system applied to GPGPU according to an embodiment of the present invention;

[0039] Figure 4 A schematic diagram of a label control module in a multi-granularity cache management system applied to GPGPU according to an embodiment of the present invention;

[0040] Figure 5 This is a diagram showing an example of a multi-granularity cache architecture within a single thread in a multi-granularity cache management system applied to a GPGPU according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it. However, the embodiments are not intended to limit the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments may be combined with each other.

[0042] The embodiment of the present invention provides a multi-granularity cache management system applied to GPGPU, which is used to solve the technical problem of how to dynamically adjust the cache granularity according to the memory access mode and improve the performance of GPGPU in processing applications with irregular memory access patterns.

[0043] Embodiment: The present invention is a multi-granularity cache management system applied to GPGPU, which is applied between memory and cache for partitioning and storing data in the form of data blocks, and includes a label decoding module, a label management module, a hit judgment module, a hit recording module and a granularity controller.

[0044] In this embodiment, the memory and cache use data partitioning technology to divide the data in the memory into data blocks of fixed size, and the data blocks are mapped to the cache according to a predetermined mapping rule, such as Figure 2 As shown in the figure, the cache is divided into several data sets (4 data sets in the figure), each data set contains several cache lines (4 lines in the figure), and each cache line is further divided into several data blocks (4 blocks in the figure). This partitioning technology can be used to support multi-granularity cache management and allow the size of the data block in the cache line to be dynamically adjusted. Calculated as 32 bytes per data block, the attached Figure 2 The illustrated cache line size is 4*32=128 bytes, the size of a single data set is 128*4=512 bytes, and the total data size is 4*512=2048 bytes=2KB.

[0045] The tag decoding module is used to generate a matching tag for the address information specified in the original tag, wherein the original tag is information sent to the tag decoding module when the Load / Store unit receives a memory access request from an application, the address information is the address of the data specified in the memory access request in the memory, and the matching tag includes a set tag for identifying the data set where the data in the cache is located, a row tag for identifying the cache line where the data in the data set is located, a block tag for identifying the data block where the data in the cache line is located, and a byte index for identifying the position of a specific byte in the cache line.

[0046] In this embodiment, the tag decoding module is used to decode the data bits corresponding to the original tag (such as Figure 3 As shown), decode to obtain the matching tag corresponding to the address information in the original tag.

[0047] In this embodiment, the tag decoding module is used to generate matching tags. The matching tags will be sent to the tag management module for further modification, and then sent to the hit judgment module to determine whether the address is hit. First, when the Load / Store unit receives a memory access request, it will send an initial tag containing address information to the cache management system. These initial tags carry the key information required for memory access and are used in the subsequent hit judgment process. After the tag decoding module receives the tag, it will generate the corresponding matching tag according to the current cache granularity setting. The specific tag format is shown in Figure 3 shown.

[0048] The tag management module is used to receive the block tag sent by the tag decoding module, the current cache granularity sent by the granularity controller and the original tag sent by the Load / Store unit, and adjust the block tag based on the current cache granularity and the specified data size in the original tag to obtain a processed block tag.

[0049] In this embodiment, the tag management module is used to perform the following operations to adjust the block tag based on the current cache granularity and the data size specified in the original tag:

[0050] (1) Selecting a granularity mode based on the block tag, the current cache granularity, and the data size specified in the original tag;

[0051] (2) Based on the selected granularity mode, the data block size, and the data size specified in the original label, the block label is assigned a value. The methods of assigning a value to the block label include direct assignment, +n assignment, and no assignment.

[0052] When assigning a value to a block tag, the value is assigned to the block tag in the block tag array corresponding to the data block address.

[0053] As a specific implementation of the tag control module, after receiving the block tag sent from the tag decoding module, it selects the mode according to the data size in the original tag and the granularity sent by the granularity controller, such as Figure 4 As shown, in this embodiment, there are three cache granularity modes: 32 bytes, 64 bytes, and 128 bytes. Since 4 data blocks are 128 bytes, one data block is 32 bytes (it can be adjusted according to actual conditions, for the convenience of explanation here). When the granularity is 32 bits, each group of data blocks is 1 granularity. When the granularity is 64 bytes, two data blocks are regarded as 1 granularity. When the granularity is 128 bytes, a whole row of cache lines is regarded as 1 granularity. After selecting according to the granularity, it will be further selected according to the data size. It is worth noting that if the granularity is larger than the data size, although it will cause space waste, the control and subsequent matching of the label can still be performed, which increases the stability under irregular memory access mode. After judging according to the granularity and data size, the block label adjustment stage is entered. At this time, the decoded block label will be directly assigned / +n assigned / not assigned according to the situation.

[0054] When the granularity is 32 bytes, if the data is also 32 bytes, it means that it is stored in a data block of the cache line. Then the block tag is directly aligned and assigned to obtain 4 groups of identical block tags for subsequent matching; if the data is 64 bytes, the data needs to be assigned to two consecutive data blocks at this time, so the block tag also needs to be assigned once every two to ensure that they can be correctly identified and matched. Therefore, the block tag of the latter block is +1, and the previous block is directly assigned, and the assignment is performed twice to ensure the accuracy of the match; similarly, if the data is 128 bytes, it needs to occupy 4 consecutive data blocks. Therefore, the block tag is increased by 1 in sequence when assigned to ensure its continuity. Generally speaking, a complete cache line should be matched at this time.

[0055] When the granularity is 64 bytes, no matter the data is 32 bytes or 64 bytes, it is matched with two consecutive data blocks. Although this causes the data blocks to be wasted in the cache line, it provides backward compatibility and simplifies the complexity of granularity adjustment. When the data is 128 bytes, it is equivalent to occupying two "large data blocks composed of two data blocks", and their tags should be in a +2 relationship, similar to the 32-byte granularity. At this time, it should be necessary to match a complete cache line.

[0056] When the granularity is 128 bytes, no further classification is required. Regardless of the size of the data, it is placed in a cache line, and the block tag is assigned only to the first data block.

[0057] In this embodiment, all the above operations are operations on the block tag. "Assigning / assigning to the data block" means assigning / assigning it to the tag array address corresponding to the data block address. The subsequent hit judgment module also matches the block tag after the operation with the block tag array in the module to determine whether it is a hit. If the starting address of the data is not aligned with the data block, it may start in the middle of a block. In this case, the remaining part of the first block and the entire second block need to be specially processed.

[0058] The hit judgment module maintains a block tag array corresponding to all data block addresses in the cache, which is used to receive the processed block tag sent by the tag management module and the row tag, set tag and byte index sent by the tag decoding module, determine the data address based on the row tag, set tag and byte index, search for the corresponding block tag from the block tag array based on the data address, and compare and match the searched block tag with the processed block tag to obtain hit information.

[0059] In this embodiment, if the block tag found is the same as the processed block tag, the hit determination module is used to determine it as a hit, otherwise it is determined as a miss.

[0060] In the cache management process, when it is judged as a hit, if the memory access request is for data reading, the address corresponding to the hit block tag is sent to the cache storage for value retrieval; if the memory access request is for data writing, check whether the data at the address corresponding to the hit block tag is consistent with the data specified in the memory access request. If they are consistent, the data write operation is not performed. If they are consistent, the hit block tag is written into the block tag array, and the data is written into the cache storage. If the block tag does not match, it is considered a miss, and the data is read and written to the next level cache or storage. At the same time, regardless of whether it is a hit or not, the hit judgment module will send the hit information to the hit recording module for counting the hits and deciding whether the granularity needs to be changed.

[0061] In this embodiment, the next level in "if a miss, go to the next level cache or storage" refers to searching for data in the next slower and larger cache level, from the zero-level cache (L0 Cache) to the third-level cache (L3 Cache) in order, and is not a physical up-down relationship. "Storage" refers to GPGPU external storage.

[0062] The hit recording module is used to receive the hit information sent by the hit judgment module, and judge whether the current granularity meets the requirements based on the hit information. If it does not meet the requirements, it is used to send a granularity non-compliance signal to the granularity controller.

[0063] The calculation formula of hit rate is:

[0064] Hit rate = number of hits / (number of hits + number of misses).

[0065] As a specific implementation of the hit recording module, the hit recording module counts the hit information emitted by the hit judgment module. If the current hit rate is lower than the predicted threshold (such as 40%), it is considered that the current granularity does not meet the requirements. At this time, the hit recording module transmits a granularity non-conformity signal to the granularity controller. After receiving the signal, the controller compares the data size with the current granularity.

[0066] The granularity controller is used to receive a granularity non-conformance signal and an original tag sent by a Load / Store unit, and adjust the current cache granularity based on the current cache granularity and a data size specified in the original tag.

[0067] In this embodiment, the granularity controller is used to detect the frequency of data size changes, and adjust the cache granularity based on the frequency of data size changes and the current cache granularity. If the data change frequency is higher than the threshold and the current data size is smaller than the current cache granularity, the cache line is divided into smaller data blocks. If the data change frequency is lower than the threshold and the current data size is smaller than the current cache granularity, the cache line is expanded by merging data blocks.

[0068] When the granularity controller detects that the data size changes frequently (for example, the data size changes greatly from 32B to 128B multiple times in the memory access task) and the current data size is smaller than the current granularity, it means that the memory access is mainly fine-grained and irregular, and the controller will dynamically divide the cache line into smaller blocks to reduce the cache miss rate and improve the utilization of the cache space. For example, if the application performs many small-scale data accesses, the system may decide to further subdivide each 128B cache line into four 32B blocks, allowing access requirements to be met more accurately and reducing unnecessary data loading. In this way, high-performance cache operations are ensured even when the granularity changes.

[0069] On the contrary, if the granularity controller detects that the data size does not change frequently (for example, the data size does not change or only changes a few times throughout the memory access task) and the current data size is larger than the current granularity, it can be considered that the memory access pattern is coarse-grained and regular at this time, and the controller will decide to merge small blocks to form a larger cache line, thereby reducing the overhead of address translation and improving the cache line fill rate.

[0070] This dynamic adjustment not only optimizes the efficiency of the cache, but also reduces the performance loss caused by cache granularity mismatch. In this way, the system can adapt to a variety of different workloads, whether compute-intensive or memory-intensive, and provide the optimal cache granularity to maximize the computing performance of GPGPU.

[0071] This embodiment allows the GPU to adjust the granularity of the cache in real time according to the memory access pattern of the application. This flexibility is achieved by monitoring the size and frequency of memory access requests. The cache management system analyzes the access pattern to determine whether a coarser granularity (e.g., 128B) should be used to optimize continuous access to large blocks of data, or whether a finer granularity (e.g., 32B or 64B) should be switched to improve the efficiency of access to scattered data. The dynamic granularity adjustment mechanism is implemented through a hit record module and a granularity controller.

[0072] As an improvement of this embodiment, the cache management system further includes a conflict detection module, which is used to identify and synchronize concurrent memory access requests to the cache line, and lock the data block being operated in the cache line through a lock mechanism.

[0073] When multiple concurrent memory access requests write data to the same cache line at the same time, a write lock is added to the data block being written by the current thread memory access request. After the current thread memory access request is executed, the write lock is released, allowing the next thread memory access request to write to the data block; when multiple concurrent memory access requests read data from the same cache line at the same time, a shared lock is added to the corresponding data block, allowing all thread concurrent memory requests to perform read operations on the data block at the same time.

[0074] The conflict detection module is configured with a request queue and a first-in-first-out scheduling strategy. The memory access requests of the same cache line are queued through the request queue, and the concurrent access to the cache line is managed through the first-in-first-out scheduling strategy.

[0075] The conflict detection module manages data through the dirty data management mechanism of the cache. Dirty data refers to the state of data when it is updated in the cache but not written to the cache. The dirty data management mechanism marks the data updated to the cache as dirty data, writes the data marked as dirty data to the cache during the non-access phase of the cache, and updates the block label and status information corresponding to the data.

[0076] Due to the parallel characteristics of GPGPU, when multiple threads perform memory access operations simultaneously, it is possible that multiple threads access a cache line at the same time, or even access the same data block at the same time. The conflict detection module is added to solve this problem. The core function of this module is to identify and synchronize concurrent access requests to cache lines to ensure data consistency and avoid data competition. By implementing a lock mechanism, the data block being operated is locked. The read operation may use a shared lock to allow multiple threads to read at the same time, but the write operation requires an exclusive lock to ensure that only one thread can write the data to ensure data accuracy. The module adds request queuing and first-in-first-out scheduling strategies to queue access requests to the same cache line and access them in sequence according to the first-in-first-out method to manage concurrent access to the cache line. In addition, the conflict detection module uses the dirty data management mechanism of the cache. Dirty data refers to the state of data after it has been updated in the cache but has not yet been written to the cache. Through the dirty data management mechanism, the data is recorded as dirty data and updated in the non-memory access phase. At the same time, the corresponding block tags and status information are updated to ensure that the data can be identified and written back in a timely and correct manner, keeping the cache data synchronized among all processors.

[0077] For the system of this embodiment, Figure 5 As shown, the following specific example is given: Assume that the address emitted by the Load / Store unit is 0x10200000, according to the attached Figure 3 The tag decoding module decodes it into set tag (0x02), row tag (0x01), block tag (0x0000), and byte index (0x00). The block tag will be sent to the tag management module. Assuming that the current cache granularity is 64 bytes, the data size is 128 bytes, and each data block is 32 bytes, the tag management module will Figure 4 The block tag processing method shown processes the block tag and finally obtains the processed block tag (0x0000, -, 0x0002, -).

[0078] The processed block tag is sent to the hit judgment module. In this module, according to the set tag (0x02), the module can lock the data in data set 2 (corresponding to data set 0 when the set tag is 0), and according to the row tag (0x01), lock cache line 1 (similarly, when the row tag is 0, it corresponds to cache line 0). After determining the location of the data, the hit matching module will compare and match the block tag at that position in the tag array with the processed block tag. Since the byte index is 0, the hit matching will start from the end. At the same time, the unassigned block tag (the block tag with the mark bit "-") does not participate in the match. It can be found that the same block tag means a successful match, and the system will read the data at that position.

[0079] Subsequently, the hit matching module will transmit the hit information back to the hit recording module to determine the hit rate and perform a series of subsequent access pattern analyses, and ultimately determine whether to adjust the granularity. This embodiment has been explained here and will not be repeated in detail.

[0080] The above example is a multi-granularity cache dynamic management system for a single thread. In GPGPU, due to the parallelism of multiple threads, a write lock needs to be added to prevent access conflicts among multiple threads. When multiple threads request to write to cache line 1 of data set 2, only the first thread to access can continue to write until the write is completed and the write lock is released, and then the second thread is allowed to continue the operation. If multiple threads request to read the same cache line data, since each thread has a hit judgment module that maintains a thread-unique tag array, the block tags of all threads will be processed and matched in parallel, making full use of the parallelism of GPGPU and saving memory access time.

[0081] The present invention has been shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the means in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. A multi-granularity cache management system applied to GPGPU, characterized in that: Applied between memory and cache for partitioning and storing data in the form of data blocks, the data in the memory is divided into multiple data blocks of fixed size, the data blocks are mapped to the cache based on predefined mapping rules, the cache is divided into multiple data sets, each data set contains multiple cache lines, and each cache line is divided into multiple data blocks; the system includes a label decoding module, a label management module, a hit judgment module, a hit recording module and a granularity controller; The tag decoding module is used to decode and obtain a matching tag corresponding to the address information in the original tag based on the data bits corresponding to the original tag, wherein the original tag is information sent to the tag decoding module when the Load / Store unit receives a memory access request from the application, the address information is the address of the data specified in the memory access request in the memory, and the matching tag includes a set tag for identifying a data set where the data in the cache is located, a row tag for identifying a cache line where the data in the data set is located, a block tag for identifying a data block where the data in the cache line is located, and a byte index for identifying the position of a specific byte in the cache line; The tag management module is used to receive the block tag sent by the tag decoding module, the current cache granularity sent by the granularity controller, and the original tag sent by the Load / Store unit, and adjust the block tag based on the current cache granularity and the data size specified in the original tag to obtain a processed block tag; The hit judgment module maintains a block tag array corresponding to all data block addresses in the cache, which is used to receive the processed block tags sent by the tag management module and the row tags, set tags and byte indexes sent by the tag decoding module, determine the data address based on the row tags, set tags and byte indexes, find the corresponding block tags from the block tag array based on the data address, and compare and match the found block tags with the processed block tags to obtain hit information. In the cache management process, if the found block tag is the same as the processed block tag, it is judged as a hit, otherwise it is judged as a miss; when it is judged as a hit, if the memory access request is a data read, the address corresponding to the hit block tag is sent to the cache storage for value retrieval, and if the memory access request is a data write, , check whether the data at the address corresponding to the hit block tag is consistent with the data specified in the memory access request. If they are consistent, do not perform the data write operation. If they are inconsistent, write the hit block tag into the block tag array and write the executed data into the cache storage; when it is determined to be a miss, if the memory access request is a data read, take a value from the next level cache or storage based on the address corresponding to the hit block tag; if the memory access request is a data write, check whether the data in the next level cache or storage at the address corresponding to the hit block tag is consistent with the data specified in the memory access request. If they are consistent, do not perform the data write operation. If they are inconsistent, write the hit block tag into the block tag array and write the data into the next level cache or storage at the address corresponding to the hit block tag; The hit recording module is used to receive the hit information sent by the hit judgment module, and judge whether the current granularity meets the requirements based on the hit information. If not, it is used to send a granularity non-compliance signal to the granularity controller; The granularity controller is used to receive the granularity non-conformity signal and the original tag sent by the Load / Store unit, and adjust the current cache granularity based on the current cache granularity and the data size specified in the original tag; The tag management module is used to perform the following to adjust the block tag based on the current cache granularity and the data size specified in the original tag: Selecting a granularity mode based on the block tag, the current cache granularity, and the data size specified in the original tag; Based on the selected granularity mode, the data block size and the data size specified in the original label, the block label is assigned a value, and the block label is assigned a value directly, assigned a value of +n, or not assigned a value. When assigning a value to a block tag, the value is assigned to the block tag in the block tag array corresponding to the data block address.

2. The multi-granularity cache management system for GPGPU according to claim 1, characterized in that: The formula for calculating the hit rate is: Hit rate = number of hits / (number of hits + number of misses).

3. The multi-granularity cache management system for GPGPU according to claim 1, characterized in that: The granularity controller is used to detect the frequency of data size changes and adjust the cache granularity based on the frequency of data size changes and the current cache granularity. If the data change frequency is higher than the threshold and the current data size is smaller than the current cache granularity, the cache line is divided into smaller data blocks. If the data change frequency is lower than the threshold and the current data size is larger than the current cache granularity, the cache line is expanded by merging data blocks.

4. The multi-granularity cache management system for GPGPU according to any one of claims 1 to 3, characterized in that: The system further includes a conflict detection module, which is used to identify and synchronize concurrent memory access requests to the cache line, and lock the data block being operated in the cache line through a lock mechanism; When multiple concurrent memory access requests write data to the same cache line at the same time, a write lock is added to the data block being written by the current thread's memory access request. After the current thread's memory access request is executed, the write lock is released, allowing the next thread's memory access request to write to the data block. When multiple concurrent memory access requests perform data read operations on the same cache line at the same time, a shared lock is added to the corresponding data block, allowing all thread concurrent memory requests to perform read operations on the data block at the same time.

5. The multi-granularity cache management system for GPGPU according to claim 4, characterized in that: The conflict detection module is configured with a request queue and a first-in-first-out scheduling strategy. The memory access requests of the same cache line are queued through the request queue, and the concurrent access to the cache line is managed through the first-in-first-out scheduling strategy.

6. The multi-granularity cache management system for GPGPU according to claim 4, characterized in that: The conflict detection module manages data through a cached dirty data management mechanism; Dirty data refers to the state of data when it is updated in the cache but not written to the cache. The data updated to the cache is marked as dirty data through the dirty data management mechanism, and the data marked as dirty data is written to the cache during the non-access phase of the cache, and the block tag and status information corresponding to the data are updated.

Images