A multi-level cache system and method for GPGPU multi-core system

Abstract

The present invention relates to the field of chip cache, and specifically to a multi-level cache system and method for GPGPU multi-core systems. In view of the limitation of memory access bandwidth, an L0 instruction cache based on a self-loop data prefetch mechanism is designed. Through an efficient prefetch access mechanism for instruction data, the access frequency of the computing core to the L1 cache data and the bandwidth pressure of the cache system are reduced. In view of the problems of cache consistency and memory access delay, a multi-level cache architecture based on shared tag state retention is provided. By deeply studying the relationship between tags in the multi-level cache system, the tag comparison process is optimized and the number of unnecessary tag comparisons is reduced. At the same time, a status marking mechanism is introduced in each level of cache, which can maintain data consistency between different cache levels. This status marking not only includes data validity information, but also covers data source and destination information, ensuring the accuracy and timeliness of data in the multi-level cache system.

Description

Technical Field

[0001] The present invention relates to the field of GPGPU and chip cache, and in particular to a multi-level cache system and method for a GPGPU multi-core system. Background Art

[0002] In the field of high-performance computing, the use of General-Purpose Compution on Graphics Processing Unit (GPGPU) has become an important means to accelerate scientific and engineering computing. The core advantage of GPGPU is that it uses SIMT (single-instruction multiple-thread architecture), which significantly improves data computing efficiency by executing multiple computing tasks in parallel through multiple computing cores. However, while multiple computing cores in parallel improve the computing performance of the system, it also brings new challenges to memory access.

[0003] Specifically, during the execution of GPGPU, each computing core needs to load and write data independently. These operations rely on a shared cache unit (cache), which leads to three major problems. First, the limitation of memory access bandwidth. With the continuous growth of computing power demand, the number of computing cores in GPGPU continues to increase, resulting in a large number of cores accessing the cache at the same time. The bandwidth of the existing cache can no longer meet the data request bandwidth requirements in this multi-core environment. Second, the cache consistency problem. In a GPGPU system, multiple computing cores may need to process the same data. In order to ensure that all cores can access a consistent data state, an efficient consistency protocol is required to achieve synchronous data updates, and the existing consistency protocol has an efficiency bottleneck in this regard. Finally, the memory access delay problem. During the memory access process, the tag comparison operation is not only time-consuming and power-consuming, but also a key step in determining cache hits. Therefore, optimizing the tag comparison mechanism and reducing its impact on system performance is a key link in improving the performance of future GPGPUs. The L0 cache uses a self-loop data prefetching mechanism to reduce the frequency of computing core access to L1 cache data; based on the GPGPU multi-level cache architecture with shared tag state retention, the tag comparison process is optimized, status tags are introduced in each level of cache, and data consistency is maintained between different cache levels. Summary of the invention

[0004] In view of the defects of the prior art, the present invention provides a multi-level cache system and method for GPGPU multi-core systems. In view of the limitation of memory access bandwidth, especially the instruction data memory access of multi-cores, the present invention designs an L0 instruction cache cache based on a self-loop data pre-fetch mechanism. Through the efficient pre-fetch access mechanism for instruction data, the access frequency of the computing core to the L1 cache data is reduced, and the bandwidth pressure of the GPGPU cache system is effectively reduced. In view of the problems of cache consistency and memory access delay, the present invention provides a GPGPU multi-level cache architecture based on shared tag state retention. The architecture optimizes the tag comparison process and reduces the number of unnecessary tag comparisons by deeply studying the relationship between tags in the multi-level cache system. At the same time, by introducing a status marking mechanism in each level of cache, the present solution can maintain data consistency between different cache levels. This status marking not only includes data validity information, but also covers data source and destination information, ensuring the accuracy and timeliness of data in the multi-level cache system.

[0005] In order to solve the technical problem, the technical solution adopted by the present invention is: a multi-level cache system for GPGPU multi-core system, L0 cache, L1 cache, L2 cache and L3 cache, each computing core is connected to a core-private L0 cache, and the L0 cache is used to store data accessed from the L1 cache; multiple computing cores share an L1 cache and access the L1 cache in a pipeline manner at the same level, multiple computing cores and L1 cache constitute a computing core cluster, multiple computing core clusters share an L2 cache and access the L2 cache in a pipeline manner at the same level, multiple computing core clusters and L2 cache constitute a computing core group, multiple computing core groups share an L3 cache and access the L3 cache in a pipeline manner at the same level, multiple computing core groups and L3 cache constitute a multi-level cache system for GPGPU, the data bit width of the L0 cache is the same as the data request bit width issued by the computing core, and the data bit widths of the L0 cache, L1 cache, L2 cache and L3 cache increase in sequence.

[0006] Furthermore, the data path of the present system is as follows: the computing core issues data request address information, and the data request address information is first retrieved in the core's private L0 cache. If the retrieval is a hit, the computing core data response is directly fed back. If the L0 cache is a miss, the L0 cache sends the data request address information to the L1 cache. After the L1 cache receives the data request address information sent by the L0 cache, the address information retrieval is completed internally. If a hit is found, the L0 cache data response information is fed back to update the L0 cache data. The L0 cache then sends the data response to the computing core that issued the data request address information. If the L1 cache is a miss, the data request address information is sent to the corresponding L2 cache. After the L2 cache receives the data request address information sent by the L1 cache, the address information retrieval is completed internally. If a hit is found, the L1 cache data response information is fed back to update the L1 cache data. The L1 cache then sends the data response to the L0 cache, updates the L0 cache data, and then the L0 cache sends the data response to the computing core that issued the data request address information. If the L2 cache is a miss, the data request address information is sent to the corresponding L3 cache.

[0007] The present invention also discloses a multi-level cache method for a GPGPU multi-core system. The method is implemented based on the above system. The L0 cache adopts a memory query based on a self-loop data prefetch mechanism. The L1 cache, L2 cache, and L3 cache all adopt an improved tag comparison memory query mechanism. The memory query based on the self-loop data prefetch mechanism caches the data required by multiple groups of computing cores at one time. The improved tag comparison memory query mechanism is the first memory access strategy based on shared tag state retention. According to the relationship between tags in the multi-level cache system, tag comparisons of unreliable data and tag comparisons of upper-level tags that have passed are omitted. At the same time, status tags are added to caches at each level. The status tags include data validity information, data source and destination information.

[0008] Furthermore, the memory access query based on the self-loop data prefetch mechanism is implemented based on the self-loop data prefetch module. The self-loop data prefetch module includes the headPC register, the LoopRam storage body, and the PC counter. The upper layer of the self-loop data prefetch module is the instruction fetch PC issued by the computing core, and the lower layer of the self-loop data prefetch module is the L1 cache. Data is obtained from the L1 cache. The memory access query implementation process based on the self-loop data prefetch mechanism is as follows:

[0009] S1.1, the multi-level cache system starts working. There is no data stored in the L0 cache. The PC value issued by the computing core directly enters the L1 cache through the PC counter for memory access. At the same time, according to the PC value issued by the computing core, the bit width of the requested data and the bit width of the L1 cache line, the HeadPC register in the L0 cache is updated; including dividing the PC value into two parts: the front-end PC value and the word selection signal, where the bit width of the word selection signal is determined by Determine, where a represents the bit width of the request data issued by the computing core, and b represents the bit width of the L1 cache line. Then, the value of the HeadPC register is updated to the combination of the front-end PC value and the zero value corresponding to the word select signal bit width, that is, the update formula is headPC = [front-end PC value, log2 (b / a) bit width 0].

[0010] S1.2, after the PC sent by the computing core is converted into a data request address, the response data is obtained after tag comparison and channel selection. This step does not pass word selection judgment, and returns a cache line data of the L1 cache to the L0 cache;

[0011] S1.3, the L0 cache obtains the feedback response data, first refreshes the value of the PC counter as the word selection signal, and then slices the feedback response data according to the data bit width of the L0 cache, and stores them in the corresponding rows of the LoopRam storage body respectively, where the bit width of the PC counter and the depth of the LoopRam storage body are obtained by b / a, where a represents the bit width of the request data issued by the computing core, and b represents the bit width of the L1 cache line;

[0012] S1.4, at the same time, feeding back the corresponding feedback response data to the computing core according to the word selection signal;

[0013] S1.5, when the computing core issues a sequential PC instruction again, the PC calculator increases by one, and at the same time, the word selection signal is cut out according to the PC, and the data corresponding to the word selection signal is selected from the LoopRam storage body and sent back to the computing core;

[0014] S1.6. When the PC counter is 3, it means that the LoopRam storage body has reached the storage upper limit, then steps S1.1 to S1.5 are repeated for the next PC issued by the computing core.

[0015] Further, in step S1.5, when the computing core issues a jump PC again, the system first performs a numerical comparison process to determine whether the program counter value after the jump is located in the address space of the LoopRam storage body. The numerical comparison process involves three data: the value of the jump PC, the value of the HeadPC register, and the depth of the LoopRam storage body. The numerical comparison process is: first compare the value of the jump PC with the value of the HeadPC register. If the value of the jump PC is greater than the value of the HeadPC register, and is less than or equal to the value of the HeadPC register plus the depth of the LoopRam storage body, it means that the value of the jump PC is within the valid address range of the LoopRam storage body. In this case, update the value of the PC calculator to make it equal to the current word select signal, and then extract data from the corresponding row in the LoopRam storage body according to the word select signal;

[0016] If the jump PC value is greater than the HeadPC register value plus the depth of the LoopRam memory bank, or the jump PC value is less than the HeadPC register value, it means that the jump PC value is not within the address space of the LoopRam memory bank. At this time, repeat steps S1.1 to S1.5.

[0017] Furthermore, the tag information maintained by the L1 cache, L2 cache, and L3 cache includes tag data, source data entries, and destination data entries. The source data entry information includes data_valid, way_valid, and source_way signals, which respectively indicate data validity, channel validity, and the subsequent cache channel as the source of the data. The destination data entry includes dest_cache and multiple dest_ways, which respectively indicate the destination cache and destination channel of the data mapping. The data reading process based on the improved tag comparison memory query mechanism is as follows:

[0018] S2.1, after receiving the data request signal from the upper-level cache, the cache at this level first obtains the cache tag data, source data entries, and destination data entries of the corresponding rows of multiple channels according to the cache row addresses obtained by clipping;

[0019] S2.2. If any one of data_valid and way_valid of a channel is 0, it means that the data is unreliable and the label data in the channel is not compared;

[0020] S2.3. If data_valid and way_valid in a channel are both 1, and the data in the requested data tag matches the cache tag data, then the data is read from the cache at this level and fed back to the upper level cache, and the destination data entry of the cache at this level is updated;

[0021] S2.4, after the upper cache obtains the data, it updates the cache tag and cache data, and also updates its source data entry;

[0022] S2.5. If the cache tag data of all channels in the cache at this level do not successfully match the data in the requested data tag, determine whether the lower-level cache hits according to the association relationship of the tag data in the multi-level cache and the tag judgment result in the cache at this level; the association relationship of the tag data in the multi-level cache means that the cache tag data is repeated and decreases level by level in the multi-level cache, and the tag data of the lower-level cache is composed of the first several bits of the tag data of the upper-level cache. Based on this association relationship, by comparing the tag data of the current level cache, the matching status of the corresponding tag in the lower-level cache is inferred, and then determine whether the lower-level cache hits;

[0023] S2.6. If there is a cache hit in the lower level in the channel where data_valid and way_valid are both 1 in the cache at this level, the data in the lower level cache is directly accessed according to the source_way data maintained in the channel;

[0024] S2.7, if there is no lower-level cache hit in the channel where data_valid and way_valid are both 1 in the cache at this level, the cache at this level normally sends a data request signal to the lower-level cache, and the lower-level cache performs tag comparison to determine whether the data request is a hit;

[0025] S2.8. The lower-level cache sends a data response to the current-level cache while updating its own destination data entry. The current-level cache receives the data response from the lower-level cache while updating its own source data entry.

[0026] Furthermore, in step S2.3, the destination data entry of the cache at this level is updated by raising the dest_cache position in the cache at this level according to the corresponding upper-level cache access source, and assigning the channel number of the upper-level cache fed back to the dest_way corresponding to the cache at this level; in step S2.4, the upper-level cache updates the source data entry by raising its own data_valid and way_valid, and writing the channel number in the data response into source_way.

[0027] Furthermore, in step S2.8, the cache at this level accepts the data response from the lower-level cache and updates its own source data entries in the following manner: according to the data source of the lower-level cache, the data_valid and way_valid in the cache at this level are pulled high, and the source_way signal is filled in the way_num of the lower-level cache. At the same time, a feedback signal is sent to other upper-level caches except the memory access cache in the dest_cache maintained in the cache at this level, and the way_valid signal of the corresponding upper-level cache is pulled low, indicating that the data mapping relationship between the upper-level cache and the cache at this level is updated.

[0028] Furthermore, the tag information maintained by the L1 cache, L2 cache, and L3 cache includes tag data, source data entries, and destination data entries. The source data entry information includes data_valid, way_valid, and source_way signals, which respectively indicate data validity, channel validity, and the subsequent cache channel as the source of the data. The destination data entry includes dest_cache and multiple dest_ways, which respectively indicate the destination cache and destination channel of the data mapping. The data writing process based on the improved tag comparison memory query mechanism is as follows:

[0029] S3.1, after receiving the memory access signal from the lower-level cache, the cache at this level first obtains the cache tag data of the corresponding rows of multiple channels and the source data entry and the destination data entry according to the cache line address obtained by clipping;

[0030] S3.2, if the way_valid of the corresponding channel in the current level cache is equal to 0, it means that the data is unreliable and the tag data in the channel is not compared;

[0031] S3.3, if there is a channel in the cache at this level where way_valid is equal to 1, and the data in the data request tag matches the cache tag data, then the data is written to the corresponding cache data page according to the channel number and cache line address, and the corresponding destination data entry is updated, indicating that the data is written to the cache at this level;

[0032] S3.4, when the write data hits, the data is directly sent to the corresponding channel of the subsequent cache according to the source_way maintained by the tag, and the data is written to the corresponding cache data page;

[0033] S3.5. If the cache tag data in all channels where way_valid is equal to 1 do not successfully match the data in the request data tag, determine whether the lower-level cache hits according to the association relationship of the tag data in the multi-level cache and the tag judgment result in the cache at this level. The association relationship of the tag data in the multi-level cache means that the cache tag data is repeated and decreases level by level in the multi-level cache. The tag data of the lower-level cache is composed of the first several bits of the tag data of the upper-level cache. Based on this association relationship, by comparing the tag data of the current level cache, the matching status of the corresponding tag in the lower-level cache is inferred, and then whether the lower-level cache hits;

[0034] S3.6, if the lower-level cache hits, the data location corresponding to the lower-level cache is directly operated according to the source_way maintained in the cache at this level, and the destination data entry of the lower-level cache is updated, indicating that the data is written to the lower-level cache;

[0035] S3.7. If the lower-level cache does not hit, a data request signal is sent to the lower-level cache as normal, and the lower-level cache performs tag comparison to determine whether the data request hits.

[0036] Furthermore, in step S3.3, the method for updating the corresponding destination data entry is: sending feedback signals according to other upper-level caches except the memory access cache in the dest_cache maintained in the current-level cache, pulling down the data_valid signal of the corresponding upper-level cache, indicating that the data is written to the current-level cache; in step S3.6, the method for updating the destination data entry of the lower-level cache is: sending feedback signals according to the dest_cache maintained in the lower-level cache, pulling down the data_valid signal of the corresponding upper-level cache, indicating that the data of the lower-level cache is written.

[0037] Beneficial effects of the present invention: The present invention proposes a multi-level cache implementation scheme for GPGPU multi-core systems. Aiming at the problem of excessive bandwidth load of the original cache scheme, a L0cache based on a self-loop data pre-fetching mechanism is proposed. Multiple groups of computing core required data are cached at one time without the need for a time-consuming tag comparison mechanism, which effectively reduces the memory access frequency of the computing core to the shared L1 cache. Aiming at the problems of time-consuming tag comparison and multi-level cache data consistency of the original cache scheme, a memory access strategy based on a shared tag state retention mechanism is proposed. According to the tag characteristics between the multi-level caches, redundant tag comparisons are reduced and the memory access rate is improved. At the same time, a special state retention register is set to maintain the data mapping relationship between the multi-level caches, which solves the cache consistency problem in the GPGPU multi-core system. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is the overall architecture diagram of the multi-level cache system for GPGPU multi-core systems;

[0039] Figure 2 It is a schematic diagram of the implementation of existing L1, L2, and L3 caches;

[0040] Figure 3 Cache_line selection mechanism and tag stored procedures for existing multi-level caches;

[0041] Figure 4 It is a schematic diagram of an implementation method of L0 execution cache based on a self-loop data prefetching mechanism;

[0042] Figure 5 A schematic diagram of a state register based on a shared tag state retention mechanism;

[0043] Figure 6 Schematic diagram of the data reading process of L1, L2, and L3 caches based on the shared tag state retention mechanism;

[0044] Figure 7 Schematic diagram of the data writing process of L1, L2, and L3 caches based on the shared tag state retention mechanism. DETAILED DESCRIPTION

[0045] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. First, the English expressions appearing in this application are explained:

[0046] GPGPU: General-Purpose computation on Graphics Processing Unit, general-purpose graphics processing unit;

[0047] SIMT: single-instruction multiple-thread, single instruction multiple thread;

[0048] Cache: cache;

[0049] tag: tag;

[0050] cache_tag: cache tag;

[0051] cache_data: cache data;

[0052] way: channel;

[0053] cache_line: cache line;

[0054] way_num: channel number;

[0055] req_addr: data request address;

[0056] word_sel1: word selection;

[0057] cache_line_addr: cache line address;

[0058] req_tag: data request tag;

[0059] rsp_data: response data;

[0060] PC; program counter, used to store the address of instructions;

[0061] Core: computing core;

[0062] PC_cnt: PC counter, which counts the number of PCs that L0cache has responded to;

[0063] HeadPC: head PC data;

[0064] LoopRam: loop RAM;

[0065] size: the depth of the LoopRam memory;

[0066] miss: cache miss;

[0067] data_valid: data validity;

[0068] way_valid: channel validity;

[0069] source_way: source channel;

[0070] dest_cache: destination cache;

[0071] dest_way: destination channel.

[0072] Example 1

[0073] This embodiment discloses a multi-level cache system for a GPGPU multi-core system. Figure 1 As shown, it includes L0 cache, L1 cache, L2 cache and L3 cache. The core in the figure is a computing core that can independently perform computing tasks. Each computing core is connected to a core-private L0 cache. The L0 cache is used to store data accessed from the L1 cache. In this embodiment, it is mostly used to store instruction data. Multiple computing cores share an L1 cache and access the L1 cache in a pipelined manner at the same level. Multiple computing cores and L1 caches constitute a computing core cluster. Multiple computing core clusters share an L2 cache and access the L2 cache in a pipelined manner at the same level. Multiple computing core clusters and L2 caches constitute a computing core group. Multiple computing core groups share an L3 cache and access the L3 cache in a pipelined manner at the same level. Multiple computing core groups and L3 caches constitute a multi-level cache system of GPGPU.

[0074] In this embodiment, the data path of the multi-level cache system of the GPGPU multi-core system is as follows: the computing core sends out data request address information, and first the data request address information is retrieved in the core's private L0 cache. If the retrieval hits, the computing core data response is directly fed back. If the L0 cache misses, the L0 cache sends the data request address information to the L1 cache. After the L1 cache receives the data request address information sent by the L0 cache, it completes the address information retrieval internally. If it hits, the L0 cache data response information is fed back to update the L0 cache data, and then the L0 cache sends the data response to the L1 cache. The computing core that issues the data request address information sends the data request address information to the corresponding L2 cache if the L1 cache does not hit. After the L2 cache receives the data request address information sent by the L1 cache, it completes the address information retrieval internally. If it hits, it feeds back the L1 cache data response information, updates the L1 cache data, and then the L1 cache sends the data response to the L0 cache, updates the L0 cache data, and then the L0 cache sends the data response to the computing core that issued the data request address information. If the L2 cache does not hit, the data request address information is sent to the corresponding L3 cache.

[0075] In this embodiment, the data bit width of L0 is the same as the data request bit width issued by the computing core, and the data bit widths of the L0 cache, L1 cache, L2 cache, and L3 cache increase in sequence.

[0076] Example 2

[0077] This embodiment discloses a multi-level cache method for a GPGPU multi-core system. This method is implemented based on the system described in Example 1. In this method, except that the L0 cache adopts a memory access query based on a self-loop data prefetch mechanism, other L1 caches, L2 caches, and L3 caches all adopt an improved tag comparison memory access query mechanism.

[0078] First, let's introduce the basic tag comparison memory access mechanism. Figure 2As shown. In the basic tag comparison memory query mechanism, each level of cache maintains two sets of data, namely cache_tag and cache_data. Each cache_tag and cache_data maintains four ways and cache_line data. When the data request address arrives, it will first enter the cache_tag, and determine which row of the cache_line the requested data should be placed in based on the high bit of the request address. Then, the data of the cache line in the four ways will be read at the same time and compared with the input address. If the data stored in the four ways does not match the input address, it means that the address is not hit, and a data response needs to be sent to the next level cache; if the data in that way matches successfully, the corresponding data will be read directly from the cache_data based on way_num and cache_line. When the data response arrives, the corresponding cache_line is also determined based on the response address, and then the way_num of the updated data is selected through the way selection mechanism, and the data in the corresponding cache_line and way_num in the cache_tag and cache_data are updated synchronously.

[0079] For example, the L0 cache data request address req_addr0 is 30 bits, the corresponding request data width is 32 bits, the cache_line depth is 16, and the L1 and L2 cache data widths are 128 bits and 512 bits respectively. Figure 3 The cache_line selection mechanism and tag storage process inside the multi-level cache in the basic tag comparison memory access mechanism are given, which are divided into the following processes:

[0080] S01, according to the L1 cache data bit width of 128 bits and the L0 cache request data bit width of 32 bits, 0-1 bits are cut from the L0 cache data request address req_addr0 as word_sel1.

[0081] S02. According to the cache_line depth of 16 in the L1 cache, bits 2 to 5 are cut from the L0 cache data request address req_addr0 as cache_line_addr1, and bits 6 to 29 are used as req_tag1 to compare with the data in cache_tag.

[0082] The above-mentioned cutting is a conventional technical means in this field and will not be described in detail in this embodiment.

[0083] S03. Read the tag data of four ways from the corresponding cache_line in cache_tag according to cache_line_addr1;

[0084] S04, compare req_tag1 data with the read tag data to see if they are consistent, if they are consistent, feedback the way_num signal; if they are inconsistent, feedback the miss signal;

[0085] S05, if they are consistent, read 128-bit data from the corresponding cache_data according to the way_num and cache_line_addr1 signals, select the corresponding 32-bit data according to word_sel1, and feed it back to the L0 cache;

[0086] S06. In case of inconsistency, cache_line_addr1 is sent as the data request signal req_addr1 of the L1 Cache to the L2 cache, and the corresponding data request bit width is 128 bits.

[0087] S07 , according to the L2 cache data bit width of 512 bits and the L1 cache request data bit width of 128 bits, 0-1 bits are cut from the L1 cache data request address req_addr1 as word_sel2 .

[0088] S08. According to the cache_line depth of 16 in the L2 cache, bits 2 to 5 are cut from the L1 cache data request address req_addr1 as cache_line_addr2, and bits 6 to 23 are used as req_tag2 to compare with the data in cache_tag.

[0089] S09. Read tag data of four ways from the corresponding cache_line in cache_tag according to cache_line_addr2;

[0090] S10, compare req_tag2 data with the read tag data to see if they are consistent, if they are consistent, feedback the way_num signal; if they are inconsistent, feedback the miss signal;

[0091] S11, if consistent, read 512-bit data from the corresponding cache_data according to the way_num and cache_line_addr2 signals, select the corresponding 128-bit data according to word_sel2, and feed it back to the L1 cache;

[0092] S12, the signal fed back to the L1 cache, finds the cache_line corresponding to the L1 cache according to req_addr1, and writes the req_tag1 signal into the cache_tag of the cache_line corresponding to the way according to the set way selection mechanism, and writes the response 128-bit data obtained from the L2 cache into the cache_data of the cache_line corresponding to the way.

[0093] S13. At the same time, according to word_sel1, 32-bit data is selected from the response 128-bit data obtained from the L2 cache and written back to the L0 cache.

[0094] In the multi-level cache architecture of the original GPGPU multi-core system, multiple computing cores access the L1 cache at the same time, which not only brings bandwidth pressure to the L1 cache, but also brings huge dynamic power consumption due to continuous data reading. The present invention proposes an L0 instruction cache implementation scheme based on a self-loop data prefetching mechanism, which caches multiple groups of computing core required data at a time without the need for a time-consuming tag comparison mechanism, effectively reducing the access frequency of the computing core to the shared L1 cache.

[0095] The L0 instruction cache implementation scheme based on the self-loop data prefetching mechanism described in this embodiment is based on Figure 4 The self-loop data prefetch module is implemented as shown in the figure. The self-loop data prefetch module includes headPC register, LoopRam storage body, PC counter. The upper layer of the self-loop data prefetch module is the instruction fetch PC issued by the computing core. The lower layer of the self-loop data prefetch module is L1 cache. Data is obtained from L1 cache. Figure 3 The traditional L0 cache memory access process shown in the figure, the L0 instruction cache implementation scheme based on the self-loop data prefetch mechanism fully utilizes the storage bit width of the L1 cache, no longer performs word_sel selection, and reads one cache_line of data from the L1 cache at a time. Taking the core requirement 32-bit wide data and the L1 cache_line storage bit width of 128bit as an example, the specific implementation process is as follows:

[0096] S1.1. When the system just starts working, there is no value stored in the L0 cache. The PC sent by the computing core will directly enter the L1 cache through the PC counter for memory access, and at the same time update the value of the HeadPC register to the reorganized data of the word_sel position set to zero by the PC value sent by the computing core; in this embodiment, the update method is specifically as follows: according to the PC value sent by the computing core, the bit width of the requested data and the bit width of the L1 cache line, the HeadPC register in the L0 cache is updated; including dividing the PC value into two parts: the front-end PC value and the word selection signal, wherein the bit width of the word selection signal is composed of Determine, where a represents the bit width of the request data issued by the computing core, and b represents the bit width of the L1 cache line. Then, the value of the HeadPC register is updated to the combination of the front-end PC value and the zero value corresponding to the word selection signal bit width, that is, the update formula is headPC = [front-end PC value, bit width 0].

[0097] S1.2, the PC sent by the computing core is converted into req_addr, and then goes through a process similar to the above Figure 3 A series of operations such as tag comparison and way selection ( Figure 3 After S03 and S04 in the process), the response data is obtained. The difference is that this solution does not require word_sel judgment ( Figure 3 Instead, the 128-bit data is returned to the L0 cache together with the feedback data.

[0098] S1.3, L0 cache gets the 128-bit response data fed back, first refreshes the PC counter value to word_sel, then divides the fed back 128-bit data into 32-bit slices, and stores them in the corresponding rows of the LoopRam storage body respectively. The bit width of the PC counter and the depth of the LoopRam storage body are obtained by b / a, where a represents the bit width of the request data issued by the computing core, and b represents the bit width of the L1 cache line. In this embodiment, the bit width of the L1 cache line is 128 bits, and the bit width of the request data issued by the computing core is 32 bits, then the bit width of the PC counter and the depth of the LoopRam storage body are both 4.

[0099] S1.4, at the same time, the 32-bit data at the position of the feedback 128-bit data word_sel is fed back to the computing core;

[0100] S1.5, when the computing core issues a sequential PC instruction again, the PC counter is incremented by one, and word_sel is cut out according to PC, 32-bit data is read from the word_sel line of LoopRam, and sent back to the computing core;

[0101] S1.6. When the PC counter counts to 3, it means that the LoopRam storage body has reached the storage upper limit, and the next PC issued by the computing core will repeat the process from S1.1 to S1.5.

[0102] S1.7, when the computing core issues a jump PC, the system first performs a numerical comparison process to determine whether the program counter value after the jump is located in the address space of the LoopRam storage body. The numerical comparison process involves three data: the value of the jump PC, the value of the HeadPC register, and the depth of the LoopRam storage body. The numerical comparison process is: first compare Comparison The value of the jump PC and the value of the HeadPC register. If the value of the jump PC is greater than the value of the HeadPC register and is less than or equal to the value of the HeadPC register plus the depth of the LoopRam storage body, it means that the value of the jump PC is within the valid address range of the LoopRam storage body. In this case, update the value of the PC calculator to make it equal to the current word select signal, and then extract data from the corresponding row in the LoopRam storage body according to the word select signal;

[0103] If the jump PC value is greater than the HeadPC register value plus the depth of the LoopRam memory bank, or the jump PC value is less than the HeadPC register value, it means that the jump PC value is not within the address space of the LoopRam memory bank. At this time, repeat steps S1.1 to S1.5.

[0104] After the self-loop data prefetching mechanism of this solution, the L0 cache can cache multiple sets of instruction data, reducing the process of accessing the L1 cache and comparing tags for each instruction fetch of the computing core, reducing the L1 cache memory access frequency and improving the memory access speed.

[0105] The improved tag comparison memory query mechanism is the first memory access strategy based on shared tag state retention. According to the relationship between tags in the multi-level cache system, tag comparisons of unreliable data and tag comparisons of upper-level tags that have passed are omitted. At the same time, status tags are added to caches at all levels. The status tags include data validity information, data source and destination information.

[0106] The format of the status register based on the shared tag status mechanism is as follows Figure 5 As shown, in this scheme, the tag information maintained by the L1 cache, L2 cache, and L3 cache includes the tag itself, the source data entry, and the destination data entry. The source data entry information includes data_valid, way_valid, and source_way signals, which respectively represent the data validity, the channel validity, and the subsequent cache channel that is the source of the data. The destination data entry includes dest_cache and multiple dest_ways, which respectively represent the destination cache and destination channel of the data mapping.

[0107] Taking the L2 cache memory access as an example, the L1 cache, L2 cache, and L3 cache memory access process of this solution is as follows: Figure 6 , 7 As shown, it is divided into two sub-processes of reading and writing, wherein the data reading process is shown in 6. In this embodiment, the L2 cache is the cache at this level, the L1 cache is the upper cache, and the L3 is the lower cache;

[0108] S2.1, after receiving the memory access signal (i.e., data request signal) from the L1 cache, the L2 cache first obtains the cache_tag data and source data entries and destination data entries of the corresponding lines of multiple ways according to the clipped cache_line_addr;

[0109] S2.2. If either data_valid or way_valid of the corresponding way is 0, it means that the data is unreliable and there is no need to compare the tag data in the way;

[0110] S2.3, if there is a way in which data_valid and way_valid are both 1, and the data in req_tag matches the cache_tag data, then read the data from cache_data according to cache_line_addr and way_num, and feed it back to the L1 cache. At the same time, according to the corresponding L1 cache access source, pull up the dest_cache position in the L2 cache, and assign the feedback input L1 cache way_num to the dest_way corresponding to the L2 cache.

[0111] S2.4, after the L1 cache receives the data response, it updates the cache_tag and cache_data data, and at the same time, it also raises its own data_valid and way_valid, and writes the way_num from the L2 cache into the source_way.

[0112] S2.5. If the cache_tag of all ways in the L2 cache does not match the req_tag successfully, then according to the association relationship between the tag bit widths of the multi-level caches, it is only necessary to find out whether the first fixed bits in the cache_tag and req_tag match according to the matching results to know whether there is a hit in the L3 cache; in this embodiment, the association relationship of the tag data in the multi-level cache means that the data of the cache tag is repeated and decreases level by level in the multi-level cache, and the tag data of the lower-level cache is composed of the first several bits of the tag data of the upper-level cache. Based on this association relationship, by comparing the tag data of the current level cache, the matching status of the corresponding tag in the lower-level cache is inferred, and then it is determined whether the lower-level cache hits. If the tag data of the current level cache consists of 26 bits, the tag data of the lower-level cache will be composed of the first several bits of these 26 bits. Based on this, by comparing the 26-bit tag data of the current level cache, the matching status of the corresponding tag in the lower-level cache can be indirectly inferred, and then it is determined whether the lower-level cache hits.

[0113] S2.6. If there is an L3 cache hit in the way where data_valid and way_valid are both 1 in the L2 cache, the data in the L3 cache is directly accessed based on the source_way data maintained in the way, without the need for repeated tag comparisons in the L3 cache.

[0114] S2.7. If there is no L3 cache hit in the way where data_valid and way_valid are both 1 in the L2 cache, a data request signal is sent to the L3 cache normally, and the L3 cache needs to perform tag comparison to determine whether the data request is a hit.

[0115] S2.8, After receiving the data response information from the L3 cache, the L2 cache needs to update its own tag and data data, and at the same time, pull up the data_valid and way_valid in the L2 cache according to the data source of the L3 cache, and fill the source_way signal with the way_num of the L3 cache. At the same time, it also needs to send feedback signals to other caches in the dest_cache maintained in the L2 cache except the memory access cache, and pull down the way_valid signal of the corresponding upper cache, indicating that the data mapping relationship between the upper cache and the current level L2 cache has been updated.

[0116] The process of writing data is as follows Figure 7 As shown:

[0117] S3.1. After the L2 cache receives the memory access signal (data request signal) from the L1 cache, it first obtains the cache_tag data and source data entries and destination data entries of the corresponding rows of multiple ways according to the clipped cache_line_addr.

[0118] S3.2. If the way_valid of the corresponding way is equal to 0, it means that the data is unreliable and there is no need to compare the tag data in the way.

[0119] S3.3, if there is a way in which way_valid is equal to 1, and the data in req_tag matches the cache_tag data, then the data is written to the corresponding cache_data according to way_num and cache_line_addr. And according to the other caches in dest_cache maintained in the L2 cache except the memory access cache, feedback signals are sent to lower the data_valid signal of the corresponding upper cache, indicating that the data has been written.

[0120] S3.4, when the write data hits, the data is directly sent to the corresponding way of the subsequent L3 cache according to the source_way maintained by the tag, and the data is written to the corresponding cache_data.

[0121] S3.5. If there is no match between cache_tag and req_tag in all ways with way_valid=1, then according to the association relationship between the tag bit widths of the multi-level caches, it is only necessary to find out whether the first fixed bits in cache_tag and req_tag match according to the matching results to know whether there is a hit in the L3 cache. The association relationship of the tag data in the multi-level cache means that the cache tag data is repeated and decreases level by level in the multi-level cache. The tag data of the lower-level cache is composed of the first several bits of the tag data of the upper-level cache. Based on this association relationship, by comparing the tag data of the current level cache, the matching status of the corresponding tag in the lower-level cache is inferred, and then it is determined whether the lower-level cache hits. If the tag data of the current level cache consists of 26 bits, the tag data of the lower-level cache will consist of the first several bits of these 26 bits. Based on this, by comparing the 26-bit tag data of the current level cache, the matching status of the corresponding tag in the lower-level cache can be indirectly inferred, and then it is determined whether the lower-level cache hits.

[0122] S3.6. If the L3 cache hits, the data location corresponding to the L3 cache is directly operated according to the source_way maintained in the L2 cache, and a feedback signal is sent according to the dest_cach maintained in the L3 cache, and the data_valid signal of the corresponding upper cache is pulled down, indicating that the data of the current level L3 cache has been written.

[0123] S3.7. If the L3 cache does not hit, a data request signal is sent to the L3 cache normally. The L3 cache needs to perform a tag comparison to determine whether the data request hits.

[0124] The shared tag state retention mechanism of this solution effectively reduces the continuous tag comparison between multi-level caches by updating the register state, effectively reducing the memory access power consumption and memory access latency.

[0125] The above description is only the basic principle and preferred embodiments of the present invention. Improvements and substitutions made by those skilled in the art based on the present invention belong to the protection scope of the present invention.

Claims

1. A multi-level cache method for GPGPU multi-core system, characterized by: The method is implemented based on a cache system, which includes L0 cache, L1 cache, L2 cache and L3 cache. Each computing core is connected to a core-private L0 cache, and the L0 cache is used to store data accessed from the L1 cache. Multiple computing cores share an L1 cache and access the L1 cache in a pipelined manner at the same level, multiple computing cores and the L1 cache constitute a computing core cluster, multiple computing core clusters share an L2 cache and access the L2 cache in a pipelined manner at the same level, multiple computing core clusters and the L2 cache constitute a computing core group, multiple computing core groups share an L3 cache and access the L3 cache in a pipelined manner at the same level, multiple computing core groups and the L3 cache constitute a multi-level cache system of the GPGPU, the data bit width of the L0 cache is the same as the data request bit width issued by the computing core, and the data bit widths of the L0 cache, the L1 cache, the L2 cache, and the L3 cache increase in sequence; The L0 cache uses a memory query based on a self-loop data prefetch mechanism, and the L1 cache, L2 cache, and L3 cache all use an improved tag comparison memory query mechanism. The memory query based on the self-loop data prefetch mechanism caches the data required by multiple groups of computing cores to the L0 cache at one time, and each group of computing cores accesses the data from the L0 cache. The improved tag comparison memory query mechanism is a memory access strategy based on shared tag state retention. According to the relationship between tags in the multi-level cache system, the tag comparison of unreliable data and the tag comparison of the upper tag comparison that has passed are omitted. At the same time, status tags are added to each level of cache. The status tags include data validity information, data source and destination information. Memory access to the L1 cache, L2 cache, and L3 cache is achieved based on the improved tag comparison memory query mechanism.

2. The multi-level cache system for GPGPU multi-core system according to claim 1, characterized in that: The data path of this system is as follows: the computing core sends out data request address information, and first the data request address information is retrieved in the core's private L0 cache. If the retrieval hits, the computing core data response is directly fed back. If the L0 cache misses, the L0 cache sends the data request address information to the L1 cache. After the L1 cache receives the data request address information sent by the L0 cache, it completes the address information retrieval internally. If it hits, it feeds back the L0 cache data response information, updates the L0 cache data, and then the L0 cache sends the data response to the computing core that sent the data request address information. If the L1 cache misses, it sends the data request address information to the corresponding L2 cache. After the L2 cache receives the data request address information sent by the L1 cache, it completes the address information retrieval internally. If it hits, it feeds back the L1 cache data response information, updates the L1 cache data, and then the L1 cache sends the data response to the L0 cache, updates the L0 cache data, and then the L0 cache sends the data response to the computing core that sent the data request address information. If the L2 cache misses, it sends the data request address information to the corresponding L3 cache.

3. The multi-level cache method for GPGPU multi-core system according to claim 1, characterized in that: The memory access query based on the self-loop data prefetch mechanism is implemented based on the self-loop data prefetch module. The self-loop data prefetch module includes the headPC register, the LoopRam storage body, and the PC counter. The upper layer of the self-loop data prefetch module is the instruction fetch PC issued by the computing core, and the lower layer of the self-loop data prefetch module is the L1 cache. Data is obtained from the L1 cache. The memory access query implementation process based on the self-loop data prefetch mechanism is as follows: S1.1, the multi-level cache system starts working. There is no data stored in the L0 cache. The PC value issued by the computing core directly enters the L1 cache through the PC counter for memory access. At the same time, the HeadPC register in the L0 cache is updated according to the PC value issued by the computing core, the bit width of the requested data, and the bit width of the L1 cache line. It includes dividing the PC value into two parts: the front-end PC value and the word select signal, where the bit width of the word select signal is Determine, where a represents the bit width of the request data issued by the computing core, and b represents the bit width of the L1 cache line, and then update the value of the HeadPC register to a zero value combination corresponding to the front-end PC value and the word selection signal bit width; S1.2, after the PC sent by the computing core is converted into a data request address, the response data is obtained after tag comparison and channel selection. This step does not pass word selection judgment, and returns a cache line data of the L1 cache to the L0 cache; S1.3, the L0 cache obtains the feedback response data, first refreshes the value of the PC counter as the word selection signal, and then slices the feedback response data according to the data bit width of the L0 cache, and stores them in the corresponding rows of the LoopRam storage body respectively, where the bit width of the PC counter and the depth of the LoopRam storage body are obtained by b / a, where a represents the bit width of the request data issued by the computing core, and b represents the bit width of the L1 cache line; S1.4, at the same time, feeding back the corresponding feedback response data to the computing core according to the word selection signal; S1.5, when the computing core issues a sequential PC instruction again, the PC calculator increases by one, and at the same time, the word selection signal is cut out according to the PC, and the data corresponding to the word selection signal is selected from the LoopRam storage body and sent back to the computing core; S1.

6. When the PC counter is 3, it means that the LoopRam storage body has reached the storage upper limit, then steps S1.1 to S1.5 are repeated for the next PC issued by the computing core.

4. The multi-level cache method for GPGPU multi-core system according to claim 3, characterized in that: In step S1.5, when the computing core issues a jump PC again, the system first performs a numerical comparison process to determine whether the program counter value after the jump is located in the address space of the LoopRam storage body. The numerical comparison process involves three data: the value of the jump PC, the value of the HeadPC register, and the depth of the LoopRam storage body. The numerical comparison process is as follows: first compare Comparison The value of the jump PC and the value of the HeadPC register. If the value of the jump PC is greater than the value of the HeadPC register and is less than or equal to the value of the HeadPC register plus the depth of the LoopRam storage body, it means that the value of the jump PC is within the valid address range of the LoopRam storage body. In this case, update the value of the PC calculator to make it equal to the current word select signal, and then extract data from the corresponding row in the LoopRam storage body according to the word select signal; If the jump PC value is greater than the HeadPC register value plus the depth of the LoopRam memory bank, or the jump PC value is less than the HeadPC register value, it means that the jump PC value is not within the address space of the LoopRam memory bank. At this time, repeat steps S1.1 to S1.

5.

5. The multi-level cache method for GPGPU multi-core system according to claim 1, characterized in that: The tag information maintained by the L1 cache, L2 cache, and L3 cache includes tag data, source data entries, and destination data entries. The source data entry information includes data_valid, way_valid, and source_way signals, which respectively indicate data validity, channel validity, and the subsequent cache channel as the source of the data. The destination data entry includes dest_cache and multiple dest_ways, which respectively indicate the destination cache and destination channel of the data mapping. The data reading process based on the improved tag comparison memory query mechanism is as follows: S2.1, after receiving the data request signal from the upper-level cache, the cache at this level first obtains the cache tag data, source data entries, and destination data entries of the corresponding rows of multiple channels according to the cache row addresses obtained by clipping; S2.

2. If any of data_valid and way_valid of a channel is 0, it means that the data is unreliable and the label data in the channel is not compared; S2.

3. If data_valid and way_valid in a channel are both 1, and the data in the requested data tag matches the cache tag data, then the data is read from the cache at this level and fed back to the upper level cache, and the destination data entry of the cache at this level is updated; S2.4, after the upper cache obtains the data, it updates the cache tag and cache data, and also updates its source data entry; S2.

5. If the cache tag data of all channels in the cache at this level do not successfully match the data in the requested data tag, determine whether the lower-level cache hits according to the association relationship of the tag data in the multi-level cache and the tag judgment result in the cache at this level; the association relationship of the tag data in the multi-level cache means that the cache tag data is repeated and decreases level by level in the multi-level cache, and the tag data of the lower-level cache is composed of the first several bits of the tag data of the upper-level cache. Based on this association relationship, by comparing the tag data of the current level cache, the matching status of the corresponding tag in the lower-level cache is inferred, and then determine whether the lower-level cache hits; S2.

6. If there is a cache hit in the lower level in the channel where data_valid and way_valid are both 1 in the cache at this level, the data in the lower level cache is directly accessed according to the source_way data maintained in the channel; S2.7, if there is no lower-level cache hit in the channel where data_valid and way_valid are both 1 in the cache at this level, the cache at this level normally sends a data request signal to the lower-level cache, and the lower-level cache performs tag comparison to determine whether the data request is a hit; S2.

8. The lower-level cache sends a data response to the current-level cache while updating its own destination data entry. The current-level cache receives the data response from the lower-level cache while updating its own source data entry.

6. The multi-level cache method for GPGPU multi-core system according to claim 5, characterized in that: In step S2.3, the destination data entry of the cache at this level is updated by raising the dest_cache position in the cache at this level according to the corresponding upper-level cache access source, and assigning the channel number of the upper-level cache fed back to the dest_way corresponding to the cache at this level; in step S2.4, the upper-level cache updates the source data entry by raising its own data_valid and way_valid, and writing the channel number in the data response into source_way.

7. The multi-level cache method for GPGPU multi-core system according to claim 5, characterized in that: In step S2.8, the cache at this level accepts the data response from the lower-level cache and updates its own source data entries in the following way: according to the data source of the lower-level cache, the data_valid and way_valid in the cache at this level are pulled high, and the source_way signal is filled with the way_num of the lower-level cache. At the same time, a feedback signal is sent to other upper-level caches except the memory access cache in the dest_cache maintained in the cache at this level, and the way_valid signal of the corresponding upper-level cache is pulled low, indicating that the data mapping relationship between the upper-level cache and the cache at this level is updated.

8. The multi-level cache method for GPGPU multi-core system according to claim 1, characterized in that: The tag information maintained by the L1 cache, L2 cache, and L3 cache includes tag data, source data entries, and destination data entries. The source data entry information includes data_valid, way_valid, and source_way signals, which respectively indicate data validity, channel validity, and the subsequent cache channel as the source of this data. The destination data entry includes dest_cache and multiple dest_ways, which respectively indicate the destination cache and destination channel of the data mapping. The data writing process based on the improved tag comparison memory query mechanism is as follows: S3.1, after receiving the memory access signal from the lower-level cache, the cache at this level first obtains the cache tag data of the corresponding rows of multiple channels and the source data entry and the destination data entry according to the cache line address obtained by clipping; S3.2, if the way_valid of the corresponding channel in the current level cache is equal to 0, it means that the data is unreliable and the tag data in the channel is not compared; S3.3, if there is a channel in the cache at this level where way_valid is equal to 1, and the data in the data request tag matches the cache tag data, then the data is written to the corresponding cache data page according to the channel number and cache line address, and the corresponding destination data entry is updated, indicating that the data is written to the cache at this level; S3.4, when the write data hits, the data is directly sent to the corresponding channel of the subsequent cache according to the source_way maintained by the tag, and the data is written to the corresponding cache data page; S3.

5. If the cache tag data in all channels with way_valid equal to 1 do not successfully match the data in the request data tag, determine whether the lower-level cache hits according to the association relationship of the tag data in the multi-level cache and the tag judgment result in the cache at this level; the association relationship of the tag data in the multi-level cache means that the cache tag data is repeated and decreases level by level in the multi-level cache, and the tag data of the lower-level cache is composed of the first several bits of the tag data of the upper-level cache. Based on this association relationship, by comparing the tag data of the current level cache, the matching status of the corresponding tag in the lower-level cache is inferred, and then determine whether the lower-level cache hits; S3.6, if the lower-level cache hits, the data location corresponding to the lower-level cache is directly operated according to the source_way maintained in the cache at this level, and the destination data entry of the lower-level cache is updated, indicating that the data is written to the lower-level cache; S3.

7. If the lower-level cache does not hit, a data request signal is sent to the lower-level cache as normal, and the lower-level cache performs tag comparison to determine whether the data request hits.

9. The multi-level cache method for GPGPU multi-core system according to claim 8, characterized in that: In step S3.3, the method for updating the corresponding destination data entry is: according to the dest_cache maintained in the current level cache, except for the memory access cache, a feedback signal is sent to the upper level cache, and the data_valid signal of the corresponding upper level cache is pulled down, indicating that the data is written to the current level cache; in step S3.6, the method for updating the destination data entry of the lower level cache is: according to the dest_cache maintained in the lower level cache, a feedback signal is sent to the upper level cache, and the data_valid signal of the corresponding upper level cache is pulled down, indicating that the data of the lower level cache is written.

Images