Multi-path group connection cache path prediction circuit, method, device and processor

Abstract

The invention discloses a path prediction circuit, method and device for a multi-path group connection cache and a processor, and belongs to the field of processors. The circuit prediction circuit comprises an operational circuit unit, a read logic circuit unit, a plurality of counting lookup tables and a judgment circuit unit, the multi-path group connection cache comprises a plurality of cache groups, each cache group comprises a plurality of cache paths, the plurality of cache paths are in one-to-one correspondence with the plurality of counting lookup tables, and each counting lookup table in the plurality of counting lookup tables has a plurality of entries; the operational circuit unit is used for generating an index value corresponding to the memory access address based on the memory access address; the read logic circuit unit is used for accessing an entry corresponding to the index value in each counting lookup table to obtain a plurality of count values, and the ith count value of the plurality of count values indicates the number of cache lines matched with the index value in the ith cache path; and the judgment circuit unit is used for respectively judging the relationship between the plurality of count values and zero. The path prediction circuit reduces the power consumption of the access cache.

Description

Technical Field

[0001] This application relates to the field of processors, and particularly to a way prediction circuit, method, device, and processor for a multi-way set associative cache. Background Art

[0002] A multi-way set associative cache is an organization form of a cache. Figure 1 The structure of a two-way set associative cache is shown. The two-way set associative cache is divided into two cache ways, Way 0 and Way 1, and four cache sets, Set 0 to Set 3.

[0003] In the related art, data is read from a multi-way set associative cache according to an input memory address. With reference to Figure 1 , the memory address has a Tag field segment, a Set field segment, and a Byte Offset field segment. Through the set 2 indicated by the Set field segment of the memory address, the Tag segment and the Data block of each cache way in the two-way set associative cache are accessed in parallel. According to the Tag read from each cache way, it is compared with the Tag in the memory address. If they are the same, it indicates a cache hit. The comparison result is used as the selection signal of a two-to-one selector, and the two-to-one selector also inputs the Data blocks read from each cache way.

[0004] In the related art, when reading data from a multi-way set associative cache, the data blocks and tag segments of each way are read in parallel, but ultimately only one way will be hit, and the read operations of the remaining ways are all invalid, which generates a large number of invalid read operations. When the number of cache ways is large, the wasted power consumption increases accordingly. Summary of the Invention

[0005] This application provides a way prediction circuit, method, device, and processor for a multi-way set associative cache. The way prediction circuit ensures that when reading data from the cache, only the cache ways with a predicted count value not equal to zero are read, rather than necessarily reading all cache ways, thereby reducing the power consumption of accessing the cache and improving the performance.

[0006] According to one aspect of this application, a way prediction circuit for a multi-way set associative cache is provided. The way prediction circuit includes an arithmetic circuit unit, a read logic circuit unit, a plurality of count lookup tables, and a judgment circuit unit. The four circuit units are sequentially connected in series through wires; the multi-way set associative cache includes a plurality of cache sets, and each group of the plurality of cache sets includes a plurality of cache ways. The plurality of cache ways correspond to the plurality of count lookup tables one by one, and each count lookup table in the plurality of count lookup tables has a plurality of entries;

[0007] An arithmetic circuit unit for generating an index value corresponding to a memory access address based on the memory access address;

[0008] A read logic circuit unit for accessing an entry corresponding to the index value in each count lookup table to obtain a plurality of count values. The plurality of count values correspond to the plurality of count lookup tables one by one. The i-th count value of the plurality of count values indicates the number of cache lines matching the index value in the i-th cache way. The i-th cache way is the cache way corresponding to the count lookup table where the i-th count value is located, and i is an integer value not greater than the number of tables of the plurality of count lookup tables;

[0009] A judgment circuit unit for respectively judging the relationship between the plurality of count values and zero. When the i-th count value is not zero, there is a possibility that the i-th cache way stores a first cache line; when the i-th count value is zero, the i-th cache way does not store the first cache line, and the first cache line is the cache line corresponding to the memory access address.

[0010] According to one aspect of the present application, there is provided a way prediction method for a multi-way set-associative cache. The multi-way set-associative cache includes a plurality of cache groups, and each group of the plurality of cache groups includes a plurality of cache ways. The plurality of cache ways correspond to a plurality of count lookup tables one by one, and each count lookup table of the plurality of count lookup tables has a plurality of entries. The method includes the following steps.

[0011] Obtain a memory access address; generate an index value corresponding to the memory access address based on the memory access address.

[0012] Access an entry corresponding to the index value in each count lookup table to obtain a plurality of count values. The plurality of count values correspond to the plurality of count lookup tables one by one. The i-th count value of the plurality of count values indicates the number of cache lines matching the index value in the i-th cache way. The i-th cache way is the cache way corresponding to the count lookup table where the i-th count value is located, and i is an integer value not greater than the number of tables of the plurality of count lookup tables.

[0013] When the i-th count value is not zero, determine that there is a possibility that the i-th cache way stores a first cache line, and the first cache line is the cache line corresponding to the memory access address.

[0014] According to another aspect of the present application, there is provided a way prediction device for a multi-way set-associative cache. The multi-way set-associative cache includes a plurality of cache groups, each group of the plurality of cache groups includes a plurality of cache ways, the plurality of cache ways correspond to a plurality of count lookup tables one by one, and each count lookup table of the plurality of count lookup tables has a plurality of entries. The device includes the following modules.

[0015] An acquisition module for acquiring a memory access address.

[0016] A generation module for generating an index value corresponding to the memory access address based on the memory access address.

[0017] A search module, configured to access entries corresponding to the index value in each of the count search tables, to obtain a plurality of count values, where the plurality of count values correspond one by one to the plurality of count search tables, and the i-th count value of the plurality of count values indicates the number of cache lines matching the index value in the i-th cache way, the i-th cache way is the cache way corresponding to the count search table where the i-th count value is located, and i is an integer value not greater than the number of the count search tables.

[0018] A determination module, configured to determine, when the i-th count value is not zero, that the i-th cache way has a possibility of storing a first cache line, where the first cache line is the cache line corresponding to the memory access address.

[0019] According to one aspect of the present application, there is provided a processor, which includes a way prediction circuit for implementing the way prediction method of the multi-way set-associative cache as described above.

[0020] According to one aspect of the present application, there is provided a chip, which includes a processor, and the processor includes a way prediction circuit for implementing the way prediction method of the multi-way set-associative cache as described above.

[0021] According to one aspect of the present application, there is provided a computer device, which includes: a processor, and the processor includes a way prediction circuit for implementing the way prediction method of the multi-way set-associative cache as described above. The above computer device may be a terminal device. The terminal device may be a mobile phone, a computer, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal, etc., but is not limited thereto.

[0022] The beneficial effects brought by the technical solution provided by the embodiments of the present application at least include the following content.

[0023] In the embodiments of the present application, a plurality of count search tables are added, and the plurality of count search tables correspond one by one to a plurality of cache ways. Before reading data in the multi-way set-associative cache, the possible cache ways are predicted through the plurality of count search tables first, so as to exclude some cache ways. Compared with the related art that needs to read all cache ways from the cache (multi-way set-associative cache) every time, the way prediction circuit of the present application ensures that when reading from the cache, only the cache ways with non-zero predicted count values are read, rather than necessarily reading all cache ways, thereby reducing the power consumption of accessing the cache and improving the performance of accessing the cache.

[0024] Moreover, compared with the way predictor provided by the related art, the cache way predicted by the related art may be incorrect, and at this time, the access operation needs to be re-executed, while the cache way predicted by the way prediction circuit of the present application must have a correct cache way, ensuring the accuracy of the accessed cache way and eliminating the need to re-execute the access operation. Brief Description of the Drawings

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0026] Figure 1 It is a schematic diagram of a method for a parallel memory access multi-way set-associative cache provided by the related art.

[0027] Figure 2 It is a schematic diagram of the structure of a way prediction circuit for a multi-way set-associative cache provided by an embodiment of the present application.

[0028] Figure 3 It is a schematic diagram of the structure of a way prediction circuit for a multi-way set-associative cache provided by an embodiment of the present application.

[0029] Figure 4 It is a schematic diagram of the principle of a way prediction method for a multi-way set-associative cache provided by the present application.

[0030] Figure 5 It is a flowchart of a way prediction method for a multi-way set-associative cache provided by an embodiment of the present application.

[0031] Figure 6 It is a schematic diagram of performing an exclusive OR operation based on a memory access address provided by an embodiment of the present application.

[0032] Figure 7 It is a flowchart of a method for updating multiple count lookup tables based on a cache line for backfill provided by an embodiment of the present application.

[0033] Figure 8 It is a flowchart of a method for updating multiple count lookup tables based on a replaced cache line provided by an embodiment of the present application.

[0034] Figure 9 It is a flowchart of a method for further accessing a cache provided by an embodiment of the present application.

[0035] Figure 10 It is a schematic diagram of a method for further accessing a cache provided by an embodiment of the present application.

[0036] Figure 11 It is a schematic diagram of a way prediction method for a multi-way set-associative cache provided by an embodiment of the present application.

[0037] Figure 12 It is a schematic diagram of the change process of multiple count lookup tables provided by an embodiment of the present application.

[0038] Figure 13 It is a schematic diagram of the change process of multiple counting lookup tables provided by another embodiment of the present application.

[0039] Figure 14 It is a structural block diagram of the way prediction device of the multi-way set associative cache provided by an embodiment of the present application.

[0040] Figure 15 It is a structural block diagram of the computer device provided by an embodiment of the present application. Specific embodiments

[0041] To make the objectives, technical solutions, and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0042] First, a brief introduction to the nouns involved in the embodiments of the present application is given.

[0043] Cache includes direct mapped cache, N-way set associative cache, and full associative cache.

[0044] Direct mapped cache has only one cache way but multiple cache sets. Full associative cache has multiple cache ways but only one cache set. N-way set associative cache is a cache organization form with multiple cache ways and multiple cache sets. With reference to Figure 1 , Figure 1 shows a two-way set associative cache. The two-way set associative cache has two cache ways, namely way0 and way1, and four cache sets from set0 to set3. The present application mainly focuses on the N-way set associative cache.

[0045] Very Long Instruction Word (VLIW) architecture executes one instruction packet per clock cycle. An instruction packet contains multiple instructions, and the VLIW architecture executes multiple instructions in parallel. These instructions can be scalar instructions, vector instructions, or tensor instructions. Multiple instructions are calculated in parallel to maximize the parallelism of the system. The characteristic of the VLIW architecture is that the instruction packet loaded each time is relatively large, usually up to 128 bit. The instruction packet is stored in the cache. Usually, the processor uses the N-way set associative cache to store the instruction packet, and 128 bit of instructions need to be read from each way of the N-way set associative cache. In the structure of the N-way set associative cache, at most one way hits, and the data read from the remaining ways is invalid.

[0046] To achieve high instruction fetch performance, reading the tag segment and data block in the multi-way set-associative cache is performed in parallel. When reading the data block, since the hit information of the tag segment has not been obtained yet, a common approach is to read out all the data blocks of all cache ways in parallel. Referring to Figure 1 , Figure 1 which shows the scheme of parallelly reading the data blocks of all cache ways.

[0047] Another approach is to add a simple way predictor in the processor to predict the cache ways that may be hit. Specifically, a historical tag table is set up to record the tag fields of the recent memory access addresses in history, as well as the cache ways corresponding to each tag field. However, limited by the size of the table which is always finite, when the number of accesses is large enough, there will always be a situation of tag name duplication, that is, multiple different memory access addresses have the same tag field, but different memory access addresses correspond to different cache ways. When the name duplication problem occurs, an overwrite operation will be performed to overwrite the cache way of the current memory access address with the cache way of the previous memory access address with the same tag. However, after overwriting the previous cache way, the prediction of the table may be wrong. If the wrong cache way is read, subsequent repair needs to be done by re-executing the instruction, that is, re-executing the Figure 1 scheme of parallelly reading all cache ways shown.

[0048] Figure 2 FIG. shows a schematic structural diagram of a way prediction circuit for a multi-way set-associative cache provided by an exemplary embodiment of the present application. The way prediction circuit is located on the processor and logically at the front end of the multi-way set-associative cache. The processor can be a CPU (Central Processing Unit), GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), NPU (Neural-network Processing Unit), etc.

[0049] The way prediction circuit includes an arithmetic circuit unit 201, a read logic circuit unit 202, multiple counting lookup tables 203, and a judgment circuit unit 204. The four circuit units are sequentially connected in series through wires. The multi-way set-associative cache includes multiple cache groups, each of the multiple cache groups includes multiple cache ways, and the multiple cache ways correspond to the multiple counting lookup tables one by one. Each of the multiple counting lookup tables 203 has multiple entries. Referring to Figure 1 , Figure 1Shows a schematic diagram of a two-way set-associative cache. The two-way set-associative cache has two cache ways, namely Way0 and Way1, and four cache sets from Set0 to Set3.

[0050] The arithmetic circuit unit 201 is used to generate an index value corresponding to the memory access address based on the memory access address. The memory access address is the address where the processor accesses the memory. Referring to Figure 1 , the memory access address includes a tag field segment (Tag), a set index field segment (Set), and a byte offset field segment (Byte offset). The tag field segment is used to determine the cache way, the set index field segment is used to indicate the cache set, and the byte offset field segment is used to determine a certain byte in the cache line.

[0051] In one embodiment, the arithmetic circuit unit 201 is further used to perform a hash calculation on at least one field segment in the memory access address to obtain a hash value. The hash value is the index value corresponding to the memory access address, and the number of bits occupied by the hash value is less than the number of bits occupied by the memory access address.

[0052] Optionally, the arithmetic circuit unit 201 is further used to perform a hash calculation on the tag field segment and the set index field segment in the memory access address to obtain a hash value; the set index field segment is used to indicate the first cache set where the first cache line is located in the multi-way set-associative cache, and the tag field segment is used to determine the first cache way where the first cache line is located in the first cache set.

[0053] The read logic circuit unit 202 is used to access the entries corresponding to the index value in each count lookup table to obtain a plurality of count values. The plurality of count values correspond one-to-one to the plurality of count lookup tables 203. The i-th count value of the plurality of count values indicates the number of cache lines matching the index value in the i-th cache way. The i-th cache way is the cache way corresponding to the count lookup table where the i-th count value is located.

[0054] Figure 2 Shows four count lookup tables (Table 0, Table 1, Table 2, and Table 3). Each count lookup table includes a plurality of entries (cnt0, cnt1 to cnt_n-1). The corresponding entries are found from each count lookup table according to the index value. For example, if the index value is 0010, the second entry is obtained from each count lookup table to get the count value cnt2, and a total of four cnt2 are obtained.

[0055] The determination circuit unit 204 is configured to determine whether multiple count values are zero respectively. When the i-th count value is not zero, there is a possibility that the i-th cache line stores the first cache line; when the i-th count value is zero, the i-th cache line does not store the first cache line, and the first cache line is the cache line corresponding to the memory access address. The determination circuit unit 204 is configured to determine whether multiple count values are zero respectively to generate a way enable signal. The way enable signal includes multiple bit positions, and each bit position corresponds to each of the multiple cache ways. When the i-th count value is not zero, it is determined that the bit position corresponding to the i-th cache way is one; when the i-th count value is zero, it is determined that the bit position corresponding to the i-th cache way is zero.

[0056] For example, the multiple count values include four count values, the 0-th count value is zero, the 1-st count value is one, the 2-nd count value is two, and the 3-rd count value is zero, then the way enable signal is 0110.

[0057] In summary, in the embodiment of the present application, multiple count lookup tables are added. The multiple count lookup tables correspond to the multiple cache ways one by one. Before reading data in the multi-way set-associative cache, the possible cache ways are predicted first through the multiple count lookup tables, so as to exclude some cache ways. Compared with the related art where all cache ways need to be read from the cache (multi-way set-associative cache) each time, the method of the present application ensures that when reading from the cache, only the cache ways with non-zero predicted count values are read, rather than necessarily reading all cache ways, thereby reducing the power consumption of accessing the cache and improving the performance of accessing the cache.

[0058] Moreover, compared with the way predictor provided by the related art, the predicted cache ways in the related art may be incorrect, and at this time, the access operation needs to be re-executed. However, the cache ways predicted by the method of the present application must include the correct cache ways, ensuring the accuracy of the accessed cache ways and eliminating the need to re-execute the access operation.

[0059] Figure 3 The structural schematic diagram of the way prediction circuit of the multi-way set-associative cache provided by an exemplary embodiment of the present application is shown. The way prediction circuit 200 further includes an update logic circuit unit 206.

[0060] For the case of cache line backfill (at this time, the cache line backfill logic circuit unit has run):

[0061] The arithmetic circuit unit 201 is further configured to calculate the first index value corresponding to the first cache line when all the multiple count values are zero and the first cache line is backfilled to the multi-way set-associative cache.

[0062] If multiple count values are all zero, indicating that the first cache line is not stored in the current set-associative cache (i.e., a cache miss occurs), and the first cache line is the cache line corresponding to the memory access address, then the first cache line needs to be filled back from the lower-level cache or the main memory into the set-associative cache.

[0063] In one embodiment, the arithmetic circuit unit 201 is further configured to perform a hash calculation on the first flag field segment and the first set index field segment to obtain a first hash value, where the first hash value is the first index value corresponding to the first cache line, the first flag field segment is the flag field segment corresponding to the first cache line, and the first set index field segment is the set index field segment corresponding to the first cache line.

[0064] The update logic circuit unit 206 is configured to increment the count value of the entry indicated by the first index value in the count lookup table corresponding to the first cache way, where the first cache way is the cache way where the first cache line is filled back into the set-associative cache.

[0065] Illustratively, the first cache way is Way1, Way1 corresponds to HCT1 (count lookup table 1), the first index value is 0010, then the count value of the entry indicated by the index value 0010 in HCT1 is incremented by one.

[0066] In summary, the above embodiments provide a method for synchronously maintaining multiple count lookup tables when "cache line filling back" occurs. Based on the cache set and tag segment corresponding to the filled-back cache line, the above embodiments generate an index value, and based on the index value, the corresponding entry in the count lookup table can be determined, providing a method for maintaining the count lookup table.

[0067] For the case of cache line replacement (when the cache line replacement logic circuit unit has already run):

[0068] The arithmetic circuit unit 201 is further configured to calculate a second index value corresponding to the second cache line in the case where the second cache line in the set-associative cache is replaced, where the second cache line is a valid cache line;

[0069] In one embodiment, if it is determined to perform the filling-back operation of the first cache line, and at this time there are valid cache lines in each cache way of the first cache set of the set-associative cache, where the first cache set is the cache set corresponding to the first cache line, then a cache line to be replaced needs to be determined from the first cache set.

[0070] Optionally, the least recently used cache line is determined to be replaced from the first cache set.

[0071] Optionally, the replaced cache line is saved to the lower-level cache of the set-associative cache.

[0072] The second cache line is the cache line that is replaced by the first cache line when there is a valid cache line in each cache way of the first cache group. The second cache line is the least recently used cache line among the multiple cache ways of the first cache group. The first cache group is the cache group corresponding to the first cache line, and the second cache line is saved to the lower-level cache of the multi-way set-associative cache.

[0073] The update logic circuit unit 206 is further configured to decrement the count value of the entry indicated by the second index value in the count lookup table corresponding to the second cache way, where the second cache way is the cache way in the multi-way set-associative cache where the second cache line is located.

[0074] Illustratively, the second cache way is Way2, Way2 corresponds to HCT2 (count lookup table 2), and the second index value is 0010. Then, in HCT2, the count value of the entry indicated by the index value 0010 is decremented.

[0075] In summary, the above embodiments provide a method for synchronously maintaining multiple count lookup tables when a "cache line replacement" occurs. The above embodiments generate an index value based on the cache group (set) and tag segment corresponding to the replaced cache line, and the corresponding entry in the count lookup table can be determined based on the index value, providing a way to maintain the count lookup table.

[0076] In the above embodiment, the arithmetic circuit unit 201 performs the hash calculation in the following manner.

[0077] The arithmetic circuit unit 201 is further configured to divide every p bits in the to-be-operated address into a group, obtaining q bit groups; in the j-th exclusive-or process, perform an exclusive-or operation on every two of the (j - 1)-th exclusive-or results to obtain the j-th exclusive-or result. The (j - 1)-th exclusive-or result includes multiple bit groups, and the j-th exclusive-or result includes at least one bit group. The pairwise exclusive-or operation means exhausting the multiple bit groups included in the (j - 1)-th exclusive-or result and performing an exclusive-or operation on the bits in the same position in every two bit groups. When j is equal to 1, the (j - 1)-th exclusive-or result is q bit groups. When the j-th exclusive-or result includes a single bit group, the value indicated by the single bit group is determined as the hash value. The to-be-operated address includes a tag field segment and a set index field segment.

[0078] Figure 4 FIG. shows a schematic diagram of the principle of the cache way prediction method of the multi-way set-associative cache provided by an exemplary embodiment of the present application. Figure 4The path prediction method shown is implemented by a path prediction circuit (circuit structure) in a processor. The processor can be a CPU (Central Processing Unit), GPU (Graphics Processing Unit), TPU (Tensor Processing Unit), NPU (Neural-network Processing Unit), and so on.

[0079] Obtain the memory access address 401, in combination with reference Figure 1 , the memory access address 401 includes a tag field segment (Tag field segment), a set index field segment (Set field segment), and a byte offset field segment (Byte offset field segment). The memory access address 401 is the address where the processor accesses the memory. The memory (broad sense of memory) includes multiple levels of caches and the main memory. The processor preferentially obtains data from the cache. If the first cache line (the first cache line is the cache line corresponding to the memory access address 401) is not stored in the cache, then the first cache line is obtained from the lower-level cache (or the main memory). In this application, the considered cache is a multi-way set-associative cache. The multi-way set-associative cache includes multiple cache sets, and each cache set includes multiple cache ways. In combination with reference Figure 1 , Figure 1 shows a schematic diagram of a two-way set-associative cache. The two-way set-associative cache has two cache ways, namely Way0 and Way1, and four cache sets from Set0 to Set3.

[0080] A cache line is the minimum transfer unit of data between the cache and the main memory. A cache line is a Data block determined by the intersection of a cache way and a cache set. The set index field segment in the memory access address 401 is used to indicate the first cache set where the first cache line is located in the multi-way set-associative cache, and the tag field segment in the memory access address 401 is used to determine the first cache way where the first cache line is located in the first cache set. The byte offset field segment in the memory access address 201 is used to indicate reading a certain byte from the first cache line.

[0081] Based on the memory access address 401, generate an index value 402 shared by multiple count lookup tables. The multiple count lookup tables 403 correspond one-to-one with multiple cache ways in the multi-way set-associative cache. In combination with reference Figure 4 , taking the multi-way set-associative cache as a four-way set-associative cache as an example, the four-way set-associative cache has four cache ways (way0, way1, way2, and way3). Figure 4Four count lookup tables (HCT0, HCT1, HCT2, and HCT3) corresponding to four cache ways are shown. The index value 402 is generated based on the memory access address 401, that is, there is a mapping relationship between the index value 402 and the memory access address 401. Optionally, multiple memory access addresses are mapped to the same index value, or different memory access addresses are mapped to different index values.

[0082] Read the entries corresponding to the index value 402 in each count lookup table to obtain multiple count values. Each count lookup table includes multiple entries, and each entry stores a count value. Figure 4 Shows the count values cnt0, cnt1 to cnt in one count lookup table n-1 . For one count lookup table, based on the index value 402, a single entry is uniquely determined from multiple entries to obtain the count value in that entry. For example, if the index value is 0010, the second entry is determined from multiple entries, and the count value cnt2 of the second entry is obtained. Therefore, multiple cnt2 values of the second entry can be obtained from multiple count lookup tables.

[0083] Multiple count values are obtained from multiple count lookup tables. The i-th count value among the multiple count values represents the number of cache lines in the i-th cache way that match the index value. The i-th cache way is the cache way corresponding to the count lookup table where the i-th count value is located. If the i-th count value is not zero, it indicates the possibility that the i-th cache way stores the first cache line (the first cache line is the cache line corresponding to the memory access address). At this time, it cannot be determined whether the i-th cache way stores the first cache line. Then, the i-th cache way in the multi-way set-associative cache is accessed to further determine whether the i-th cache way is the correct cache way. If the i-th count value is zero, it indicates that the i-th cache way does not store the first cache line, and then there is no need to access the i-th cache way in the multi-way set-associative cache.

[0084] In one example, if two memory access addresses are mapped to the same index value, the i-th count value can be used as the basis for judging whether both memory access addresses exist in the i-th cache way. Suppose the two memory access addresses are memory access address 0 and memory access address 1, and memory access address 0 is input at the current moment. If the i-th count value is zero, it can be determined that the cache line corresponding to memory access address 0 must not be in the i-th cache way; if the i-th count value is not zero, it cannot be determined whether the cache line corresponding to memory access address 0 is in the i-th cache way. In this case, the tag segment in the i-th cache way also needs to be accessed, and it is judged whether the i-th cache way is the correct cache way by whether the tag segment is consistent with the tag field segment in memory access address 0 (for the method introduction, please refer to Figure 1 .

[0085] It should be noted that the information involved in this application (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions. For example, the memory access addresses involved in this application are obtained under sufficient authorization.

[0086] Moreover, regarding the relevant information, the relevant information processors will follow the principles of legality, propriety, and necessity, clarify the purpose, method, and scope of relevant information processing, obtain the consent of the relevant information subjects, and take necessary technical and organizational measures to ensure the security of relevant information.

[0087] Figure 5 The flowchart of the way prediction method for the multi-way set-associative cache provided by an exemplary embodiment of this application is shown. Taking the implementation of this method by the way prediction circuit in the processor as an example, this method includes:

[0088] Step 510, obtain the memory access address;

[0089] The memory access address is the address where the processor accesses the memory. In a broad sense, the memory includes multiple levels of caches (Cache) and the main memory (Main Memorry). The cache includes direct mapped cache, multi-way set-associative cache, and fully associative cache. The multi-way set-associative cache includes multiple cache groups, and each group of the multiple cache groups includes multiple cache ways. In this application, multiple counting lookup tables are added, and the multiple counting lookup tables correspond to the multiple cache ways one by one. Each counting lookup table in the multiple counting lookup tables has multiple entries.

[0090] In this application, the memory access address will be used to access the multi-way set-associative cache. Optionally, the multi-way set-associative cache is the instruction cache in the AI processor. The AI processor is usually designed using the very long instruction word architecture. Under the very long instruction word architecture, one instruction packet is executed per clock cycle, and one instruction packet contains multiple instructions. Multiple instructions are executed in parallel under the very long instruction word architecture. These instructions can be scalar instructions, vector instructions, or tensor instructions. Multiple instructions are calculated in parallel to improve the parallelism of the system as much as possible. The characteristic of the very long instruction word architecture is that the loaded instruction packet is relatively large each time, usually up to 128bit, and the instruction packet is stored in the multi-way set-associative cache.

[0091] Step 520, based on the memory access address, generate an index value corresponding to the memory access address;

[0092] Based on the memory access address, map to obtain the index value shared by the multiple counting lookup tables. Optionally, multiple memory access addresses can be mapped to the same index value; optionally, different memory access addresses are mapped to different index values.

[0093] In one embodiment, a hash calculation is performed on at least one field segment in the memory access address to obtain a hash value, where the number of bits occupied by the hash value is less than the number of bits occupied by the memory access address; and the hash value is determined as the index value corresponding to the memory access address.

[0094] At this time, the hash calculation is an address folding operation of the memory access address, and the hash calculation maps the memory access address with more bits to the hash value with fewer bits. The hash calculation may result in two or more memory access addresses corresponding to the same hash value (index value). Therefore, the hash calculation reduces the depth of the counting lookup table, and instead of setting up an entry for each memory access address, it allows multiple memory access addresses to share one entry.

[0095] Schematically, the memory access address includes a tag field segment and a set index field segment. The set index field segment is used to indicate the first cache set where the first cache line is located in the multi-way set-associative cache, and the tag field segment is used to determine the first cache way where the first cache line is located in the first cache set. The first cache line is the cache line corresponding to the memory access address. The tag field segment and the set index field segment in the memory access address are obtained; and a hash calculation is performed on the tag field segment and the set index field segment to obtain a hash value. The specific hash calculation process will be introduced below.

[0096] Step 530: Access the entries corresponding to the index value in each counting lookup table to obtain a plurality of count values. The plurality of count values correspond to the plurality of counting lookup tables one by one. The i-th count value of the plurality of count values indicates the number of cache lines matching the index value in the i-th cache way. The i-th cache way is the cache way corresponding to the counting lookup table where the i-th count value is located, and i is an integer value not greater than the number of tables of the plurality of counting lookup tables.

[0097] In this application, a plurality of counting lookup tables are added. Each counting lookup table in the plurality of counting lookup tables has a plurality of entries, and each entry can be accessed through a uniquely corresponding index value. Schematically, for the counting lookup table HCT0, if the index value is 0010, the count value cnt2 is obtained from the second entry. For each counting lookup table, the count value cnt2 is obtained respectively, and finally a plurality of count values can be obtained.

[0098] The i-th count value among multiple count values indicates the number of cache lines in the i-th cache line that match the index value. For example, when accessing the entry with an index value of 0010 in each count lookup table among multiple count lookup tables, the 0-th count value (corresponding to the 0-th cache way), the 1-st count value (corresponding to the 1-st cache way), the 2-nd count value (corresponding to the 2-nd cache way), and the 3-rd count value (corresponding to the 3-rd cache way) are obtained. If the 0-th count value is zero, it means the number of cache lines in the 0-th cache way that match the index value 0010 is zero. If the 0-th count value is one, it means the number of cache lines in the 0-th cache way that match the index value 0010 is one. If the 0-th count value is two, it means the number of cache lines in the 0-th cache way that match the index value 0010 is two.

[0099] Step 540, when the i-th count value is not zero, determine that there is a possibility that the i-th cache way stores the first cache line, where the first cache line is the cache line corresponding to the memory access address.

[0100] If the i-th count value is not zero, determine that the i-th cache way may store the first cache line. If the i-th count value is zero, determine that the i-th cache way does not store the first cache line.

[0101] Suppose the 0-th count value is zero, the 1-st count value is one, the 2-nd count value is two, and the third count value is zero. Then determine that the 0-th cache way (Way0) and the 3-rd cache way (Way3) do not store the first cache line, and the 1-st cache way (Way1) and the 2-nd cache way (Way2) may store the first cache line. Optionally, afterwards, the final cache way is also determined in the 1-st cache way and the 2-nd cache way.

[0102] In one example, if two memory access addresses are mapped to the same index value, the i-th count value can be used as the basis for judging whether both memory access addresses exist in the i-th cache way. Suppose the two memory access addresses are memory access address 0 and memory access address 1, and memory access address 0 is input at the current moment. If the i-th count value is zero, it can be determined that the cache line corresponding to memory access address 0 must not be in the i-th cache way; if the i-th count value is not zero, it can be determined that at least one of the cache lines of memory access address 0 and memory access address 1 exists in the i-th cache way, but it is still impossible to determine whether the cache line corresponding to memory access address 0 is in the i-th cache way. In this case, the tag segment in the i-th cache way still needs to be accessed, and it is judged whether the i-th cache way is the target cache way by whether the tag segment is consistent with the tag field segment in memory access address 0 (for the method introduction, refer to Figure 1 ).

[0103] In summary, in the embodiments of the present application, multiple counting lookup tables are added. The multiple counting lookup tables correspond to multiple cache ways one by one. Before reading data in a multi-way set-associative cache, the possible cache ways are predicted first through the multiple counting lookup tables, so as to exclude some cache ways. Compared with the related art that needs to read all cache ways from the cache (multi-way set-associative cache) each time, the method of the present application ensures that when reading from the cache, only the cache ways with non-zero predicted count values are read, rather than necessarily reading all cache ways, thereby reducing the power consumption of accessing the cache and improving the performance of accessing the cache.

[0104] Moreover, compared with the way predictor provided by the related art, the cache ways predicted by the related art may be incorrect, and at this time, the access operation needs to be executed again. However, among the cache ways predicted by the method of the present application, there must be correct cache ways, ensuring the accuracy of the accessed cache ways and eliminating the need to execute the access operation again.

[0105] Based on Figure 5 the optional embodiments shown, optionally, the hash calculation includes the following calculation process.

[0106] 1. Combine the tag field segment and the set index field segment to obtain the address to be operated on;

[0107] 2. Divide every p bits in the address to be operated on into a group to obtain q bit groups;

[0108] 3. In the j-th exclusive OR process, perform an exclusive OR operation on every two of the (j - 1)-th exclusive OR results to obtain the j-th exclusive OR result. The (j - 1)-th exclusive OR result includes multiple bit groups, and the j-th exclusive OR result includes at least one bit group. The exclusive OR operation on every two means exhausting the multiple bit groups included in the (j - 1)-th exclusive OR result and performing an exclusive OR operation on the bits in the same position in every two bit groups; when j is equal to 1, the (j - 1)-th exclusive OR result is q bit groups, and when the j-th exclusive OR result includes a single bit group, the value indicated by the single bit group is determined as the hash value.

[0109] Illustratively, if the depth of the counting lookup table is set to 16 (that is, each counting lookup table includes 16 entries), then the value of p is determined to be 4. Assume that the address to be operated on is 16 bits, then every 4 bits (p bits) are divided into a group to obtain 4 bit groups (q bit groups). Perform an exclusive OR operation on the 4 bit groups until finally a single bit group is obtained, and the value indicated by the single bit group is determined as the hash value, and the hash value is determined as the shared index value of the multiple counting lookup tables.

[0110] Illustratively, with reference to Figure 6, the flag field segment and the set index field segment in the memory access address are combined to obtain the address "0110 0010 1100 0101" (the address to be operated on) participating in the hash calculation. The address to be operated on includes 16 bits. In the address to be operated on, every four bits are divided into a group, and a total of four bit groups are obtained.

[0111] Among the four bit groups, every two bit groups are subjected to an exclusive OR operation to obtain an exclusive OR result. At this time, the exclusive OR result includes two bit groups. Figure 6 The middle bit group "0110" and the bit group "0010" are subjected to an exclusive OR operation to obtain the bit group "0100", and the bit group "1100" and the bit group "0101" are subjected to an exclusive OR operation to obtain the bit group "1001". Then, an exclusive OR operation is performed between the two bit groups "0100" and "1001" to obtain the bit group "1101". The exclusive OR operation between two bit groups refers to the operation of performing an exclusive OR on two bits with the same position between the two bit groups.

[0112] The present application provides a plurality of counting lookup tables. The plurality of counting lookup tables correspond one by one to a plurality of cache ways in the set-associative cache. The plurality of counting lookup tables are used to reflect the current state of the set-associative cache in real time. Therefore, when the cache lines stored in the set-associative cache change, it is also necessary to update and maintain the plurality of counting lookup tables. Next, the update methods of the plurality of counting lookup tables when "cache line filling" and "cache line replacement" occur will be introduced.

[0113] Figure 7 The flowchart of the maintenance method of the plurality of counting lookup tables provided by an exemplary embodiment of the present application is shown. Taking the implementation of this method by the way prediction circuit in the processor as an example, this method includes:

[0114] Step 710, when all the plurality of count values are zero, determine to perform a cache line filling operation;

[0115] Based on Figure 5 The method shown, a plurality of count values are obtained by looking up in the plurality of counting lookup tables. If all the plurality of count values are zero, it means that the first cache line is not stored in the current set-associative cache (that is, the case of cache miss occurs). The first cache line is the cache line corresponding to the memory access address, and then the first cache line needs to be filled back from the lower-level cache or the main memory to the set-associative cache.

[0116] Step 720, when the first cache line is filled back to the set-associative cache, calculate the first index value corresponding to the first cache line;

[0117] After the first cache line is backfilled into the set-associative cache, multiple count lookup tables need to be updated. In one embodiment, an index value shared by multiple count lookup tables corresponding to the first cache line is calculated to obtain a first index value. The first index value is used to determine the corresponding entry from the count lookup table.

[0118] In one embodiment, the tag and set index corresponding to the first cache line are known. Determine the tag field segment corresponding to the first cache line to obtain a first tag field segment; and, determine the set index field segment corresponding to the first cache line to obtain a first set index field segment; perform a hash calculation on the first tag field segment and the first set index field segment to obtain a first hash value; determine the first hash value as the first index value corresponding to the first cache line. Specifically, the calculation process of the hash calculation can refer to the above text.

[0119] Step 730, determine the first cache way to which the first cache line is backfilled into the set-associative cache;

[0120] The first cache line is backfilled into the first cache way according to the way allocation policy.

[0121] In one case, if there is an idle cache way in the first cache set of the set-associative cache, then backfill the first cache line into the idle cache way, and the first cache set is the cache set where the first cache line is located.

[0122] In another case, if all the cache ways in the first cache set of the set-associative cache are occupied, then replace the least recently used cache way.

[0123] Step 740, in the count lookup table corresponding to the first cache way, increment the count value of the entry indicated by the first index value by one.

[0124] Illustratively, the first cache way is Way1, Way1 corresponds to HCT1 (count lookup table 1), and the first index value is 0010, then increment the count value of the entry indicated by the index value 0010 in HCT1 by one.

[0125] In summary, the above embodiments provide a way to synchronously maintain multiple count lookup tables when "cache line backfill" occurs. The above embodiments generate an index value based on the cache set (set) and tag segment (tag) corresponding to the backfilled cache line, and can determine the corresponding entry in the count lookup table based on the index value, providing a way to maintain the count lookup table.

[0126] Figure 8 The flowchart of the method for maintaining multiple count lookup tables provided by an exemplary embodiment of the present application is shown. Taking the method being implemented by the way prediction circuit in the processor as an example, the method includes:

[0127] Step 810, when the second cache line in the multi-way set-associative cache is replaced, calculate the second index value corresponding to the second cache line, where the second cache line is a valid cache line;

[0128] In one embodiment, if it is determined to perform the backfill operation of the first cache line, and at this time there is a valid cache line in each cache way of the first cache set of the multi-way set-associative cache, and the first cache set is the cache set corresponding to the first cache line, then a cache line to be replaced needs to be determined from the first cache set.

[0129] Optionally, determine that the least recently used cache line in the first cache set is replaced.

[0130] Optionally, save the replaced cache line to the lower-level cache of the multi-way set-associative cache.

[0131] When the second cache line is replaced, calculate the second index value corresponding to the second cache line.

[0132] In one embodiment, the tag and set corresponding to the second cache line are known. Determine the tag field segment corresponding to the second cache line to obtain the second tag field segment; and, determine the set index field segment corresponding to the second cache line to obtain the second set index field segment; perform a hash calculation on the second tag field segment and the second set index field segment to obtain a second hash value; determine the second hash value as the second index value corresponding to the second cache line. Specifically, the calculation process of the hash calculation can refer to the above.

[0133] Step 820, determine the second cache way where the second cache line is located in the multi-way set-associative cache;

[0134] The second cache line is in the second cache way in the multi-way set-associative cache.

[0135] Step 830, in the count lookup table corresponding to the second cache way, decrement the count value of the entry indicated by the second index value by one.

[0136] Illustratively, the second cache way is Way2, Way2 corresponds to HCT2 (count lookup table 2), and the second index value is 0010, then in HCT2, decrement the count value of the entry indicated by the index value 0010 by one.

[0137] In summary, the above embodiments provide a method for synchronously maintaining multiple count lookup tables when a "cache line replacement" occurs. The above embodiments generate an index value based on the cache set (set) and tag segment corresponding to the replaced cache line, and the corresponding entry in the count lookup table can be determined based on the index value, providing a method for maintaining the count lookup table.

[0138] The above Figure 5In the illustrated method embodiment, multiple count values will respectively indicate the possibility that multiple cache ways store the first cache line (the cache line corresponding to the memory access address). A count value of zero indicates that the corresponding cache way does not store the first cache line; a non-zero count value indicates that the corresponding cache line may store the first cache line. Therefore, the multiple count values may predict all cache ways, some cache ways, or even no cache ways in a multi-way set-associative cache. However, the predicted cache ways must include the first cache line.

[0139] Since there is a situation where more than one cache way is predicted, it is necessary to further determine the correct cache way based on the predicted more than one cache way.

[0140] Based on Figure 5 In the optional embodiment shown, Figure 9 The flowchart of a method for further determining the final cache way is shown. The method includes:

[0141] Step 910, determining at least one non-zero count value among the multiple count values; and, based on the set index field segment in the memory access address, determining the first cache set in the multi-way set-associative cache;

[0142] In one case, there is at least one non-zero count value among the multiple count values. For example, there are 1, 2, 3, etc. non-zero count values. The set index field segment in the memory access address is used to indicate the first cache set where the first cache line is located in the multi-way set-associative cache.

[0143] In one embodiment, based on the multiple count values, a way enable signal is generated. The way enable signal includes multiple bit positions, and each bit position corresponds to each cache way among the multiple cache ways. If the i-th count value among the multiple count values is non-zero, determine that the bit position of the i-th cache way is one; if the i-th count value is zero, determine that the bit position of the i-th cache way is zero. For example, the multiple count values include four count values, the 0-th count value is zero, the 1-st count value is one, the 2-nd count value is two, and the 3-rd count value is zero, then the way enable signal is 0110.

[0144] Combined with reference to Figure 10 , Figure 10 It shows that the count value of HCT0 is 0, the count value of HCT1 is 1, the count value of HCT2 is 2, the count value of HCT3 is 0, and the way enable signal is 0110.

[0145] Step 920, for the k-th count value among the at least one non-zero count values, based on the first cache set and the k-th cache way corresponding to the count lookup table where the k-th count value is located, determine the k-th tag segment and the k-th data block in the multi-way set-associative cache;

[0146] For each of at least one non-zero count value, based on the first cache group and the cache way corresponding to each count value, determine the flag segment and data block corresponding to each count value in the multi-way set-associative cache. Refer to Figure 8 , based on the set field segment in the memory access address, determine the first cache group (set2), and way1 and way2 indicated by the way enable signal, and determine the Tag segment and data block of way1, and the Tag segment and data block of way2 in the four-way set-associative cache.

[0147] Step 930, access the k-th flag segment and the k-th data block;

[0148] Access the flag segment and data block corresponding to each count value.

[0149] Step 940, when the flag indicated by the k-th flag segment is consistent with the flag indicated by the flag field segment in the access address, determine the k-th data block as the first cache line.

[0150] From at least one cache way corresponding to at least one non-zero count value, determine the cache way in which the flag indicated by the flag segment is consistent with the flag indicated by the flag field segment, and determine the data block in this cache way as the first cache line. Schematically, refer to Figure 10 , from way1 and way2, determine the cache way in which the tag is consistent with the tag field segment obtained from the memory access address, and determine the data block of this cache way as the first cache line.

[0151] In one embodiment, in the first cache group, based on the way enable signal, read the flag segment and data block of at least one way with the bit being one. Each cache way in the at least one way corresponds to a flag segment and a data block; from the at least one way, determine the cache way in which the flag indicated by the flag segment is consistent with the flag indicated by the flag field segment of the memory access address, generate a way selection signal. The way selection signal includes multiple bits, each bit corresponding to each of the multiple cache ways, and a bit being one indicates that the corresponding cache way is selected; input the way selection signal into a multiplexer; and input at least one data block of the at least one way into the multiplexer; under the indication of the way selection signal, select the data block of the selected cache way from the at least one data block to obtain the first cache line.

[0152] Schematically, if the way enable signal is 0110, then the tag segment and data block of the first cache way are read, and the tag segment and data block of the second cache way are read. If the tag indicated by the tag segment of the first cache way is consistent with the tag indicated by the tag field segment in the memory access address, then it is determined that the first cache way is selected, and a way selection signal 0010 is generated. The way selection signal is input into a four-to-one selector (4-to-1 MUX). The data block of the first cache way and the data block of the second cache way are also input into the four-to-one selector, and under the indication of the way selection signal, the data block of the first cache way is selected as the first cache line.

[0153] Figure 11 The figure shows a schematic diagram of a way prediction method for a multi-way set-associative cache provided by an exemplary embodiment of the present application. The memory access address 1101 is obtained, and a hash operation 1102 is performed according to the tag field segment (tag field segment) and set index field segment (set field segment) in the memory access address to obtain a hash value. The hash value is the index value of multiple count lookup tables (HCTs), and a read logic 1103 is executed in multiple count lookup tables through the index value. Figure 11 Taking a four-way set-associative cache as an example, Figure 11 The figure shows four count lookup tables 1104, and each count lookup table corresponds to one way in the four-way set-associative cache. In one embodiment, if the index value is represented by four bits, then the depth of each HCT is 16, that is, each HCT includes entries from cnt0 to cnt15.

[0154] Assume the index value is m, and the m-th entry is looked up from each count lookup table to obtain the m-th count value. The m-th count value represents the number of cache lines corresponding to the index value m in the current way. A cache line is the minimum unit of data transfer between the cache and the main memory. A cache line, that is, Figure 1 the data block determined by the intersection of the way and the set.

[0155] For each count lookup table, if the m-th count value is zero, it means that there is no cache line of the memory access address in the current way. If it is not zero, it means that there may be a cache line of the memory access address in the current way.

[0156] Figure 11 In this case, it is respectively determined whether the four count values obtained by looking up the 4 HCTs are zero. If it is zero, the corresponding bit in the way enable signal 1105 is 0. If it is not zero, the corresponding bit in the way enable signal 1105 is 1. For example, if the way enable signal 1105 is "0110", it means that there may be cache lines of the memory access address in the first way and the second way, and there are no cache lines of the memory access address in the zero-th way and the third way.

[0157] Under the instruction of the way enable signal 1105, the flag segments 1106 of the four ways in the four-way set-associative cache and the data blocks 1107 of the four ways in the four-way set-associative cache are read in parallel. If the way enable signal 1105 indicates that the first way and the second way are predicted ways, only the flag segments and data blocks of the first way and the second way are read. For example, the flag segments and data blocks are read in parallel in the first way and the second way of set 2.

[0158] According to the read flag segment, a flag comparison is performed to generate a way selection signal 1108, and the way selection signal 1108 is input into the four-to-one selector 1109. Specifically, whether the correct way is hit is determined based on whether the read flag segment is consistent with the tag indicated by the tag field segment in the access address. For example, in the flag segment of the first way and the flag segment of the second way, the flag segment of the first way is consistent with the tag indicated by the tag field segment in the access address, and the first way is determined to be the correct way. Based on the fact that the first way is the correct way, the way selection signal 1108 is generated as "0010".

[0159] Under the instruction of the path enable signal 1105, at least one of the read data blocks is input into the four-to-one selector 1109. For example, under the instruction of the path enable signal 1105, only the data blocks of the first path and the data blocks of the second path are read, and the data blocks of the first path and the data blocks of the second path are input into the four-to-one selector 1109.

[0160] Under the instruction of the way selection signal 1108, the four-to-one selector 1109 selects the data block of the correct way, for example, selects the data block of the first way.

[0161] Figure 11 Two situations where updates to the four count lookup tables 1104 are required are also shown.

[0162] In one case, if there is no cache line corresponding to the memory access address in the current four-way set-associated cache, it is necessary to backfill the cache line corresponding to the memory access address from the lower-level cache. The backfill operation will cause changes in the data in the four-way set-associated cache, so the four additional count lookup tables 1104 need to be updated. The four count lookup tables 1104 reflect the real-time status of the four-way set-associated cache. When updating, a hash operation 1102 is performed based on the backfilled cache line to calculate the hash value, and the update logic 1110 is executed based on the hash value. Specifically, the count value of the entry corresponding to the hash value is increased by one.

[0163] In another case, a replacement operation is also performed. If there is valid data in all four ways (i.e., they are occupied), then one of the cache lines needs to be replaced to accommodate the cache line corresponding to the memory access address backfilled from the lower-level cache. The replaced cache line will be written out to the lower-level cache, and the replaced cache line will also cause changes to the data in the multi-way set-associative cache. Therefore, it is necessary to update the four additional counting lookup tables 1104. When updating, a hash operation 1102 is performed according to the replaced cache line to calculate the hash value, and an update logic 1110 is executed according to the hash value. Specifically, the count value of the entry corresponding to the hash value is decremented by one.

[0164] Figure 12 FIG. shows a schematic diagram of the change process of typical multiple counting lookup tables provided by an exemplary embodiment of the present application.

[0165] At time T0, just powered on, as Figure 12 shown in part (A) of, at this time, all four counting lookup tables (HCT0, HCT1, HCT2, and HCT3) are empty, that is, there is no valid cache line in the cache. At this time, an access request for the memory access address addr0 is received, and a hash calculation is performed on the memory access address addr0 to obtain a hash value of 2. The four counting lookup tables are accessed, and the entry corresponding to the hash value of 2 is read, that is, the four cnt2 of the four counting lookup tables. It is found that they are all 0, indicating that the cache line (cacheline) corresponding to addr0 has not been loaded into the cache yet, and a backfill operation is required.

[0166] At time T1, the cache miss backfill of addr0 is completed, and way1 is selected for backfill according to the way selection policy. The multiple counting lookup tables are updated. Way1 corresponds to HCT1, so the cnt2 in HCT1 is incremented by 1, and cnt2 is updated to 1, as Figure 12 shown in part (B) of.

[0167] At time T2, the memory access address addr0 requests the cache again. At this time, it is found that the cnt2 of HCT1 is 1, and the cnt2 of the other HCTs are all 0, indicating that the data is in way1, and a way enable signal way_sel = 0010 is generated.

[0168] At time T3, an access request for the memory access address addr1 is received. At this time, a cache miss also occurs for the memory access address addr1. Therefore, a backfill is required, and the cache line corresponding to addr1 is backfilled to way2 of the cache, and the multiple counting lookup tables are updated. Way2 corresponds to HCT2 in the multiple counting lookup tables, so the cnt2 in HCT2 is incremented by 1, and cnt2 is updated to 1, as Figure 12 shown in part (C) of.

[0169] At time T4, the memory access address addr0 requests the cache again. At this time, it is found that the cnt2 of HCT1 and HCT2 is 1, and the cnt2 of the remaining HCTs is 0, indicating that the data is in way1 or way2. The way enable signal way_sel = 0110 is generated.

[0170] At time T5, since the memory access address addr0 is replaced out of the cache, the counting lookup table is updated, and the count value of the corresponding entry in the counting lookup table is decremented by 1. The memory access address addr0 corresponds to the cnt2 in HCT1. Therefore, the cnt2 in HCT1 is decremented by 1, and the cnt2 is updated to 0, as Figure 12 shown in part (D).

[0171] Next, the update method of multiple counting lookup tables in the case of name duplication will be introduced. Figure 13 It shows a schematic diagram of the change process of multiple counting lookup tables in the case of name duplication provided by an exemplary embodiment of the present application.

[0172] At time T0, just powered on, as Figure 13 shown in part (A). At this time, all four counting lookup tables (HCT0, HCT1, HCT2, and HCT3) are empty, that is, there is no valid cache line in the cache. At this time, a memory access request for the memory access address addr0 is received, and a hash calculation is performed on the memory access address addr0 to obtain a hash value of 2. The four counting lookup tables are accessed, and the entries corresponding to the hash value of 2 are read, that is, the four cnt2 of the four counting lookup tables. It is found that they are all 0, indicating that the cache line (cacheline) corresponding to addr0 has not been loaded into the cache yet, and a fill operation is required.

[0173] At time T1, the cache miss for addr0 is filled. According to the way selection strategy, way1 is selected for filling, and multiple counting lookup tables are updated. Way1 corresponds to HCT1. Therefore, the cnt2 in HCT1 is incremented by 1, and the cnt2 is updated to 1, as Figure 13 shown in part (B).

[0174] At time T2, a memory access request for the memory access address addr1 is received. At this time, the memory access address addr1 also has a cache miss. Therefore, a fill operation is required. The cache line corresponding to addr1 is filled into way1 of the cache, and multiple counting lookup tables are updated. The cnt2 in HCT2 is incremented by 1, and the cnt2 is updated to 2, as Figure 13 shown in part (C).

[0175] At this time, there is a situation where the memory access addresses add0 and addr1 have the same name. Their hash values are the same, both being 2, and both are mapped to way1.

[0176] At time T3, the cache line of the memory access address addr1 is replaced out of the cache. Therefore, the counting lookup table is updated, and the count value of the corresponding entry in the counting lookup table is decremented by 1. The memory access address addr1 corresponds to cnt2 in HCT1. Therefore, cnt2 in HCT1 is decremented by 1, and cnt2 is updated to 1, as Figure 13 shown in part (D) of

[0177] In summary, the multiple counting lookup tables provided by the present application can be well compatible with the situation of the same name. Compared with the related art, when there is the same name, the predicted cache way may be incorrect and the memory access operation needs to be re-executed. The present application ensures the accuracy of the counting lookup table in real time through replacement operations and backfill operations, without the need to re-execute the memory access operation.

[0178] Figure 14 The structure block diagram of the way prediction device of the multi-way set-associative cache provided by an exemplary embodiment of the present application is shown. The multi-way set-associative cache includes multiple cache groups. Each of the multiple cache groups includes multiple cache ways, and the multiple cache ways correspond one-to-one to multiple counting lookup tables. Each of the multiple counting lookup tables has multiple entries. The device includes:

[0179] An acquisition module 1401, configured to acquire a memory access address;

[0180] A generation module 1402, configured to generate an index value corresponding to the memory access address based on the memory access address;

[0181] A lookup module 1403, configured to access the entries corresponding to the index value in each counting lookup table to obtain multiple count values. The multiple count values correspond one-to-one to the multiple counting lookup tables. The i-th count value of the multiple count values indicates the number of cache lines matching the index value in the i-th cache way. The i-th cache way is the cache way corresponding to the counting lookup table where the i-th count value is located, and i is an integer value not greater than the number of tables of the multiple counting lookup tables;

[0182] A determination module 1404, configured to determine that there is a possibility that the i-th cache way stores a first cache line when the i-th count value is not zero. The first cache line is the cache line corresponding to the memory access address.

[0183] In an optional embodiment, the generation module 1402 is further configured to perform a hash calculation on at least one field segment in the memory access address to obtain a hash value. The number of bits occupied by the hash value is less than the number of bits occupied by the memory access address; and determine the hash value as the index value corresponding to the memory access address.

[0184] In an alternative embodiment, the generating module 1402 is further configured to obtain a flag field segment and a set index field segment in the memory access address, where the set index field segment is used to indicate a first cache set in which a first cache line is located in the multi-way set associative cache, and the flag field segment is used to determine a first cache way in which the first cache line is located in the first cache set; perform a hash calculation on the flag field segment and the set index field segment to obtain a hash value.

[0185] In an alternative embodiment, the generating module 1402 is further configured to combine the flag field segment and the set index field segment to obtain an address to be operated on; divide every p bits in the address to be operated on into a group to obtain q bit groups; in the j-th exclusive OR process, perform an exclusive OR operation on every two of the (j - 1)-th exclusive OR results to obtain the j-th exclusive OR result, the (j - 1)-th exclusive OR result includes a plurality of bit groups, the j-th exclusive OR result includes at least one bit group, and the exclusive OR operation on every two refers to exhausting the plurality of bit groups included in the (j - 1)-th exclusive OR result and performing an exclusive OR operation on bits at the same position in every two bit groups; when j is equal to 1, the (j - 1)-th exclusive OR result is q bit groups, and when the j-th exclusive OR result includes a single bit group, determine the value indicated by the single bit group as the hash value.

[0186] In an alternative embodiment, the apparatus further includes an updating module 1405. The updating module 1405 is configured to determine to perform a cache line filling operation when multiple count values are all zero; calculate a first index value corresponding to the first cache line when the first cache line is filled back into the multi-way set associative cache; determine a first cache way in which the first cache line is filled back into the multi-way set associative cache; and increment the count value of the entry indicated by the first index value in the count lookup table corresponding to the first cache way.

[0187] In an alternative embodiment, the updating module 1405 is further configured to determine a flag field segment corresponding to the first cache line to obtain a first flag field segment; and determine a set index field segment corresponding to the first cache line to obtain a first set index field segment; perform a hash calculation on the first flag field segment and the first set index field segment to obtain a first hash value; and determine the first hash value as the first index value corresponding to the first cache line.

[0188] In an alternative embodiment, the updating module 1405 is configured to calculate a second index value corresponding to a second cache line when the second cache line in the multi-way set associative cache is replaced, where the second cache line is a valid cache line; determine a second cache way in which the second cache line is located in the multi-way set associative cache; and decrement the count value of the entry indicated by the second index value in the count lookup table corresponding to the second cache way.

[0189] In an alternative embodiment, the update module 1405 is further configured to determine a flag field segment corresponding to the second cache line to obtain a second flag field segment; and determine a set index field segment corresponding to the second cache line to obtain a second set index field segment; perform a hash calculation on the second flag field segment and the second set index field segment to obtain a second hash value; and determine the second hash value as a second index value corresponding to the second cache line.

[0190] In an alternative embodiment, the second cache line is a cache line replaced by the first cache line when valid cache lines exist in each cache way of the first cache set, and the second cache line is the least recently used cache line included in multiple cache ways of the first cache set; the first cache set is the cache set corresponding to the first cache line, and the second cache line is saved to a lower-level cache of the multi-way set-associative cache.

[0191] In an alternative embodiment, the determination module 1404 is further configured to generate a way enable signal based on multiple count values. The way enable signal includes multiple bits, and each bit corresponds to each cache way among the multiple cache ways. If the i-th count value among the multiple count values is not zero, determine the bit corresponding to the i-th cache way as one; if the i-th count value is zero, determine the bit corresponding to the i-th cache way as zero.

[0192] In summary, the embodiments of the present application add multiple count lookup tables, and the multiple count lookup tables correspond to the multiple cache ways one by one. Before reading data from the multi-way set-associative cache, first predict the possible cache ways through the multiple count lookup tables, so as to exclude some cache ways. Compared with the related art that needs to read all cache ways from the cache (multi-way set-associative cache) each time, the way prediction device of the present application ensures that when reading from the cache, only the cache ways with non-zero predicted count values are read, rather than necessarily reading all cache ways, thereby reducing the power consumption of accessing the cache and improving the performance of accessing the cache.

[0193] Moreover, compared with the way predictor provided by the related art, the cache ways predicted by the related art may be incorrect, and in this case, the access operation needs to be re-executed, while the cache ways predicted by the way prediction device of the present application must have correct cache ways, ensuring the accuracy of the accessed cache ways and eliminating the need to re-execute the access operation.

[0194] Figure 15The structural block diagram of a computer device 1500 provided by an exemplary embodiment of the present application is shown. The computer device 1500 may be a portable mobile terminal, such as: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop computer or a desktop computer. The computer device 1500 may also be referred to by other names such as user equipment, portable terminal, laptop terminal, desktop terminal, etc.

[0195] Generally, the computer device 1500 includes: a processor 1501 and a memory 1502.

[0196] The processor 1501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1501 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), PLA (Programmable Logic Array). The processor 1501 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 1501 may be integrated with a GPU (Graphics Processing Unit), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1501 may also include an AI (Artificial Intelligence) processor, and the AI processor is used to process computational operations related to machine learning.

[0197] The processor 1501 includes a way prediction circuit, and the way prediction circuit is a circuit structure. The way prediction circuit is used to implement the way prediction method of the multi-way set-associative cache provided by the method embodiment in the present application.

[0198] The memory 1502 may include one or more computer-readable storage media, which may be non-transitory. The memory 1502 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices and flash storage devices.

[0199] In some embodiments, the computer device 1500 may further optionally include a peripheral device interface 1503 and at least one peripheral device. Those skilled in the art can understand that Figure 15 the structure shown in does not constitute a limitation on the computer device 1500, and it may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0200] This application also provides a processor, which includes a circuit structure for implementing the way prediction method of the multi-way set-associative cache provided in the above method embodiment.

[0201] This application also provides a chip, which includes: a processor, and the processor includes a circuit structure for implementing the way prediction method of the multi-way set-associative cache provided in the above method embodiment.

[0202] The serial numbers of the above embodiments of this application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0203] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above embodiments can be completed by hardware, or can be completed by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and the above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disc, etc.

[0204] The above are only optional embodiments of this application and are not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included in the protection scope of this application.

Claims

1. A way prediction circuit for a multi-way set-associative cache, characterized in that The path prediction circuit includes an arithmetic circuit unit, a read logic circuit unit, a plurality of count lookup tables, and a judgment circuit unit. The four circuit units are sequentially connected in series by wires. The multi-way set-associative cache includes a plurality of cache sets. Each of the plurality of cache sets includes a plurality of cache ways. The plurality of cache ways correspond to the plurality of count lookup tables one by one. Each of the plurality of count lookup tables has a plurality of entries. The arithmetic circuit unit is configured to generate an index value corresponding to the memory access address based on the memory access address. The read logic circuit unit is configured to access the entry corresponding to the index value in each count lookup table to obtain a plurality of count values. The plurality of count values correspond to the plurality of count lookup tables one by one. The i-th count value of the plurality of count values indicates the number of cache lines matching the index value in the i-th cache way. The i-th cache way is the cache way corresponding to the count lookup table where the i-th count value is located. i is an integer value not greater than the number of tables of the plurality of count lookup tables. The judgment circuit unit is configured to respectively judge the relationship between the plurality of count values and zero. When the i-th count value is not zero, there is a possibility that the i-th cache way stores a first cache line. When the i-th count value is zero, the i-th cache way does not store the first cache line. The first cache line is the cache line corresponding to the memory access address.

2. The path prediction circuit according to claim 1, wherein The arithmetic circuit unit is further configured to perform a hash calculation on at least one field segment in the memory access address to obtain a hash value. The hash value is the index value corresponding to the memory access address. The number of bits occupied by the hash value is less than the number of bits occupied by the memory access address.

3. The path prediction circuit according to claim 2, wherein The arithmetic circuit unit is further configured to perform a hash calculation on the flag field segment and the set index field segment in the memory access address to obtain the hash value. The set index field segment is used to indicate the first cache set where the first cache line is located in the multi-way set-associative cache. The flag field segment is used to determine the first cache way where the first cache line is located in the first cache set.

4. The path prediction circuit according to claim 3, wherein The arithmetic circuit unit is further configured to divide every p bits in the to-be-operated address into a group to obtain q bit groups. In the j-th exclusive OR process, perform an exclusive OR operation on the bit groups of the (j - 1)-th exclusive OR result in pairs to obtain the j-th exclusive OR result. The (j - 1)-th exclusive OR result includes a plurality of bit groups. The j-th exclusive OR result includes at least one bit group. The pairwise exclusive OR operation means exhausting the plurality of bit groups included in the (j - 1)-th exclusive OR result and performing an exclusive OR operation on the bits at the same position in every two bit groups. When j is equal to 1, the (j - 1)-th exclusive OR result is the q bit groups. When the j-th exclusive OR result includes a single bit group, determine the value indicated by the single bit group as the hash value. The to-be-operated address includes the flag field segment and the set index field segment.

5. The road prediction circuit according to any one of claims 1 to 4, characterized in that The path prediction circuit further includes an update logic circuit unit. The arithmetic circuit unit is further configured to calculate a first index value corresponding to the first cache line when all the plurality of count values are zero and the first cache line is backfilled to the set-associative cache. The update logic circuit unit is configured to increment the count value of the entry indicated by the first index value in the count lookup table corresponding to the first cache way, where the first cache way is the cache way to which the first cache line is backfilled to the set-associative cache.

6. The way prediction circuit according to claim 5, wherein The arithmetic circuit unit is further configured to perform a hash calculation on a first flag field segment and a first set index field segment to obtain a first hash value, where the first hash value is the first index value corresponding to the first cache line, the first flag field segment is the flag field segment corresponding to the first cache line, and the first set index field segment is the set index field segment corresponding to the first cache line.

7. The road prediction circuit according to any one of claims 1 to 4, characterized in that The way prediction circuit further includes an update logic circuit unit; The arithmetic circuit unit is further configured to calculate a second index value corresponding to the second cache line when the second cache line in the set-associative cache is replaced, where the second cache line is a valid cache line. The update logic circuit unit is further configured to decrement the count value of the entry indicated by the second index value in the count lookup table corresponding to the second cache way, where the second cache way is the cache way in which the second cache line is located in the set-associative cache.

8. The way prediction circuit according to claim 7, wherein The arithmetic circuit unit is further configured to perform a hash calculation on a second flag field segment and a second set index field segment to obtain a second hash value, where the second hash value is the second index value corresponding to the second cache line, the second flag field segment is the flag field segment corresponding to the second cache line, and the second set index field segment is the set index field segment corresponding to the second cache line.

9. The way prediction circuit according to claim 7, wherein The second cache line is the cache line replaced by the first cache line when there are valid cache lines in each cache way of the first cache set, and the second cache line is the least recently used cache line included in the plurality of cache ways of the first cache set; The first cache set is the cache set corresponding to the first cache line, and the second cache line is saved to the lower-level cache of the set-associative cache.

10. A way prediction method for a multi-way set-associative cache, characterized in that, The set-associative cache includes a plurality of cache sets, each of the plurality of cache sets includes a plurality of cache ways, the plurality of cache ways correspond to a plurality of count lookup tables one by one, each of the plurality of count lookup tables has a plurality of entries, and the method includes: Obtain a memory access address; based on the memory access address, generate an index value corresponding to the memory access address; Access the entries corresponding to the index value in each of the count lookup tables to obtain a plurality of count values. The plurality of count values correspond one-to-one with the plurality of count lookup tables. The i-th count value of the plurality of count values indicates the number of cache lines matching the index value in the i-th cache way. The i-th cache way is the cache way corresponding to the count lookup table where the i-th count value is located, and i is an integer value not greater than the number of tables of the plurality of count lookup tables; When the i-th count value is not zero, determine that there is a possibility that the i-th cache way stores a first cache line, where the first cache line is the cache line corresponding to the memory access address.

11. The method according to claim 10, wherein The method further includes: Generate a way enable signal based on the plurality of count values. The way enable signal includes a plurality of bits, and each bit corresponds to each of the plurality of cache ways. When the i-th count value is not zero, the bit corresponding to the i-th cache way is one; when the i-th count value is zero, the bit corresponding to the i-th cache way is zero.

12. A way prediction device for a multi-way set-associative cache, characterized in that, The multi-way set-associative cache includes a plurality of cache sets, each of the plurality of cache sets includes a plurality of cache ways, the plurality of cache ways correspond one-to-one with a plurality of count lookup tables, and each of the plurality of count lookup tables has a plurality of entries. The apparatus includes: An acquisition module, configured to acquire a memory access address; A generation module, configured to generate an index value corresponding to the memory access address based on the memory access address; A lookup module, configured to access the entries corresponding to the index value in each of the count lookup tables to obtain a plurality of count values. The plurality of count values correspond one-to-one with the plurality of count lookup tables. The i-th count value of the plurality of count values indicates the number of cache lines matching the index value in the i-th cache way. The i-th cache way is the cache way corresponding to the count lookup table where the i-th count value is located, and i is an integer value not greater than the number of tables of the plurality of count lookup tables; A determination module, configured to determine that there is a possibility that the i-th cache way stores a first cache line, where the first cache line is the cache line corresponding to the memory access address, when the i-th count value is not zero.

13. A processor, characterized in that, The processor includes a way prediction circuit of the multi-way set-associative cache according to any one of claims 1 to 9.

14. A chip, characterized in that, The chip includes a processor, and the processor includes a way prediction circuit of the multi-way set-associative cache according to any one of claims 1 to 9.

15. A computer device, characterized in that, The computer device includes a processor, and the processor includes a way prediction circuit of the multi-way set-associative cache according to any one of claims 1 to 9.

Images