Instruction cache management method, instruction cache, computing device and system
By downsampling the original age value of cache lines in the instruction cache, a baseline age value is obtained, which solves the failure problem of the LRU replacement strategy in the parallel access environment of high-performance computing devices and achieves more efficient cache management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI BIREN TECH CO LTD
- Filing Date
- 2026-04-13
- Publication Date
- 2026-07-24
AI Technical Summary
In high-performance computing devices, LRU-based instruction cache replacement strategies are prone to failure in parallel access environments, leading to unfairness and reduced efficiency in the replacement strategy.
By downsampling the original age value of the cache line in the instruction cache, a baseline age value is obtained, and this baseline age value is used to replace the original age value to represent the replacement priority of the cache line, thereby reducing the risk of replacement strategy failure.
This effectively alleviates the problem of the cache line baseline age value growing too quickly to the upper limit due to hot access, ensuring the effectiveness and fairness of the replacement strategy in a concurrent access environment.
Smart Images
Figure CN122018985B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of chip technology, and more specifically, to an instruction cache management method, an instruction cache, a computing device, and a computing system. Background Technology
[0002] High-performance computing devices, represented by Graphics Processing Units (GPUs), occupy a central position in fields such as artificial intelligence inference and scientific computing. To meet the access demands of massive parallel computing units, the storage architecture of computing devices typically integrates a multi-level cache system. Among them, the instruction cache (I-Cache), as a key storage unit for temporarily storing recently accessed instructions, directly determines the instruction fetching efficiency of the computing units.
[0003] Due to capacity constraints, instruction caches need to improve utilization efficiency through replacement strategies. When a cache set in the instruction cache is full and a new instruction needs to be loaded, a cache line within that set must be selected for replacement. A common replacement strategy is Least Recently Used (LRU), which selects the cache line that has not been accessed for the longest time in the set for replacement. To implement LRU, the instruction cache maintains an age value for each cache line to represent its replacement priority. In traditional implementations, when a cache line within a set is hit, its age value is set to 0, while the age values of all other cache lines in the same set are incremented by 1. The smaller the age value, the more recently the cache line has been accessed; conversely, the larger the age value, the longer the cache line has not been accessed.
[0004] Based on this replacement strategy, in highly parallel computing architectures such as GPUs, multiple computing units may access the same cache line within the same time period. This high-frequency concentrated access within a short period of time will cause the age values of most cache lines in the same group to quickly reach the set upper limit, thus losing the ability to distinguish replacement priorities and causing the replacement strategy to fail for a long time. Summary of the Invention
[0005] One objective of this disclosure is to provide a new technical solution for instruction cache management, reducing the risk of long-term failure of LRU-based replacement strategies.
[0006] According to a first aspect of this disclosure, an instruction cache management method is provided, the method comprising: Receive a memory access request for the instruction cache; wherein the instruction cache contains multiple cache groups, and each cache group contains multiple cache lines; Determine the target cache group mapped by the memory access request, and detect whether the target cache group contains the target instruction indicated by the memory access request; If the target instruction is missing in the target cache group and the target cache group is full, a target cache line to be replaced is selected from the target cache group based on the baseline age value of each cache line in the target cache group; wherein, the baseline age value of the cache line is obtained by downsampling the original age value of the corresponding cache line with a set sampling step size, the original age value represents the number of consecutive misses of the corresponding cache line, and the baseline age value characterizes the replacement priority of the corresponding cache line in the target cache group; The target instruction is read from the lower-level memory of the instruction cache and written into the target cache line.
[0007] Optionally, the extraction step size is an integer power of 2; the original age value of the cache line is encoded in binary, the low-order field of the binary encoding is used to accumulate the number of consecutive misses within the range limited by the extraction step size, and the high-order field of the binary encoding is used to accumulate the base age value of the corresponding cache line.
[0008] Optionally, after writing the target instruction into the target cache line, the method further includes: Set the original age value of the target cache line to 0; and, Based on the weight value corresponding to the memory access request, determine the weighted miss count of the target instruction that was not hit in this cache line; Based on the weighted miss count, update the original age values of other cache lines in the target cache group.
[0009] Optionally, the weight value corresponding to the memory access request is determined based on at least one of the request type of the memory access request and the instruction address carried by the memory access request.
[0010] Optionally, the weight value corresponding to the memory access request is determined based on the request type of the memory access request. In the case that the memory access request is a prefetch instruction request, the weight value is a first weight value set for the prefetch instruction request, and the first weight value is greater than the default weight value.
[0011] Optionally, the first weight value is greater than or equal to the extraction step size, but less than twice the extraction step size.
[0012] Optionally, the weight value corresponding to the memory access request is determined based on the instruction address carried by the memory access request. If the instruction address belongs to any address segment in the weight configuration table, the weight value is a second weight value configured for the address segment, and the second weight value is greater than the default weight value.
[0013] Optionally, if the memory access request is a prefetch instruction request, the weight value is a first weight value set for the prefetch instruction request; When the memory access request is an instruction fetch request, the weight value corresponding to the memory access request is determined based on the instruction address carried by the memory access request.
[0014] Optionally, before determining the weighted miss count of the target instruction that was not hit in the current cache line, the method further includes: Detect whether there are any full-age cache lines in the other cache lines whose original age value has reached the upper limit; If the full-age cache line exists, then keep the original age values of the other cache lines unchanged; If the full-age cache line does not exist, then the step of updating the original age value of other cache lines in the target cache group is performed.
[0015] According to a second aspect of this disclosure, an embodiment of an instruction cache is provided, the instruction cache comprising multiple cache groups, each cache group comprising multiple cache lines; the instruction cache stores age information for each cache line, the age information including an original age value and a baseline age value, the baseline age value of the cache line being obtained by downsampling the original age value of the corresponding cache line with a set sampling step size, the original age value representing the number of consecutive misses of the corresponding cache line, and the baseline age value characterizing the replacement priority of the corresponding cache line within its cache group.
[0016] According to a third aspect of this disclosure, another embodiment of an instruction cache is provided, which is configured to perform the method described according to a first aspect of this disclosure.
[0017] According to a fourth aspect of this disclosure, a computing device is provided, comprising: Multiple computing units; An instruction cache is connected to the computing unit; the instruction cache is the instruction cache described in the second or third aspect of this disclosure.
[0018] According to a fifth aspect of this disclosure, a computing system is provided, the computing system including a control device and a computing device according to a fourth aspect of this disclosure, the control device being used to control the computing device to perform computing tasks.
[0019] In this embodiment of the LRU replacement strategy, the original age value of a cache line in the instruction cache is downsampled to obtain a baseline age value. This baseline age value is then used to replace the original age value, representing the replacement priority of the corresponding cache line, and the LRU replacement strategy is executed accordingly. This process is equivalent to reducing the growth rate of the baseline age value by sampling a larger step size, thereby effectively alleviating the problem that the baseline age value of other cache lines in the group grows too quickly to the upper limit due to hot access to a certain cache line, ensuring the effectiveness and fairness of the replacement strategy in a concurrent access environment.
[0020] The features and advantages of the embodiments of this specification will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description
[0021] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of this specification and, together with their description, serve to explain the principles of these embodiments.
[0022] Figure 1 This is a schematic diagram of the structure of a computing system to which the methods of the embodiments of this disclosure can be applied; Figure 2 This is a schematic diagram of the internal structure of a computing device; Figure 3 This is a flowchart illustrating an instruction cache management method according to some embodiments; Figure 4 This is a flowchart illustrating an instruction cache management method according to some other embodiments; Figure 5 This is a schematic diagram of the instruction cache structure according to some embodiments; Figure 6 This is a microarchitecture diagram of an instruction cache according to some other embodiments; Figure 7 This is a schematic diagram of the structure of a computing device according to some embodiments. Detailed Implementation
[0023] Various exemplary embodiments of this specification will now be described in detail with reference to the accompanying drawings.
[0024] The following description of at least one exemplary embodiment is merely illustrative and is not intended to limit the embodiments of this specification or their application or use.
[0025] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.
[0026] Figure 1This is a schematic diagram of the structure of a computing system to which the instruction cache management method provided in the embodiments of this disclosure can be applied. Figure 1 As shown, the computing system 1000 may include a computing device 1100, a control device 1200, and a storage device 1300. The computing device, the control device, and the storage device may be connected to each other via a bus network or other means.
[0027] The storage device can be used to store instructions and / or data, which can be retrieved and used by a control device or a computing device. For example, the storage device can store program instructions executed by the control device or computing device, as well as data such as text, images, audio, and configuration parameters. The control device can control the computing device to execute related processes or tasks to achieve related system functions, such as artificial intelligence, scientific computing, or image processing functions based on user needs. Exemplarily, the control device can be a central processing unit (CPU); the computing device can be a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a neural network processing unit (NPU), or a tensor processing unit (TPU); and the storage device can be static random access memory (SRAM), read-only memory (ROM), or erasable programmable read-only memory (EPROM). This storage device can also be called memory, and this computing device can also be called a chip, processor, or artificial intelligence chip.
[0028] Continue as Figure 1 As shown, the computing device 1100 may include a computing unit 1110 and a cache 1120. The computing unit may be the core component in the computing device that performs computing tasks, and may also be referred to as a computing core, arithmetic core, or processing core. The cache may be used to cache instructions and / or data, and may also be referred to as a cache circuit, on-chip memory circuit, or on-chip memory.
[0029] This computing device contains multiple computing units, which can execute a process or task together or individually. Taking a GPU as an example, the computing unit can be a GPU Core, and a GPU consists of multiple GPU Cores.
[0030] The processing efficiency of the cache in this computing device is greater than that of external storage devices, but the storage capacity is smaller. This cache can be used to store recently used instructions and / or data to improve the processing efficiency of the computing device. For example, the computing device may include multi-level caches (at least two levels), such as a Level 1 cache (L1 Cache) and a Level 2 cache (L2 Cache), or a Level 1 cache (L1 Cache), a Level 2 cache (L2 Cache), and a Level 3 cache (L3 Cache), etc. The Level 2 or Level 3 cache, acting as the Last Level Cache (LLC), is the Level 1 cache with the largest capacity in the computing device and the closest to the external storage device 1300. It can directly access the storage device 1300, for example, by reading instructions from or writing data to the storage device 1300 via a bus. For example, each computing unit can have its own independent level-one cache, allowing each level-one cache to be used independently by a single computing unit. Alternatively, multiple computing units within the computing device can be grouped into multiple computing unit groups (e.g., SPCs), with each computing unit group having its own independent level-one cache, allowing each level-one cache to be shared by the computing units within that group. The last-level cache can be configured to be shared by all or some of the computing units in the computing device; this last-level cache can also be called the final level cache, shared cache, etc.
[0031] For example, each cache level can adopt a set-associative organization structure. The cache can be divided into multiple cache sets, each containing multiple ways, and each way containing one cache line (also called a cache block). In this way, a cache line can be uniquely identified by an index of (set, way). A specific memory address in memory can be mapped to any cache line. Thus, when a computing unit needs to access a specific memory address, it can improve processing efficiency by accessing the mapped cache line.
[0032] Figure 2 This diagram illustrates the memory access path of a computing unit in a computing device. For example... Figure 2 As shown, the computing device 1100 may include: a group of multiple computing units 1110, a configuration unit 1130, an instruction cache 1121 shared by the group of computing units 1110, and a lower-level cache 1122 of the instruction cache 1121. Figure 2 The instruction cache 1121 is usually a level 1 cache, and the lower-level cache 1122 includes at least a level 2 cache. The lower-level cache 1122 is also referred to as the external cache of the instruction cache 1121.
[0033] When executing program instructions, the configuration unit 1130 first sends instruction information, including the base address of the instruction in memory, into the instruction cache 1121. Subsequently, the computing unit 1110 initializes multiple thread bundles, and each thread bundle sends a memory access request to the instruction cache 1121 through its respective computing unit 1110. The instruction cache 1121 determines the target cache group mapped to the memory access request based on the instruction address (program counter PC, i.e., the instruction's index in the assembler) carried in the memory access request, and checks whether the target instruction indicated by the instruction address exists in the target cache group. If it exists, the instruction cache 1121 fetches the target instruction and returns it to the corresponding computing unit; if it does not exist, it reads the target instruction from the lower-level memory of the instruction cache, writes it into the target cache group, and after writing, fetches the target instruction and returns it to the corresponding computing unit. The lower-level memory includes a lower-level cache 1122 and a storage device 1300 as memory. If the target instruction is hit in the lower-level cache 1122, it is loaded from the cache into the instruction cache 1121. If the lower-level cache 1122 also misses the instruction, the target instruction is loaded from memory through the lower-level cache 1122 and simultaneously written to both the instruction cache 1121 and the lower-level cache 1122.
[0034] Here, due to the limited capacity of the instruction cache 1121, to improve its utilization efficiency, when the cache is full and a new instruction needs to be loaded from the lower-level memory, the instruction cache 1121 needs to select the cache line with the highest replacement priority from the target cache group based on the LRU replacement policy. The instruction cache management scheme provided in this embodiment of the disclosure reduces the risk of LRU policy failure in a parallel access environment by implementing hierarchical management of cache line age values in the cache line replacement process.
[0035] The following combination Figure 1 The computing system described herein illustrates various embodiments.
[0036] <First Embodiment> Figure 3 A flowchart illustrating an instruction cache management method according to some embodiments is shown. This method is applied to... Figure 2 The instruction cache is 1121. (For example...) Figure 3 As shown, the method of this embodiment may include the following steps S310 to S340.
[0037] Step S310: Receive a memory access request for the instruction cache.
[0038] The instruction cache is organized into multiple cache sets using a set-associative structure, with each set containing multiple cache lines. A cache line, as the basic unit for storing instructions in the instruction cache, has a set number of bytes, such as 64 bytes or 128 bytes. When loading instructions, the instruction cache performs the loading operation on a whole cache line basis.
[0039] Memory access requests are issued by the thread bundle of the computing unit to fetch instructions from the instruction cache. The memory access request carries the instruction address, indicating the target instruction requested.
[0040] Step S320: Determine the target cache group mapped to the memory access request, and check whether the target instruction indicated by the memory access request is stored in the target cache group.
[0041] After receiving a memory access request, the instruction cache can decode the cache set mapped to the memory access request based on the index field in the instruction address. In this embodiment, it is referred to as the target cache set.
[0042] The instruction cache determines whether the target instruction is present in a cache line of the target cache group by comparing the tag in the instruction address with the tags of each cache line in the target cache group and checking the validity bit of the cache line.
[0043] Step S330: If the target instruction is missing in the target cache group and the target cache group is full, then select the target cache line to be replaced from the target cache group based on the baseline age value of each cache line in the target cache group.
[0044] In this embodiment, the instruction cache maintains an original age value `full_age` and a baseline age value `bm_age` for each cache line. The original age value `full_age` represents the number of consecutive misses for the corresponding cache line. The baseline age value `bm_age` is obtained by downsampling the original age value `full_age` of the corresponding cache line according to a set sampling step size, so that the growth rate of the baseline age value is lower than that of the original age value. The sampling step size represents the interval step size for sampling consecutive miss events, reflecting the ratio of the growth rate of the baseline age value to the original age value, that is, reflecting the deceleration factor.
[0045] In this embodiment, the baseline age value bm_age of a cache line is used to characterize the replacement priority of the corresponding cache line within the target cache group, rather than the original age value full_age. The larger the baseline age value bm_age of a cache line, the higher its replacement priority, and the more likely it will be replaced.
[0046] Taking a extraction step size of 4 as an example, the baseline age value `bm_age` of the cache line is defined as the integer part of the original age value `full_age` divided by 4, and the remainder is denoted as the sub-age value `sub_age`, which is used to implement fine-grained counting within the range defined by the extraction step size. Only when the number of consecutive misses of the cache line reaches 4, i.e., when the sub-age value `sub_age` overflows, is the baseline age value `bm_age` incremented by 1, and the sub-age value `sub_age` is reset to zero. In other words, if the number of consecutive misses is less than 4, only the sub-age value `sub_age` is updated, while the baseline age value `bm_age` used for replacement decisions remains unchanged, thereby reducing the growth rate of the baseline age value.
[0047] In some examples, the extraction step size can be set to an integer power of 2, such as 2, 4, or 8. Setting the extraction step size to an integer power of 2 here ensures that the mapping relationship between the original age value, the baseline age value, and the sub-age value satisfies the binary bit field partitioning. This eliminates the need for complex operations such as division and modulo; the original age value, baseline age value, and sub-age value can be obtained solely through shifting and masking operations. This significantly reduces the hardware complexity of the decoding and computation logic, shortens critical path latency, and improves timing performance.
[0048] Therefore, in these examples, the original age value of the cache line can be encoded in binary. The low-order bits of the binary encoding are used to accumulate the number of consecutive misses within the extraction step size, while the high-order bits are used to accumulate the baseline age value of the corresponding cache line. Here, the overall bit width of the binary encoding and the bit width of the low-order bits can be configured based on the upper limit of the original age value and the extraction step size. Taking an extraction step size of 4 as an example, the low-order bits have a bit width of 2 bits, and the remaining bits represent the high-order bits that characterize the baseline age value.
[0049] In this embodiment, if the target cache group contains the target instruction, the instruction cache can directly extract the target instruction from the hit cache line and return the extracted target instruction to the computing unit that issued the memory access request. At this time, the instruction cache sets the original age value full_age of the hit cache line to 0, and its base age value bm_age also becomes 0 accordingly; and increments the original age value full_age of other cache lines in the target cache group by 1.
[0050] If the target instruction is missing from the target cache set, the instruction cache will load the target instruction from its lower-level memory and write it to the target cache set.
[0051] If the target instruction is missing in the target cache group and the target cache group has free cache lines, the instruction cache can write the target instruction into any free cache line, set the original age value full_age of the cache line where the target instruction is written to to 0, and increment the original age value full_age of other cache lines in the target cache group that contain instruction data by 1.
[0052] If the target instruction is missing in the target cache group and the target cache group is full, that is, the target cache group does not have a free cache line, then the target cache line to be replaced is selected from the target cache group based on the LRU policy.
[0053] In this embodiment, the instruction cache selects the cache line with the largest current baseline age value from the target cache group as the target cache line to be replaced; that is, the baseline age value of the target cache line is the maximum value in the target cache group. If there are two or more candidate lines with the same baseline age value in the target cache group, one of them can be determined as the target cache line through arbitration.
[0054] Step S340: Read the target instruction from the lower-level memory of the instruction cache and write the read target instruction into the target cache line.
[0055] After the read target instruction is written to the target cache line selected based on the LRU policy, the instruction cache sets the original age value of the target cache line to 0, and its base age value bm_age also becomes 0 accordingly; in addition, the original age value of other cache lines in the target cache group can be increased by 1.
[0056] According to steps S310 to S340, the instruction cache maintains a baseline age value bm_age for each cache line. Since the baseline age value bm_age is obtained by downsampling the original age value full_age of the corresponding cache line according to a set sampling step size, it has a low growth rate. Therefore, executing the LRU strategy based on the baseline age value of the cache line can reduce the failure risk of the replacement strategy in high-frequency concentrated access scenarios. At the same time, the baseline age value of the cache line can maintain the same bit width as the traditional age value, and the above optimization can be achieved without increasing the hardware resources used for LRU comparison.
[0057] Furthermore, if there are other cache lines in the target cache group whose original age values have reached the upper limit, that is, if there are cache lines in the target cache group that have reached the maximum age, the original age values of the other cache lines can be kept unchanged. Only when the original age values of the other cache lines have not reached the upper limit will the operation of updating the original age values of the other cache lines be performed. This can further reduce the risk of failure of the replacement strategy.
[0058] <Second Embodiment> Figure 4A flowchart illustrating an instruction cache management method according to some other embodiments is shown. This method is also applied to... Figure 2 The instruction cache is 1121. Unlike the first embodiment, after the target instruction is written to the target cache line, the original age value of other cache lines in the target cache group no longer increases by a fixed 1. Instead, it is calculated based on the weight value corresponding to the memory access request, resulting in a weighted miss count, and the original age value `full_age` of other cache lines is updated accordingly. This embodiment adjusts the impact of each missed requested instruction on the original age value `full_age`, thereby controlling the update of the baseline age value `bm_age` under the extraction step size constraint. This allows intervention in the selection of future cache lines to be replaced, enabling the intervened instruction to remain in the target cache group for a longer period, so that when a memory access request for the intervened instruction is received from the computing unit later, it can be quickly returned to the computing unit.
[0059] like Figure 4 As shown, the method of this embodiment may include the following steps S410 to S460.
[0060] Step S410: Receive a memory access request for the instruction cache.
[0061] Step S420: Determine the target cache group mapped to the memory access request, and check whether the target instruction indicated by the memory access request is stored in the target cache group.
[0062] Step S430: If the target instruction is missing in the target cache group and the target cache group is full, then select the target cache line to be replaced from the target cache group based on the baseline age value of each cache line in the target cache group.
[0063] Step S440: Read the target instruction from the lower-level memory of the instruction cache and write the read target instruction into the target cache line.
[0064] Step S450: Set the original age value of the target cache line to 0.
[0065] Step S460: Based on the weight value corresponding to the memory access request, determine the weighted miss count of the target instruction that the cache line missed in this cache line, and update the original age value of other cache lines in the target cache group based on the weighted miss count.
[0066] In this embodiment, the weighted number of misses is equal to the product of the weight value and 1 miss. Therefore, the weighted number of misses is numerically equal to the weight value.
[0067] Taking an extraction step size of 4 and a memory access request with a weight of 4 as an example, in this scenario, the miss will increase the original age value of other cache lines by 4. Since the increase is equal to the extraction step size, the baseline age value of these cache lines will be updated accordingly, thus giving them a higher replacement priority than the target cache line, making them more likely to be selected as replacement targets in subsequent memory access requests.
[0068] In some examples, before executing step S460, the instruction cache can detect whether there are any cache lines in other cache lines that have reached the upper limit of the original age value; if there are no cache lines that have reached the upper limit of the original age value, then step S460 is executed; if there are cache lines that have reached the upper limit of the original age value, then the original age values of other cache lines are kept unchanged to maintain the effectiveness of the replacement strategy and further reduce the risk of failure of the replacement strategy.
[0069] During instruction cache execution step S460, if the difference between the current value of the original age value of a cache line and the upper limit value is less than the required increase in the weighted miss count, then the original age value of the cache line will be updated to the upper limit value.
[0070] In this embodiment, instructions that do not have a high-frequency centralized access state can be set as intervention instructions, and a special weight value greater than the default weight value can be configured for them to cooperate with the age-level management of the first embodiment and realize the control of the update of the baseline age value bm_age. The default weight value is the weight value that has not been specially set. For example, the default weight value can be configured as "1". In this case, the number of weighted misses is still 1, and the impact on the original age value is not changed.
[0071] In some examples, the instruction requested by the prefetch instruction request can be set as the instruction to be intervened, so that the weight value corresponding to the memory access request can be determined according to the request type of the memory access request.
[0072] Request types include prefetch instructions and fetch instructions. Fetch instructions are issued by the compute unit and require retrieving instructions from the instruction cache for execution within the compute unit. Prefetch instructions can be issued by the compute unit or generated spontaneously by the instruction cache. For prefetch instructions, the instruction cache performs a hit check and reads the instruction data from the lower-level memory if a miss occurs, but it does not immediately return the data to the compute unit. Instead, it waits for a subsequent fetch instruction request to arrive before returning the data. Since the prefetching of the same instruction typically occurs only once, it does not create a high-frequency concentrated access. Therefore, by using a replacement strategy based on the reduced baseline age value, setting the instruction requested by the prefetch instruction request as the intervention instruction can make other cache lines more likely to be selected as replacement targets in subsequent accesses, thereby extending the retention time of the target instruction in the cache set and achieving the purpose of prefetching.
[0073] In these examples, the weight value corresponding to the prefetch instruction request can be configured to a first weight value greater than the default weight value. When the memory access request is a prefetch instruction request, the weight value corresponding to the memory access request is the first weight value set for the prefetch instruction request. However, when the memory access request is a fetch instruction request, the weight value corresponding to the memory access request can be the default weight value.
[0074] In these examples, the first weight value can be further greater than or equal to the extraction step size, but less than twice the extraction step size. For example, if the extraction step size is 4, the first weight value can be set to be greater than or equal to 4, but less than 8. This way, when updating the original age value of other cache lines based on the weighted miss count determined by the first weight value, the baseline age value of the other cache lines will be updated.
[0075] In other examples, instructions in specific address ranges can be designated as the intervention targets, allowing them to remain in the cache longer and thus improving the fetch speed of high-frequency instructions. If an instruction is accessed by more threads, its weight should be configured to a smaller value, and vice versa.
[0076] In these examples, the weight value corresponding to the memory access request can be determined based on the instruction address carried in the memory access request. If the instruction address belongs to any address range in the weight configuration table, the weight value corresponding to the memory access request is the second weight value configured for that address range, which is also greater than the default weight value. If the instruction address does not belong to any address range in the weight configuration table, the weight value corresponding to the memory access request can be the default weight value.
[0077] In these examples, weight configuration tables can be written in software and sent to the computing device during hardware initialization to complete the configuration. Weight configuration examples are as follows: start_addr_0 end_addr_0 update_weight_0 start_addr_1 end_addr_1 update_weight_1 ...... start_addr_m end_addr_m update_weight_m The example above configures a total of m address segments, each with a corresponding weight value. Taking the 0th address segment as an example, its starting address is start_addr_0, its ending address is end_addr_0, and its corresponding weight value is update_weight_0.
[0078] In other examples, the weight value of a memory access request can be determined by combining the request type and the instruction address it carries, where the request type has higher priority. In these examples, when the memory access request is a prefetch instruction request, the weight value is the first weight value set for prefetch instruction requests; while when the memory access request is a fetch instruction request, the weight value corresponding to the memory access request is determined based on the instruction address carried by the memory access request.
[0079] <Third Embodiment> This disclosure also provides an instruction cache. The instruction cache includes multiple cache groups, and each cache group includes multiple cache lines. The instruction cache stores age information for each cache line. This age information adopts a hierarchical structure, including an original age value and a baseline age value. The baseline age value of a cache line is obtained by downsampling the original age value of the corresponding cache line with a set sampling step size. The original age value represents the number of consecutive misses for the corresponding cache line, and the baseline age value characterizes the replacement priority of the corresponding cache line within its cache group. This allows for the selection of cache lines to be replaced based on the baseline age value of each cache line within the target cache group, thereby reducing the risk of replacement strategy failure.
[0080] Based on this hierarchical structure, in some embodiments, the instruction cache is configured to execute the method according to the first embodiment or the second embodiment.
[0081] In other embodiments, such as Figure 5 As shown, the instruction cache 1121 may include a request receiving unit 11211, a hit detection unit 11212, a replacement decision unit 11213, and a memory access unit 11214.
[0082] exist Figure 5 In the example shown, the request receiving unit 11211 is used to receive memory access requests for the instruction cache. The hit detection unit 11212 is used to determine the target cache group mapped by the memory access request and to detect whether the target cache group contains the target instruction indicated by the memory access request. The replacement decision unit 11213 is used to select a target cache line to be replaced from the target cache group based on the reference age value of each cache line in the target cache group when the target instruction is missing from the target cache group and the target cache group is full. The memory access unit 11214 is used to read the target instruction from the lower-level memory of the instruction cache and write the read target instruction into the target cache line when the target instruction is missing from the target cache group.
[0083] Figure 6 A microarchitecture diagram of an instruction cache according to some other embodiments is shown. Figure 6In the illustrated embodiment, the instruction cache 1121 includes a request receiving unit 11211, a hit detection unit 11212, a replacement decision unit 11213, a memory access unit 11214, a request scheduling unit 11215, and a request return unit 11216.
[0084] like Figure 6 As shown, the instruction cache 1121 receives memory access requests from N computing units. The request receiving unit 11211 arbitrates the concurrent requests and temporarily stores them in the request scheduling queue. Subsequently, these memory access requests sequentially enter the hit detection unit 11212, which determines whether the target instruction is in the target cache group mapped to the memory access request by comparing tags. If a hit occurs, the memory access request is sent to the request scheduling unit 11215, which reads the target instruction from the hit cache line and sends it to the request return unit 11216, which then returns it to the computing unit. If a cache miss occurs, the request scheduling unit 11215 first selects the target cache line to be replaced based on the baseline age value of each cache line in the target cache group, and sends a read request to the lower-level cache 1122 through the memory access unit 11214. After receiving the returned target instruction, it writes it into the target cache line. At the same time, the replacement decision unit 11213 updates the original age value of the cache line in the target cache group, and returns the target instruction to the calculation unit through the request return unit 11216.
[0085] <Fourth Embodiment> This disclosure also provides a computing device, such as... Figure 7 As shown, the computing device 1100 includes a plurality of computing units 1110 and an instruction cache 1121 connected to the computing units 1110; the instruction cache 1121 may have a microarchitecture according to the third embodiment for executing the method according to the first embodiment or the second embodiment.
[0086] <Fifth Embodiment> This disclosure also provides a computing system, which includes a control device and a computing device. The computing device may include an instruction cache according to a third embodiment, or a computing device according to a fourth embodiment, and may be configured to execute an instruction cache management method according to a first or second embodiment. The control device, such as a CPU, is used to control the computing device to perform computing tasks.
[0087] The computing device of this computing system is, for example, Figure 1 The computing device 1100 and the control device are, for example, Figure 1 The control device 1200 in the middle.
[0088] The computing system can be any type of electronic device, such as any type of terminal device, workstation, or server, etc., without any limitation.
[0089] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0090] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and apparatuses according to various embodiments of this specification. In this regard, each block in a flowchart or block diagram may represent a module, unit, or part of a circuit. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using hardware that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. Unless otherwise specified, implementation in hardware, implementation in software, and implementation using a combination of software and hardware can be equivalent.
[0091] Various embodiments of this specification have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. An instruction cache management method, characterized in that, include: Receive a memory access request for the instruction cache; wherein the instruction cache contains multiple cache groups, and each cache group contains multiple cache lines; Determine the target cache group mapped by the memory access request, and detect whether the target cache group contains the target instruction indicated by the memory access request; If the target instruction is missing in the target cache group and the target cache group is full, a target cache line to be replaced is selected from the target cache group based on the baseline age value of each cache line in the target cache group. The baseline age value of the cache line is obtained by downsampling the original age value of the corresponding cache line with a set sampling step size. The original age value represents the number of consecutive misses of the corresponding cache line. The baseline age value characterizes the replacement priority of the corresponding cache line in the target cache group. The larger the baseline age value of the cache line, the higher the replacement priority of the corresponding cache line. The target instruction is read from the lower-level memory of the instruction cache and written into the target cache line.
2. The method according to claim 1, characterized in that, The extraction step size is an integer power of 2; the original age value of the cache line is encoded in binary, the low-order field of the binary encoding is used to accumulate the number of consecutive misses within the range limited by the extraction step size, and the high-order field of the binary encoding is used to accumulate the base age value of the corresponding cache line.
3. The method according to claim 1, characterized in that, After writing the target instruction into the target cache line, the method further includes: Set the original age value of the target cache line to 0; and, Based on the weight value corresponding to the memory access request, determine the weighted miss count of the target instruction that was not hit in this cache line; Based on the weighted miss count, update the original age values of other cache lines in the target cache group.
4. The method according to claim 3, characterized in that, The weight value corresponding to the memory access request is determined based on at least one of the request type of the memory access request and the instruction address carried by the memory access request.
5. The method according to claim 4, characterized in that, The weight value corresponding to the memory access request is determined based on the request type of the memory access request. In the case that the memory access request is a prefetch instruction request, the weight value is a first weight value set for the prefetch instruction request, and the first weight value is greater than the default weight value.
6. The method according to claim 5, characterized in that, The first weight value is greater than or equal to the extraction step size, and less than twice the extraction step size.
7. The method according to claim 4, characterized in that, The weight value corresponding to the memory access request is determined based on the instruction address carried by the memory access request. If the instruction address belongs to any address segment in the weight configuration table, the weight value is a second weight value configured for the address segment, and the second weight value is greater than the default weight value.
8. The method according to claim 7, characterized in that, When the memory access request is a prefetch instruction request, the weight value is a first weight value set for the prefetch instruction request; When the memory access request is an instruction fetch request, the weight value corresponding to the memory access request is determined based on the instruction address carried by the memory access request.
9. The method according to any one of claims 3 to 8, characterized in that, Before determining the weighted miss count of the target instruction that was not hit in the current cache line, the method further includes: Detect whether there are any full-age cache lines in the other cache lines whose original age value has reached the upper limit; If the full-age cache line exists, then keep the original age values of the other cache lines unchanged; If the full-age cache line does not exist, then the step of updating the original age value of other cache lines in the target cache group is performed.
10. An instruction cache, characterized in that, The instruction cache contains multiple cache groups, and each cache group contains multiple cache lines. The instruction cache stores age information for each cache line, which includes an original age value and a baseline age value. The baseline age value of the cache line is obtained by downsampling the original age value of the corresponding cache line with a set sampling step size. The original age value represents the number of consecutive misses of the corresponding cache line, and the baseline age value characterizes the replacement priority of the corresponding cache line within its cache group. The larger the baseline age value of the cache line, the higher the replacement priority of the corresponding cache line.
11. An instruction cache, characterized in that, The instruction cache is configured to execute the method of any one of claims 1 to 9.
12. A computing device, characterized in that, include: Multiple computing units; An instruction cache is connected to the computing unit; the instruction cache is the instruction cache as described in claim 10 or 11.
13. A computing system, characterized in that, include: The control device and the computing device of claim 12, wherein the control device is used to control the computing device to perform computing tasks.