input to oldest operation latency indication in set contention

Abstract

Systems, apparatuses, and methods are disclosed for dynamically adjusting cache policies to reduce execution core latency. A processor includes a cache subsystem. The cache subsystem includes one or more cache levels and one or more cache controllers. A cache controller divides a cache level into two test portions and a remaining portion. The cache controller applies a first policy to a first test portion and a second policy to a second test portion. The cache controller determines an amount of time it takes for the execution core to wait to access the first and second test portions. If the measured latency of the first test portion is less than the measured latency of the second test portion, the cache controller applies the first policy to the remaining portion. Otherwise, the cache controller applies the second policy to the remaining portion.

Description

Background Technology

[0001] Description of related technologies In addition to the main memory in a computing device, most modern computing devices also provide at least one level of cache memory (or cache). Typically, a cache is a small, high-speed memory used to store a limited number of copies of data and instructions used to perform various operations on functional blocks within the computing device closer to the execution operation. Caches are typically implemented using high-speed memory circuitry, such as static random-access memory (SRAM) integrated circuits or other types of memory circuitry. When a processor requests access to data stored in memory, the processor first determines whether a copy of the data is stored in the cache. If so, the processor accesses the cache, thereby facilitating more efficient data access. Attached Figure Description

[0002] The advantages of the methods and mechanisms described herein can be better understood by referring to the following description in conjunction with the accompanying drawings, in which: Figure 1 It is a block diagram of one implementation method of a computing system.

[0003] Figure 2 It is a block diagram of one implementation of a processor.

[0004] Figure 3 This is a block diagram of an implementation of a strategy for determining the optimal performance of a cache.

[0005] Figure 4 This is a flowchart outlining an implementation of a method for determining the optimal performance strategy for a cache.

[0006] Figure 5 This is a flowchart illustrating an implementation of a method for monitoring the oldest pending cache accesses.

[0007] Figure 6 This is a flowchart illustrating an implementation of a method for dynamically adjusting caching strategies to improve performance. Detailed Implementation

[0008] Numerous specific details are set forth in the following description to provide a thorough understanding of the methods and mechanisms presented herein. However, those skilled in the art should recognize that various implementations can be practiced without these specific details. In some cases, well-known structures, components, signals, computer program instructions, and techniques have not been shown in detail to avoid obscuring the methods described herein. It should be understood that, for the sake of simplicity and clarity, the elements shown in the figures are not necessarily drawn to scale. For example, the dimensions of some elements may be enlarged relative to other elements.

[0009] This document discloses various systems, devices, methods, and computer-readable media for dynamically adjusting cache policies to reduce execution core wait times. In one implementation, the processor includes at least an execution core and a cache subsystem. The cache subsystem includes one or more cache levels and one or more cache controllers. The cache controller divides the cache level into three parts, including two test parts and a third part different from the two test parts. The cache controller applies a first policy to the first test part and a second policy to the second test part. Each of the first and second policies specifies behaviors and attributes, such as cache line replacement and insertion policies, request priority, speculative hints, preemptive response generation, etc. For example, the preemptive response generation policy specifies whether the cache should preemptively send a response before it is known to be in a cache hit / miss state or a DRAM page open / close state. The cache controller also determines the amount of time spent by the execution core waiting to access the first and second test parts. In various embodiments, the cache controller itself monitors and / or measures this amount of time. In other embodiments, another component monitors and / or measures this amount of time and the cache controller accesses the monitored and / or measured amount of time (e.g., stored in a memory location, register, etc.). Various such implementations are possible and various such implementations are envisioned. If the measured latency of the first test portion is less than the measured latency of the second test portion, the cache controller applies the first policy to the third portion. Otherwise, if the measured latency of the second test portion is less than the measured latency of the first test portion, the cache controller applies the second policy to the third portion. In some implementations, the third portion represents the remaining portion of the cache (i.e., all the remaining portion of the cache), and in some implementations, the third portion represents less than the remaining portion of the cache. Although the remaining portion of the cache is frequently referred to in the following description for ease of discussion, it should be understood that, depending on the implementation, all or less of the remaining portion is expected.

[0010] Now for reference Figure 1 The diagram illustrates a block diagram of one implementation of a computing system 100. In one implementation, the computing system 100 includes at least processors 105A-N, input / output (I / O) interfaces 120, a bus 125, memory controllers 130A-N, a network interface 135, and memory devices 140A-N. In other implementations, the computing system 100 includes other components (e.g., a display controller) and / or the computing system 100 is arranged differently. Processors 105A-N represent any number of processors included in the system 100.

[0011] In one implementation, processor 105A is a general-purpose processor, such as a central processing unit (CPU). In another implementation, processor 105N is a data-parallel processor with a highly parallel architecture. Data-parallel processors include graphics processing units (GPUs), digital signal processors (DSPs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc. In one implementation, processor 105A includes at least an execution core 110A, a cache subsystem 115A, and a memory controller 130A. Similarly, processor 105N includes at least an execution core 110N, a cache subsystem 115N, and a memory controller 130N. It should be noted that execution cores 110A-N may also be referred to herein as execution units. It should also be noted that processors 105A-N may include other components, which are not shown to avoid obscuring the diagram.

[0012] In one implementation, cache subsystems 115A-N include control logic for monitoring the amount of time spent by execution cores 110A-N waiting for cache accesses. In one implementation, the control logic applies different strategies to different parts of a given cache level. Furthermore, in this implementation, the control logic tracks the amount of time spent by the execution cores waiting for access to two separate parts of a given cache level. If the amount of time spent waiting for a first cache part is less than the amount of time spent waiting for a second cache part, the control logic adopts the strategy applied to the first cache part and applies this strategy to the remaining parts of the given cache level (a third part different from the first and second parts). Otherwise, if the amount of time spent waiting for the second cache part is less than the amount of time spent waiting for the first cache part, the control logic adopts the strategy applied to the second cache part and applies this strategy to the remaining parts of the given cache level. In other implementations, the control logic monitors other numbers of parts of a given cache level and applies different strategies to these parts.

[0013] Memory controllers 130A-N represent any number and type of memory controllers coupled to any number and type of memory devices 140A-N. While memory controllers 130A-N are shown as residing within processors 105A-N, this indicates only one implementation. In another implementation, memory controllers 130A-N are external to processors 105A-N. It should also be noted that a portion of the cache subsystems 115A-N may be residing within execution cores 110A-N, while another portion may be residing outside execution cores 110A-N. For example, in one implementation, the Level 1 (L1) cache and Level 2 (L2) cache are located within execution cores 110A-N, while the Level 1 cache (LLC) is located outside execution cores 110A-N.

[0014] Memory devices 140A-N represent any number and type of memory devices. For example, the types of memory in memory devices 140A-N include Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), NAND flash memory, NOR flash memory, ferroelectric random access memory (FeRAM), or other memory. I / O interface 120 represents any number and type of I / O interface (e.g., Peripheral Component Interconnect (PCI) bus, PCI Expansion (PCI-X), PCIE (PCI High Speed) bus, Gigabit Ethernet (GBE) bus, Universal Serial Bus (USB)). Various types of peripheral devices (not shown) are coupled to I / O interface 120. Such peripheral devices include (but are not limited to) displays, keyboards, mice, printers, scanners, joysticks or other types of game controllers, media recording devices, external storage devices, network interface cards, etc. Network interface 135 is used to receive and send network messages across a network.

[0015] In various implementations, computing system 100 is any of a computer, laptop, mobile device, game console, server, streaming device, wearable device, or various other types of computing system or device. It should be noted that the number of components in computing system 100 varies depending on the implementation. For example, there may be more components than in... Figure 1 Each component may be more or less than the number shown. It should also be noted that the computing system 100 may include... Figure 1 Other components not shown. Additionally, in other implementations, it can be achieved through... Figure 1 The computing system 100 is constructed in different ways as shown.

[0016] Now transferred to Figure 2The diagram illustrates a block diagram of one implementation of processor flow 200. In one implementation, processor 200 includes at least an execution core 205, a cache subsystem 210, a control unit 215, and a memory subsystem 220. It should be noted that processor 200 may also include other components, which are not shown to avoid obscuring the diagram. Core 205 represents any number of processor cores for executing instructions of one or more programs and / or an operating system. In one implementation, the circuitry of processor 200 includes processor 105A and / or processor 105N (…). Figure 1 )middle.

[0017] In one implementation, core 205 continuously transmits the oldest pending physical address (or some other subset of identifiable information, such as a cache index) to control unit 215. For example, in one implementation, the instruction cache or data cache sends the oldest pending address for an access to the L2 cache of cache subsystem 210 to control unit 215. In one implementation, a valid bit is also transmitted to indicate that core 205 is waiting for this operation in the performance-critical path. Control unit 215 monitors the IDs of these instructions and divides these instruction IDs into two or more buckets. Control unit 215 applies different strategies to these buckets, such as cache replacement / insertion strategies, request prioritization for the entire memory subsystem 220, speculative hints (e.g., whether a response should be preemptively sent before a known cache hit / miss state or DRAM page open / close state), etc. Control unit 215 counts the number of cycles spent waiting for instructions in a given bucket to cache subsystem 210, and uses the count to determine the performance-optimal strategy. It should be noted that although the control unit 215 is shown as separate from the cache subsystem 210, in other implementations, the control unit 215 is integrated into the cache subsystem 210.

[0018] The strategies for allocating cache lines to different buckets affect various settings of the cache subsystem 210. For example, in one implementation, there is a usage period associated with each cache line in an index, where the usage period is related to other cache lines in the index. The usage period can be set to various values ​​when accessing a corresponding cache line or when accessing different cache lines. Alternatively, the usage period can be adjusted when performing various operations on the same index. Different strategies can adjust these settings in different ways. For example, in a first strategy, a line may be inserted into the cache and assigned an intermediate usage period, while in a second strategy, a line may be inserted as the shortest usage period. In a third strategy, the line may not be inserted into the cache but may be inserted into the next level of cache. For example, in one implementation, the line may not be allocated in the L2 cache but in the third-level (L3) cache. Different strategies can adjust these settings as well as other settings and parameters associated with caches, cache portions, cache lines, etc.

[0019] In one implementation, a cache subsystem 210 of a given level is divided into a first part, a second part, and a remaining part. The first part corresponds to a first subset of cache indexes, the second part corresponds to a second subset of cache indexes, and the remaining part corresponds to all other cache indexes not in the first or second subset. In one implementation, the first and second parts are smaller than the remaining part. In one implementation, a control unit 215 applies a first policy to the first part and a second policy to the second part. The control unit 215 monitors the latency of instructions accessing cache indexes in the first and second parts. For example, the control unit 215 calculates a first total latency for the first part of the cache subsystem 210. The control unit 215 also calculates a second total latency for the second part of the cache subsystem 210. If the first total latency is less than the second total latency, the control unit 215 applies the first policy to the remaining part of the cache subsystem 210. Otherwise, if the second total latency is less than the first total latency, the control unit 215 applies the second policy to the remaining part of the cache subsystem 210. By tracking latency rather than cache hit / miss measurements, control unit 215 is better able to determine which behaviors are impacting performance. This allows for the application of better performance-optimizing strategies to the remainder of cache subsystem 210 compared to conventional methods.

[0020] Now for reference Figure 3 This diagram illustrates one implementation of a strategy for determining the optimal performance of cache 300. Cache 300 represents any level of cache in the cache hierarchy (e.g., ...). Figure 2The cache subsystem 210. Cache 300 includes at least cache memory 310 and control unit 315. In one implementation, cache memory 310 is divided into portion 320A, portion 320B, and remaining portion 320C. For example, in one implementation, portion 320A includes a first subset of cache indexes, portion 320B includes a second subset of cache indexes, and remaining portion 320C includes all remaining cache indexes not in the first or second subset. In another implementation, portion 320A includes a first subset of cache indexes, portion 320B includes a second subset of cache indexes, and remaining portion 320C includes cache indexes not in the first or second subset, but not all remaining cache indexes. Control unit 315 applies policy 330A to portion 320A, and control unit 315 applies policy 330B to portion 320B. For the purposes of discussion, it is assumed that policy 330A differs from policy 330B.

[0021] Depending on the implementation, policies 330A and 330B assign different types of behaviors and / or rules to their corresponding parts 330A and 330B, respectively. For example, policies 330A and 330B may specify one or more of the following: replacement policy, insertion policy, request prioritization, speculative hints, bypass decisions, etc. Control unit 315 counts the number of cycles for a given part to have associated processor core wait times. For example, in one implementation, wait time counter 325A counts the number of cycles spent waiting for a response from cache 300 for a request corresponding to a cache index in part 320A, and wait time counter 325B counts the number of cycles spent waiting for a response from cache 300 for a request corresponding to a cache index in part 320B. In one implementation, control unit 315 monitors the values ​​of wait time counters 325A and 325B, and counters with lower values ​​are considered to have a better performance-optimized policy. This better performance-optimized policy is then applied to the remaining part 320C. For example, after a given test interval, if the wait time counter 325A is less than the wait time counter 325B, then the cache memory 310 applies policy 330A to the cache index of the remaining portion 320C. Otherwise, if the wait time counter 325B is less than the wait time counter 325A after a given test interval expires, then the cache memory 310 applies policy 330B to the cache index of the remaining portion 320C. If the wait time counter 325B is equal to the wait time counter 325A, then the control unit 315 extends the test interval and then checks the value of counter 325A-B after the additional amount of time. Alternatively, instead of extending the test interval, in some embodiments, the control unit 315 simply selects one of the policies to apply to the remaining portion 320C.

[0022] In one implementation, control unit 315 resets the latency counters 325A-B according to a given schedule, and then control unit 315 runs the monitoring test again. In one implementation, control unit 315 uses the same strategy 330A-B as the previous test, while in another implementation, control unit 315 changes one or both of the strategies 330A-B for the new test. It should be noted that the example with two separate parts 320A-B and two corresponding strategies 330A-B only indicates one implementation. In other implementations, cache memory 310 is divided into other numbers of parts, each with its own strategy. Control unit 315 then selects the best strategy from three or more strategies to apply to the remaining parts 320C based on which part has the lowest latency counter.

[0023] Turn now Figure 4This paper illustrates one implementation of a method 400 for determining the optimal performance strategy for a cache. For illustrative purposes, the steps and... Figures 5 to 6 The steps are shown in sequence. However, it should be noted that in various implementations of the described method, they may be performed simultaneously, in a different order than shown, or one or more of the described elements may be omitted entirely. Additional elements may also be performed as needed. Any of the various systems or devices described herein is configured to implement method 400.

[0024] The cache controller applies a first policy to a first cache portion and a second policy to a second cache portion (box 405). For the purposes of this discussion, it is assumed that the second policy differs from the first policy. Each of the first and second policies specifies various types of settings and behaviors followed by the corresponding cache portion. Policies specify behaviors such as cache replacement / insertion policies, request priorities across the entire memory subsystem, speculative hints (e.g., whether a response should be preemptively sent before a known cache hit / miss state or DRAM page open / close state), etc. For example, in one implementation, there is a usage period associated with each cache line at a given index, where the usage period is related to other cache lines at the given index. Different policies may specify which value the usage period is set to when accessing a corresponding cache line or when accessing different cache lines at the same index. In one case, in the first policy, when a line is inserted, the line is set to an intermediate usage period, while in the second policy, when the line is inserted, the line is set to a minimum usage period. In this scenario, a third strategy can specify that the line should not be inserted into the cache, but rather into the next level cache.

[0025] Furthermore, the cache controller monitors the amount of time spent by the execution core waiting to access the first and second cache portions (box 410). In one implementation, the cache controller monitors the time from when the core issues a request to when the request is serviced (e.g., when the core receives a response from the cache). For example, in one implementation, a first time point is recorded when the core issues a request to the cache. Next, a second time point is recorded when the core receives a response from the cache. The difference between the second time point and the first time point is then calculated and used as a measurement of the amount of time in box 410. In one implementation, this difference is measured based on the number of clock cycles that have elapsed between the first and second time points.

[0026] After a given test interval has elapsed, the cache controller determines whether a first amount of time spent by the execution core waiting for a request to access a first cache portion is greater than a second amount of time spent by the execution core waiting for a request to access a second cache portion (box 415). In one implementation, the first amount of time is the average time spent by the execution core waiting for each access to the first cache portion. In this implementation, the second amount of time is the average time spent by the execution core waiting for each access to the second cache portion.

[0027] If the first time value is greater than the second time value (condition box 420, "Yes" branch), the cache controller applies the second policy to the remainder of the cache (box 430). Otherwise, if the first time value is less than the second time value (condition box 420, "No" branch), the cache controller applies the first policy to the remainder of the cache (box 425). After boxes 425 and 430, method 400 ends. If the first time value equals the second time value, the cache controller can extend the test interval. Alternatively, the cache controller can change either the first or second policy and rerun the test.

[0028] Now for reference Figure 5 This illustrates one implementation of method 500 for monitoring the oldest pending cache access. The execution unit transmits an identifier (ID) of the oldest pending cache access to the control unit (box 505). For example, in one implementation, the execution unit's instruction cache or data cache sends the ID of the oldest pending address to the L2 cache. The ID can be a physical address, a virtual address, a cache index, or some other attribute of the access. The execution unit also sends valid bits with the ID to indicate whether the core is waiting for this access in a performance-critical path (box 510).

[0029] The control unit monitors the received IDs and divides them into two separate groups (box 515). In some cases, the control unit divides the IDs into more than two separate groups. The control unit applies a first behavior policy to the first group and a second behavior policy to the second group (box 520). The control unit then counts the number of cycles that an access to each group causes the execution unit to wait (box 525). Next, the control unit applies the behavior policy of the group with fewer wait cycles to the remainder of the cache (box 530). After box 530, method 500 ends.

[0030] Turn now Figure 6This illustrates one implementation of a method 600 for dynamically adjusting a cache strategy to improve performance. The control unit tests a first strategy for a first cache portion and a second strategy for a second cache portion (box 605). The duration of the test period can vary depending on the implementation. If the latency associated with the first cache portion is greater than the latency associated with the second cache portion (condition box 610, "Yes" branch), the control unit applies the strategy for the second cache portion to the remaining portion of the cache (box 630). Furthermore, the control unit generates a new (third) strategy, different from the first strategy, to apply to the first cache portion (box 635). In one implementation, one or more attributes selected for the new third strategy are randomly generated. In another implementation, one or more attributes are set to values ​​predicted to improve the performance of the corresponding execution unit. The control unit then reruns the test using the new third strategy applied to the first cache portion and the unchanged strategy for the second cache portion (box 640). After box 640, method 600 returns to condition box 610.

[0031] If the latency associated with the first cache portion is not greater than the latency associated with the second cache portion (condition box 610, "No" branch), the control unit applies the policy of the first cache portion to the remaining portion of the cache (box 615). Furthermore, the control unit generates a new third policy to apply to the second cache portion (box 620), which differs from the second policy. The control unit then reruns the test using the unchanged policy of the first cache portion and the new third policy applied to the second cache portion (box 625). After box 625, method 600 returns to condition box 610. By using method 600, the control unit is able to dynamically adjust the policy at runtime to achieve higher performance. The policy is not a fixed or predetermined policy, but rather adaptable to the operating conditions of the host computing system.

[0032] In various implementations, the methods and / or mechanisms described herein are implemented using program instructions from a software application. For example, program instructions executable by a general-purpose processor or a special-purpose processor are envisioned. In various implementations, such program instructions are represented by a high-level programming language. In other implementations, the program instructions are compiled from the high-level programming language into binary, intermediate, or other forms. Alternatively, program instructions describing the behavior or design of the hardware are written. Such program instructions are represented by a high-level programming language such as C. Alternatively, a hardware design language (HDL) such as Verilog is used. In various implementations, the program instructions are stored on any of a variety of non-transitory computer-readable storage media. During use, the computing system can access the storage media to provide the program instructions to the computing system for program execution. Generally, such a computing system includes at least one or more memories and one or more processors configured to execute the program instructions.

[0033] It should be emphasized that the above implementations are merely non-limiting examples. Many variations and modifications will be apparent to those skilled in the art once the above disclosure is fully understood. The appended claims are intended to cover all such variations and modifications.

Claims

1. A cache subsystem, comprising: Cache; as well as A cache controller coupled to the cache, wherein the cache controller is configured to: Receive the identifier of the oldest pending cache access from the execution core; The received identifiers are divided into a first bucket of identifiers and a second bucket of identifiers; Apply the first strategy to the first bucket of the identifier and apply the second strategy to the second bucket of the identifier; The cache is divided into a first cache portion corresponding to the first bucket with an identifier and a second cache portion corresponding to the second bucket with an identifier; Monitor the first amount of time spent by the execution core waiting for instructions to access the first cache portion; as well as In response to determining that the latency associated with the first cache portion is less than the latency associated with the second cache portion, the first policy is applied to the third cache portion.

2. The caching subsystem as described in claim 1, wherein the first strategy is one or more of the following: a cache line replacement strategy, a cache line insertion strategy, a request priority, speculative hint generation, or preemptive response generation specified by the first cache portion.

3. The cache subsystem of claim 1, wherein each of the identifiers corresponds to the oldest pending cache access transferred from the execution core.

4. The cache subsystem as described in claim 1, wherein the cache controller is configured to: Monitor the second amount of time spent by the execution core waiting for instructions to access the second cache portion; The determining condition is that the first time value is less than the second time value; as well as In response to the first time amount being not less than the second time amount, the second strategy is applied to the third cache portion.

5. The cache subsystem of claim 4, wherein in response to the first time amount for the first cache portion being less than the second time amount for the second cache portion, the cache controller is configured to: Create a third strategy; The third strategy is applied to the second cache portion, while the first strategy is applied to the first cache portion; as well as In response to the third wait time for the second cache portion being less than the fourth wait time for the first cache portion, the third strategy is applied to the remaining portion of the cache.

6. The cache subsystem of claim 1, wherein the cache controller includes a plurality of counters, each of the counters being configured to indicate the number of cycles spent waiting for a response for a request corresponding to a portion of the cache.

7. The cache subsystem of claim 1, wherein each identifier includes a valid bit indicating that the execution core is waiting for a corresponding access in a performance-critical path.

8. A method comprising: The cache controller receives the identifier of the oldest pending cache access from the execution core; The cache controller divides the received identifiers into a first bucket and a second bucket. The cache controller applies a first policy to the first bucket of the identifier and a second policy to the second bucket of the identifier; The cache controller divides the cache into a first cache portion corresponding to the first bucket and a second cache portion corresponding to the second bucket. Monitor the first amount of time spent by the execution core waiting for instructions to access the first cache portion; as well as In response to determining that the latency associated with the first cache portion is less than the latency associated with the second cache portion, the first policy is applied to the third portion of the cache.

9. The method of claim 8, wherein the first strategy is one or more of the following: a cache line replacement strategy, a cache line insertion strategy, a request priority, speculative hint generation, or preemptive response generation for the first cache portion.

10. The method of claim 8, wherein each of the identifiers corresponds to the oldest pending cache access transferred from the execution core.

11. The method of claim 8, further comprising: Monitor the second amount of time spent by the execution core waiting for instructions to access the second cache portion; The determining condition is that the first time value is less than the second time value; as well as In response to the first time amount being not less than the second time amount, the second strategy is applied to the third part of the cache.

12. The method of claim 11, wherein in response to the first time amount for the first cache portion being less than the second time amount for the second cache portion, the method further comprises: Create a third strategy; The third strategy is applied to the second cache portion, while the first strategy is applied to the first cache portion; as well as In response to the third wait time for the second cache portion being less than the fourth wait time for the first cache portion, the third strategy is applied to the remaining portion of the cache.

13. The method of claim 12, wherein one or more attributes of the third strategy are randomly generated.

14. The method of claim 8, wherein the first time quantity is equal to the average number of clock cycles spent by the execution core waiting to access the first cache portion.

15. A system comprising: Memory; as well as A processor, the processor being coupled to the memory; The processor described herein includes the cache subsystem as claimed in claim 1.

Images