Caching system and method

Abstract

The invention relates to a cache system and method. A cache system 110 is provided to a system having a CPU 102 and a memory 200, the CPU having a stack pointer register 108 for storing a stack pointer representing an address in the main memory 200 at the top of the stack. A cache system 110 has a cache memory 116 structured into cache lines and cache controller circuitry operable to receive a stack pointer (SP), store a first cache line containing content of a first address range of bytes of a main memory, and store a second cache line containing content of a second address range of bytes of the main memory, the first address range comprises a stack pointer; and locking the first cache line to protect the first cache line from cache eviction.

Description

Technical Field

[0001] This disclosure relates to the use of caches and processor systems using caches. Background Art

[0002] Modern processors use processor cores which use caches to access data from memory. The purpose of a cache is to maintain CPU performance by accelerating access to frequently used data. A cache is fast memory. Some data from the main memory is stored in the cache so that it can be accessed more quickly than the data stored in the main memory. The cache size is limited and is organized as a set of lines, each line having a fixed size.

[0003] A cache may use a cache controller (also simply referred to as a "controller") which reads data from the main memory into the cache when needed. Generally, a line of data needs to be removed from the cache to make room for a new line of data.

[0004] CPU performance depends critically on cache operations because accessing data causes a performance penalty when data needs to be read from the slow main memory. Summary of the Invention

[0005] In an example, there is provided a cache system for a processor having a stack pointer register for storing a stack pointer which is the main memory address of the top of a stack, the cache system comprising:

[0006] a cache memory structured as cache lines; and

[0007] a cache controller circuitry operable to:

[0008] receive the stack pointer (SP),

[0009] store a first cache line containing the contents of a first address range of bytes of the main memory, the first address range including the stack pointer address; and

[0010] lock the first cache line to protect the first cache line from cache eviction.

[0011] Note that the stack pointer may point to an address in the address space of the device, i.e., the address is an address in the main memory. By storing a data cache line including the data at the address pointed to by the stack pointer, the contents of the stack can be quickly accessed in the cache without having to load data from the main memory. By locking this data line, this data line remains in the cache even if this data line would otherwise be replaced.

[0012] Those skilled in the art will recognize additional features and advantages upon reading the following detailed description and upon viewing the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The present disclosure is illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like reference numerals refer to similar or identical elements. The elements of the figures are not necessarily drawn to scale relative to each other. Features of the examples of the various figures can be combined unless they are mutually exclusive.

[0014] Figure 1 Illustrating a system including a CPU, a cache, and a memory;

[0015] Figure 2 Illustrating various parts of a memory address;

[0016] Figure 3 Illustrating a tag memory;

[0017] Figure 4 Illustrating a data cache line; and

[0018] Figure 5 Illustrating the hardware circuitry in a cache controller. DETAILED DESCRIPTION

[0019] Data cache

[0020] In an example, the computer system 100 has a CPU 102. The CPU has registers 106 that can be accessed very quickly, but these registers have a limited size. Therefore, a main memory 200 is also provided. The CPU has a load / store interface 104 that is connected to a cache 110 through a number of interconnects 118. The cache 110 includes a cache controller 112, a tag RAM 114, and a cache RAM 116. The cache 110 is connected to the main memory 200 through a bus 210.

[0021] The cache RAM 116 contains a number of data cache lines 400, and each data cache line contains a number of data words. For example, each data word can include 32 bits, and each data cache line can include 32 such words. In this case, 5 bits are needed to identify each individual word within the cache line (2 5 ^5 = 32); these five bits can be referred to as the word address.

[0022] The interconnect 118 includes address lines 220, read / write lines 222, and enable lines 224, all of which are connected to be driven by the load / store interface. The interconnect also includes data write lines 226 and data read lines 228. Note that while the address lines 220, write lines 226, and read lines 228 are schematically shown as single lines, these may in fact be multiple lines, such as 32 parallel lines to carry 32-bit data items or addresses.

[0023] To enable the CPU 102 to access the contents of the memory 200, which may contain program code or data, it outputs an address, e.g., a 32-bit address, on the address lines 220. The CPU also controls the read / write lines 222 to indicate whether it is going to read the contents of that address or write to it. The CPU also controls the enable lines 224 to enable the cache.

[0024] At least some (usually, most) of the possible memory locations in the main memory 200 are cacheable memory locations, that is, the data stored at that memory location in the main memory 200 can be stored in the cache 110 for faster access. When the CPU 102 accesses a cacheable memory location for writing or reading, the request can be output by the CPU load / store interface 104 and received by the cache controller 112. The cache controller 112 checks the memory address to determine whether the addressed location already exists in the cache (cache lookup).

[0025] In the example, the memory address 210 in the memory 200 is divided into a tag part 250, an index part 252, and a byte address part 254, see Figure 2 For example, the 16 most significant bits of the memory address can represent the tag part 250, the next 11 bits represent the index part 252, and the remaining 5 bits represent the write address part 254, which represents the location within the cache line. Note that these 5 bits correspond to the location within the cache line, i.e., the word address. The cache controller 112 calculates the tag RAM address from the index part 250 of the memory address 210 and compares the contents of that part of the tag RAM with the tag part 252 of the memory address 220. If and only if the corresponding tag RAM address contains the tag part 252, the cache already contains the contents of the memory address.

[0026] The cache RAM 116 can be, for example, a two-way cache, which contains two cache lines. The tag RAM reflects this because each tag RAM addressable word 300 specified by the index contains two tags 310, 320 ( Figure 3)。In addition, each of these two tags corresponds to a separate data cache line 400. Those skilled in the art will realize that, as an alternative, a three-way, four-way cache architecture can be used, where each index corresponds to three, four, or more cache lines 400 in the cache memory 116, and in this case, three, four, or more corresponding tags can be stored in each tag RAM addressable word.

[0027] If the tag matches the tag RAM content in either way, this indicates that the requested data is already in the cache (cache hit), and the read or write can be performed immediately using the cache data.

[0028] Alternatively, if the tag does not match, a cache miss has occurred. To access the requested data, the data must be downloaded from the memory 200. To achieve this, the cache controller must first load the data line containing the requested location from the main memory into the cache (refill operation); such a data line is called a cache line 400 and includes the data line that includes the data at the requested memory address. Note that the data line is the data within the address range of the data stored in the main memory 200. The address range has a lower value and an upper value. For example, a cache line can contain 32 words of data, each word having 32 bits, and the upper value of the address range is 31 greater than the lower value.

[0029] During refill, the cache controller will allocate one of the ways available at that index, which is used to store the newly read data from the main memory. Usually, the cache is fully occupied, so the cache controller must evict a line that has already been stored in the cache to make room (cache line eviction). If the evicted cache data has been modified, it is written back to the main memory (write-back), alternatively, the cache line is simply invalidated before being rewritten.

[0030] According to the least recently used (LRU) algorithm, cache line eviction can be controlled. The cache controller maintains a tag bit for each index, which indicates which way (among the 2 available ways) has been least recently accessed and thus which way should be preferentially evicted. The intention is to keep the most recently accessed data in the cache for performance reasons.

[0031] Program stack

[0032] The program stack is a commonly used data structure in memory. It is maintained by the microprocessor system and the operating software to temporarily store data (and program code addresses), and it is accessed very frequently. One of the registers 106 is the stack pointer register 108, which is used to store the stack pointer 410. The stack pointer 410 is the address in the main memory at the top of the stack. Data is pushed onto the stack to store data and popped from the stack to retrieve data, while the stack grows and shrinks in memory accordingly. In both cases, data is stored or accessed at the location of the stack pointer, and the stack pointer is then incremented or decremented accordingly.

[0033] By keeping the most recently and frequently accessed part of the stack (the top of the stack) in a fast local cache, the system maintains high performance by avoiding the degradation that would occur if the top of the stack were evicted from the cache.

[0034] The top of the stack is accessed very frequently, and because of this, it is often retained in the data cache by the normal operation of the LRU mechanism. However, there is no guarantee that the LRU mechanism will keep the cache line with the top of the stack in the cache.

[0035] Lock

[0036] In an example, by using the stack pointer 410 stored in the stack register 108, a portion of the data cache is identified as containing the top of the stack. This cache line is then locked so that it cannot be evicted even if it would normally be evicted by the LRU. As the top of the stack moves up and down through the memory address space, the stack pointer automatically indicates the address of the data that is to be locked in the cache. Thus, even if the LRU algorithm would otherwise remove the cache line for the data included in the stack pointer (i.e., the address in the main memory pointed to by the stack pointer), the cache line is preserved in the cache memory.

[0037] When a cache line is locked, it is protected from eviction by overriding the LRU tag at that index, and the replacement way is chosen for eviction even if the replacement way has been more recently used.

[0038] In terms of the reduced execution time of some test programs, the benefits of automatically locking the cache line containing the top of the stack as described here can be directly measured.

[0039] The cache line locking function can be performed by a cache controller. It is supplied with the tag and index portions of the locked memory address and a tag bit for indicating the validity of the locked address. The cache controller compares the lock tag with the lock index on each cache query. When the lock tag matches, if the cache query results in a miss, a subsequent refill operation, and a cache line eviction, the cache controller sets the tag bit to override the LRU. Thus, the locked cache line is protected from eviction.

[0040] Figure 5 Illustrated is a cache controller design that can be implemented in hardware that implements a two-way cache and locking.

[0041] Address line 220 is connected to index address line 512 that carries index portion 252 and tag address line 510 that carries tag portion 250.

[0042] Index address line 512 is connected to tag ram 114 and LRU 500. Tag RAM 114 has a first write output 506 and a second write output 508, which are connected to first tag comparator 502 and second tag comparator 504 respectively. Tag address line 510 is also connected to first and second tag comparators 502, 504.

[0043] Locking subsystem 526 receives a lock address on lock address line 528. The locking subsystem has a lock tag comparator 530 connected to address tag line 510 and a lock index comparator 532 connected to index tag line 512. The lock tag and index comparators 530, 532 are also connected to respective portions of lock address line 532. The outputs of lock comparators 530, 532 are connected to lock output 536 through an AND gate 534.

[0044] In use, when an address is input, LRU 500 outputs an LRU signal 520 that indicates which of the first and second cache lines corresponding to the address index has been most recently used. Tag RAM 114 outputs the tag in the first way 310 of the index location stored in tag ram 114 on first write output 506 and outputs the tag in the second way 320 of the index location stored in tag ram 114 as tag data on second write output 508. These are then compared in first and second comparators 502, 504 to output a first output 522 indicating a hit in the first way on first tag comparator 502 and a second output 524 indicating a hit in the second way on second tag comparator 504.

[0045] Similarly, the lock tags and index comparators 530, 532 compare the input on the lock address line 528 with the tag and index portions of the address input on the address line 220, and output a lock signal on the lock output 536 if the address is locked. The lock address line can deliver the locked address.

[0046] If a hit is indicated at the first tag output 522 or the second tag output 524, the cache controller can then simply access the data in the cache.

[0047] If no hit exists, data from the main memory must be loaded into the cache. If the lock output does not indicate a lock, the way of the data in the cache RAM 116 indicated by the LRU output is cleared and replaced with the data loaded from the main memory 200. If, in contrast, the lock output indicates that one of the ways is locked, the other way is cleared and replaced with the data loaded from the main memory.

[0048] An alternative version of this design allows multiple lines to be locked. The cache controller is supplied with multiple locked memory addresses, and for each locked memory address, the above comparison is repeated. If any lock tag matches the requested memory address at the same index during a cache query, in the case of a cache line miss, another tag bit is set to indicate that the LRU should be overridden. If all available ways in the available ways at that index are so locked, the cache controller resorts to using the LRU again to select a way for eviction.

[0049] The system can keep track of one or more previous values of the stack pointer. These are used again to selectively and dynamically lock data in the fast local cache so that the most recently accessed data at the top of the stack is not evicted.

[0050] For example, a lower limit and an upper limit can be pre-set in the address range of the data cache line. For example, in the case where the first cache line 400 having a first address range as in the above example contains 32 words represented by a five-bit word address, the predetermined lower limit can be three higher than the lower value of the address range, and the predetermined upper limit can be three lower than the upper value of the address range. See Figure 4 , Figure 4 illustrating cache line 400, lower value 402, upper value 404, lower threshold 406, and upper threshold 408. The first address range 414 in the main memory corresponding to the lower value 402 to the upper value 404 is the address range represented in this cache line 400.

[0051] When the stack pointer 410 is between the lower threshold and the upper threshold, the stack pointer then increases and decreases as data is stored in and retrieved from the stack, but there is no need to access the main memory 200 because the data in this address range is stored in the cache line 400 in the cache RAM 116.

[0052] However, if the stack pointer 410 exceeds the upper threshold 408, there is a risk that the stack pointer will continue to rise and exceed the upper value 404. This will then cause a stack miss and further latency. To mitigate this risk, the cache controller 110 can be arranged to read another data cache line corresponding to another address range immediately above the first address range 414. In some examples, this additional data line can also be locked. In this way, the risk of cache misses is reduced.

[0053] Similarly, if the stack pointer 410 drops below the lower threshold 406, the cache controller 110 can load another data cache line corresponding to another address range immediately below the first address range in this case. In some examples, this data line can be locked. In this way, the risk of cache misses is reduced.

[0054] Although specific examples have been illustrated and described herein, those of ordinary skill in the art will understand that various alternatives and / or equivalent implementations may be substituted for the specific examples shown and described without departing from the scope of the present invention. This application is intended to cover any adaptations or variations of the specific examples discussed herein. Accordingly, it is intended that the present invention be limited only by the claims and their equivalents.

[0055] In a first example, there is provided a cache system for a processor, the processor having a stack pointer register for storing a stack pointer, the stack pointer being an address in the main memory at the top of a stack, the cache system comprising: a cache memory structured as cache lines; and a cache controller circuitry operable to: receive the stack pointer (SP), store a first cache line containing the contents of a first address range of bytes of the main memory, the first address range including the stack pointer; and lock the first cache line to protect the first cache line from cache eviction.

[0056] When the stack pointer register contains a new stack pointer having an address outside the first address range, the cache controller circuitry is further operable to: store another cache line containing the contents of another address range including the new stack pointer in the cache; and lock the another cache line.

[0057] The cache system may lock the first cache line and the other cache line, but unlock any previous cache line corresponding to the previous stack pointer address.

[0058] The cache controller circuitry may also be operable to: lock a plurality of addresses of a corresponding cache line corresponding to the stack pointer.

[0059] The cache controller circuitry may also be operable to: determine a low threshold address in the first address range; determine a high threshold address in the first address range; determine whether the stack pointer exceeds the high threshold address, and if the stack pointer exceeds the high threshold address, prefetch another data cache line of an address range from main memory immediately above the first address range into the cache; or determine whether the stack pointer is below the second threshold, and if the stack pointer is below the second threshold, prefetch another data cache line of an address range from main memory immediately below the low threshold address.

[0060] The cache controller circuitry may also be operable to: lock the address of the other cache line including the prefetched data.

[0061] The low threshold address and / or the high threshold address may be located at a programmable position within the first cache line.

[0062] The cache controller circuitry may be operable to lock the address of the cache line in a dynamic manner.

[0063] The cache controller circuitry may be operable to: if a cache query results in a miss, cache line eviction, and cache line refill, lock the address by overriding the output of the least recently used (LRU) logic to protect the cache line from eviction.

[0064] The cache controller circuitry may include: a lock detection circuit operable to determine whether a requested address received from a processor corresponds to a locked address.

[0065] The cache memory may be organized as a multi-way set associative memory.

[0066] In an example, a microcontroller may be provided, the microcontroller including the cache system as described above.

[0067] The microcontroller may further include: a processor; and a main memory, wherein the cache system is communicatively coupled between the processor and the main memory to manage the data stream therebetween.

[0068] In an example, a method for managing a cache memory structured into cache lines may also be provided, the method comprising: receiving, by a cache controller circuit system, a stack pointer from a processor, the stack pointer indicating an address in main memory of a top of a stack; and locking, by the cache controller circuit system, a cache line including the top of the stack to protect the cache line from cache eviction.

[0069] The method may further include updating the stack pointer to an address not stored in the cache line in the cache memory, storing another cache line corresponding to the updated address; and locking the another cache line.

[0070] The method may further include locking a plurality of cache lines pointed to and / or pointed to by the SP.

[0071] In an example, the cache line corresponds to an address range in the main memory, and the method further comprises: determining whether the stack pointer exceeds a predetermined threshold address when the stack pointer is updated; and pre-fetching another data cache line from the main memory if the stack pointer exceeds the predetermined threshold address.

[0072] The method may further include locking the at least one cache line including the prefetched data.

[0073] The method may further include locking the address to protect the cache line from eviction by overwriting an output of least recently used (LRU) logic if the cache query results in a miss, cache line eviction, and cache line refill operation.

[0074] The method may further include determining, by the lock detection circuit, whether a request address received from the processor corresponds to a locked address.

[0075] It should be noted that the methods and apparatus as outlined in this document including preferred embodiments thereof may be used independently or in combination with other methods and apparatus disclosed in this document. In addition, features outlined in the context of an apparatus may also be applicable to the corresponding method and vice versa. In addition, all aspects of the methods and apparatus outlined in this document may be combined arbitrarily. In particular, the features of the claims may be combined with each other in any manner.

[0076] It should be noted that the description and the drawings merely illustrate the principles of the proposed method and system. Those skilled in the art will be able to implement various arrangements, which, although not explicitly described or shown herein, embody the principles of the present invention and are included within the spirit and scope of the present invention. Additionally, all examples and embodiments outlined in this document are primarily intended to be explicitly for explanatory purposes only to assist the reader in understanding the principles of the proposed method and system. Additionally, all statements providing the principles, aspects, and embodiments of the present invention, as well as specific examples of the present invention herein, are intended to include their equivalents.

Claims

1. A cache system for a processor having a stack pointer register for storing a stack pointer, the stack pointer being the address in main memory of the top of a stack, the cache system comprising: Cache memory,structured into cache lines; and A cache controller circuitry operable to: receiving the stack pointer (SP), storing a first cache line containing contents of a first address range of bytes of the main memory, the first address range including the stack pointer; and The first cache line is locked to protect the first cache line from cache eviction.

2. The cache system of claim 1 , wherein when the stack pointer register contains a new stack pointer having an address outside the first address range, the cache controller circuitry is further operable to: storing in the cache another cache line containing contents of another address range including the new stack pointer; and The other cache line is locked.

3. The cache system of claim 2, wherein the cache system locks the first cache line and the another cache line but unlocks any previous cache line corresponding to a previous stack pointer address.

4. A cache system as claimed in any preceding claim, wherein the cache controller circuitry is further operable to: A plurality of addresses of respective cache lines corresponding to the stack pointer are locked.

5. The cache system of claim 1 , wherein the cache controller circuitry is further operable to: determining a low threshold address in the first address range; determining a high threshold address in the first address range; determining whether the stack pointer exceeds the high threshold address, and if the stack pointer exceeds the high threshold address, prefetching into the cache another cache line of data from an address range in main memory immediately above the first address range; or A determination is made as to whether the stack pointer is below the second threshold, and if so, another cache line of data is pre-fetched from an address range in main memory immediately below the low threshold address.

6. The cache system of claim 5, wherein the cache controller circuitry is further operable to: The address of the other cache line including the prefetched data is locked.

7. A cache system as claimed in claim 5 or 6, wherein the low threshold address and the high threshold address are located at programmable locations within the first cache line.

8. A cache system as claimed in any preceding claim, wherein the cache controller circuitry is operable to lock the address of the cache line in a dynamic manner.

9. A cache system as claimed in any preceding claim, wherein the cache controller circuitry is operable to: If a cache query results in a miss, a cache line eviction, and a cache line refill, the address is locked to protect the cache line from eviction by overwriting the output of the least recently used (LRU) logic.

10. A cache system as claimed in any preceding claim, wherein the cache controller circuitry comprises: A lock detection circuit is operable to determine whether a request address received from the processor corresponds to a locked address.

11. A cache system as claimed in any preceding claim, wherein the cache memory is organised as a multi-way set associative memory.

12. A microcontroller comprising: A cache system as claimed in any preceding claim.

13. The microcontroller of claim 12, further comprising: processor; and Main memory, The cache system is communicatively coupled between the processor and the main memory to manage data flow therebetween.

14. A method for managing a cache memory structured into cache lines, comprising: receiving, by the cache controller circuitry from the processor, a stack pointer indicating an address in main memory of a top of a stack; and A cache line comprising a top of the stack is locked by the cache controller circuitry to protect the cache line from cache eviction.

15. The method of claim 14, further comprising: updating the stack pointer to an address not stored in the cache line in the cache memory, storing another cache line corresponding to the updated address; and The other cache line is locked.

16. The method of claim 14 or 15, further comprising: Lock multiple cache lines that are and / or were pointed to by the SP.

17. The method of claim 14, 15 or 16, wherein the cache line corresponds to an address range in the main memory, the method further comprising: When the stack pointer is updated, determining whether the stack pointer exceeds a predetermined threshold address; and If the stack pointer exceeds a predetermined threshold address, another data cache line is pre-fetched from main memory.

18. The method of claim 17, further comprising: The at least one cache line including the prefetched data is locked.

19. The method of any one of claims 14 to 18, further comprising: If the cache query results in a miss, cache line eviction, and cache line refill operation, the address is locked to protect the cache line from eviction by overwriting the output of the least recently used (LRU) logic.

20. The method of any one of claims 14 to 19, further comprising: A determination is made by the lock detection circuit whether a request address received from the processor corresponds to a locked address.

Images