Method and apparatus for transferring data within a hierarchical cache circuit - Patents.com

Abstract

An aspect of the present disclosure relates to an apparatus including a processing circuit, a first cache circuit, and a second cache circuit, the second cache circuit having an access latency higher than an access latency of the first cache circuit, wherein the second cache circuit, in response to receiving a request for data stored in the second cache circuit, identifies the data as pseudo-invalid data and provides the data to the first cache circuit. The second cache circuit identifies the pseudo-invalid data as valid data in response to receiving an eviction instruction indicating that the first cache circuit is to evict the data, and in response to determining that the data has not been modified since the data was provided to the first cache circuit.

Description

[Technical Field]

[0001] The present technology relates to the field of data storage, and more particularly to cache storage circuits.

[0002] In some processing systems, a processor is coupled to a memory. The memory stores data accessed by the processor. For example, the memory may be dynamic random access memory (DRAM). The latency associated with accessing data stored in this memory may be relatively long. Therefore, some such systems provide a cache circuit that conceptually resides "between" the processor and the memory. The cache stores temporary copies of data from the memory for provision to the processing circuit. The cache has a smaller capacity than the memory, but as a tradeoff, the latency associated with accessing the cache is less than the latency associated with the memory. Therefore, data access requests from the processor for data stored in the cache may be processed more quickly than requests for data not stored in the cache.

[0003] In some systems, multiple caches are provided. These may have a hierarchical structure with multiple levels of cache. Each level is progressively "closer" to the processor, with closer levels having lower access latency but also lower capacity. Data travels back and forth between memory, levels of cache, and the processor according to a caching scheme. For example, in response to a processor issuing a data request, the request may first propagate through cache levels and then to memory until the data is found. The data is then provided to the processor and may be stored, for example, in the cache level closest to the processor. Later, the data may be evicted to a cache level further away from the processor or returned to memory.

[0004] The use of such hierarchical cache structures consumes power and transmission bandwidth, and it would be desirable to improve the efficiency of operation of such hierarchical cache structures. Summary of the Invention

[0005] At least some embodiments comprise an interconnection device, comprising: a processing circuit configured to issue an access request for the data; a first cache circuit for storing a temporary copy of data from the memory for providing to the processing circuitry in response to a corresponding access request; a second cache circuit for storing a temporary copy of data from the memory for providing to the first cache circuit in response to a corresponding access request, the second cache circuit having an access latency higher than the access latency of the first cache circuit; The second cache circuit, in response to receiving a request for data stored in the second cache circuit, Identifying the data as pseudo-invalid data; providing the data to a first cache circuit; The second cache circuit, in response to receiving an eviction instruction indicating that the first cache circuit is to evict the data, An interconnection device is provided that identifies the pseudo-invalid data as valid data in response to determining that the data has not been modified since the data was provided to the first cache circuit.

[0006] A further embodiment is a method comprising: storing a temporary copy of data from the memory in a first cache circuit for providing to the processing circuit in response to a corresponding access request; storing a temporary copy of the data from the memory in a second cache circuit for providing to the first cache circuit in response to a corresponding access request, the second cache circuit having a higher access latency than the access latency of the first cache circuit; transmitting a request for data stored in the second cache circuit from the first cache circuit to the second cache circuit; In response to the second cache circuit receiving a request for the data, identifying the data as pseudo-invalid data by a second cache circuit; providing the data from the second cache circuit to the first cache circuit; sending an eviction notification from the first cache circuit to the second cache circuit, the eviction notification indicating that the first cache circuit will evict the data; and identifying, by the second cache circuit, the pseudo-invalid data as valid data in response to the second cache circuit receiving the eviction notification and in response to determining that the data has not been modified since it was provided to the first cache circuit.

[0007] A further embodiment is a system comprising: A memory circuit; a processing circuit configured to issue access requests relating to data stored in the memory circuit; a first cache circuit for storing a temporary copy of data from the memory circuit for providing to the processing circuit in response to a corresponding access request; a second cache circuit for storing a temporary copy of data from the memory circuit for providing to the first cache circuit in response to a corresponding access request, the second cache circuit having an access latency higher than the access latency of the first cache circuit; The second cache circuit, in response to receiving a request for data stored in the second cache circuit, Identifying the data as pseudo-invalid data; providing the data to a first cache circuit; The second cache circuit, in response to receiving an eviction instruction indicating that the first cache circuit is to evict the data, A system is provided that identifies the pseudo-invalid data as valid data in response to determining that the data has not been modified since the data was provided to the first cache circuit.

[0008] Further aspects, features, and advantages of the present technique will become apparent from the following description of examples, read in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0009] [Figure 1] 1 illustrates a schematic representation of an exemplary device. [Figure 2] 2 illustrates an operational flow by which a data request may be processed by the device of FIG. 1. [Figure 3A] 10 is a schematic diagram illustrating data movement between caches according to a comparative example; [Figure 3B] 10 is a schematic diagram illustrating data movement between caches according to a comparative example; [Figure 3C] 10 is a schematic diagram illustrating data movement between caches according to a comparative example; [Figure 4A] 1 illustrates an exemplary method for storing and moving data between caches. [Figure 4B] 1 illustrates an exemplary method for storing and moving data between caches. [Figure 4C] 1 illustrates an exemplary method for storing and moving data between caches. [Figure 4D] 1 illustrates an exemplary method for storing and moving data between caches. [Figure 4E] 1 illustrates an exemplary method for storing and moving data between caches. [Figure 5A] 1 illustrates an exemplary operational flow associated with the eviction process. [Figure 5B]1 illustrates an exemplary operational flow associated with the eviction process. DETAILED DESCRIPTION OF THE INVENTION

[0010] As described above, a processing unit may include a processing circuit and a cache circuit that provides a cache hierarchy. The hierarchy may include a first cache circuit closest to the processing circuit and a second cache circuit that has a higher access latency than the first cache circuit.

[0011] In such a comparative example, data may be transferred from the second cache circuit to the first cache circuit (e.g., in response to a request from the processing circuit). The data is then marked as "invalid" in the second cache circuit (a non-inclusive cache hierarchy system). Later, the (possibly modified) data is evicted from the first cache circuit and sent back to the second cache circuit. In some operating situations, for example, when a working data set (on which a processor is performing data processing operations and thus repeatedly accessing data) is larger than the capacity of the first cache circuit but smaller than the capacity of the second cache circuit, a "ping-pong" effect may occur, in which given data is repeatedly transmitted between the first cache circuit and the second cache circuit. This consumes power and transmission bandwidth.

[0012] In one example of the present disclosure in which this ping-pong effect is reduced, an apparatus includes a processing circuit configured to issue access requests for data. The apparatus also includes a first cache circuit for storing a temporary copy of the data from a memory for providing to the processing circuit in response to the corresponding access request. The apparatus further includes a second cache circuit for storing a temporary copy of the data from the memory for providing to the first cache circuit in response to the corresponding access request, the second cache circuit having a higher access latency than the access latency of the first cache circuit. Thus, the first cache circuit and the second cache circuit form a hierarchical structure, which may be, for example, a non-inclusive cache hierarchy in which given data is not held as valid in the first cache circuit and the second cache circuit simultaneously. Providing given data between the first cache circuit and the second cache circuit may be performed by transmitting a cache line containing the given data.

[0013] In response to receiving a request for data stored in the second cache circuit, the second cache circuit provides the data to the first cache circuit and identifies the data as pseudo-invalid data, which is effectively a provisional indication that the data may become invalid.

[0014] The first cache circuit then evicts the data, e.g., to make space for the data newly contained in the first cache circuit. The second cache circuit then receives an eviction instruction indicating that the first cache circuit is evicting the data. For example, the first cache circuit may send an eviction instruction to the second cache circuit.

[0015] In response to receiving this indication, the second cache circuit determines whether the data has been modified since it was provided to the first cache circuit. If not, the second cache circuit identifies the pseudo-invalid data as valid data. Thus, the data can be deleted from the first cache circuit without being sent back to the second cache circuit, and because the data has not been modified, the pseudo-invalid copy of the data in the second cache circuit still has the correct value and can therefore be marked as valid.

[0016] In this way, power and bandwidth are saved compared to the comparative example described above in which no transfer of data from the first cache circuit to the second cache circuit is performed and therefore no pseudo-invalidation is implemented.

[0017] Eviction from the first cache circuit is as described above. However, the second cache circuit may also apply eviction logic to determine data to evict therefrom. In one example, the second cache circuit is configured to select data to evict, where the selecting includes preferentially selecting pseudo-invalid data over valid data. The selecting may further include preferentially selecting invalid data over pseudo-invalid data. Thus, the second cache circuit may apply an eviction priority in which invalid data has the highest priority for eviction, followed by pseudo-invalid data, and finally valid data has the lowest eviction priority. This allows pseudo-invalid data to be preferentially retained in the second cache circuit instead of invalid data, but not at the expense of evicting valid data.

[0018] The second cache circuit may be configured to maintain a priority indicator associated with valid data, with lower priority data being evicted before higher priority data. The second cache circuit may be configured to identify data as pseudo-invalid by assigning the lowest priority indicator value to that data. This allows the identification of pseudo-invalidity to be configured within the system without significant system modifications. However, this also removes a potential priority level that might otherwise have been assigned to valid data.

[0019] Alternatively, the second cache circuit may be configured to identify the data as pseudo-invalid data by setting a pseudo-invalidity indicator for the data, which provides an effective way of identifying data as pseudo-invalid without compromising performance in terms of the number of priority levels that can be assigned to valid data.

[0020] The above describes functionality associated with eviction from a first cache circuit where data has not been modified since being provided by a second cache circuit. In one example, in response to determining that the data has been modified since being provided to the first cache circuit, the second cache circuit sends a return request for the data to the first cache circuit. Then, in response to receiving the return request, the first cache circuit provides the (modified) data to the second cache circuit to restore the pseudo-invalid data. This example can therefore process not only unchanged data, but also modified data. Although data is still sent from the first cache circuit to the second cache circuit if it has been modified, in many cases the data is unchanged, thus still resulting in significant power and bandwidth savings compared to a comparable system in which the present disclosure is not implemented.

[0021] In one example, the second cache circuit is configured to provide data along with an associated change indicator to the first cache circuit. The first cache circuit is then configured to, in response to modifying the data, set the change indicator to indicate the change. For example, the change indicator may be a bit that is inverted by the first cache circuit. When evicting the data, the first cache circuit sends an eviction instruction to the second cache circuit that includes the change indicator. The second cache circuit can then determine that the data has been modified based on the change indicator.

[0022] The modification indicator may be provided, for example, in response to a determination by the second cache circuit that the cache line containing the data is a dirty cache line. If the cache line is a clean cache line, the modification indicator may optionally not be provided, since its status as a clean cache line indicates that the cache line has not been modified. Thus, the second cache circuit can absorb multiple dirty evictions of the data from the first cache circuit without the data being transmitted and without writing the data back to memory. Thus, power and bandwidth incurred are reduced.

[0023] In one example, when a first cache circuit evicts data identified as pseudo-invalid data in a second cache circuit, the first cache circuit is configured not to provide the data to the second cache circuit unless there is a request for the data from the second cache circuit. Thus, power and bandwidth savings can be achieved by not allowing data to be transmitted from the first cache circuit to the second cache circuit unless requested by the second cache circuit.

[0024] In one example, the second cache circuit is configured to operate as a shared cache for the processing circuit and the second processing circuit, which allows the present disclosure to be implemented in a system with multiple processors, each having a respective first cache circuit, but sharing a single second cache circuit.

[0025] Embodiments of the present disclosure will now be described with reference to the drawings.

[0026] FIG. 1 schematically illustrates an apparatus 100 according to an embodiment of the present disclosure. The apparatus 100 includes a processor 105 that issues access requests for data stored in a memory 110 connected to the apparatus 100. The apparatus 100 includes a first cache 115 and a second cache 120. Each of the first cache 115 and the second cache 120 is configured to store a temporary copy of data from the memory for faster access by the processor 105. Specifically, the first cache 115 is relatively small in size and has a significantly smaller access latency than the memory 110. The second cache 120 has a larger capacity than the first cache and an access latency that is longer than the first cache 115 but shorter than the memory 110. Therefore, specifically, the processing speed of data access requests can be significantly improved when the processor 105 repeatedly accesses the same data item.

[0027] First cache 115 and second cache 120 form a non-inclusive cache hierarchy in which a given data item is typically not validly stored in first cache 115 and second cache 120 simultaneously. For example, if a given data item in second cache 120 is provided to first cache 115, the second cache invalidates its copy of the data. This avoids duplicate caching and allows second cache 120 to be smaller.

[0028] FIG. 2 illustrates an operational flow by which a data request may be processed by the device of FIG.

[0029] The processor 105 issues a request for a given data item. The request is received by the first cache 115, which determines whether the requested data is stored therein. If the data is stored in the first cache 115, the data is provided to the processor 105.

[0030] If the data is not stored in the first cache 115, the request is forwarded to the second cache 120, which also determines whether the requested data is stored therein. If the data is stored in the second cache 120, the data is provided to the processor 105 via the first cache 115. Additionally, the data may be forwarded to the first cache 115 and invalidated in the second cache 120.

[0031] If the data is not stored in the first cache 115 or the second cache 120, the request is forwarded to the memory 110. The memory 110 provides the data to the processor 105 via the first cache 115 and the second cache 120. Additionally, the data may be cached in the first cache 115. In some embodiments, the data is not cached in the second cache 120.

[0032] 3A-3C illustrate a schematic diagram of data movement between caches, such as caches 115, 120 of FIG. 1, according to a comparative example.

[0033] In the initial configuration shown in Figure 3A, a data item is stored in the second cache. The data is stored with a valid bit V, indicating a valid copy of the data. In one example, prior to this configuration, the first cache had a valid copy of the data item and the second cache did not. The data item is then evicted from the first cache, resulting in the initial configuration of Figure 3A.

[0034] The data item is the subject of a data request received from a first cache. As shown in Figure 3B, the data, along with an associated valid bit V, is sent to the first cache and stored therein. Simultaneously, the data in the second cache is invalidated by flipping the valid bit I to indicate invalidity I (e.g., a bit value of 1 may indicate "valid" and a bit value of 0 may indicate "invalid"). Later, when data items are evicted from the second cache to make space for newly added data items, invalid data items are evicted first.

[0035] Later, the data item is evicted from the first cache to make room for the new data item. As shown in Figure 3C, the data, along with an associated valid bit V, is sent to and stored in the second cache.

[0036] In this way, high priority (e.g., more frequently used data) may be moved to the first cache and then moved back to the second cache when other data has a higher priority. However, if the size of the working data set on which the processor is performing operations is between the size of the first cache and the size of the second cache, this can result in a "ping-pong" effect in which data is repeatedly transferred between the first cache and the second cache: a given data item in the second cache is requested, thus filling it into the first cache, then evicted to make room for new data, returned to the second cache, then requested again, thus refilling it into the first cache, etc. At each of these steps, data is transmitted between the first cache and the second cache, consuming power and bandwidth.

[0037] An exemplary method for storing and moving data between caches according to the present disclosure, in which this ping-pong effect is reduced or eliminated, will now be described with reference to Figures 4A-4E. This may be achieved using caches 115, 120 of device 100, as shown in Figure 1.

[0038] In the initial configuration shown in Figure 4A, data items are stored in second cache 120. The data is stored with a validity indicator V, which indicates a valid copy of the data.

[0039] The data item is the subject of a data request received from the first cache 115. As shown in FIG. 4B , the data is transmitted to the first cache 115 along with an associated validity indicator V, similar to that shown in FIG. 3B , and stored therein. A change indicator U is also associated with the data item in the first cache, indicating that it has not (yet) been changed. However, instead of invalidating the data in the second cache 120, the data in the second cache is marked as pseudo-invalid by updating the validity indicator to indicate pseudo-invalidity PI. Thus, the validity indicator can take at least three values, and may be, for example, a two-bit value. Later, when data items are evicted from the second cache, invalid data items are evicted first, followed by pseudo-invalid data items, and then valid data items. That is, pseudo-invalid data items have a higher priority for remaining invalid data items in the cache, but a lower priority than valid data items.

[0040] Later, the data item is evicted from the first cache 115 .

[0041] 4C illustrates eviction when a data item has not been modified since it was placed in the first cache 115. The first cache 115 issues an eviction notification to the second cache 120 indicating that the data item is to be evicted. The eviction notification includes the current state of the modification indicator (i.e., "U" for "unmodified"). In response to receiving the eviction notification, the second cache 120 determines that it still contains a pseudo-invalid copy of the data item and validates the data item by updating the validity indicator to "V." In this way, a valid copy of the data item is provided in the second cache 115 without the data item itself being sent to the second cache. Thus, the ping-pong effect described above is avoided, with corresponding power and bandwidth savings.

[0042] 4C and 4E illustrate eviction when a data item is modified after being placed in first cache 115. In FIG. 4D, the data item is stored with a validity indicator "V" and a modification indicator "M," indicating that it is valid and has been modified. First cache 115 sends an eviction notification, including the modification indicator "M," to second cache 120. In response, second cache 120 requests the data from first cache 115.

[0043] 4E, the first cache 115 evicts the data item and sends it to the second cache 120, which stores the data item with a validity indicator "V." Thus, in this situation, the data item is still sent from the first cache 115 to the second cache 120. However, the power and bandwidth savings in situations where the data item has not changed result in an overall savings.

[0044] 5A and 5B illustrate an exemplary operational flow associated with the above-described eviction process.

[0045] 5A illustrates an operational flow between the first cache 115 and the second cache 120, corresponding to FIGS. 4A-4C. The first cache 115 requests data from the second cache 120. The second cache 120 responds by providing the requested data to the first cache 115. The second cache 120 then marks the data as pseudo-invalid.

[0046] Later, the first cache 115 evicts the data and issues an eviction notification to the second cache indicating that the data has not been modified. The second cache 120 still has a pseudo-invalid copy of the data and determines that the data has not been modified. In response, the second cache 120 marks its copy of the data as valid.

[0047] FIG. 5A shows the operational flow between the first cache 115 and the second cache 120, corresponding to FIGS. 4D and 4E (prior to this the flow proceeds as the first part of FIG. 5A).

[0048] The first cache 115 issues an eviction notification to the second cache. Unlike Figure 5A, this eviction notification indicates that the data has been modified since it was provided to the first cache 115. Upon receiving it, the second cache 120 determines that the data has been modified and therefore its copy does not have the correct value (even though it has not been evicted). Therefore, the second cache 120 sends a data request to the first cache 115, which in response sends the modified data to the second cache 120. The second cache 120 is therefore provided with the most up-to-date copy of the data.

[0049] Accordingly, an apparatus and method are provided for reducing power and bandwidth consumption within a cache hierarchy.

[0050] In this application, the term "configured to..." is used to mean that an element of an apparatus has a configuration that is capable of performing a defined operation. In this context, "configuration" refers to a method of arranging or interconnecting hardware or software. For example, an apparatus may have dedicated hardware that provides the defined operation, or a processor or other processing device may be programmed to perform the function. "Configured to" does not imply that an apparatus element needs to be modified in any way to provide the defined operation.

[0051] Although exemplary embodiments of the present invention are described in detail herein with reference to the accompanying drawings, it will be understood that the invention is not limited to these precise embodiments, and that various changes and modifications can be made to the embodiments by those skilled in the art without departing from the scope of the invention as defined by the appended claims.

Claims

1. 1. An apparatus comprising: a processing circuit configured to issue an access request for the data; a first cache circuit for storing a temporary copy of data from the memory for providing to said processing circuitry in response to corresponding access requests; a second cache circuit for storing a temporary copy of data from the memory for providing to the first cache circuit in response to a corresponding access request, the second cache circuit having an access latency higher than the access latency of the first cache circuit; the second cache circuit, in response to receiving a request for data stored in the second cache circuit, identifying the data as pseudo-invalid data; providing the data to the first cache circuit; In response to receiving an eviction instruction indicating that the first cache circuit is to evict the data, the second cache circuit: In response to determining that the data has not been modified since it was provided to the first cache circuit, the apparatus identifies the pseudo-invalid data as valid data.

2. 2. The apparatus of claim 1, wherein the second cache circuitry is configured to select data for eviction, the selecting including preferentially selecting pseudo-invalid data over valid data.

3. The apparatus of claim 2 , wherein said selecting comprises preferentially selecting invalid data over pseudo-invalid data.

4. 4. The apparatus of claim 1, wherein the second cache circuit is configured to identify the data as pseudo-invalid data by setting a pseudo-invalidity indicator for the data.

5. the second cache circuit maintaining a priority indicator for said data; An apparatus according to any preceding claim, configured to identify said data as pseudo invalid data by: assigning a minimum priority indicator value to said data.

6. the second cache circuit, in response to receiving the eviction instruction and determining that the data has been modified since it was provided to the first cache circuit, sends a return request for the data to the first cache circuit; 6. The apparatus according to claim 1, wherein the first cache circuit provides the data to the second cache circuit in response to receiving the return request to restore the pseudo-invalid data.

7. the second cache circuit is configured to provide the data, along with an associated modification indicator, to the first cache circuit; the first cache circuit responsive to modifying the data, setting the modification indicator to indicate the modification; configured to send the eviction instruction including the modification indicator to the second cache circuit; 7. The apparatus of claim 6, wherein the second cache circuitry is configured to perform the determining that the data has been modified based on the modification indicator.

8. 8. The apparatus of claim 7, wherein the second cache circuit is configured to provide the modification indicator to the first cache circuit in response to determining that a cache line containing the data is a dirty cache line.

9. 9. The device of claim 1, wherein the first cache circuit is configured to, when evicting the data identified as pseudo-invalid data in the second cache circuit, not provide the data to the second cache circuit if there is no request for the data from the second cache circuit.

10. The apparatus of any preceding claim, wherein the first cache circuitry and the second cache circuitry are configured to implement a non-inclusive cache hierarchy.

11. The apparatus of any preceding claim, wherein the second cache circuitry is configured to operate as a shared cache for the processing circuitry and the second processing circuitry.

12. 12. The apparatus of claim 1, wherein the first cache circuit and the second cache circuit are configured to provide the data by providing a cache line containing the data.

13. 1. A method comprising: storing a temporary copy of data from the memory in a first cache circuit for providing to the processing circuit in response to a corresponding access request; storing a temporary copy of data from the memory in a second cache circuit for providing to the first cache circuit in response to corresponding access requests, the second cache circuit having a higher access latency than the access latency of the first cache circuit; transmitting a request for data stored in the second cache circuit from the first cache circuit to the second cache circuit; In response to said second cache circuit receiving said request for said data, identifying the data as pseudo-invalid data by the second cache circuit; providing the data from the second cache circuit to the first cache circuit; sending an eviction notification from the first cache circuit to the second cache circuit indicating that the first cache circuit will evict the data; and identifying, by the second cache circuit, the pseudo-invalid data as valid data in response to the second cache circuit receiving the eviction notification and in response to determining that the data has not been modified since it was provided to the first cache circuit.

14. 1. A system comprising: A memory circuit; a processing circuit configured to issue access requests relating to data stored in the memory circuit; a first cache circuit for storing a temporary copy of data from said memory circuit for providing to said processing circuit in response to a corresponding access request; a second cache circuit for storing a temporary copy of data from the memory circuit for providing to the first cache circuit in response to a corresponding access request, the second cache circuit having an access latency higher than the access latency of the first cache circuit; the second cache circuit, in response to receiving a request for data stored in the second cache circuit, identifying the data as pseudo-invalid data; providing the data to the first cache circuit; In response to receiving an eviction instruction indicating that the first cache circuit is to evict the data, the second cache circuit: The system identifies the pseudo-invalid data as valid data in response to determining that the data has not been modified since the data was provided to the first cache circuit.

Images