Method and apparatus for transferring data within a hierarchical cache circuit

Abstract

Aspects of the disclosure relate to an apparatus comprising processing circuitry, a first cache circuit, and a second cache circuit, wherein the second cache circuit has an access latency that is longer than an access latency of the first cache circuit. The second cache circuit, in response to receiving a request for data stored within the second cache circuit, identifies the data as pseudo-invalid data and provides the data to the first cache circuit. The second cache circuit, in response to receiving an eviction indication indicating that the first cache circuit is to evict the data, in response to determining that the data has not been modified after the data was provided to the first cache circuit, identifies the pseudo-invalid data as valid data.

Description

Background Technology

[0001] This technology relates to the field of data storage, and more specifically to cache storage circuitry.

[0002] In some processing systems, the processor is coupled to memory. Memory stores data that will be accessed by the processor. For example, the memory can be dynamic random access memory (DRAM). The latency associated with accessing data stored in this memory can be relatively long. Therefore, in some such systems, cache circuitry is provided, conceptually positioned "between" the processor and memory. The cache stores temporary copies of data from memory for use by the processing circuitry. The cache has a smaller capacity than memory, but as a trade-off, the latency associated with accessing the cache is less than the latency associated with memory. Therefore, data access requests from the processor for data stored in the cache can be served faster than requests for data not stored in the cache.

[0003] In some systems, multiple caches are provided. These caches may have a hierarchical structure with multiple cache layers. Each layer progressively "closer" to the processor, with closer layers having lower access latency but also lower capacity. Depending on the caching scheme, data is transferred between memory, cache layers, and the processor. For example, in response to a data request from the processor, the request may first propagate through the cache layers, then to memory, until the data is found. The data can then be provided to the processor, and stored, for example, in the cache layer closest to the processor. Subsequently, this data can be evicted to cache layers further away from the processor, or evicted back from memory.

[0004] Using a hierarchical cache structure in this way consumes power and bandwidth. The goal is to improve the efficiency of operations using such hierarchical cache structures. Summary of the Invention

[0005] At least some examples provide an interconnection device comprising: A processing circuit configured to issue an access request for data; A first cache circuit is used to store a temporary copy of data from the memory in order to provide it to the processing circuit in response to a corresponding access request. A second cache circuit is used to store a temporary copy of the data from the memory for provision to the first cache circuit in response to a corresponding access request, wherein the second cache circuit has a longer access latency than the first cache circuit, wherein: The second cache circuit responds to a request for data stored in the second cache circuit as follows: The data is identified as pseudo-invalid data; and The data is provided to the first cache circuit. The second cache circuit responds to receiving an evictation instruction instructing the first cache circuit to evict the data: In response to determining that the data has not been modified after it has been provided to the first cache circuit, the pseudo-invalid data is identified as valid data.

[0006] Another example provides a method that includes: A temporary copy of the data from the memory is stored in the first cache circuit so that it can be provided to the processing circuit in response to the corresponding access request; A temporary copy of the data from the memory is stored in a second cache circuit for provision to the first cache circuit in response to a corresponding access request, wherein the second cache circuit has a longer access latency than the first cache circuit. The request for data stored in the second cache circuit is transmitted from the first cache circuit to the second cache circuit; In response to the second cache circuit receiving the request for the data: The second cache circuit identifies the data as pseudo-invalid data; and The data is provided from the second cache circuit to the first cache circuit. Transmitting an eviction notification from the first cache circuit to the second cache circuit, instructing the first cache circuit to evict the data; and In response to the second cache circuit receiving the eviction notification, and in response to determining that the data has not been modified after the data was provided to the first cache circuit, the second cache circuit identifies the pseudo-invalid data as valid data.

[0007] Another example provides a system that includes: Memory circuits; A processing circuit configured to issue an access request for data stored in the memory circuit; A first cache circuit is used to store a temporary copy of the data from the memory circuit so as to provide it to the processing circuit in response to a corresponding access request. A second cache circuit is used to store a temporary copy of the data from the memory circuit for provision to the first cache circuit in response to a corresponding access request, wherein the second cache circuit has a longer access latency than the first cache circuit, wherein: The second cache circuit responds to a request for data stored in the second cache circuit as follows: The data is identified as pseudo-invalid data; and The data is provided to the first cache circuit. The second cache circuit responds to receiving an evictation instruction instructing the first cache circuit to evict the data: In response to determining that the data has not been modified after it has been provided to the first cache circuit, the pseudo-invalid data is identified as valid data.

[0008] Further aspects, features, and advantages of this technology will become apparent from the following description, which is taken in conjunction with the accompanying drawings. Attached Figure Description

[0009] Figure 1 An exemplary device is depicted schematically.

[0010] Figure 2 The operation process is shown. Figure 1 The device can provide services for data requests through this operating procedure.

[0011] Figures 3A to 3C The diagram illustrates the movement of data between caches based on a comparison example.

[0012] Figures 4A to 4E An exemplary method for storing and moving data between caches is described.

[0013] Figure 5A and Figure 5B An exemplary operational flow associated with the expulsion process is described. Detailed Implementation

[0014] As described above, the processing apparatus may include processing circuitry and cache circuitry providing a cache hierarchy. The hierarchy may include a first cache circuitry closest to the processing circuitry and a second cache circuitry having a longer access latency than the first cache circuitry.

[0015] In such an apparatus according to the comparative example, data can be transferred from a second cache circuit to a first cache circuit (e.g., in response to a request from processing circuitry). This data is then marked as "invalid" in the second cache circuit (in a system without a cache hierarchy). Subsequently, this data (which may have been modified) is evicted from the first cache circuit and transferred back to the second cache circuit. In some operating environments, such as where the working dataset (to which the processor is performing data processing operations and therefore repeatedly accesses) is larger than the capacity of the first cache circuit but smaller than the capacity of the second cache circuit, a "ping-pong" effect may occur, where given data is repeatedly transferred between the first and second cache circuits. This consumes power and bandwidth.

[0016] In one example of this disclosure, where the ping-pong effect is reduced, the apparatus includes processing circuitry configured to issue access requests for data. The apparatus also includes a first cache circuitry for storing a temporary copy of the data from the memory, to be provided to the processing circuitry in response to a corresponding access request. The apparatus further includes a second cache circuitry for storing a temporary copy of the data from the memory, to be provided to the first cache circuitry in response to a corresponding access request, wherein the second cache circuitry has a longer access latency than the first cache circuitry. Therefore, the first and second cache circuitries form a hierarchical structure, which can be, for example, a non-cache-inclusive hierarchy, where given data is not simultaneously held as valid data in both the first and second cache circuitries. Providing given data between the first and second cache circuitries can be performed by transferring cache lines containing the given data.

[0017] In response to a request for data stored in the second cache circuit, the second cache circuit provides the data to the first cache circuit and also marks the data as pseudo-invalid data. In effect, this is a provisional indication that the data may become invalid.

[0018] Subsequently, the first cache circuit will evict this data, for example, to make room for newly added data to the first cache circuit. The second cache circuit receives an evictment instruction, instructing the first cache circuit to evict the data. For example, the first cache circuit may transmit the evictment instruction to the second cache circuit.

[0019] In response to receiving this instruction, the second cache circuit determines whether the data has been modified since it was provided to the first cache circuit. If it has not been modified, the second cache circuit marks the pseudo-invalid data as valid data. Therefore, the data can be deleted from the first cache circuit without being transmitted back to the second cache circuit: since the data has not been modified, the pseudo-invalid copy of the data in the second cache circuit still has the correct value and can therefore be marked as valid.

[0020] In this way, there is no need to perform data transfer from the first cache circuit to the second cache circuit, thus saving power and bandwidth compared to the above comparative example that does not implement pseudo-invalidity.

[0021] The above mentioned eviction from the first cache circuitry. However, the second cache circuitry can also apply eviction logic to determine the data to be evicted. In one example, the second cache circuitry is configured to select data to be evicted, the selection including prioritizing pseudo-invalid data over valid data. The selection may also include prioritizing invalid data over pseudo-invalid data. Therefore, the second cache circuitry can apply eviction priorities, where invalid data has the highest eviction priority, followed by pseudo-invalid data, and finally valid data has the lowest eviction priority. This allows pseudo-invalid data, rather than invalid data, to be preferentially retained in the second cache circuitry without at the expense of evicting valid data.

[0022] The second cache circuit can be configured to maintain a priority indicator associated with valid data, wherein lower priority data is evicted relative to higher priority data. The second cache circuit can also be configured to mark data as pseudo-invalid by assigning it a minimum priority indicator value. This allows for the configuration of pseudo-invalidity flags within the system without significant system modifications. However, it also eliminates possible priority levels that might otherwise be assigned to valid data.

[0023] Alternatively, the second cache circuit can be configured to identify the data as pseudo-invalid by setting a pseudo-invalidity indicator for the data. This provides an efficient method for identifying data as pseudo-invalid without reducing the number of priority levels that can be allocated to valid data.

[0024] The above describes the functionality associated with evicting from the first cache circuit, where data has not been modified after being provided from the second cache circuit. In one example, in response to determining that the data has been modified after being provided to the first cache circuit, the second cache circuit transmits a return request for the data to the first cache circuit. Then, in response to receiving the return request, the first cache circuit provides the (modified) data to the second cache circuit to replace the pseudo-invalid data. Thus, this example can handle both modified and unmodified data. Although data is still transmitted from the first cache circuit to the second cache circuit in the case that the data has been modified, the data is typically unmodified, thus still resulting in significant power and bandwidth savings compared to comparison systems that do not implement this disclosure.

[0025] In one such example, a second cache circuit is configured to provide data along with an associated modification indicator to a first cache circuit. The first cache circuit is then configured to set the modification indicator to indicate the modification in response to a modification of the data. For example, the modification indicator could be a bit toggled by the first cache circuit. When data is evicted, the first cache circuit transmits the evicting instruction, including the modification indicator, to the second cache circuit. The second cache circuit can then determine that the data has been modified based on the modification indicator.

[0026] For example, a modification indicator may be provided in response to the second cache circuit determining that the cache line containing the data is a dirty cache line. If the cache line is a clean cache line, a modification indicator may optionally not be provided, as its status as a clean cache line indicates that it has not been modified. Therefore, the second cache circuit can absorb multiple dirty evictions of the data from the first cache circuit without transferring data or writing data back to memory. Consequently, the resulting power and bandwidth are reduced.

[0027] In one example, the first cache circuit is configured to, when evicting data identified as pseudo-invalid data from the second cache circuit, not provide the data to the second cache circuit unless a data request is received from the second cache circuit. Therefore, power and bandwidth savings are achieved by disallowing data transfer from the first cache circuit to the second cache circuit (unless requested by the second cache circuit).

[0028] In one example, the second cache circuitry is configured to operate as a shared cache relative to both the processing circuitry and the second processing circuitry. This allows the invention to be implemented within a system comprising multiple processors, each having its own first cache circuitry but all sharing a single second cache circuitry.

[0029] Examples of this disclosure will now be described with reference to the accompanying drawings.

[0030] Figure 1 An apparatus 100 according to an example of this disclosure is schematically illustrated. The apparatus 100 includes a processor 105 that issues access requests for data stored in a memory 110 connected to the apparatus 100. The apparatus 100 includes a first cache 115 and a second cache 120. Each of the first cache 115 and the second cache 120 is configured to store a temporary copy of the data from the memory so that the processor 105 can access it more quickly. Specifically, the first cache 115 is relatively small in size and has a significantly shorter access latency than the memory 110. The second cache 120 has a larger capacity than the first cache and has a longer access latency than the first cache 115 but still a shorter access latency than the memory 110. Therefore, the speed of servicing data access requests can be greatly improved, especially when the processor 105 repeatedly accesses the same data item.

[0031] The first cache 115 and the second cache 120 form a non-containment cache hierarchy. In this hierarchy, a given data item is typically not simultaneously and effectively stored in both the first cache 115 and the second cache 120. For example, if a given data item in the second cache 120 is to be provided to the first cache 115, the second cache invalidates its copy of the data. This avoids duplicate caching and allows the second cache 120 to be smaller.

[0032] Figure 2 The operation process is shown. Figure 1 The device can provide services for data requests through this operating procedure.

[0033] Processor 105 issues a request for a given data item. The request is received at a first cache 115, which determines whether the requested data is stored there. If the data is stored in the first cache 115, the data is provided to processor 105.

[0034] If the data is not stored in the first cache 115, the request is forwarded to the second cache 120, which similarly determines whether the requested data is stored there. If the data is stored in the second cache 120, the data is provided to the processor 105 via the first cache 115. Alternatively, data may be transferred to the first cache 115 and invalidated in the second cache 120.

[0035] If the data is not stored in the first cache 115 or the second cache 120, the request is forwarded to memory 110. Memory 110 provides the data to processor 105 via the first cache 115 and the second cache 120. Alternatively, the data may be cached in the first cache 115. In some examples, the data is not cached in the second cache 120.

[0036] Figures 3A to 3C This schematically illustrates data in a cache (such as, based on a comparison example) according to the data. Figure 1 The movement between caches 115 and 120.

[0037] exist Figure 3A In the initial configuration shown, data items are stored in the second cache. The data is stored along with a validity bit V, indicating a valid copy of the data. In one example, prior to this configuration, the first cache had a valid copy of the data item, and the second cache did not. Subsequently, data items are evicted from the first cache, resulting in… Figure 3A Initial configuration.

[0038] This data item is governed by data requests received from the first cache. For example... Figure 3B As shown, data is transferred to a first cache and stored therein along with its associated valid bit V. Simultaneously, the data in the second cache is invalidated by toggling the valid bit to indicate invalidity I (e.g., bit value 1 indicates "valid" and bit value 0 indicates "invalid"). Subsequently, if data items are to be evicted from the second cache to make room for newly added data items, invalid data items are evicted first.

[0039] Subsequently, data items will be evicted from the first cache to make room for new data items. For example... Figure 3C As shown, the data is transferred to the second cache and stored therein along with the associated valid bit V.

[0040] This allows higher-priority data (e.g., frequently used data) to be moved to the first cache and then moved back to the second cache when other data has higher priority. However, if the size of the working dataset for which the processor is performing operations is between the size of the first and second caches, this can lead to a "ping-pong" effect, where data is repeatedly transferred between the first and second caches: a given data item is requested from the second cache and thus populated into the first cache; then, the given data item is evicted to make room for new data and returned to the second cache; then, the given data item is requested again and thus repopulated into the first cache, and so on. With each of these steps, data is transferred between the first and second caches, consuming power and bandwidth.

[0041] Now, refer to Figures 4A to 4E This disclosure describes an exemplary method for storing and moving data between caches, wherein the ping-pong effect is reduced or eliminated. This method can be used as follows: Figure 1 The depicted device 100 is implemented using caches 115 and 120.

[0042] exist Figure 4A In the initial configuration shown, data items are stored in the second cache 120. The data is stored together with a validity indicator V, indicating a valid copy of the data.

[0043] This data item is governed by data requests received from the first cache 115. For example... Figure 4B As shown, the data is presented in the following manner. Figure 3B The data is transferred to the first cache 115 in the same manner as shown and stored therein along with the associated validity indicator V. A modification indicator U is also associated with the data item in the first cache, indicating that the data item has not yet been modified. However, instead of invalidating data in the second cache 120, the data in the second cache is marked as pseudo-invalid by updating the validity indicator to indicate pseudo-invalidity PI. Therefore, the validity indicator can use at least three values, and for example, it can be a two-digit value. Subsequently, if data items are to be evicted from the second cache, invalid data items are evicted first, followed by pseudo-invalid data items, and then valid data items. That is, pseudo-invalid data items have a higher priority than invalid data items to be retained in the cache, but a lower priority than valid data items.

[0044] Subsequently, data items will be evicted from the first cache 115.

[0045] Figure 4CThe eviction process is described if a data item has not been modified after being populated in the first cache 115. The first cache 115 sends an eviction notice to the second cache 120, indicating that the first cache will evict the data item. The eviction notice includes the current state of the modification indicator (i.e., "U" indicating "not modified"). In response to receiving the eviction notice, the second cache 120 determines that it still contains a pseudo-invalid copy of the data item and verifies the data item by updating the validity indicator to "V". In this way, a valid copy of the data item is provided to the second cache 120 without transferring the data item itself to the second cache. Therefore, the aforementioned ping-pong effect is avoided, and power and bandwidth are saved accordingly.

[0046] Figure 4C and Figure 4E This describes the eviction process if a data item is modified after it has been filled into the first cache 115. Figure 4D In the cache, a data item is stored along with a validity indicator "V" and a modification indicator "M," indicating that the data item is valid and has been modified. The first cache 115 transmits an eviction notice, including the modification indicator "M," to the second cache 120. In response, the second cache 120 requests data from the first cache 115.

[0047] Then, as Figure 4E As shown, the first cache 115 evicts the data item and transfers it to the second cache 120, which stores the data item along with the validity indicator "V". Therefore, in this case, the data item is still transferred from the first cache 115 to the second cache 120. However, the power and bandwidth savings when the data item is not modified result in an overall saving.

[0048] Figure 5A and Figure 5B An exemplary operational flow associated with the expulsion process described above is depicted.

[0049] Figure 5A Depicting the corresponding Figures 4A to 4C The operational flow between the first cache 115 and the second cache 120 is as follows: The first cache 115 requests data from the second cache 120. In response, the second cache 120 provides the requested data to the first cache 115. Then, the second cache 120 marks this data as pseudo-invalid.

[0050] Subsequently, the first cache 115 sends an eviction notice to the second cache, indicating that the first cache will evict data even though the data has not been modified. The second cache 120 determines that it still has a pseudo-invalid copy of the data, and that the data has not been modified. In response, the second cache 120 marks its data copy as valid.

[0051] Figure 5A Depicting the corresponding Figure 4D and Figure 4E The operation process between the first cache 115 and the second cache 120 (the process before this follows) Figure 5A (The first part is carried out).

[0052] The first cache (115) sends an eviction notice to the second cache. Figure 5A Unlike other caches, the eviction notification indicates that the data was modified after being provided to the first cache 115. Upon receiving this notification, the second cache 120 determines that the data has been modified and therefore its copy (even if it has not yet been evicted) does not have the correct value. Therefore, the second cache 120 sends a data request to the first cache 115, and in response, the first cache sends the modified data to the second cache 120. Thus, the latest copy of the data is provided to the second cache 120.

[0053] Therefore, apparatus and methods are provided for reducing power and bandwidth consumption within a cache hierarchy.

[0054] In this application, the phrase "configured as..." is used to mean that the elements of the device have a configuration capable of performing the defined operation. In this context, "configuration" means the arrangement or manner of interconnection of hardware or software. For example, the device may have dedicated hardware that provides the defined operation, or a processor or other processing device may be programmed to perform the function. "Configured as" does not mean that the elements of the device need to be changed in any way to provide the defined operation.

[0055] While exemplary embodiments of the invention have been described in detail herein with reference to the accompanying drawings, it should be understood that the invention is not limited to those precise embodiments, and various changes and modifications can be made therein by those skilled in the art without departing from the scope of the invention as defined by the appended claims.

Claims

1. A data storage device, the device comprising: A processing circuit configured to issue an access request for data; A first cache circuit is used to store a temporary copy of data from the memory in order to provide it to the processing circuit in response to a corresponding access request; A second cache circuit is used to store a temporary copy of the data from the memory for provision to the first cache circuit in response to a corresponding access request, wherein the second cache circuit has a longer access latency than the first cache circuit, wherein: The second cache circuit responds to a request for data stored in the second cache circuit by: The data is identified as pseudo-invalid data; and The data is provided to the first cache circuit. The second cache circuit responds to receiving an evictation instruction instructing the first cache circuit to evict the data as follows: In response to determining that the data has not been modified after the data is provided to the first cache circuit, the pseudo-invalid data is identified as valid data; In response to receiving the eviction instruction and in response to determining that the data has been modified after the data was provided to the first cache circuit, the second cache circuit transmits a request to return the data to the first cache circuit. In response to receiving the return request, the first cache circuit provides the data to the second cache circuit to replace the pseudo-invalid data; The second cache circuit is configured to provide the data, along with an associated modification indicator, to the first cache circuit; The first cache circuit is configured as follows: In response to modifying the data, the modification indicator is set to indicate the modification; and The eviction instruction, including the modification indicator, is transmitted to the second cache circuit, and The second cache circuit is configured to perform the determination that the data has been modified based on the modification indicator.

2. The apparatus of claim 1, wherein the second cache circuit is configured to select data to be evicted, the selection including prioritizing pseudo-invalid data over valid data.

3. The apparatus of claim 2, wherein the selection includes prioritizing invalid data over pseudo-invalid data.

4. The apparatus according to any one of claims 1 to 3, wherein the second cache circuit is configured to identify the data as pseudo-invalid data by setting a pseudo-invalidity indicator for the data.

5. The apparatus according to any one of claims 1 to 3, wherein the second cache circuit is configured to identify the data as pseudo-invalid data by means of the following steps: Maintain the priority indicator for the data; and Assign a minimum priority indicator value to the data.

6. The apparatus according to any one of claims 1 to 3, wherein the second cache circuit is configured to provide the modification indicator to the first cache circuit in response to determining that a cache line including the data is a dirty cache line.

7. The apparatus according to any one of claims 1 to 3, wherein the first cache circuit is configured to, when evicting the data identified as pseudo-invalid data in the second cache circuit, not provide the data to the second cache circuit in the absence of a data request from the second cache circuit.

8. The apparatus according to any one of claims 1 to 3, wherein the first cache circuit and the second cache circuit are configured to implement a non-caching hierarchical structure.

9. The apparatus according to any one of claims 1 to 3, wherein the second cache circuit is configured to operate as a shared cache relative to the processing circuit and the second processing circuit.

10. The apparatus according to any one of claims 1 to 3, wherein the first cache circuit and the second cache circuit are configured to provide the data by providing cache lines including the data.

11. A data storage method, comprising: A temporary copy of the data from the memory is stored in the first cache circuit so that it can be provided to the processing circuit in response to the corresponding access request; A temporary copy of the data from the memory is stored in a second cache circuit for provision to the first cache circuit in response to a corresponding access request, wherein the second cache circuit has a longer access latency than the first cache circuit. The request for data stored in the second cache circuit is transmitted from the first cache circuit to the second cache circuit. In response to the second cache circuit receiving the request for the data: The second cache circuit identifies the data as pseudo-invalid data; and The data is provided from the second cache circuit to the first cache circuit. Transmit an evict instruction from the first cache circuit to the second cache circuit, instructing the first cache circuit to evict the data; and In response to the second cache circuit receiving the eviction instruction, and in response to determining that the data has not been modified after the data was provided to the first cache circuit, the second cache circuit identifies the pseudo-invalid data as valid data; In response to the second cache circuit receiving the eviction instruction, and in response to determining that the data has been modified after the data was provided to the first cache circuit, a request for the return of the data is transmitted to the first cache circuit; In response to the first cache circuit receiving the return request, the data is provided to the second cache circuit to replace the pseudo-invalid data; The data, along with the associated modification indicator, is provided from the second cache circuit to the first cache circuit; In response to the modification of the data, the modification indicator is set using the first cache circuit to indicate the modification; as well as The eviction instruction, including the modification indicator, is transmitted from the first cache circuit to the second cache circuit, and The second cache circuit is used to determine that the data has been modified based on the modification indicator.

12. A data storage system, comprising: Memory circuits; A processing circuit configured to issue an access request for data stored in the memory circuit; A first cache circuit is configured to store a temporary copy of the data from the memory circuit for provision to the processing circuit in response to a corresponding access request. A second cache circuit is used to store a temporary copy of the data from the memory circuit for provision to the first cache circuit in response to a corresponding access request, wherein the second cache circuit has a longer access latency than the first cache circuit. The second cache circuit responds to a request for data stored in the second cache circuit by: The data is identified as pseudo-invalid data; and The data is provided to the first cache circuit. The second cache circuit responds to receiving an evictation instruction instructing the first cache circuit to evict the data as follows: In response to determining that the data has not been modified after the data is provided to the first cache circuit, the pseudo-invalid data is identified as valid data; In response to receiving the eviction instruction and in response to determining that the data has been modified after the data was provided to the first cache circuit, the second cache circuit transmits a request to return the data to the first cache circuit. In response to receiving the return request, the first cache circuit provides the data to the second cache circuit to replace the pseudo-invalid data; The second cache circuit is configured to provide the data, along with an associated modification indicator, to the first cache circuit; The first cache circuit is configured as follows: In response to modifying the data, the modification indicator is set to indicate the modification; and The eviction instruction, including the modification indicator, is transmitted to the second cache circuit, and The second cache circuit is configured to perform the determination that the data has been modified based on the modification indicator.

Images