Victim cache supporting emptying write missed entries

Abstract

The embodiment of the invention relates to victim cache supporting emptying write missed entries. A cache system includes a first sub-cache and a second sub-cache (700) in parallel with the first sub-cache, where the second sub-cache includes: a set of cache lines; a line type bit (708) configured to store an indication that a corresponding cache line of the set of cache lines is configured to store write miss data; and an eviction controller (714) configured to flush the stored write miss data based on the line type bit (708).

Description

[0001] Related information of divisional application

[0002] This application is a divisional application of the application patent application with the application date of May 26, 2020, the application number of 202080037620.6, and the invention name of “Victim cache supporting flushing write miss entries”. TECHNICAL FIELD

[0003] Embodiments of the present application generally relate to the field of memory technology, and more specifically, to victim cache supporting flushing write miss entries. BACKGROUND

[0004] In multi-core coherent systems, multiple processors and system components share the same memory resources, such as on-chip and off-chip memory. Memory caches, such as caches, can provide an amount of high-speed memory that is operatively positioned near (e.g., close to) a processor (as opposed to main memory). Generally, the closer a cache is operatively positioned to a processor, the lower the latency, i.e., fewer processor clock cycles are used to fulfill a memory request. Generally, the cache memory closest to a processor includes a level 1 (LI) cache that is typically directly on the die with the processor. Many processors also include a larger level 2 (L2) cache. This L2 cache is typically slower than the LI cache, but can still be on the die with the processor core. The L2 cache can be per-processor core cache or shared across multiple cores. Often, a larger, slower L3 cache that is on the die as a separate component or another part of a system on a chip (SoC) can also be used for processor cores.

[0005] Memory systems, such as caches, can be susceptible to data corruption, such as due to electronic or magnetic interference from cosmic rays, solar particles, or malicious memory access. As processors are increasingly used in critical and / or other non-tolerable systems, such as autonomous vehicles and autonomous systems, techniques to protect memory systems from data corruption are increasingly applied to memory systems. One such technique is to use error correction codes (ECC) to detect and correct memory corruption. Implementing ECC in high-speed cache memory is challenging as ECC can introduce additional timing overhead that needs to be accounted for. For example, a high-speed cache memory system can have a five-stage memory pipeline for determining whether an accessed memory address is in the cache and retrieving the contents of the cache memory. Each stage can take one clock cycle, which at 1 GHz is about 1 nanosecond. Verifying the contents of the cache memory can essentially take up one full clock cycle. What is needed are techniques for improving cache performance of fault-tolerant caches. SUMMARY

[0006] This description relates to a caching system. More specifically (and not by limitation), aspects of this description relate to a caching system comprising a first sub-cache and a second sub-cache running parallel to the first sub-cache, wherein the second sub-cache comprises: a set of cache lines; line type bits configured to store an indication that a corresponding cache line in the set of cache lines is configured to store write-miss data; and an eviction controller configured to flush the stored write-miss data based on the line type bits.

[0007] Another aspect of this description relates to a method for caching data, comprising: receiving a write-to-memory request for a memory address by a caching system; determining, by a first sub-cache of the caching system, that the memory address is not cached in the first sub-cache; determining, by a second sub-cache of the caching system, that the memory address is not cached in the second sub-cache; storing data associated with the write-to-memory request in the second sub-cache; storing an indication of a write miss corresponding to the stored data in a line-type bit of the second sub-cache; and flushing the stored data based on the indication.

[0008] Another aspect of this description relates to an apparatus comprising a first sub-cache and a second sub-cache running parallel to the first sub-cache; wherein the second sub-cache comprises: a set of cache lines; a line type bit configured to store an indication that a corresponding cache line in the set of cache lines is configured to store write-miss data; and an eviction controller configured to flush the stored write-miss data based on the line type bit.

[0009] Another aspect of this description relates to a caching system comprising a first sub-cache and a second sub-cache running parallel to the first sub-cache, wherein the second sub-cache comprises: a set of cache lines; line type bits configured to store an indication that a corresponding line in the set of cache lines is configured to store an indication of a write miss; and an eviction controller configured to evict the cache lines of the second sub-cache storing the write miss based on an indication that the cache lines have been fully written.

[0010] Another aspect of this description relates to a method for caching data, comprising: receiving a write-to-memory request for a memory address by a caching system; determining, by a first sub-cache of the caching system, that the memory address is not cached in the first sub-cache; determining, by a second sub-cache of the caching system, that the memory address is not cached in the second sub-cache; storing data associated with the write-to-memory request in the second sub-cache; storing an indication of a write miss corresponding to the stored data in a line-type bit of the second sub-cache; and evicting the cache line storing the write miss in the second sub-cache based on an indication that the cache line has been fully written.

[0011] Another aspect of this description relates to an apparatus comprising: a processor; a first sub-cache and a second sub-cache running parallel to the first sub-cache, wherein the second sub-cache comprises: a set of cache lines; line type bits configured to store an indication that a corresponding line in the set of cache lines is configured to store an indication of writing miss data; and an eviction controller configured to evict the cache lines of the second sub-cache storing the write miss data based on an indication that the cache lines have been fully written.

[0012] Another aspect of this description relates to a caching system comprising: a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write memory commands not cached in the first sub-cache, wherein the second sub-cache comprises: a color-mark bit configured to store an indication associated with the color-mark of a corresponding cache line of the second sub-cache storing write-miss data; and an evict controller configured to evict cache lines of the second sub-cache storing write-miss data based on the color-mark associated with the cache line.

[0013] Another aspect of this description relates to a method for caching data, comprising: receiving a write-to-memory command for a memory address by a caching system; determining, by a first sub-cache of the caching system, that the memory address is not cached in the first sub-cache; determining, by a second sub-cache of the caching system, that the memory address is not cached in the second sub-cache, wherein the second sub-cache is configured to store cached data evicted from the first sub-cache and the write-to-memory command not cached in the first sub-cache in parallel with the first sub-cache; storing data associated with the write-to-memory command in the second sub-cache; storing a color scale associated with the data in the second sub-cache; and evicting the stored data based on the color scale.

[0014] Another aspect of this description relates to an apparatus comprising: a processor; a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write memory commands not cached in the first sub-cache, wherein the second sub-cache comprises: a color-mark bit configured to store an indication associated with the color-mark of a corresponding cache line of the second sub-cache storing write-miss data; and an evict controller configured to evict the cache line of the second sub-cache storing write-miss data based on the color-mark associated with the cache line.

[0015] Another aspect of this description relates to a technique for caching data via a caching system comprising: a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write-memory commands not cached in the first sub-cache, the second sub-cache comprising: a color-mark bit configured to store an indication of a corresponding line of the second sub-cache associated with a color-mark; and an evict controller configured to evict cache lines of the second sub-cache storing write-memory data based on the color-mark associated with the line, wherein the second sub-cache is further configured to: receive a first write-memory command for a memory address, the write-memory command being associated with a first color-mark; store a first data associated with the first write-memory command in a cache line of the second sub-cache; store the first color-mark in the second sub-cache; receive a second write-memory command for the cache line, the write-memory command being associated with a second color-mark; merge the second color-mark with the first color-mark; store the merged color-mark; and evict the cache line based on the merged color-mark.

[0016] Another aspect of this description relates to a method for caching data, comprising: receiving a write memory command for a memory address associated with a first color mark by a caching system; determining, by a first sub-cache of the caching system, that the memory address is not cached in the first sub-cache; determining, by a second sub-cache of the caching system, that the memory address is not cached in the second sub-cache, wherein the second sub-cache is configured to store cached data evicted from the first sub-cache and the write memory command not cached in the first sub-cache in parallel with the first sub-cache; storing, in a cache line of the second sub-cache, a first data associated with the first write memory command; storing, in the second sub-cache; receiving, for the cache line, a second write memory command associated with a second color mark; merging, incorporating, the merged color mark; and evicting, based on, the merged color mark.

[0017] Another aspect of this description relates to an apparatus comprising: a processor; a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write-memory commands not cached in the first sub-cache, wherein the second sub-cache includes: a color-mark bit configured to store an indication of a corresponding line of the second sub-cache associated with a color-mark; and an evict controller configured to evict cache lines of the second sub-cache storing write-memory data based on the color-mark associated with the line, wherein the second sub-cache is further configured to: receive a first write-memory command for a memory address, the write-memory command being associated with a first color-mark; store a first data associated with the first write-memory command in a cache line of the second sub-cache; store the first color-mark in the second sub-cache; receive a second write-memory command for the cache line, the write-memory command being associated with a second color-mark; merge the second color-mark with the first color-mark; store the merged color-mark; and evict the cache line based on the merged color-mark.

[0018] Another aspect of this description relates to a caching system comprising: a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write-memory commands not cached in the first sub-cache, the second sub-cache comprising: a privilege bit configured to store an indication of a corresponding cache line of the second sub-cache associated with a privilege level, wherein the second sub-cache is further configured to receive a first write-memory command for a memory address, the first write-memory command being associated with a first privilege level; storing first data associated with the first write-memory command in a cache line of the second sub-cache; storing the privilege level associated with the cache line in the second sub-cache; receiving a second write-memory command for the cache line, the second write-memory command being associated with a second privilege level; merging the first privilege level and the second privilege level; storing the merged privilege level; and outputting the merged privilege level using the cache line.

[0019] Another aspect of this description relates to a method for caching data, comprising: receiving, by a caching system, a first write-to-memory command for a memory address, the first write-to-memory command being associated with a first privilege level; determining, by a first sub-cache of the caching system, that the memory address is not cached in the first sub-cache; determining, by a second sub-cache of the caching system, that the memory address is not cached in the second sub-cache, wherein the second sub-cache is configured to store, in parallel with the first sub-cache, cached data evicted from the first sub-cache and the write-to-memory command not cached in the first sub-cache; storing, by storing, a first data associated with the first write-to-memory command in a cache line of the second sub-cache; storing, by storing, the first privilege level in the second sub-cache; receiving, by receiving, a second write-to-memory command for the cache line, the second write-to-memory command being associated with a second privilege level; merging, by merging, the first privilege level and the second privilege level; storing, by storing, the merged privilege level; and outputting, by the cache line, the merged privilege level.

[0020] Another aspect of this description relates to an apparatus comprising: a processor; a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write-memory commands not cached in the first sub-cache, the second sub-cache including: privilege bits configured to store an indication of a corresponding cache line of the second sub-cache associated with a privilege level, and wherein the second sub-cache is further configured to receive a first write-memory command for a memory address, the first write-memory command being associated with a first privilege level; storing first data associated with the first write-memory command in a cache line of the second sub-cache; storing the privilege level associated with the cache line in the second sub-cache; receiving a second write-memory command for the cache line, the second write-memory command being associated with a second privilege level; merging the first privilege level and the second privilege level; storing the merged privilege level; and outputting the merged privilege level using the cache line.

[0021] Another aspect of this description relates to a caching system comprising a first sub-cache and a second sub-cache coupled in parallel with the first sub-cache; wherein the second sub-cache includes line-type bits configured to store an indication of a corresponding line in the second sub-cache being configured to store an indication of a write miss.

[0022] Another aspect of this description relates to a method for caching data, comprising: receiving a write-to-memory request for a memory address by a caching system; determining, by a first sub-cache of the caching system, that the memory address is not cached in the first sub-cache; determining, by a second sub-cache of the caching system, that the memory address is not cached in the second sub-cache, the second sub-cache being coupled in parallel with the first sub-cache; storing data associated with the write-to-memory request in the second sub-cache; and storing an indication of a write miss corresponding to the stored data in a line-type bit of the second sub-cache.

[0023] Another aspect of this description relates to an apparatus comprising: a processor; a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache; wherein the second sub-cache includes line-type bits configured to store an indication of writing a miss on a corresponding line of the second sub-cache.

[0024] Another aspect of this description relates to a caching system comprising: a first sub-cache; a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write-memory commands not cached in the first sub-cache; and a cache controller configured to: receive two or more cache commands; determine that a conflict exists between the received two or more cache commands; determine conflict resolution between the received two or more cache commands; and send the two or more cache commands to the first sub-cache and the second sub-cache.

[0025] Another aspect of this description relates to a method for caching data, comprising: receiving two or more cache commands; determining that a conflict exists between the two or more cache commands; determining conflict resolution between the received two or more cache commands; and sending the two or more cache commands to a first sub-cache and a second sub-cache, wherein the second sub-cache is configured to store cached data evicted from the first sub-cache and write-to-memory commands not cached in the first sub-cache in parallel with the first sub-cache.

[0026] Another aspect of this description relates to an apparatus comprising: a processor; a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write-memory commands not cached in the first sub-cache; and a cache controller configured to: receive two or more cache commands; determine that a conflict exists between the received two or more cache commands; determine conflict resolution between the received two or more cache commands; and send the two or more cache commands to the first sub-cache and the second sub-cache. Attached Figure Description

[0027] To describe each example in detail, reference will now be made to the accompanying drawings, in which:

[0028] Figure 1 It is a block diagram of a computer system.

[0029] Figure 2 This is a block diagram illustrating a simplified cache memory pipeline used to handle read requests.

[0030] Figure 3 This is a block diagram of a cache memory pipeline with a storage queue, based on the aspects described herein.

[0031] Figure 4 This is a block diagram illustrating a cache architecture that includes a victim cache.

[0032] Figure 5 This is a block diagram illustrating a cache architecture that includes a parallel victim cache, based on the aspects described herein.

[0033] Figure 6 This is a block diagram illustrating the victim cache memory pipeline according to the aspects described herein.

[0034] Figure 7 This is a block diagram illustrating the victim cache pipeline that includes writes to the miss buffer, according to the aspects described herein.

[0035] Figure 8 This is a block diagram of MESIRAM based on the aspects described herein.

[0036] Figure 9 This is an architectural block diagram of a cache system that includes a write-miss buffer, based on the aspects described herein.

[0037] Figures 10 to 16 This is a flowchart illustrating a technique for caching data based on the aspects described herein. Detailed Implementation

[0038] Figure 1 This is a block diagram of computer system 100. Computer system 100 includes a data cache 102, such as a level 1 (L1) data cache. Data cache 102 stores a subset of system data to reduce the time spent accessing (e.g., reading and / or writing) the cached subset. By effectively caching the most frequently used data, data cache 102 can significantly improve system performance.

[0039] Data cache 102 may be coupled to one or more processing resources 104 (e.g., processor cores) and extended memory 106. Extended memory 106 includes other levels of memory hierarchy, such as L2 cache, storage devices, etc. Data cache 102 may be incorporated into the same die as processing resource 104 (e.g., on-die cache) or may be on a separate die. In either case, cache 102 is coupled to the processing resource via one or more interfaces for exchanging data between cache 102 and each processing resource 104. In this example, cache 102 is coupled to each processing resource 104 via scalar interfaces and vector interfaces. In instances with more than one interface, commands can be served using another interface if one interface is busy. For example, when a scalar read command is received by the cache via a scalar interface, the associated data may be provided to processing resource 104 via a vector interface based on interface utilization, data size, and / or other considerations. Similarly, cache 102 may also be coupled to extended memory 106 via one or more interfaces. When there are more than one interface, the interface can be selected based on utilization, data size and / or other considerations.

[0040] Each interface can have any suitable width. The widths of the interfaces can differ from one another, although in many instances, they are integer multiples of the narrowest interface. In one such instance, the scalar interface is 64 bits wide, the vector interface is 512 bits wide, and the extended memory interface is 1024 bits wide.

[0041] The interface can be bidirectional or unidirectional. A bidirectional interface can contain two independent unidirectional interfaces, allowing data to be transmitted and received simultaneously. In one example, the vector interface contains two 512-bit unidirectional buses, one for receiving data and performing operations from processing resource 104, and the other for sending data to processing resource 104.

[0042] The data cache 102 may include several pipelines for processing operations received via these interfaces. Figure 2This is a block diagram illustrating a simplified cache memory pipeline 200 used to process read requests. As shown in cache memory pipeline 200, processor 202 sends a memory request to cache memory 204. Although cache memory 204 is described in the context of an L1 cache, the concepts described herein are applicable to any type of cache memory. In certain cases, memory requests may be sent via a cache or a memory controller, not shown in the diagram. In this example, the cache memory pipeline comprises five stages: E1, E2, E3, E4, and E5. Each cache memory pipeline stage can be allocated a specific number of clock cycles to complete, and in some instances, each stage is allocated one clock cycle, such that cached data can be returned to processor 202 after five clock cycles of the E5 memory pipeline stage. In the E1 memory pipeline stage, a memory request is received by cache memory 204. The memory request contains the memory address from which data is to be retrieved. In the E2 pipeline, tag random access memory (RAM) 206 is read to determine which memory addresses are currently stored in cache memory. Tag RAM 206 stores a table recording which entries in record memory 208 correspond to which memory addresses in extended memory. Tag RAM can be a set or portion of memory used to maintain the memory address table. In a particular case, the cache can be an N-way associative cache, where each cache set can hold N rows of memory addresses. As N increases, the number of addresses searched also increases, which in turn increases the amount of time required to determine whether the requested memory address is in the tag RAM. In the E3 memory pipeline, the received memory address is compared with the stored address read from the tag RAM to determine if there is a cache hit or miss. A cache hit occurs when the data associated with the requested memory address is stored in the cache, and a cache miss occurs when the data associated with the requested memory address is not stored in the cache. In the E4 memory pipeline level, a portion of memory 208 associated with the requested memory address is read, and in the E5 memory pipeline level, the requested memory address is provided to processor 202. Memory 208 can be any type of memory suitable for cache memory, such as dynamic random access memory (DRAM), static random access memory (SRAM), registers, etc. The pipeline level describes how the memory pipeline can be configured, and therefore, specific sub-steps and features are omitted. In certain embodiments, the levels in which specific activities, such as memory access, are performed may differ.

[0043] Figure 3This is a block diagram illustrating a cache pipeline 300 that supports read-modify-write operations using an error correction code storage queue architecture, according to the aspects described herein. Pipeline 300 includes read paths (with read path latch 302, tag RAM 306, memory 308, etc.) and write paths (write path latch 304, storage queue 314, etc.).

[0044] Regarding the read path, pipeline 300 includes tag RAM 306 and memory 308 (e.g., DRAM or other suitable memory). The cache can have any degree of associativity, and in this example, the cache is a direct-mapped cache such that each extended memory address corresponds to exactly one entry in cache memory 308.

[0045] In certain scenarios, cache pipeline 300 may include ECC support, and memory 308 may be coupled to error detection and correction circuitry 310. In an ECC instance, memory 308 stores data in blocks along with a set of ECC check bits corresponding to those blocks. Upon receiving a read operation, memory 308 may provide the stored data block and corresponding ECC check bits to error detection and correction circuitry 310. Error detection and correction circuitry 310 may regenerate ECC check bits based on a data block as read from memory 208 and compare the regenerated ECC check bits with previously stored ECC check bits. Any discrepancies may indicate that the data block has been read incorrectly, and the ECC check bits can be used to correct errors in the data block. This ability to detect and correct errors makes caches highly suitable for mission-critical applications.

[0046] Arbitration unit 312 may be coupled to memory 308 to arbitrate conflicting accesses to memory 308. When multiple operations attempt to access memory 308 in the same cycle, arbitration unit 312 may select which operations(s) are allowed to access memory 308 according to a priority scheme. Many different priority schemes may be used. As an example of a priority scheme, arbitration prioritizes read operations over write operations because write data in pipeline 300 can be used by subsequent operations even before write data is written to memory 308, for example, via data forwarding multiplexer 318 of storage queue 314, as will be described in more detail below. Therefore, there is minimal performance impact when write data is allowed to wait in pipeline 300. However, as pipeline 300 is filled with write data that has not yet been written back, the priority of write operations may increase until it takes precedence over read operations.

[0047] The read path can extend in parallel with memory queue 314. Because a read operation can refer to data in a write operation that may not yet be completed, pipeline 300 may include write forwarding functionality, which allows the read path to obtain data from memory queue 314 that has not yet been written back to memory 308. In an example, pipeline 300 includes: a pending memory address table 316 that records the addresses of operations at each stage of memory queue 314; a data forwarding multiplexer 318 that selects data from a stage of memory queue 314 for forwarding; and a memory queue hit multiplexer 320 that selects between the output of memory 308 (via error detection and correction circuitry 310) and the forwarded data from memory queue 314 from data forwarding multiplexer 318.

[0048] An example flow of a read operation via pipeline 300 will now be described. In the first loop indicated by stage E1, the cache retrieves the record associated with the address of the read operation from tag RAM 306 to determine whether the data is stored in the cache's memory 308. In the direct-mapped instance, the cache does not need to wait for the tag RAM comparison before requesting data from memory 308, and therefore, the tag RAM comparison between the address of the read operation and the record at the cached address does not need to extend to the second (E2) or third (E3) loop.

[0049] In the second loop, at level E2, if arbitration unit 312 grants permission, the cache can request data and ECC check bits from memory 308. In this loop, the cache can also determine if newer data is available in storage queue 314 by comparing the read address with the pending memory address table 316. If available, the data forwarding multiplexer 318 is configured to forward the appropriate data from storage queue 314.

[0050] Data and the ECC checksum bit can be provided by memory 308 in the third cycle of stage E3. However, this data may or may not correspond to the memory address specified by the read operation, because the cache can allocate multiple extended memory addresses to the same entry in cache memory 308. Therefore, in the third cycle, the cache determines whether the provided data and ECC bit from memory 308 correspond to the memory address based on a comparison of the tag RAM record during a read operation (e.g., a cache hit). In the case of a cache hit, the data and the ECC bit are received by error detection and correction circuitry 310, which corrects for any errors in the data, in the fourth cycle of stage E4.

[0051] As described above, newer data that has not yet been written to memory 308 may exist in storage queue 314 and may be forwarded from storage queue 314 by data forwarding multiplexer 318. If so, then storage queue hit multiplexer 320 selects forwarded data instead of corrected data from memory 308.

[0052] Corrected data from memory 308 or forwarded data from storage queue 314 is provided to the requesting processor in the fifth loop of stage E5. In this way, the instance cache can provide data to the processor with full ECC checksum and correction in approximately five loops, provided there is a cache hit.

[0053] If the data and ECC bit are not present in memory 308 (e.g., a cache miss), pipeline 300 can pause until the data can be retrieved from extended memory. At this point, the data can be written to memory 308, and tag RAM 306 can be updated so that subsequent reads of the data will hit the cache.

[0054] The cache can also support multiple operations that read data from the cache and make changes to the data before rewriting it. For example, the cache can support a Read-Modify-Write (RMW) operation. An RMW operation reads existing data, modifies at least a portion of the data, and rewrites that portion of the data. In an ECC implementation, the RMW operation can be performed when the write is less than the full group width. Since the write is not to the full group width, performing an ECC operation on only a portion of the written data would result in an incorrect ECC checksum. Therefore, the read functionality of RMW is used because the portion of the data in the group that will not be rewritten still contributes to the ECC checksum bits.

[0055] The RMW operation can be divided into write and read operations, and pipeline 300 can be structured to keep read operations in the read path synchronized with write operations in storage queue 314. Read and write operations remain synchronized until read-modify-write merging circuit 322 rewrites at least a portion of the read data with the written data to produce merged data. The merged data is provided to ECC generation circuit 324, which generates new ECC check bits for the merged data, and then the merged data and ECC check bits can be provided to arbitration unit 312 for storage in memory 308.

[0056] An example flow of the RMW operation via pipeline 300 will now be described. The read portion of the operation continues essentially as described above in stages E1 through E4, where the cache compares the address of the read operation with the record in tag RAM 306, and the cache requests data and ECC check bits from memory 308 and / or storage queue 314. Because the RMW operation modifies data, in instances where the MESI (Modified, Exclusive, Shared, and Invalid) states of entries in memory 308 are tracked, a cache hit not in a modified or exclusive state can be considered a cache miss. When the data is obtained in the correct state and any errors are corrected, it is provided to the read-modify-write merging circuit 322 in loop E4 (or later in the case of a cache miss). In this same loop, the read-modify-write merging circuit 322 can rewrite at least a portion of the corrected data with the write data to produce merged data. ECC generation circuit 324 generates a new ECC checksum bit for the merged data in stage E5 (or later in the event of a cache miss). The merged data and ECC checksum bit are provided to arbitration unit 312 for writing to cache memory 308.

[0057] In some instances, sequential RMW operations involving the same address are received. Instead of waiting for merged data from a previous RMW operation to be written to memory 308, memory queue 314 may include an address comparator 326 for write forwarding, which can feed the merged data back to the previous stage of memory queue 314 for use by the subsequent RMW operation. This can be referred to as "pigging-backing." Data can be fed back before or after ECC generation circuitry 324. Because the feedback effectively merges the RMW operations, the final RMW operation has a complete set of data and ECC check bits. Therefore, the earlier RMW operation can be canceled before it is written back to memory 308. This avoids pausing other operations due to writing outdated data.

[0058] In certain cases, memory 308 may represent the entire data cache. By way of example only, in this embodiment, the data cache (which may be an L1 data cache) is associated with a single storage queue structure 314. As an example, the data cache may contain 256 rows, with each row having 1024 bits (1Kb).

[0059] In other instances, the cache can be divided into multiple independently addressable groups, each group having its own corresponding storage queue structure 314. For example, consider the following embodiment: the aforementioned data cache has 256 rows, each with a line width of 1024 bits, but is divided into 16 groups, with each row having 64 bits in a given group. In this embodiment, there will be 16 storage queues, each for each group of the data cache. Therefore, read and write operations can be sent to the groups in parallel, and each group arbitrates its own process in response to read and / or write operations. By allowing each group in multiple caches to operate independently, cache operations are more efficient because the entire cache line is not locked when a request is received. Specifically, only the portion of the cache line allocated to the group receiving the request will be locked. Of course, the cache size described above is just one example, and the description is not limited to any particular cache line width, number of groups, or number of rows, etc.

[0060] The examples above also apply to writing and / or reading vector data. For instance, vector data can be 512 bits wide. For multiple cache sets, a write request containing vector data that is a cache hit can be processed as 8 parallel writes to 8 sets (e.g., 8 x 64 bits = 512 bits). Similarly, a read request for this multiple cache set can be executed as 8 parallel reads from 8 sets.

[0061] Another feature that may exist in a contemplated embodiment of a caching system is support for in-transit forwarding and invalidation. For example, suppose in one instance the cache is a two-way set-associative cache. In a two-way set-associative implementation, each cache line within the cache may be mapped to two different addresses in higher-level memory (e.g., L2 cache or main system memory, for example).

[0062] Consider the following scenario: a given cache line, referred to as "line 1" in this example, is a cache line in a two-way set-associative cache and maps to two different addresses in memory, referred to as "address A" and "address B". Now, suppose the cache receives a first request that is a partial write (e.g., a write to less than the entire cache line) and after a read request, it behaves like this:

[0063] Write(Address A): Data 1 / / Writes Data1 to the cache line corresponding to address A.

[0064] Read(Address B) / / Reads the data contained in the cache line corresponding to address B.

[0065] In this example, let's assume the write request is a hit, meaning the cache line corresponding to address A (which we'll assume to be line 1 in this example) is in the cache. In response, the caching system will begin the process of writing data 1 to line 1.

[0066] Let's now assume the read request is a miss. In other words, while line 1 could be mapped to either address A or address B in this two-way set-associative cache, in this example, it is currently mapped to address A. The miss occurs because address B is not found in tag RAM 306. In response to the miss, the cache system will serve the miss by retrieving data from a higher-level memory, such as the L2 cache, and the data currently in line 1 will be evicted to store data 2 when the data, referred to as "data 2" in this instance, is returned. This can happen before the previous write has completed. In this case, there is no need to complete the write operation because line 1 will be evicted anyway. Therefore, the write can be invalidated before it completes (referred to as "in transit").

[0067] For example, atomic comparison and swap operations are a further example of memory operations that include read operations. An atomic comparison and swap operation provides an address, a key value, and a new set of data to be written. The current data at the address is compared with the key, and if they match, the current data at the address is overwritten with the new data. Therefore, in some instances, pipeline 300 includes atomic comparison and swap circuitry 328 to perform a comparison and forward new data for writing if the current data matches the key value. Atomic comparison and swap circuitry 328 may be coupled in parallel with read-modify-write merging circuitry 322, and a function multiplexer 330 may be used to determine which output is provided to ECC generation circuitry 324.

[0068] An example flow of atomic and swap operations via pipeline 300 will now be described. The read portion of the operation continues essentially as described above in stages E1 through E4, where the cache compares the address of the read operation with the record in tag RAM 306, and the cache requests data and ECC check bits from memory 308 and / or storage queue 314. Because atomic compare and swap operations can modify data, in instances where the MESI state of entries in tracking memory 308 is not in a modified or exclusive state, a cache hit can be considered a cache miss. When data is obtained in the correct state, it is provided to atomic compare and swap circuit 328 in stage E4 (or later, in the case of a cache miss). During these cycles, new data and keys can be stored in write path latch 304. During stage E4, atomic compare and swap circuit 328 can receive corrected read data, keys, and new data. If the read data matches the key after calibration, the atomic comparison and exchange circuit 328 provides the new data to the ECC generation circuit 324. If the read data matches the key, the atomic comparison and exchange circuit 328 can also change the state of the new data to modified. The ECC generation circuit 324 generates a new ECC checksum bit for the new data in stage E5 (or later, in the event of a cache miss). The new data and the ECC checksum bit are provided to the arbitration unit 312 for writing to the cache memory 308.

[0069] When a sequential atomic comparison and swap operation is received, similar to what is described in the context of an RMW operation, pipeline 300 can also use address comparator 326 to perform write forwarding.

[0070] A further example of memory operations for reading existing data includes histogram operations. Histogram operations can specify values ​​to add to or subtract from existing data. To support histogram operations, pipeline 300 may include an arithmetic unit 332 coupled in parallel with read-modify-write merging circuitry 322 and / or atomic comparison and switching circuitry 328. Arithmetic unit 332 can support operands of any size, and in one example, it supports addition and subtraction of 16-bit values ​​from a 64-bit set. Multiplexer 330 can be used to select between arithmetic unit 332, read-modify-write merging circuitry 322, and atomic comparison and switching circuitry 328.

[0071] An example flow of the histogram operation via pipeline 300 will now be described. The read portion of the operation continues essentially as described above in stages E1 through E4, where the cache compares the address of the read operation with the record in tag RAM 306, and the cache requests data and ECC check bits from memory 308 and / or storage queue 314. Because the histogram operation modifies the data, in instances where the MESI state of entries in memory 308 is tracked, a cache hit not in a modified or exclusive state can be considered a cache miss. When the data is obtained in the correct state, it is provided to arithmetic unit 332 in stage E4 (or later in the case of a cache miss). During these cycles, the value to be added to the corrected data can be stored in write path latch 304. During stage E4, arithmetic unit 332 can receive the corrected read data and the value to be added to or subtracted from the read data. Arithmetic unit 332 performs the specified addition or subtraction and provides the new data to ECC generation circuit 324. ECC generation circuit 324 generates a new ECC checksum bit for the new data in stage E5 (or later in the event of a cache miss). The new data and ECC checksum bit are provided to arbitration unit 312 for writing to cache memory 308.

[0072] When a sequential histogram operation is received, similar to what is described in the context of an RMW operation, pipeline 300 can also use address comparator 326 to perform write forwarding.

[0073] Although Figure 3 This describes a single pipeline 300 with a single example of a memory queue 314, latches 302 and 304, error detection and correction circuitry 310, etc. However, as described above, the memory 308 can be arranged to support more than one independent copy of the pipeline 300. Therefore, the pipeline 300 can be reproduced multiple times. In one such example, the memory 308 entries are 1024-bit blocks based on the width of the extended memory interface, and the memory 308 is arranged in 16 parallel groups, where each 1024-bit block in each of the 16 groups contains 64 bits. In this particular example, each of the 16 groups has its own independent pipeline 300. Read or write operations performed from the processor are divided into several group-level operations, and each is provided to its corresponding pipeline 300 so that they can be processed simultaneously. For example, a 512-bit vector read operation can be divided into eight 64-bit read operations provided to eight independent pipelines 300 for parallel processing. Meanwhile, another operation can be received via a scalar or vector interface of another processor and divided into operations that are provided to the remaining pipeline 300. Therefore, the data cache can simultaneously serve multiple independent operations, each allocated a subset of pipeline 300.

[0074] In certain situations, cache pipeline 300 can be coupled to a victim cache. A victim cache can help improve cache miss rates, such as cache misses due to conflicts. In certain cases, a victim cache may have an impact on cache performance, similar to increasing the N-way associativity (e.g., cache associativity procedures). This benefit is more pronounced when the victim cache is added to a direct-mapped cache, as direct-mapped caches typically have the highest conflict miss rates. Adding a victim cache can help improve overall cache hit rates.

[0075] Figure 4 This is a block diagram illustrating a cache architecture 400 including a victim cache. Here, cache architecture 400 includes a main cache 404 coupled to a victim cache 402, which resides after the main cache 404. When a row of data is evicted from the main cache 404, instead of moving the row to the next level cache (e.g., L2), the row is moved to the victim cache 402 and held there until there is a reason to evict it, such as due to capacity or other reasons. In this cache architecture 400, in the event of a cache miss from the main cache 404, the victim cache 402 can serve the miss and return the row to the processor 414, thereby reducing the overall latency experienced during the read miss.

[0076] As an example, suppose processor 414 attempts to access address A in both main cache 404 and victim cache 402, resulting in a cache miss. Next, main cache 404 may evict address B and create a victim cache line (address B) in victim cache 402. The read miss at address B then proceeds to 406 to a higher-level memory 412 that will be serviced, such as an L2 cache, another cache, and / or other memory. Once address A is accessed, its contents can be transferred back to 410 and stored in main cache 404.

[0077] If a subsequent access attempt to address B is made, the main cache 404 is checked to see if address B is cached, resulting in a cache miss. Next, the victim cache 402 is checked at 408, and a hit is returned if address B was stored in the victim cache 402 after being evicted from the main cache 404. Address B can then be restored 410 to the main cache 404 and returned as a result to the processor 414. The read latency in this system, while faster than fetching an address from higher-level memory 414 (e.g., L2 cache, main memory, etc.), is still greater than the latency if a cache hit exists in the main cache 404.

[0078] To help address the latency experienced in caching architectures that place the victim cache after the main cache, in certain cases, the victim cache can be placed in parallel with the main cache. Figure 5 This is a block diagram illustrating a cache architecture 500 including a parallel victim cache according to the aspects described herein. The cache architecture 500 can be viewed as having multiple parallel sub-caches. Each of the main cache 504 and the victim cache 502 has its own independent associativity, and the overall cache system 500 can have the appearance of a hybrid-way cache system. In a particular case, the main cache 504 or the victim cache 502 can be n-way associative, while the other caches can be fully associative. In this example, the main cache 504 can be a direct-mapped (1-way associative) cache, and the victim cache 502 can be fully associative.

[0079] In this example, if processor 514 issues a read request for address A, which misses in main cache 504, the miss request is forwarded to a higher-level cache 512 for servicing, and the victim line (here, address B) is moved to victim cache 502. Next, let's assume processor 514 wants to read address B. In this example, the address to be read is checked in parallel against both main cache 504 and victim cache 502. This results in a read miss in main cache 504 but a hit in victim cache 502. The result of the read at address B can then be directly passed back to processor 514 from victim cache 502. Because both caches are checked in parallel, the read latency remains the same, as if the read request were a hit in main cache 504. The overall effect of having a victim cache running in parallel with the main cache is that in cases where subsequent CPU accesses hit the victim cache, we gain the benefit of reduced cache miss rates without affecting latency.

[0080] Figure 6This is a block diagram illustrating the victim cache memory pipeline 600 according to the aspects described herein. As described above, the victim cache can be a fully associative cache, meaning there are no mapped addresses and cache lines. Specifically, any address can be stored in any cache line of the victim cache. Therefore, for an address received at block 602 in E1 stage, the received address is checked against the address stored in each location of the victim cache tag RAM 604 in E2 stage. In certain cases, these locations in the victim cache tag RAM 604 correspond to lines of the victim cache 606. For example, if address A is received at block 602, then address A is compared with the address in the victim cache tag RAM 604 to see if there is a match. If, for example, location 5 of the victim cache tag RAM 604 corresponds to address A, then the data corresponding to address A is stored at address 5 of the victim cache 606. This data is finally passed back to the processor at block 608.

[0081] In cases where the victim cache operates in parallel with the main cache, it is expected that the victim cache supports specific features of the main cache, such as ECC and RMW operations. In certain situations, the victim cache may also include a storage queue 610 structure to facilitate operations similar to those of a combined cache. Figure 3 The described storage queue 314 provides support for RMW and ECC operations.

[0082] In certain cases, a cache may include a write miss buffer. A write miss can occur when a memory write to a memory address not in the cache is issued to the cache. Memory writes can be atomic operations, and the processor issuing the memory write can pause until a memory write complete message is returned to the processor. Instead of pausing the processor until a write to a higher-level memory is complete, the cache can store the memory write in a write miss buffer and return a memory write complete indication. The write miss buffer buffers this information and then outputs the memory write (e.g., clears) to the target memory address. Once the memory write to the target memory address is complete, the memory write is removed from the write miss buffer. To output a memory write, the write miss buffer may contain three pieces of information for each memory write: the memory address to be written, the corresponding data, and a byte enable (byteification) mask for the memory write. Typically, the byte enable defines a logical OR mask of the portion of the memory line to be written. Typically, write miss buffers are implemented as first-in-first-out (FIFO) buffers using a dedicated buffer circuitry system. However, FIFO write miss buffers are configured more or less frequently to be flushed to higher-level memory and do not support features such as write merging. In certain cases, write miss buffers can be allocated to the cache system using the victim cache.

[0083] Figure 7 This is a block diagram illustrating a victim cache pipeline 700 including a write-miss buffer according to the aspects described herein. As described above, the write-miss buffer may store the memory address for writing, the data to be written, and byte-ification logic. In this example victim cache pipeline 700, the memory address of the write-miss buffer entry may be stored in the victim cache tag RAM 704, and the corresponding data may be stored in the victim cache RAM 706. Byte-enabled memory 702 may also be added to the victim cache to store the byte-ification logic associated with the write-miss buffer line. In certain cases, MESI RAM 732 may be modified to include additional bits indicating the line type 708 associated with the entry in the victim cache RAM 706. Line type 708 indicates whether a particular line in the victim cache RAM 706 is a cache line or a write-miss buffer line. In certain cases, line type 708 may be added to another memory, such as the victim cache tag RAM 704 instead of MESI RAM 718.

[0084] Moreover, for example, such as combination Figure 6The described victim cache system can be configured as a write miss buffer by adding control logic to the read and write miss state machine to handle the allocation of write miss entries within the victim cache, logic to handle various address hazards in cases where reads are performed on the same line as the write miss entries within the victim cache, and logic to resolve the same address load / store in the victim cache's data path. The storage queue 710 structure can also be provided to implement support for RMW and ECC support for write misses. In specific cases, the storage queue 710 can be used to implement support for RMW and ECC support for write miss operations and cache operations.

[0085] As an example of performing a write-miss buffer operation in the cache, when a write is received at box 712, the address of the write is checked against the memory address stored in each location of the victim cache tag RAM 704. A write miss occurs when the address to be written is not found in the victim cache tag RAM 704. When the operation is a write-miss cache, the location in the victim cache where the write miss will be stored is determined. In certain cases, this determination may be based on entries in the victim cache tag RAM 704. If an empty location exists in the victim cache tag RAM 704, then the address of the write miss is stored in the empty location in the victim cache tag RAM 704. If no empty location exists, then the line may be evicted based on an eviction scheme to allocate write-miss buffer space. In other cases, the line may have previously been used as a write-miss buffer or cache line. The line type 708 corresponding to the location in the victim cache tag RAM 704 where the write miss occurred may be updated to indicate that the location was used as a write-miss buffer. For example, a 0 in line type 708 can indicate that the corresponding line is a cache entry, while a 1 in line type 708 can indicate that the corresponding line is a write miss buffer entry. In certain cases, line type 708 may be stored in another memory location, such as MESIRAM 732, or in a dedicated memory space, such as a set of registers. Write miss data is stored in a line of victim cache RAM 706 corresponding to the location of the victim cache tag RAM 704 used. Byte enable logic may be stored in a corresponding slot of byte enable memory 702. In certain cases, byte enable memory 702 may contain bits indicating which locations in byte enable memory 702 are valid. Line type bits facilitate storing write miss entries in lines of victim cache RAM 706 without dedicating a portion of victim cache RAM 706 to a write buffer.

[0086] According to the aspects described herein, for example, the victim cache eviction logic in eviction controller 714 can be updated to handle write miss entries stored in the victim cache to prioritize the eviction of write miss entries to support stored cache entries. Furthermore, the control logic for the read and write state machine of the victim cache controller 734 can be configured to determine if an incoming read request hits a cache line of the victim cache configured to store write misses. In such cases, the victim cache controller 734 can be configured to handle a read hit as if it were a read miss and evict the stored write miss command to higher-level memory, and then send the read miss out to higher-level memory. Moreover, cache systems are typically configured to pause on misses, while write miss buffers typically do not pause the processor on their buffered write misses. To help resolve this discrepancy, the victim cache controller 734 can be configured to speculatively read incoming write data, such that if the data is a write miss, the data can be directly stored in the victim cache without pausing, and write miss entries can be allocated to the victim cache without pausing the CPU.

[0087] In certain scenarios, write misses can be cached until they need to be evicted to higher-level memory by placing them in the cache system. In other scenarios, write misses stored in the victim cache can remain in the cache system until a defined condition is met, a barrier operation is requested, or an address / hardware conflict exists that requires the eviction of the specific write miss entry.

[0088] Maintaining a write miss buffer until a definitive fence operation is requested can be problematic. For example, if the fence operation is not raised at the end of the operation, any write misses stored in the victim cache can get stuck inside the victim cache. Furthermore, fence operations can introduce latency issues because flushing (e.g., evicting all) write miss buffer entries in response only to a fence operation can cause delays, as the cache needs to flush the stored write misses to the appropriate memory and then wait for acknowledgments. In certain cases, automatic flushing or wiping of write miss entries stored in the victim cache can be provided. Automatic flushing can be implemented to help clear write miss entries from the victim cache while reducing potential latency that may occur during the wiping process. In certain cases, the eviction controller 714 can monitor activity on the processor interface. If the eviction controller 714 detects idle activity on the processor interface, it can proceed to begin clearing write miss entries from the victim cache. For example, if the eviction controller 714 detects that the processor has not issued any commands within N clock cycles, then the eviction controller 714 may internally initiate an automatic flushing process to begin clearing the storage cached within the victim cache. In certain cases, N may be predetermined. In other cases, N may be configurable. In certain cases, the automatic flushing process may traverse (e.g., sequentially check) the VTag RAM to find write miss entries and flush the corresponding entries. In other cases, the automatic flushing process may include logic based on FIFO, Least Recently Used (LRU), random, or other fundamental principles for flushing entries.

[0089] During automatic flushing of write miss entries, the eviction controller 714 may send an eviction trigger request to the victim cache controller 734 of the selected write buffer cache line. The victim cache controller 734 verifies the memory queue to see if any entries exist in the memory queue corresponding to the write buffer cache line to be flushed. If a corresponding entry exists in the memory queue, the victim cache controller 734 may wait for the memory queue to complete. Once the memory queue is empty, the victim cache controller 734 may evict the selected cache line. The victim cache controller 734 sets up the creation of memory miss packets to be sent to a higher-level cache or other memory. These memory miss packets may contain the corresponding memory address stored in the victim cache tag RAM 704, the byte-enabled logic in the byte-enabled memory 702, and the data in the victim cache RAM 706. The victim cache may then send the memory miss packets, for example, to the cache controller of a higher-level cache or other memory. When a write-miss entry is flushed, the write-miss data packet can be sent to other memory, invalidating the corresponding entry in the victim cache RAM 706, the victim cache tag RAM 704, and the byte-enabled memory 702. The write-miss data packet can be temporarily placed in the flush buffer 716 until it is accepted by other memory (e.g., acknowledged). Once a write-miss cache line fails, the eviction controller 714 can select the next write-miss cache line to evict and continue evicting the next write-miss cache line. If activity is subsequently detected on the processor interface, this automatic flushing operation can be canceled. In this way, the victim cache can utilize idle periods and avoid impacting processor performance when activity exists on the interface by attempting to flush write-miss cache entries.

[0090] In certain situations, write misses stored in the victim cache can be evicted based on the state of the write miss line storing the specific write miss (rather than based on the order in which cache lines are allocated, such as FIFO, LRU, random, or other schemes). When the cache is configured to merge data in the cache, a cache write from the processor to a memory line does not imply that the processor needs to read from other parts of the memory line. Furthermore, compared to a case where the processor has written only a small portion of the memory line, it is less likely that the processor will need to access (e.g., merge additional data into) that portion of the memory line again when it has written a large portion (e.g., half or the entire memory line). Because the processor is unlikely to need to access a given write miss cache line after writing half or a full write miss cache line, this write miss cache line can be evicted with a smaller potential impact on performance. Therefore, a full or half-full cache line can be evicted once it has been fully or half-written, and the write has begun on the next entry in the victim cache on another write miss cache line. This helps resolve situations where the processor is issuing a stream of cache write misses that is collectively larger than the victim cache. Furthermore, aggressively evicting full or partial write misses from the cache helps avoid performance issues that can arise when a large number of write misses stored in the victim cache are flushed all at once due to a gate operation at the end of the operation.

[0091] To facilitate the removal of full or partial write miss cache lines, byte-enabled memory 702 may include two status bits: a partial line status bit 718 and a full line status bit 720. Each line of byte-enabled memory 702 corresponds to a cache line in victim cache RAM 206, and each line of byte-enabled memory 702 may include a partial line status bit 718 and a full line status bit 720. These status bits may be updated in each write cycle, where bits are written to cache lines configured for write miss buffers, and the status bits may be calculated based on bits already stored in the corresponding line of victim cache RAM 706 and bits written to victim cache RAM 706 during the write cycle. In certain cases, this update may be performed for each cache line of byte-enabled memory 702 within each cycle, and the update may be omitted when there is no write cycle. As indicated above, each line of byte-enabled memory 702 may contain a bitmask indicating which locations (e.g., bytes) in byte-enabled memory 702 are valid (e.g., written to). Determining which lines are complete or partial can be performed using an XOR operation on the corresponding portions of a bitmask. The partial-complete line status bit 718 can be set to either the least significant or most significant bit when half of a given cache line has been written. Similarly, the complete line status bit 720 can be set when the entire cache line has been written. The cache lines can then be traversed (e.g., checked sequentially) to determine the address that returns a complete or partial cache line. In a particular case, the address corresponding to a complete cache line is selected first, and if no complete cache line exists, the address corresponding to a partial cache line is returned. If multiple cache lines are complete (or, if no complete cache line exists, then partial), the first cache line found to be complete (or partial) during cache line traversal can be selected. If no cache lines are complete or partial, the write-missed cache line can be selected based on FIFO, LRU, or other logic used to determine which cache line to select when multiple available lines exist.

[0092] Evicting write-miss entries can be similar to automatic flushing, except that the memory queue is not checked to see if any entry exists corresponding to the full or partial write-miss cache line to be evicted, because this operation is initiated by a read or write miss and performed after the line has been written to. Once a full or partial write-miss cache line to be evicted has been selected, the eviction controller 714 instructs the victim cache controller 734 to evict the selected cache line. The victim cache controller 734 sets up the creation of memory-miss packets to be sent to a higher-level cache or other memory. These memory-miss packets may contain the corresponding memory address stored in the victim cache tag RAM 704, the byte-enabled logic in the byte-enabled memory 702, and the data in the victim cache RAM 706. The memory-miss packets are sent, for example, to the cache controller of a higher-level cache or other memory, and may invalidate the corresponding entries in the victim cache RAM 706, the victim cache tag RAM 704, and the byte-enabled memory 702. Storage miss packets can be temporarily placed in the clear buffer 716 until the storage miss packet is accepted by other memory (e.g., acknowledged).

[0093] Once a write-miss cache line has been invalidated, the cache line address can be sent to replacement policy logic 722 in each write cycle. Available cache line addresses can be fed into mux 724 in replacement address line 726. Address generation line 728 can contain the address assigned to the received write instruction. If hit / miss logic 730 determines that the address of the received write instruction is a hit, then the address has been stored in the victim cache line and the data from the write instruction is merged into said line. If hit / miss logic 730 determines that the address of the received write instruction is a miss, then the address from replacement address line 728 can be used to write the write-miss command to the victim cache / write-miss buffer. If there is no empty cache line in the victim cache / write-miss buffer for allocating a write-miss, then an eviction process can be triggered. During the eviction process, the centralized controller can detect that a write-miss that needs to be stored in the victim cache has been received. The centralized controller can instruct the victim cache controller 734 to ensure that an available cache line exists. This cache line can be based on the location provided by replacement address 726 or address generation 728. If a write miss already exists at the identified location, then the write miss can be evicted.

[0094] Figure 8This is a block diagram of a MESIRAM 800 according to the aspects described herein. The MESIRAM 800 may be a memory (e.g., SRAM) or a set of registers (e.g., a register file) used to track the MESI state of corresponding cache lines. As an example, in a particular case, the victim cache may track the MESI state in MESI state bit 802 in the MESIRAM 800 as part of the written data for each cache entry. In a particular case, the MESIRAM 800 contains lines corresponding to each line of the corresponding cache (here, the victim cache). The main cache may contain the corresponding MESI RAM. In a particular case, a single MESI RAM may be shared by both the main cache and the victim cache. The MESIRAM 800 may have a set of bits for tracking the MESI state of each cache line stored in the victim cache. In a particular case, the MESIRAM 800 may be extended with additional bits to track other states of cache lines in the victim cache. The MESIRAM 800 may be extended to track additional states by including additional columns for additional states. In this example, the MESIRAM has been extended to include status bits in the bit fields to track the privilege level 804, red tag 806, and blue tag 808 of each cache line, regardless of whether the line is valid 810 or not, and regardless of whether the line is a write-miss buffer or a victim cache line 812. It should be noted that although shown as a portion of MESIRAM, in other embodiments, the status bits may be placed in other memory, such as tag RAM 704, or together in a separate memory. The MESIRAM can be configured to track additional statuses not shown here.

[0095] In certain situations, it may be useful to flush all write-miss entries of a specific type stored in the victim cache. As described below, entries in the write-miss buffer can be associated with a "color" scale. This color scale can be represented by one or more bits such that when a bit is set for a write-miss buffer entry, the color associated with that entry can be "blue," and when the bit is not set or another bit is set, the entry can be "red." Tagging helps allow specific write-miss buffer entries to be associated with a specific process running on the processor, while other write-miss buffer entries can be associated with another process running on the processor. In certain situations, the processor may want to flush all write-miss buffer entries associated with a specific process (e.g., selective fence operations) but not write-miss buffer entries associated with another process.

[0096] To perform selective barrier operations, the processor can assert a clear signal. Along with the asserted clear signal, the processor can provide an indication that specific tags should be cleared first (red or blue tags). Return to Reference Figure 7 A clear signal is passed to the eviction controller 714. The eviction controller 714 can check the availability of the higher-level cache or memory to see if the higher-level cache or memory is busy. If the higher-level cache or memory is unavailable, the eviction controller 714 can suspend the processor. If the higher-level cache or memory is available, the eviction controller 714 can send an eviction trigger request indicating a priority tag to the victim cache controller 734. The victim cache controller 734 can then check the memory queue to see if there are any outstanding entries in the memory queue of any group of the victim cache RAM 706. The victim cache controller 734 can also instruct the memory queue 710 to escalate the priority of clearing the memory queue. If, for example, there is data in the memory queue that has not yet been stored in the victim cache, the eviction controller 714 can wait for the memory queue 710 to complete. Once the memory queue 710 is empty, the eviction controller 714 instructs the victim cache controller to evict the selected cache line. The victim cache controller 734 sets up the creation of memory miss packets to be sent to the higher-level cache or other memory. These memory miss packets may contain the corresponding memory address stored in the victim cache tag RAM 704, the byte enable logic in the byte enable memory 702, and the data in the victim cache RAM 706. The victim cache may then send the memory miss packets to, for example, a cache controller for a higher-level cache or other memory. When flushing write miss entries, write miss packets may be sent to other memories and may invalidate the corresponding entries in the victim cache RAM 706, the victim cache tag RAM 704, and the byte enable memory 702. Write miss packets may be temporarily placed in the flush buffer 716 until the write miss packets are accepted by other memories (e.g., acknowledged). This flushing continues until the processor stops asserting the flush signal. If the processor stops asserting the flush signal before all write miss buffer entries of the indicated color have been flushed, then the flushing of the write miss buffer stops at any point during the flushing process of the victim cache when the flush signal stops being asserted. Once all write miss buffer entries for the indicated color mark have been cleared, the idle signal is asserted back to the processor. If the processor continues to assert the clear signal, other write miss buffer entries associated with the other color mark can be cleared.

[0097] Placing write misses within a cache system helps provide an opportunity to merge additional write misses into those already in the cache. Because write misses remain within the cache for a longer period, the cache has the ability to merge any future write misses into those already existing in the cache system, thereby reducing write miss traffic leaving for higher-priority memory. Reducing write misses sent to higher-priority memory reduces stress on those memories and helps improve overall performance.

[0098] Write merging can occur when the processor issues a second store or write command to a specific address at box 712. In the case that the victim cache is fully associative, the address of the second write can be verified by hit / miss logic 730 against a set of memory addresses stored in each location of the victim cache tag RAM 704. Replacement policy logic 722 determines in parallel the location where write transactions within the victim cache can be allocated in the event of a write miss. Typically, when a write miss is stored in the victim cache, the write miss is allocated an entire cache line, even if the data in the write miss does not require the entire cache line. Therefore, a set of memory addresses can be associated with each write miss stored in the victim cache (including the memory address in the write miss) along with a set of memory addresses preceding, following, or both preceding and following the memory address in the write miss. If the address of the second write is determined to be within a set of memory addresses associated with a write miss stored in the victim cache, then a cache hit is detected and the hit / miss logic is passed back to the location in the victim cache where the write miss is currently stored. At the E3 level, the color type and privilege level associated with the second write are merged with the currently stored color type and privilege level, for example, via the victim cache controller 734, and MESIRAM 732 is updated. In certain cases, merging privilege levels may be performed with reference to the privilege merging table 736. The data of the second write may also be merged, for example, via the victim cache controller 734, into the corresponding line in the victim cache 706 and the corresponding entry in the byte-enabled memory 702 updated based on the byte-enabled logic associated with the second write. In certain cases, the storage queue 710 may also include write merging functionality. For example, when back-to-back writes are issued to the same address, the storage queue 710 may internally merge them to help minimize the number of writes issued to the victim cache RAM 706.

[0099] In certain scenarios, write merging can be performed across privilege levels. Merging privilege levels can be performed during write merging. For example, a first write miss stored in the victim cache can be associated with a first privilege level. A second write associated with a second privilege level can be merged with the stored first write miss. For merging to be performed, privilege levels should also be merged. In certain scenarios, privilege levels can be merged by merging privilege levels to a lower privilege level. For example, in an embodiment, a write operation can be associated with two privilege levels (e.g., root / guest user / supervisor). A first write miss can be received with the root user's privilege level. This first write miss can be written to a write miss buffer in the victim cache. A second write with the guest supervisor's privilege level can be received, and the second write is directed to an address within the set of addresses associated with the first write miss. When merging the privilege levels of two writes, the lower privilege level can be used, resulting in the merged privilege level of the guest user. In certain scenarios, merging privilege levels can operate in the same manner when merging is performed against multiple data paths. In certain cases, privilege level merging can be encoded in privilege merging table 736, indicating how different privilege levels can be merged. In certain cases, privilege merging table 736 can be predefined. An example of a predefined privilege merging table 736 is shown in Table 1. In certain cases, privilege merging table 736 is configurable.

[0100] Table 1

[0101]

[0102]

[0103] In certain situations, write merging can be performed on color marks. For example, the first write miss stored in the victim's cache could be associated with, for example, the first color mark, red. (See reference) Figure 8A first write miss can be associated with an entry in MESIRAM 800 that has bits set in the red label 806 of the bit field. A second write associated with a second color label can be merged with the stored first write miss. For merging to be performed, the color labels should also be merged. In certain cases, color labels can be merged by combining the color labels of the writes. For example, if both writes are red-labeled, the merged write retains the red label. Similarly, if both writes are blue-labeled, the merged write will be blue-labeled. If one write is red-labeled and the other is blue-labeled, the merged write is both red-labeled and blue-labeled. Therefore, if the second write to be merged is associated with a blue label, the merged write can have bits set in the red label 806 of the bit field and bits set in the blue label 808 of the bit field. In certain cases, if a clear operation is then performed, for example for the blue label, the merged write will be cleared. Similarly, if a clear operation is performed on a red label, then merged writes will also be cleared, because merged writes are labeled with both red and blue labels.

[0104] Figure 9 This is an architectural block diagram of a cache system 900 including a write-miss buffer, according to the aspects described herein. As shown, cache 902 can receive cache requests from processor 904 via processor interface 906. In certain cases, cache 902 can be configured to process scalar or vector data, or both. Cache 902 is logically divided into parallel sub-caches including a main cache 908 and a victim cache 910. Although a single main cache 908 and victim cache 910 are shown, certain embodiments may include multiple pairs of main cache 908 and victim cache 910. In certain cases, a different number of main cache 908 and victim cache 910 may exist. Regarding the main cache 908, the main cache tag RAM access controller 912 accesses the main cache tag RAM 914 and verifies the main cache tag RAM 914 against the memory address contained in the cache request to determine whether the memory address is stored in the main cache 908. The main cache controller 916 guides the appropriate allocation, eviction, arbitration, and merging processes, as well as read / write operations into the main cache memory 918. Combined with... Figure 3 The described main cache storage queue structure 920 provides features such as RMW, ECC, write merging, etc.

[0105] The overall architecture of the victim cache 910 is similar to that of the main cache 908. For example, in response to a received memory request, the victim cache tag RAM access controller 922 accesses the victim cache tag RAM 924 and verifies the victim cache tag RAM 924 against the memory address contained in the cache request to determine if the memory address is stored in the victim cache 910. The victim cache controller 926 similarly guides appropriate allocation, eviction, arbitration, merging, and read / write operations to the victim cache memory 928. The victim cache storage queue 930 provides RMW, ECC, and other features, such as... Figure 3 Description. Victim cache logic 932 helps coordinate operations between victim cache 910 and main cache 908. While victim cache logic 932 is shown separately, it may be integrated with or part of other components, such as victim cache controller 926. In certain cases, victim cache 908 includes specific components that help enable victim cache 908 to support write miss buffer functionality. For example, auto-flush logic 934 may monitor processor interface 906 and, via one or more memory interfaces 936, flush portions of victim cache memory 928 allocated for write miss buffering, for example, to higher-level caches and memories. As another example, MESIRAM 938 or victim cache tag RAM may be configured to include one or more bit fields to store status information about write miss buffer entries stored in victim cache 910.

[0106] In certain scenarios, the victim cache and main cache may contain multiple data paths configured simultaneously as a single cache to the processor in a parallel cache configuration. Each data path is configured to process a specific command in parallel with the other data paths. For example, a first data path 910 may be processing a read command to the main cache, while a second data path 942 is processing a write miss to the victim cache. In embodiments with two data paths for the victim cache, four combinations of cache commands for the first data path 940 and the second data path 942 are possible: read / read, read / write, write / read, and write / write. Furthermore, each command combination may result in a cache hit / hit, hit / miss, miss / hit, or miss / miss. Additionally, two commands may potentially be addressed to the same memory address with different privilege levels and color codes. In certain scenarios, to facilitate control of the parallel cache, a centralized controller 944 may help coordinate the processing of read and write requests between the main cache 908 and the victim cache 910. In certain situations, the centralized controller 944 can be configured to handle coordination between the main cache 908 and the victim cache 910, and to delegate tasks for the main cache 908 and the victim cache 910 to the main cache controller 916 and the victim cache controller 926, respectively. The centralized controller 944 can also detect and handle potential conflicts. For example, upon receiving a read request from the processor, the centralized controller 944 can check the main cache 908 and the victim cache 910 to determine that the addressed memory location of the read request is neither stored in the main cache 908 nor in the victim cache 910, and therefore constitutes a read miss. The centralized controller 944 can also check the victim cache 910 to determine that the addressed memory address was not stored in the victim cache 910 as a write miss, or was flushed out of the victim cache 910. If the addressed memory location was stored in the victim cache 910 as a write miss, the centralized controller 944 can suspend the read request that flushes the addressed memory location. Next, the centralized controller 944 can issue a request for an addressable memory location to a higher-level cache or memory.

[0107] In certain situations, if a read or write request results in a hit in the primary cache 908, the result in the victim cache 910 can be ignored. When the primary cache 908 is configured as a single-way associative cache, cache request conflicts across multiple data paths are explicit. Cache request conflicts from multiple data paths can be issues in the victim cache 910, which is fully associative and stores both write misses and cache entries. Cache request conflicts between two data paths can be handled as described in the following examples. It is important to note that in the following examples, cache operations are described within the context of a specific data path. However, the examples involve combinations of specific cache operations, and the specific data path on which a particular cache operation occurs is irrelevant.

[0108] In the first instance, there is a cache read miss for the first data path and a cache write hit for the second data path, with two different memory addresses. In the specific case where the victim cache 910 is fully associative, cache requests can reach any location within the victim cache 910. (See reference) Figure 7 Address generation 928 for a location within the victim cache occurs at the E2 level along with an address lookup in the victim cache tag RAM 904. Therefore, address generation 728 for a location within the victim cache is performed before it is known whether the requested cache address is a hit or a miss. Once the centralized controller 944 determines that a read miss exists on the main cache 908, a request to obtain the memory address of the read request is issued to a higher-level cache or memory. In the case that the main cache 908 is single-path associated, the memory address of the read request for the first data path is mapped to a single location in the main cache 908. If cached data exists in the single location, the cached data is evicted from the main cache 908 to a pre-generated location within the victim cache 910. If this pre-generated location is the same as the location of a cache write hit on the second data path, a conflict occurs. This conflict can be detected by the centralized controller 944. As indicated above, the address generated for the location within the victim cache 910 occurs before it is known whether the cache request address is a hit or a miss. Therefore, a cache write for the second data path exists with an address generated for the second location within the victim cache 910 before a cache write is determined to be a hit. Based on the collision detection, this second location within the victim cache can be used to store data evicted from the main cache 908 due to a read miss.

[0109] In another instance, similar to the first instance, there may be a cache read miss for the first data path and a cache write hit for the second data path. However, in this instance, the cache read misses in the main cache, and the cache write hits in the main cache. In a specific case, the main cache is a single-path associative cache (e.g., a direct-mapped cache), where a specific memory address is mapped to a specific location within the main cache. In this instance with a direct-mapped cache, both the cache read and cache write are mapped to the same location in the main cache. In this instance, due to the read miss, a request for the memory address of the read request is issued to a higher-level cache or memory, and the cached data is evicted from main cache 908 to a pre-generated location within victim cache 910, which is location A. In this instance, the cache write for the second data path hits the same location in the evicted main cache. One possible solution to this conflict is to load the requested read miss directly from a higher-level cache or memory into a separate location in the victim cache. Another solution is to have the centralized controller 944 pause the read miss, preventing already cached data from being evicted from the victim cache 910 until the cache write of the second data path completes and reaches location A in the main cache, after which location A is evicted from the victim cache. In certain cases, the pause can occur while waiting for memory writes and evictions to complete.

[0110] As another example, a first cache write may be received on a first data path used for victim cache 910, and a second cache write may also be received on a second data path used for victim cache 910. If the first cache write and the second cache write are addressed to different memory addresses, then the two cache writes may continue in parallel with victim cache 910. If both the first cache write and the second cache write are addressed to the same memory address and both are cache misses, then victim cache controller 926 allocates a single location in victim cache write miss memory 928 and merges the first cache write and the second cache write in victim cache storage queue 930. The merged data, along with merged byte enable, privilege, and color-coded information, may then be written to victim cache write miss memory 928. If both the first cache write and the second cache write are addressed to the same memory address and both are cache hits, then the first cache write and the second cache write are merged into the hit location in the cache.

[0111] As another example, cache reads may be received on a first data path used for the victim cache 910, and cache writes may be received on a second data path used for the victim cache 910. Where cache reads are hits and cache writes are hits at different locations within the victim cache 910, cache reads and cache writes continue in parallel without conflict. Where cache reads are hits and cache writes are misses at different locations within the victim cache 910, cache reads and cache writes also continue in parallel without conflict. Where cache reads are misses and cache writes are hits at different locations within the victim cache 910, cache reads may use addresses generated for locations within the victim cache 910 for cache writes as described above. Where cache reads are misses and cache writes are misses at different locations within the victim cache 910, both cache reads and cache writes use addresses generated for locations within the victim cache 910.

[0112] In another instance, cache reads may be received on a first data path for main cache 908, and cache writes may be received on a second data path for main cache 908. In a specific case, a cache read may be a hit for a location in main cache 908 (e.g., a cache line or slot). A cache write may also be a hit for the same location in main cache 908. In this case, cache reads and cache writes can continue in parallel without conflict because cache reads can be performed relatively quickly and end before cache writes require access to the main cache location as cache writes continue through the storage queue. In another case, cache reads may be hits for locations in main cache 908. Cache writes may be misses for the same location in main cache 908. In this case, cache commands typically continue in parallel because cache reads can complete before cache writes require access to the cache location. In yet another case, cache reads may be misses for locations in main cache 908. A cache write can be a hit for the same location in main cache 910. In this case, similar to the description above, a cache read can be paused until the cache write of the second data path completes, reaches the location in main memory 908, and is evicted from the victim cache 908. Then, the cache read continues, reading a set of addresses from a higher-level cache or memory into main cache 910. In another case, a cache read can be a miss for the first location in main cache 908. A cache write can also be a miss for the same location in main cache 908. In this case, cache reads and cache writes can continue in parallel without conflict.

[0113] In another instance, cache reads may be received on a first data path used for the victim cache 910, and cache writes may be received on a second data path used for the victim cache 910. In a specific case, a cache read may be a hit for an address stored in the victim cache 910. A cache write may also be a hit for the same address stored in the victim cache 910. In this case, cache reads may proceed first, and cache writes may be paused until after the cache read is complete. Alternatively, the order of cache writes and cache reads may be based on the data paths on which cache writes and cache reads are received, where cache commands arriving on lower (or higher) numbered data paths are completed before other cache commands. In a specific case, a cache read may be a miss for an address stored in the victim cache 910. A cache write may also be a miss for the same address stored in the victim cache 910. In this scenario, a cache write command can be forwarded to a higher-level cache or memory, and then a cache read can retrieve the data from the higher-level cache or memory after the cache write command has been completed and stored in the victim cache 910.

[0114] In another instance, a first cache read may be received on a first data path used for victim cache 910, and a second cache read may be received on a second data path used for victim cache 910. If the first cache read and the second cache read target different memory addresses, then there is no conflict between a cache hit and a cache miss. In a specific case, the first cache read may be a cache miss targeting a first address in a set of addresses. The second cache read may also be a cache miss targeting a second address in the same set of addresses. If the first cache read and the second cache read have different priorities, then the higher-priority cache or memory is accessed based on the higher of the different priorities. Otherwise, the higher-priority cache or memory is accessed and a set of memory addresses is obtained for storage in victim cache 910. Cases where the first cache read and the second cache read target the same address are handled the same way.

[0115] Figure 10This is a flowchart illustrating the technique for caching data 1000 according to the aspects described herein. In block 1002, the cache system receives a write request for a memory address. For example, a cache system such as an L1 cache may receive a write request from a processor via a processor interface. In block 1004, a first sub-cache of the cache system determines that the memory address is not cached in the first sub-cache. For example, the cache system may include two parallel caches: a main cache and a victim cache. The memory address requested for caching may be checked against both the main cache and the victim cache in parallel. A write miss occurs when a write request for a memory address not included in the cache system is received. In block 1006, a second sub-cache of the cache system determines that the memory address is not cached in the second sub-cache. In block 1008, the data associated with the write request is stored in the second sub-cache. For example, the data requested for writing to memory may be stored in the victim cache. Furthermore, the address associated with the write-to-memory request can be stored in the tag RAM of the victim cache, and the byte-enabled logic associated with the write-to-memory request can be stored in the byte-enabled memory of the victim cache. In block 1010, an indication that the stored data corresponds to a write miss is stored in the line type bit of the second sub-cache. For example, the victim cache includes line type bits associated with lines in the victim cache. Bits can be set to indicate that a specific line in the victim cache contains a line type bit associated with a write miss. In block 1012, the stored data can be flushed based on the indication. For example, stored write-miss data can be flushed from the victim cache to higher-level memory by verifying the line type bits of the cache lines and flushing the lines associated with the write miss.

[0116] Figure 11This is a flowchart illustrating the technology for caching data 1100 according to the aspects described herein. In block 1102, the cache system receives a write request for a memory address. For example, a cache system such as an L1 cache may receive a write request from a processor via a processor interface. In block 1104, a first sub-cache of the cache system determines that the memory address is not cached in the first sub-cache. For example, the cache system may include two parallel caches: a main cache and a victim cache. The memory address requested for caching may be checked against both the main cache and the victim cache in parallel. A write miss occurs when a write request for a memory address not included in the cache system is received. In block 1106, a second sub-cache of the cache system determines that the memory address is not cached in the second sub-cache. In block 1108, the data associated with the write request is stored in the second sub-cache. For example, the data requested for writing to memory may be stored in the victim cache. Furthermore, the address associated with the write memory request can be stored in the tag RAM of the victim cache, and the byte enable logic associated with the write memory request can be stored in the byte enable memory of the victim cache. In block 1110, an indication that the stored data corresponds to a write miss is stored in the line type bit of the second sub-cache. For example, the victim cache contains line type bits associated with lines in the victim cache. Bits can be set in the line type bit indicating that a particular line in the victim cache contains a write miss. In block 1112, the line in the second sub-cache that experienced a write miss is evicted based on an indication that the line has been fully written. For example, the line can be calculated based on byte enable logic indicating that all or half of the bytes of the cache line have been filled with the write data. Filled or half-filled cache lines can be evicted from the victim cache based on bits.

[0117] Figure 12This is a flowchart illustrating the technology for caching data 1200 according to the aspects described herein. In block 1202, the cache system receives a write-to-memory command for a memory address, the write-to-memory command being associated with a first privilege level. For example, a cache system such as an L1 cache may receive a write request from a processor via a processor interface. In block 1204, a first sub-cache of the cache system determines that the memory address is not cached in the first sub-cache. For example, the cache system may include two parallel caches: a main cache and a victim cache. The memory address requested for caching may be checked against both the main cache and the victim cache in parallel. A write miss occurs when a write request for a memory address that is not in either the main cache or the victim cache is received. In block 1206, a second sub-cache of the cache system determines that the memory address is not cached in the second sub-cache, and the second sub-cache is configured to store cached data evicted from the first sub-cache and write-to-memory commands not cached in the first sub-cache in parallel with the first sub-cache. In box 1208, the data associated with the write-to-memory command is stored in the second sub-cache. In box 1210, the color-coded bits associated with the data are stored in the second sub-cache. For example, the color-coded bits may be stored in a bit field of the memory of, for example, MESIRAM. In box 1212, the stored data is evicted based on the color-coded bits. For example, a cache line storing write-miss data associated with a specific color-coded bit may be evicted when the processor asserts a clear signal and a color-coded priority indication. After all cache lines with a specific color-coded bit have been cleared, the cache system may assert an idle signal. If the processor continues to assert clear signals, other cache lines storing write-miss data associated with other color-coded bits may also be evicted.

[0118] Figure 13This is a flowchart illustrating the technique used for caching data 1300 according to the aspects described herein. In block 1302, the cache system receives a write-to-memory command for a memory address, the write-to-memory command being associated with a first color code. For example, a cache system such as an L1 cache may receive a write request from a processor via a processor interface. The write request may include an indication of the color code associated with the write request. In block 1304, a first sub-cache of the cache system determines that the memory address is not cached in the first sub-cache. For example, the cache system may include two parallel caches: a main cache and a victim cache. The memory address requested for caching may be checked against both the main cache and the victim cache in parallel. A write miss occurs when a write request for a memory address not included in the cache system is received. In block 1306, a second sub-cache of the cache system determines that the memory address is not cached in the second sub-cache. In block 1308, the data associated with the write-to-memory command is stored in the cache line of the second sub-cache. For example, the data for the write request can be stored in the cache line of the victim cache. Furthermore, the address associated with the write memory request can be stored in the tag RAM of the victim cache, and the byte-enabled logic associated with the write memory request can be stored in the byte-enabled memory of the victim cache. In block 1310, the first color mark is stored in the second sub-cache. For example, the color mark can be stored using 2 bits in the bit field of the memory of the second sub-cache, such as MESIRAM. In block 1312, a second write memory command for the cache line is received, the write memory command being associated with the second color mark. For example, a second write request can be received for a memory address targeting the same cache line as the first write request. The second write request is associated with the second color mark. The second color mark may be the same as or different from the first color mark. In block 1314, the second color mark and the first color mark are merged. For example, if the first and second color marks represent the same color, the two tags are merged to continue setting that color mark. If the two color marks are different, merging can set both color marks. In box 1316, merged color marks are stored. In box 1318, cache lines can be evicted based on merged color marks. For example, if the first and second color marks represent the same color, eviction can be based on which color mark? If the two color marks are different, eviction can be based on either color mark.

[0119] Figure 14This is a flowchart illustrating the technique used for caching data 1400 according to the aspects described herein. In block 1402, the cache system receives a first write-to-memory command for a memory address, the first write-to-memory command being associated with a first privilege level. For example, a cache system such as an L1 cache may receive write requests from a processor via a processor interface. In block 1404, a first sub-cache of the cache system determines that the memory address is not cached in the first sub-cache. For example, the cache system may include two parallel caches: a main cache and a victim cache. The memory address requested for caching may be checked against both the main cache and the victim cache in parallel. A write miss occurs when a write request for a memory address that is neither in the main cache nor in the victim cache is received. In block 1406, a second sub-cache of the cache system determines that the memory address is not cached in the second sub-cache, and the second sub-cache is configured to store cached data evicted from the first sub-cache and write-to-memory commands not cached in the first sub-cache in parallel with the first sub-cache. In box 1408, the first data associated with the first write memory command is stored in a cache line of the second sub-cache. In box 1410, the first privilege level is stored in the second sub-cache. For example, the first privilege level may include two privilege levels, such as root / guest user / supervisor. These two privilege levels may be stored using 2 bits in a bit field of the memory of the second sub-cache, such as MESIRAM. In box 1412, a second write memory command is received for a cache line, the second write memory command being associated with a second privilege level. For example, a second write request may be received for a memory address targeting the same cache line as the first write request. The second write request is associated with a second privilege level, which may be the same as or different from the first write request. In box 1414, the first privilege level and the second privilege level are merged. For example, the privilege levels may be merged based on a privilege merging table or by selecting the lower privilege level among the merged privilege levels. In box 1416, the merged privilege level is stored. In box 1418, the merged privilege level is output using a cache line. For example, a cache line containing merged first write requests and second write requests can be evicted to a higher-level cache or memory in a memory miss packet containing merged privilege levels.

[0120] Figure 15This is a flowchart illustrating the technique used for caching data 1500 according to the aspects described herein. In block 1502, the cache system receives a write request for a memory address. For example, a cache system such as an L1 cache may receive a write request from a processor via a processor interface. In block 1504, a first sub-cache of the cache system determines that the memory address is not cached in the first sub-cache. For example, the cache system may include two parallel caches: a main cache and a victim cache. The memory address requested for caching may be checked against both the main cache and the victim cache in parallel. A write miss occurs when a write request for a memory address not included in the cache system is received. In block 1506, a second sub-cache of the cache system determines that the memory address is not cached in the second sub-cache. In block 1508, the data associated with the write request is stored in the second sub-cache. For example, the data requested for writing to memory may be stored in the victim cache. Furthermore, the address associated with the write memory request can be stored in the tag RAM of the victim cache, and the byte enable logic associated with the write memory request can be stored in the byte enable memory of the victim cache. In block 1510, an indication that the stored data corresponds to a write miss is stored in the line type bit of the second sub-cache. For example, the victim cache includes line type bits associated with lines in the victim cache. Bits can be set in the line type bit indicating that a particular line in the victim cache contains a write miss.

[0121] Figure 16 This is a flowchart illustrating the technique used for caching data 1600 according to the aspects described herein. In block 1602, the cache system receives two or more cache commands. For example, a cache system using an L1 cache may receive multiple cache commands via a processor interface using, for example, two or more data paths. In block 1604, a conflict is determined to exist between the two or more cache commands. For example, a cache controller may determine that a cache conflict exists between two cache commands. A conflict may occur when two cache commands cause a conflicting change regarding the same cache location. In block 1606, conflict resolution is determined between the received two or more cache commands. In block 1608, two or more cache commands are sent to a first sub-cache and a second sub-cache, and the second sub-cache is configured to store cached data evicted from the first sub-cache and write-to-memory commands not cached in the first sub-cache in parallel with the first sub-cache.

[0122] Example:

[0123] Further exemplary embodiments are provided in the following paragraphs.

[0124] Example 1 includes a caching system comprising: a first sub-cache; and a second sub-cache running parallel to the first sub-cache, wherein the second sub-cache comprises: a set of cache lines; a line type bit configured to store an indication that a corresponding cache line in the set of cache lines is configured to store write-miss data; and an eviction controller configured to flush the stored write-miss data based on the line type bit.

[0125] Example 2 includes the caching system described in Example 1, wherein the second sub-caching further includes: a tag random access memory (RAM) configured to store memory addresses associated with the stored write miss data; and a byte enable memory configured to store byte enable logic associated with the stored write miss data, wherein the eviction controller is configured to flush the stored write miss data by transmitting memory write requests to a higher-level memory based on the stored memory addresses and the stored byte enable logic.

[0126] Example 3 includes the caching system described in Example 1, wherein the caching system further includes a processor interface, and wherein the eviction controller is further configured to: monitor activity on the processor interface; and flush stored write miss data based on the monitored activity.

[0127] Example 4 includes the caching system described in Example 3, wherein monitoring activity on the processor interface includes detecting idle activity on the processor interface.

[0128] Example 5 includes the caching system described in Example 4, wherein detecting idle activity includes determining whether a memory request has been received within a predetermined number of clock cycles.

[0129] Example 6 includes the caching system described in Example 4, wherein the eviction controller is further configured to: determine that activity on the processor interface has been restored; and cancel the flushing of the stored write miss data.

[0130] Example 7 includes the caching system described in Example 6, wherein determining that activity on the processor interface has resumed includes detecting a cache request received from the processor.

[0131] Example 8 includes a method for caching data, comprising: receiving a write-to-memory request for a memory address by a caching system; determining, by a first sub-cache of the caching system, that the memory address is not cached in the first sub-cache; determining, by a second sub-cache of the caching system, that the memory address is not cached in the second sub-cache; storing data associated with the write-to-memory request in the second sub-cache; storing an indication of a write miss corresponding to the stored data in a line-type bit of the second sub-cache; and flushing the stored data based on the indication.

[0132] Example 9 includes the method described in Example 8, wherein flushing the stored data includes transferring a memory write request from the second sub-cache to a higher-level memory.

[0133] Example 10 includes the method of Example 8, which further includes monitoring activity on the processor interface, and wherein flushing the stored data is further based on the monitored activity.

[0134] Example 11 includes the method described in Example 10, wherein monitoring activity on the processor interface includes detecting idle activity on the processor interface.

[0135] Example 12 includes the method described in Example 11, wherein detecting idle activity includes determining that a memory request has not been received within a predetermined number of clock cycles.

[0136] Example 13 includes the method described in Example 11, further comprising: determining that activity on the processor interface has been restored; and canceling the flushing of the stored write miss data.

[0137] Example 14 includes the method described in Example 13, wherein determining that activity on the processor interface has resumed includes detecting cache request reception.

[0138] Example 15 includes an apparatus comprising: a processor; a first sub-cache; and a second sub-cache running parallel to the first sub-cache, wherein the second sub-cache includes: a set of cache lines; a line type bit configured to store an indication that a corresponding cache line in the set of cache lines is configured to store write-miss data; and an eviction controller configured to flush the stored write-miss data based on the line type bit.

[0139] Example 16 includes the apparatus of Example 15, wherein the second sub-cache further includes: a tag random access memory (RAM) configured to store a memory address associated with the stored write miss data; and a byte enable memory configured to store byte enable logic associated with the stored write miss data, wherein the eviction controller is configured to flush the stored write miss data by transmitting a memory write request to a higher-level memory based on the stored memory address and the stored byte enable logic.

[0140] Example 17 includes the apparatus described in Example 15, wherein the cache system further includes a processor interface, and wherein the eviction controller is further configured to: monitor activity on the processor interface; and flush stored write miss data based on the monitored activity.

[0141] Example 18 includes the apparatus described in Example 17, wherein monitoring activity on the processor interface includes detecting idle activity on the processor interface.

[0142] Example 19 includes the apparatus described in Example 18, wherein detecting idle activity includes determining whether a memory request has been received within a predetermined number of clock cycles.

[0143] Example 20 includes the apparatus described in Example 18, wherein the eviction controller is further configured to: determine that activity on the processor interface has been restored; and cancel the flushing of the stored write miss data.

[0144] Example 21 includes a caching system comprising: a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write memory commands not cached in the first sub-cache, wherein the second sub-cache includes: a color-mark bit configured to store an indication associated with the color-mark of a corresponding cache line of the second sub-cache storing write-miss data; and an evict controller configured to evict cache lines of the second sub-cache storing write-miss data based on the color-mark associated with the cache line.

[0145] Example 22 includes the caching system described in Example 21, wherein the second sub-cache includes a bit field configured to store the state of the color mark.

[0146] Example 23 includes the caching system described in Example 22, wherein the bit field includes bits storing the state of each color mark.

[0147] Example 24 includes the cache system described in Example 23, wherein the bit field is stored in a memory that tracks the modification, exclusive, shared, and invalid (MESI) states of entries in the second sub-cache.

[0148] Example 25 includes the cache system described in Example 24, wherein the MESI memory is configured to store an indication that the corresponding cache line of the second sub-cache is writing miss data.

[0149] Example 26 includes the cache system described in Example 21, wherein the cache system further includes a processor interface, and wherein the eviction controller is further configured to: receive a color-coded priority indication on the processor interface; and eviction cache lines storing write-missed data in the second sub-cache based on the received color-coded priority indication.

[0150] Example 27 includes the cache system described in Example 6, wherein the eviction controller is further configured to assert an idle signal on the processor interface after all cache lines of the second sub-cache storing write-missed data having color-mark bits corresponding to the color-mark priority indication have been evicted.

[0151] Example 28 includes a method for caching data, comprising: receiving a write-to-memory command for a memory address by a caching system; determining, by a first sub-cache of the caching system, that the memory address is not cached in the first sub-cache; determining, by a second sub-cache of the caching system, that the memory address is not cached in the second sub-cache, wherein the second sub-cache is configured to store cached data evicted from the first sub-cache and the write-to-memory command not cached in the first sub-cache in parallel with the first sub-cache; storing data associated with the write-to-memory command in the second sub-cache; storing a color scale associated with the data in the second sub-cache; and evicting the stored data based on the color scale.

[0152] Example 29 includes the method of Example 28, further comprising: receiving a color mark priority indication from a processor; receiving a clear signal from the processor; and wherein the evicting of the stored data is based on the received color mark priority indication.

[0153] Example 30 includes the method of Example 29, further comprising: asserting an idle signal after all cache lines of the second sub-cache storing write-missed data having color-mark bits corresponding to the color-mark priority indication have been evicted.

[0154] Example 31 includes the method of Example 30, further comprising: continuing to receive the clear signal from the processor; and evict stored data having color mark bits that do not correspond to the color mark priority indication.

[0155] Example 32 includes the method of Example 39, further comprising: determining that storing the data has not yet been completed; and waiting until the data has been stored.

[0156] Example 33 includes the method of Example 38, which further includes storing the color mark bit in a bit field associated with the corresponding color mark bit.

[0157] Example 34 includes an apparatus comprising: a processor; a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write memory commands not cached in the first sub-cache, wherein the second sub-cache includes: a color-mark bit configured to store an indication associated with the color-mark of a corresponding cache line of the second sub-cache storing write-miss data; and an evict controller configured to evict the cache line storing write-miss data of the second sub-cache based on the color-mark associated with the cache line.

[0158] Example 35 includes the apparatus described in Example 34, wherein the second sub-cache includes a bit field configured to store the state of the color mark.

[0159] Example 36 includes the apparatus described in Example 35, wherein the bit field includes bits storing the state of each color mark.

[0160] Example 37 includes the apparatus described in Example 36, wherein the bit field is stored in a memory that tracks the modification, exclusive, shared, and invalid (MESI) states of entries in the second sub-cache.

[0161] Example 38 includes the apparatus of Example 37, wherein the MESI memory is configured to store an indication that the corresponding cache line of the second sub-cache is storing a write miss.

[0162] Example 39 includes the apparatus of Example 34, wherein the cache system further includes a processor interface, and wherein the eviction controller is further configured to: receive a color-coded priority indication on the processor interface; and eviction cache lines storing write-missed data in the second sub-cache based on the received color-coded priority indication.

[0163] Example 40 includes the apparatus described in Example 39, wherein the eviction controller is further configured to assert an idle signal on the processor interface after all cache lines of the second sub-cache storing write-miss data having color-mark bits corresponding to the color-mark priority indication have been evicted.

[0164] Example 41 includes a caching system comprising: a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write-memory commands not cached in the first sub-cache, the second sub-cache including: a color-mark bit configured to store an indication of a corresponding line of the second sub-cache associated with a color-mark; and an evict controller configured to evict cache lines of the second sub-cache storing write-memory data based on the color-mark associated with the line, wherein the second sub-cache is further configured to: receive a first write-memory command for a memory address, the write-memory command being associated with a first color-mark; store a first data associated with the first write-memory command in a cache line of the second sub-cache; store the first color-mark in the second sub-cache; receive a second write-memory command for the cache line, the write-memory command being associated with a second color-mark; merge the second color-mark with the first color-mark; store the merged color-mark; and evict the cache line based on the merged color-mark.

[0165] Example 42 includes the caching system described in Example 41, wherein the first color mark and the second color mark represent the same color mark.

[0166] Example 43 includes the cache system described in Example 41, wherein the first color mark and the second color mark represent different color marks, and wherein the merged color mark includes the first bit representing the first color mark and the second bit representing the second color mark.

[0167] Example 44 includes the caching system described in Example 43, wherein the second sub-cache is further configured to evict the cache line based on the first color mark.

[0168] Example 45 includes the caching system described in Example 44, wherein the second sub-cache is further configured to evict the cache line based on the second color mark.

[0169] Example 46 includes the cache system described in Example 41, wherein the first color mark and the second color mark are stored in bit fields in a memory that tracks the modification, exclusive, shared, and invalid (MESI) states of entries in the second sub-cache data.

[0170] Example 47 includes the caching system described in Example 46, wherein the bit field contains bits that store the state of each color mark.

[0171] Example 48 includes a method for caching data, comprising: receiving a write memory command for a memory address associated with a first color mark by a caching system; determining, by a first sub-cache of the caching system, that the memory address is not cached in the first sub-cache; determining, by a second sub-cache of the caching system, that the memory address is not cached in the second sub-cache, wherein the second sub-cache is configured to store cached data evicted from the first sub-cache and the write memory command not cached in the first sub-cache in parallel with the first sub-cache; storing first data associated with the first write memory command in a cache line of the second sub-cache; storing the first color mark in the second sub-cache; receiving a second write memory command for the cache line associated with a second color mark; merging the second color mark and the first color mark; storing the merged color mark; and evicting the cache line based on the merged color mark.

[0172] Example 49 includes the method described in Example 48, wherein the first color mark and the second color mark represent the same color mark.

[0173] Example 50 includes the method described in Example 48, wherein the first color mark and the second color mark represent different color marks, and wherein the merged color mark includes the first position representing the first color mark and the second position representing the second color mark.

[0174] Example 51 includes the method described in Example 50, wherein the eviction is based on the first color mark.

[0175] Example 52 includes the method described in Example 51, wherein the eviction is based on the second color mark.

[0176] Example 53 includes the method of Example 48, wherein the first color mark and the second color mark are stored in a bit field in a memory that tracks the modification, exclusive, shared, and invalid (MESI) states of entries in the second sub-cache data.

[0177] Example 54 includes the method described in Example 53, wherein the bit field contains bits that store the state of each color mark.

[0178] Example 55 includes an apparatus comprising: a processor; a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write-memory commands not cached in the first sub-cache, wherein the second sub-cache includes: a color-mark bit configured to store an indication of a corresponding line of the second sub-cache associated with a color-mark; and an evict controller configured to evict cache lines of the second sub-cache storing write-memory data based on the color-mark associated with the line, wherein the second sub-cache is further configured to: receive a first write-memory command for a memory address, the write-memory command being associated with a first color-mark; store first data associated with the first write-memory command in a cache line of the second sub-cache; store the first color-mark in the second sub-cache; receive a second write-memory command for the cache line, the write-memory command being associated with a second color-mark; merge the second color-mark with the first color-mark; store the merged color-mark; and evict the cache line based on the merged color-mark.

[0179] Example 56 includes the apparatus described in Example 55, wherein the first color mark and the second color mark represent the same color mark.

[0180] Example 57 includes the apparatus described in Example 55, wherein the first color mark and the second color mark represent different color marks, and wherein the merged color mark includes a first position representing the first color mark and a second position representing the second color mark.

[0181] Example 58 includes the apparatus described in Example 57, wherein the second sub-cache is further configured to evict the cache line based on the first color mark.

[0182] Example 59 includes the apparatus described in Example 58, wherein the second sub-cache is further configured to evict the cache line data based on the second color mark.

[0183] Example 60 includes the apparatus of Example 55, wherein the first color mark and the second color mark are stored in bit fields in a memory that tracks the modification, exclusive, shared, and invalid (MESI) states of entries in the second sub-cache data.

[0184] Example 61 includes a caching system comprising: a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write-memory commands not cached in the first sub-cache, the second sub-cache including: privilege bits configured to store an indication of a corresponding cache line of the second sub-cache associated with a privilege level, and wherein the second sub-cache is further configured to: receive a first write-memory command for a memory address, the first write-memory command being associated with a first privilege level; store first data associated with the first write-memory command in a cache line of the second sub-cache; store the privilege level associated with the cache line in the second sub-cache; receive a second write-memory command for the cache line, the second write-memory command being associated with a second privilege level; merge the first privilege level and the second privilege level; store the merged privilege level; and output the merged privilege level using the cache line.

[0185] Example 62 includes the cache system described in Example 61, wherein privilege level information is stored in a bit field in a memory that tracks the modification, exclusive, shared, and invalid (MESI) states of entries in the second sub-cache data.

[0186] Example 63 includes the caching system described in Example 62, wherein the privilege level comprises two privilege levels.

[0187] Example 64 includes the caching system described in Example 63, wherein the bit field includes 2 bits for storing the privilege level.

[0188] Example 65 includes the caching system described in Example 61, wherein the second sub-cache includes a privilege merging table that indicates how different privilege levels are merged.

[0189] Example 66 includes the caching system described in Example 65, wherein the privilege merging table is predefined.

[0190] Example 67 includes the caching system described in Example 61, wherein merging the first privilege level and the second privilege level includes selecting the lower privilege level.

[0191] Example 68 includes a method for caching data, comprising: receiving, by a caching system, a first write-to-memory command for a memory address, the first write-to-memory command being associated with a first privilege level; determining, by a first sub-cache of the caching system, that the memory address is not cached in the first sub-cache; determining, by a second sub-cache of the caching system, that the memory address is not cached in the second sub-cache, wherein the second sub-cache is configured to store, in parallel with the first sub-cache, cached data evicted from the first sub-cache and the write-to-memory command not cached in the first sub-cache; storing, by storing, a first data associated with the first write-to-memory command in a cache line of the second sub-cache; storing, by storing, the first privilege level in the second sub-cache; receiving, by receiving, a second write-to-memory command for the cache line, the second write-to-memory command being associated with a second privilege level; merging, by merging, the first privilege level and the second privilege level; storing, by storing, the merged privilege level; and outputting, by the cache line, the merged privilege level.

[0192] Example 69 includes the method described in Example 67, wherein the privilege level comprises two privilege levels.

[0193] Example 70 includes the method described in Example 69, wherein the second sub-cache includes 2 bits of a bit field for storing the privilege level.

[0194] Example 71 includes the method described in Example 68, wherein the merging is based on a privilege merging table that indicates how different privilege levels are merged.

[0195] Example 72 includes the method described in Example 71, wherein the privileged merging table is predefined.

[0196] Example 73 includes the method described in Example 68, wherein merging the first privilege level and the second privilege level includes selecting the lower privilege level.

[0197] Example 74 includes an apparatus comprising: a processor; a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write-memory commands not cached in the first sub-cache, the second sub-cache including: privilege bits configured to store an indication of a corresponding cache line of the second sub-cache associated with a privilege level, and wherein the second sub-cache is further configured to: receive a first write-memory command for a memory address, the first write-memory command being associated with a first privilege level; store first data associated with the first write-memory command in a cache line of the second sub-cache; store the privilege level associated with the cache line in the second sub-cache; receive a second write-memory command for the cache line, the second write-memory command being associated with a second privilege level; merge the first privilege level and the second privilege level; store the merged privilege level; and output the merged privilege level using the cache line.

[0198] Example 75 includes the apparatus described in Example 74, wherein privilege level information is stored in a bit field in a memory that tracks the modification, exclusive, shared, and invalid (MESI) states of entries in the second sub-cache data.

[0199] Example 76 includes the apparatus described in Example 75, wherein the privilege level comprises two privilege levels.

[0200] Example 77 includes the apparatus described in Example 76, wherein the bit field includes 2 bits for storing the privilege level.

[0201] Example 78 includes the apparatus described in Example 75, wherein the second sub-cache includes a privilege merging table indicating how different privilege levels are merged.

[0202] Example 79 includes the apparatus described in Example 78, wherein the privilege merging table is predefined.

[0203] Example 80 includes the apparatus described in Example 75, wherein merging the first privilege level and the second privilege level includes selecting the lower privilege level.

[0204] Example 81 includes a caching system comprising a first sub-cache and a second sub-cache coupled in parallel with the first sub-cache; wherein the second sub-cache includes line-type bits configured to store an indication of a corresponding line in the second sub-cache being written to a miss.

[0205] Example 82 includes the caching system described in Example 81, wherein the second sub-caching further includes a tag random access memory (RAM) configured to store memory addresses associated with the stored write miss data.

[0206] Example 83 includes the caching system described in Example 81, wherein the second sub-cache further includes a byte-enabled memory configured to store byte-enabled logic associated with the stored write-miss data.

[0207] Example 84 includes the caching system described in Example 83, wherein the byte-enabled memory includes a memory location corresponding to each cache line of the second sub-cache.

[0208] Example 85 includes the caching system described in Example 81, wherein the stored indication indicates whether the corresponding cache line of the second sub-cache is storing a miss entry or a cache entry.

[0209] Example 86 includes the cache system described in Example 81, wherein the line type bits are stored in a memory that tracks the modification, exclusive, shared, and invalid (MESI) states of entries in the second sub-cache.

[0210] Example 87 includes the caching system described in Example 81, wherein the first sub-cache is an n-way set-associative cache and wherein the second sub-cache is a fully associative cache.

[0211] Example 88 includes a method for caching data, comprising: receiving a write-to-memory request for a memory address by a caching system; determining, by a first sub-cache of the caching system, that the memory address is not cached in the first sub-cache; determining, by a second sub-cache of the caching system, that the memory address is not cached in the second sub-cache, the second sub-cache being coupled in parallel with the first sub-cache; storing data associated with the write-to-memory request in the second sub-cache; and storing an indication of a write miss corresponding to the stored data in a line-type bit of the second sub-cache.

[0212] Example 89 includes the method of Example 88, further comprising storing the memory address associated with the stored write-miss data in tag random access memory (RAM).

[0213] Example 90 includes the method of Example 88, which further includes storing byte enable logic associated with the stored write-miss data in a byte enable memory.

[0214] Example 91 includes the method of Example 88, wherein the line type bits are stored in a memory that tracks the modification, exclusive, shared, and invalid (MESI) states of entries in the second sub-cache.

[0215] Example 92 includes the method of Example 88, further comprising: evicting cache entries from the line of the second sub-cache, wherein storing data associated with the write memory request includes storing the data in the line.

[0216] Example 93 includes the method described in Example 88, wherein the stored indication indicates whether the corresponding cache line of the second sub-cache is a positive storage write miss entry or a cache entry.

[0217] Example 94. An apparatus comprising: a processor; a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache; wherein the second sub-cache includes line-type bits configured to store an indication of writing miss data on a corresponding line of the second sub-cache.

[0218] Example 95 includes the apparatus described in Example 94, wherein the second sub-cache further includes a tag random access memory (RAM) configured to store memory addresses associated with the stored write miss data.

[0219] Example 96 includes the apparatus described in Example 94, wherein the second sub-cache further includes a byte-enabled memory configured to store byte-enabled logic associated with the stored write-miss data.

[0220] Example 97 includes the apparatus described in Example 96, wherein the byte-enabled memory includes a memory location corresponding to each cache line of the second sub-cache.

[0221] Example 98 includes the apparatus described in Example 94, wherein the stored indication indicates whether the corresponding cache line of the second sub-cache is storing a miss entry or a cache entry.

[0222] Example 99 includes the apparatus described in Example 94, wherein the line type bits are stored in a memory that tracks the modification, exclusive, shared, and invalid (MESI) states of entries in the second sub-cache.

[0223] Example 100 includes the apparatus described in Example 94, wherein the first sub-cache is an n-way set-associative cache and wherein the second sub-cache is a fully associative cache.

[0224] Example 101 includes a caching system comprising: a first sub-cache; and a second sub-cache running parallel to the first sub-cache, wherein the second sub-cache includes: a set of cache lines; line type bits configured to store an indication that a corresponding cache line in the set of cache lines is configured to store a write miss; and an eviction controller configured to evict the cache lines of the second sub-cache storing the write miss based on an indication that the cache lines have been fully written.

[0225] Example 102 includes the caching system described in Example 101, wherein the eviction controller is further configured to evict a cache line storing a second sub-cache that has been written to a cache line based on an indication that half of the cache line has been written.

[0226] Example 103 includes the caching system described in Example 102, wherein the eviction controller is configured to determine, based on the indication that half of the cache line has been written, that no cache line in the second sub-cache storing write-missed data prior to eviction of the cache line has been fully written.

[0227] Example 104 includes the caching system described in Example 102, wherein the second sub-cache further includes a byte-enabled memory configured to store byte-enabled logic associated with the stored write-miss data, and wherein the byte-enabled memory includes a first bit storing the indication that the cache line has been fully written.

[0228] Example 105 includes the cache system described in Example 104, wherein the byte-enabled memory includes a second bit storing the indication that half of the cache line has been written.

[0229] Example 106 includes the caching system described in Example 105, wherein the first and second bits are determined based on the byte enable logic associated with the stored write miss data.

[0230] Example 107 includes the caching system described in Example 106, wherein the first and second bits are determined based on the byte enable logic each time a write miss is stored in the second sub-cache.

[0231] Example 108 includes a method for caching data, comprising: receiving a write-to-memory request for a memory address by a caching system; determining, by a first sub-cache of the caching system, that the memory address is not cached in the first sub-cache; determining, by a second sub-cache of the caching system, that the memory address is not cached in the second sub-cache; storing data associated with the write-to-memory request in the second sub-cache; storing an indication of a write miss corresponding to the stored data in a line type bit of the second sub-cache; and evicting the cache line storing the write miss in the second sub-cache based on an indication that the cache line has been fully written.

[0232] Example 109 includes the method of Example 108, further comprising evicting a cache line storing a second sub-cache that has been written to based on an indication that half of the cache line has been written.

[0233] Example 110 includes the method of Example 109, which further includes determining, based on the indication that half of the cache line has been written, that no cache line in the second sub-cache storing write-missed data prior to evicting the cache line has been fully written.

[0234] Example 111 includes the method of Example 109, which further includes storing the first bit in the byte-enabled memory of the second sub-cache indicating that the cache line has been fully written to.

[0235] Example 112 includes the method of Example 111, further comprising storing a second bit in the byte-enabled memory of the second sub-cache indicating that half of the cache line is written to the second sub-cache.

[0236] Example 113 includes the method of Example 112, wherein the first and second bits are determined based on the byte associated with the stored write miss data.

[0237] Example 114 includes the method of Example 113, wherein the first and second bits are determined based on the byte-enabled logic each time a write miss is stored in the second sub-cache.

[0238] Example 115 includes an apparatus comprising: a processor; a first sub-cache; and a second sub-cache running parallel to the first sub-cache; wherein the second sub-cache includes: a set of cache lines; line type bits configured to store an indication that a corresponding cache line in the set of cache lines is configured to store write-miss data; and an eviction controller configured to evict the cache lines of the second sub-cache storing write-miss data based on an indication that the cache lines have been fully written.

[0239] Example 116 includes the apparatus described in Example 115, wherein the eviction controller is further configured to evict a cache line storing a second sub-cache that has been written to a cache line based on an indication that half of the cache line has been written.

[0240] Example 117 includes the apparatus described in Example 116, wherein the eviction controller is configured to determine, based on the indication that half of the cache line has been written, that no cache line in the second sub-cache storing write-missed data prior to eviction of the cache line has been fully written.

[0241] Example 118 includes the apparatus described in Example 118, wherein the second sub-cache further includes a byte-enabled memory configured to store byte-enabled logic associated with the stored write-miss data, and wherein the byte-enabled memory includes a first bit storing the indication that the cache line has been fully written.

[0242] Example 119 includes the apparatus described in Example 118, wherein the byte-enabled memory includes a second bit storing the indication that half of the cache line has been written.

[0243] Example 120 includes the apparatus of Example 119, wherein the first and second bits are determined based on the byte enable logic associated with the stored write miss data.

[0244] Example 121 includes a caching system comprising: a first sub-cache; a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write-memory commands not cached in the first sub-cache; and a cache controller configured to: receive two or more cache commands; determine a conflict between the received two or more cache commands; determine conflict resolution between the received two or more cache commands; and send the two or more cache commands to the first sub-cache and the second sub-cache.

[0245] Example 122 includes the caching system described in Example 121, wherein the cache command is a read command, and wherein the cache controller is further configured to: determine, based on the memory address of the read command, that the memory address is not cached in the first sub-cache; determine, based on the memory address, that the memory address is stored as a write memory command in the second sub-cache; and suspend the read command until the write memory command is cleared from the second sub-cache.

[0246] Example 123 includes the caching system described in Example 121, wherein the cache controller is further configured to receive two or more cache commands in parallel.

[0247] Example 124 includes the caching system described in Example 123, wherein the two or more cache commands include read commands and write commands, and wherein the cache controller is further configured to: determine a first cache location in a first sub-cache based on a first memory address of the read command; determine that the first memory address is not cached at the first cache location in the first sub-cache; receive a second cache location in a second sub-cache associated with the read command; determine that the second memory address is stored as a write memory command at the second cache location in the second sub-cache based on a second memory address of the write command; receive a third cache location in the second sub-cache associated with the write command; and evict a cache memory entry from the first cache location in the first sub-cache to the third cache location in the second sub-cache.

[0248] Example 125 includes the caching system described in Example 123, wherein the two or more cache commands include read commands and write commands, and wherein the cache controller is further configured to: determine a first cache location in the first sub-cache based on a first memory address of the read command; determine that the first memory address is not cached at the first cache location in the first sub-cache; determine that the second memory address is stored at the first location in the first sub-cache based on a second memory address of the write command; suspend the read command; execute the write command; evict the executed write command to the second sub-cache; and resume the read command after the eviction.

[0249] Example 126 includes the caching system described in Example 123, wherein the two or more cache commands include a first write command and a second write command, and wherein the cache controller is further configured to: determine that a first memory address of the first write command is not cached in the first sub-cache; determine that a second memory address of the second write command is not cached in the first sub-cache; determine that the first memory address matches the second memory address; merge the first write command and the second write command; and store the merged command in the second sub-cache.

[0250] Example 127 includes the caching system described in Example 123, wherein the two or more cache commands include read commands and write commands, and wherein the cache controller is further configured to: determine that a first memory address of the read command is not cached in the first sub-cache; determine that a second memory address of the write command is cached in the first sub-cache; determine that the first memory address matches the second memory address; suspend the write command until the read command completes; and resume the write command.

[0251] Example 128 includes a method for caching data, comprising: receiving two or more cache commands; determining that a conflict exists between the two or more cache commands; determining conflict resolution between the received two or more cache commands; and sending the two or more cache commands to a first sub-cache and a second sub-cache, wherein the second sub-cache is configured to store cached data evicted from the first sub-cache and write-to-memory commands not cached in the first sub-cache in parallel with the first sub-cache.

[0252] Example 129 includes the method of Example 8, wherein the cache command is a read command and the method further includes: determining, based on the memory address of the read command, that the memory address is not cached in the first sub-cache; determining, based on the memory address, that the memory address is stored in the second sub-cache as a write memory command; and pausing the read command until the write memory command is cleared from the second sub-cache.

[0253] Example 130 includes the method described in Example 128, wherein the two or more cache commands are received in parallel.

[0254] Example 131 includes the method of Example 130, wherein the two or more cache commands include read commands and write commands, and the method further includes: determining a first cache location in a first sub-cache based on a first memory address of the read command; determining that the first memory address is not cached at the first cache location in the first sub-cache; receiving a second cache location in a second sub-cache associated with the read command; determining that the second memory address is stored as a write memory command at the second cache location in the second sub-cache based on the second memory address of the write command; receiving a third cache location in the second sub-cache associated with the write command; and evicting a cache memory entry from the first cache location in the first sub-cache to the third cache location in the second sub-cache.

[0255] Example 132 includes the method of Example 130, wherein the two or more cache commands include read commands and write commands, and the method further includes: determining a first cache location in a first sub-cache based on a first memory address of the read command; determining that the first memory address is not cached at the first cache location in the first sub-cache; determining that the second memory address is stored at the first location in the first sub-cache based on a second memory address of the write command; pausing the read command; executing the write command; evicting the executed write command to the second sub-cache; and resuming the read command after the evicting.

[0256] Example 133 includes the method of Example 130, wherein the two or more cache commands include a first write command and a second write command, and wherein the cache controller is further configured to: determine that a first memory address of the first write command is not cached in the first sub-cache; determine that a second memory address of the second write command is not cached in the first sub-cache; determine that the first memory address matches the second memory address; merge the first write command and the second write command; and store the merged command in the second sub-cache.

[0257] Example 134 includes the method of Example 130, wherein the two or more cached commands include read commands and write commands, and the method further includes: determining that a first memory address of the read command is not cached in a first sub-cache; determining that a second memory address of the write command is cached in the first sub-cache; determining that the first memory address matches the second memory address; pausing the write command until the read command completes; and resuming the write command.

[0258] Example 135 includes an apparatus comprising: a processor; a first sub-cache; and a second sub-cache coupled in parallel with the first sub-cache for storing cached data evicted from the first sub-cache and write-memory commands not cached in the first sub-cache; and a cache controller configured to: receive two or more cache commands; determine that a conflict exists between the received two or more cache commands; determine conflict resolution between the received two or more cache commands; and send the two or more cache commands to the first sub-cache and the second sub-cache.

[0259] Example 136 includes the apparatus of Example 135, wherein the cache command is a read command, and wherein the cache controller is further configured to: determine, based on the memory address of the read command, that the memory address is not cached in the first sub-cache; determine, based on the memory address, that the memory address is stored as a write memory command in the second sub-cache; and suspend the read command until the write memory command is cleared from the second sub-cache.

[0260] Example 137 includes the apparatus described in Example 135, wherein the cache controller is further configured to receive two or more cache commands in parallel.

[0261] Example 138 includes the apparatus of Example 137, wherein the two or more cache commands include read commands and write commands, and wherein the cache controller is further configured to: determine a first cache location in a first sub-cache based on a first memory address of the read command; determine that the first memory address is not cached at the first cache location in the first sub-cache; receive a second cache location in a second sub-cache associated with the read command; determine that the second memory address is stored as a write memory command at the second cache location in the second sub-cache based on a second memory address of the write command; receive a third cache location in the second sub-cache associated with the write command; and evict a cache memory entry from the first cache location in the first sub-cache to the third cache location in the second sub-cache.

[0262] Example 139 includes the apparatus of Example 137, wherein the two or more cache commands include read commands and write commands, and wherein the cache controller is further configured to: determine a first cache location in the first sub-cache based on a first memory address of the read command; determine that the first memory address is not cached at the first cache location in the first sub-cache; determine that the second memory address is stored at the first location in the first sub-cache based on a second memory address of the write command; suspend the read command; execute the write command; evict the executed write command to the second sub-cache; and resume the read command after the eviction.

[0263] Example 140 includes the apparatus of Example 137, wherein the two or more cache commands include a first write command and a second write command, and wherein the cache controller is further configured to: determine that a first memory address of the first write command is not cached in the first sub-cache; determine that a second memory address of the second write command is not cached in the first sub-cache; determine that the first memory address matches the second memory address; merge the first write command and the second write command; and store the merged command in the second sub-cache.

[0264] The term "coupling" is used throughout this specification. The term may encompass a connection, communication, or signaling path that achieves a functional relationship consistent with the description herein. For example, if device A generates a signal to control device B to perform an action, then in a first instance, device A is coupled to device B; or in a second instance, if the intermediary component C substantially does not alter the functional relationship between device A and device B, then device A is coupled to device B via the intermediary component C such that device B is controlled by the control signals generated by device A.

[0265] Modifications are possible in the described embodiments, and other embodiments are possible within the scope of the claims.

Claims

1. An eviction method, comprising: Detect idle activity on the processor interface; as well as In response to detecting the idle activity on the processor interface: Identify write miss entries within the victim's cache, wherein the write miss entries are associated with a line type; as well as The write miss entry is cleared from the victim's cache based on the line type.

2. The expulsion method according to claim 1, wherein: The idle activity is detected when the processor has not issued any commands on the processor interface for N clock cycles.

3. The expulsion method according to claim 2, wherein: The N clock cycles are predetermined.

4. The expulsion method according to claim 2, wherein: The N clock cycles are configurable.

5. The expulsion method according to claim 1, wherein: Identifying the write miss entry involves checking each line of the victim cache in logical order to find the write miss entry.

6. The expulsion method according to claim 1, wherein: The write miss entries are multiple write miss entries; Each of the plurality of write-miss entries is stored in the corresponding line of the victim cache; and Clearing the write miss entries includes clearing the multiple write miss entries based on one of first-in-first-out, least recently used, or random basis.

7. The expulsion method according to claim 1, wherein: Determining the write miss entry within the victim cache includes sending an eviction trigger request to the victim cache controller for the selected line of the victim cache associated with the write miss entry.

8. The expulsion method according to claim 7, wherein: In response to receiving the eviction trigger request, the victim cache controller determines whether an entry in the storage queue corresponds to the selected line of the victim cache; as well as In response to determining that the entry exists, the victim cache controller waits for the storage queue to be empty in order to evict the selected line from the victim cache.

9. The expulsion method according to claim 1, further comprising: The write miss entry is stored in the clear buffer until it is accepted by the memory.

10. The expulsion method according to claim 1, wherein: Clearing the write miss entry from the victim cache is in response to the processor writing half of the write miss entry.

11. An apparatus comprising: Victim cache; as well as The victim cache controller is configured to detect idle activity on the processor interface; In response to detecting idle activity on the processor interface, the victim cache controller is configured to: Identify write miss entries within the victim's cache, wherein the write miss entries are associated with a line type; as well as The write miss entry is flushed from the victim's cache based on the line type.

12. The apparatus according to claim 11, wherein: The idle activity is detected when the processor has not issued any commands on the processor interface for N clock cycles.

13. The apparatus according to claim 12, wherein: The N clock cycles are predetermined.

14. The apparatus according to claim 12, wherein: The N clock cycles are configurable.

15. The apparatus according to claim 11, wherein: The victim cache controller is configured to determine the write miss entry by checking each line of the victim cache in logical order to find the write miss entry.

16. The apparatus according to claim 11, wherein: The write miss entries are multiple write miss entries; Each of the plurality of write-miss entries is stored in the corresponding line of the victim cache; and The victim cache controller is configured to flush the write miss entries by flushing the plurality of write miss entries on a first-in-first-out, least recently used, or random basis.

17. The apparatus according to claim 11, wherein: The victim cache controller is configured to identify the write miss entry in the victim cache in response to an eviction trigger request for a selected line of the victim cache associated with the write miss entry.

18. The apparatus according to claim 17, wherein: In response to the eviction trigger request, the victim cache controller is configured to determine whether an entry in the storage queue corresponds to the selected line of the victim cache; as well as In response to determining that the entry exists, the victim cache controller is configured to wait for the storage queue to become empty in order to evict the selected line from the victim cache.

19. The apparatus according to claim 11, wherein: The victim cache controller is configured to store the write miss entry in a flush buffer until the write miss entry is accepted by the memory.

20. The apparatus according to claim 11, wherein: The victim cache controller is configured to flush the write miss entry from the victim cache in response to the processor writing half of the write miss entry.

21. An apparatus comprising: processor; as well as The cache contains: A processor interface coupled to the processor; Cache memory, which includes: A set of cache lines, including a first cache line configurable to store a first write-miss entry; and A set of line type entries, each configured to store a corresponding value, the corresponding value specifying whether a corresponding line in the set of cache lines is associated with a write miss entry; as well as A controller, coupled to the processor interface and the cache memory, and configured to: Detect the status of the processor interface; as well as Based on the aforementioned state: Select the first cache line to be evicted based on the set of line type entries; as well as The first cache line is evicted by providing data packets based on the first write miss entry stored in the first cache line.

22. The apparatus according to claim 21, wherein: The cache includes tag RAM; and The controller is configured to select the first cache line to be evicted by traversing the tag RAM.

23. The apparatus of claim 21, wherein the controller is configured to select the first cache line based on a first-in-first-out policy.

24. The apparatus of claim 21, wherein the controller is configured to select the first cache line based on a least recently used policy.

25. The apparatus of claim 21, wherein the controller is configured to select the first cache line based on a random selection policy.

26. The apparatus according to claim 21, wherein: The cache includes a flush buffer; and The controller is configured to evict the first cache line by storing the data packet in the clear buffer until an acknowledgment is received.

27. The apparatus according to claim 26, wherein: The cache is the first cache; The device further includes a second cache coupled to the processor; The controller is configured to provide the data packet to the second cache; as well as The second cache is configured to provide the confirmation.

28. The apparatus according to claim 21, wherein: The cache memory is the victim's cache memory; and Each of the set of line type entries specifies whether the corresponding line in the set of cache lines is associated with a write miss entry or with a victim cache entry.

29. The apparatus of claim 21, wherein the state is an idle state.

30. The apparatus of claim 29, wherein the controller is configured to select the first cache line to be evicted based on the processor interface being in the idle state during a plurality of clock cycles.

31. An apparatus comprising: interface; A cache memory comprising a set of cache lines, each configurable to store either a first type of entry or a second type of entry, wherein: The second type of entry is a write miss entry; and The set of cache lines includes a first cache line configurable to store a first write miss entry; and A cache controller, coupled to the interface and the cache memory, and configured to: Detect the status of the interface; and Based on the aforementioned state: Select the first cache line to be evicted; and The first cache line is evicted by providing data packets based on the first write miss entry stored in the first cache line.

32. The apparatus according to claim 31, wherein: The cache memory includes a set of line-type entries, each configured to store a corresponding value, the corresponding value specifying whether the corresponding line of the set of cache lines is configured to store a first-type entry or a second-type entry; and The cache controller is configured to select the first cache line to be evicted based on the set of line type entries.

33. The apparatus according to claim 31, wherein: The device includes a tag RAM; and The cache controller is configured to select the first cache line to be evicted by traversing the tag RAM.

34. The apparatus of claim 31, wherein the cache controller is configured to select the first cache line based on at least one of a first-in-first-out policy, a least recently used policy, or a random replacement policy.

35. The apparatus according to claim 31, wherein: The cache memory is the victim's cache memory; and The first type of entry is a victim cache entry.

36. The apparatus of claim 31, wherein the state is an idle state.

37. A method comprising: Store a set of data in the cache line of the cache memory; Store an indication of whether the set of data is associated with a write miss entry; Check the status of the cache interface; as well as Based on the aforementioned state: Based on the indicated instructions, select the set of data to be expelled; as well as Evicting the set of data, wherein the eviction includes providing data packets based on the set of data.

38. The method of claim 37, wherein selecting the set of data comprises traversing a set of tag RAM entries.

39. The method of claim 37, wherein selecting the set of data includes applying at least one of a first-in-first-out policy, a least recently used policy, or a random replacement policy.

40. The method of claim 37, wherein the indication specifies whether the set of data is associated with the write miss entry or with a victim cache entry.

Images